Move light VR interaction does the company use cell phone cameras to do gesture

Is probably a large number of companies are aware of the problem of lack of interactive mobile VR, a variety of interactive programmes to emerge this year. Outside in space directly positioned on the machine; there are Leap Motion, uSens, fretting, and so do the binocular camera gesture tracking; also useful Intel Inside out with RealSense spatial positioning, Google handle Daydream with 3 degrees of freedom, and so on. Vera Bradley wholesale

Now a domestic venture companies entered the field, made a mobile VR interaction scenarios based on cell phone cameras. The Beijing venture company named yingmeiji, its product called Hand of CV. Compared with the preceding various interactive scenarios, Hand CV is the biggest feature is the light, do not require additional hardware devices, achieved only through the phone's camera and software, and is freely licensed.

A light gesture interaction

So-called light natural gesture interaction in terms relative to the gesture interaction, Zhu Yu, CEO of the yingmeiji Plexus, Leap like Motion, uSens are gesture interaction, not only do they require additional hardware-assisted, but needs more power and performance.

Specific to the actual functionality and gesture interaction tracking hands finger joint movement, want to reproduce your hands in a virtual world, like Leap Motion pictured as a Demo presentation. Hand light CV's gesture interaction is realized through the different signs choose dragging, clicks, and other basic operations, as this video demonstrates.

Video: Hand CV single gesture recognition demo

To cite an example, by Leap in Motion in the virtual world you can use hand picked up an object and throw it out, but with a Hand CV failed to do so, because the single camera without precise depth information, while using either image recognition algorithms can calculate the depth of the fuzzy data.

According to the official, Hand CV techniques are:

Ordinary camera acquisition screen, frame-by-frame hand detection, segmentation. Segment information by k-COS and other clustering algorithms for feature extraction of target detection, and finally extracted characteristics data for gesture recognition. Gesture recognition using hidden Markov models on identifying sample data for repeating.

Hand gestures of the CV version 2.0 is expected to support several of the types shown in the following figure (proposed), and Zhu Yu Cong says manufacturers can customize the gesture according to their needs.

This product will first support iOS devices, mainly because of Apple's hardware relatively simple and stable. On power consumption, for A8 processor devices, it uses 8% to 10% performance. In order to reduce power consumption, it is "standby" State each one frame of that 2-3 second camera picture, 12 frames per second when using the frequency.

In September, the company plans on the App Store online product Demo, and SDK, the official version is expected to be released early next year by the end of this year.

Captures the pain points of mobile VR?

Small series Hand CV experience and found that the whole operation is very simple, start up soon, just remember a few gestures and gesture recognition speed and accuracy are pretty good.

In promoting this product, yingmeiji free licensing model is cooperation among the cool, odd, etc. Content makers simply Hand CV into our App, can make VR cell phone box user gesture. Drag the video is playing, pausing, and while the game is usually a light interactive games, such as card.

However, this technology will meet all computer vision (CV) technical problems, whether monocular or binocular camera, in very dim light, bright light, and in noisy environments may encounter can't identify the problem. In this regard, Zhu Yu Cong said they are building a library with 100,000 gesture model, hope to improve robustness of technology through deep learning.

Zhu Yu bundles yingmeiji's goal is to become the basis for ARVR interaction technology suppliers, and he thinks that VR is a, the technology of the future in terms of AR value will be much larger. At present the company has licensed technology to AR glasses manufacturer shadow technology. Vera Bradley cell phone case

In addition to basic interactions, Zhu Yu Cong also disclosed that the company paid with MasterCard to develop VR products, mainly sign password. In addition, the companies also hope to apply the technology to automobiles, robots and other products above.

The revenue side, the yingmeiji does not currently intend to make money from ARVR manufacturer, but hope to gain enough market share to earnings matter. Meanwhile the company in cooperation with the intellectual property agency, hopes to do some technology licensing revenue, for example, mobile phone manufacturer in the product into their technology.

Yingmeiji in 2014 by Zhu Yu Cong, CEO and CTO Xiaobo Li co-founded, has been engaged in technology development in computer vision, products previously developed AR shopping, but this year the focus has been on Hand on the CV of this product.

For the next direction, Zhu Yu bundles is first gesture recognition, other computer vision technology to explore, but there's no other product directions. He said technology was not done to virtuoso, but a good scenario. In addition, he thinks Slam is over the value of startups, now it's too late.

Bet Hand CV, Zhu Yu Cong opinion because this product captures the users of mobile VR experience pain points. He thinks that interactive single purpose market, heavy interaction is handles marketing, camera gesture tracking based on depth, which is wrong, does not have enough market.

In immersion, with gestures instead of mobile phone boxes by "turn + a click on the button" or "turn + touchpad" interaction design looks like a good solution, but can be recognized by users also need further observation of its product launch.