Hi all,

I´ve recorded different arm movement sequences, and have found that oftentimes when the forearms are in front of the torso tracking is lost, and that manual adjustments are usually not enough to regain convergence of the forward tracking algorithm, since tracking is usually lost again shortly after.
The dance demo video posted on [1] --which dates back to 2008-- seems to deal with this quite effectively (and its motions are _considerably_ more complex than mine). Was much manual intervention required to obtain these results?. Also, were these results obtained using algorithms available in the public version of DMC or only on a dev branch?. I´m using version 1.0.0.87 Beta 2, and have tried at least with two previous versions, always with similar results.

As usual, here are the details of my current setup, I use three cameras capturing at 320x240@30fps (located roughly North, NorthEast, NorthWest, with the actor facing North), good illumination, a light background, and the actor is light-skinned, dark-haired and wears blue jeans, black short-sleeved T-shirt over green long-sleeved T-shirt.

[1] http://www.youtube.com/watch?v=R-it7qdg ... _embedded#

TIA,

Adolfo Rodríguez Tsouroukdissian, Ph. D.

Robotics engineer
PAL ROBOTICS S.L
http://www.pal-robotics.com
Tel. +34.93.414.53.47
Fax.+34.93.209.11.09

Thanks for good questions. Previously you sent me one of your videos, so I can confirm that you certainly use good lighting and proper actor clothing.

Tracking is very sensitive to the accuracy of modeling of actor appearance (shape, color, scale etc.). Our current actor model has limited number of scale/shape settings and is not perfectly accurate. In particular, head and hands are a bit disproportionate. A better model requires serious re-design of actor model and re-design of tracking algorithm - a big change and a large amount of work (currently in progress).

Tracking accuracy and reliability may seriously vary for different actors. Some of our demos use custom/experimental models and obscure fine-tuning for particular actor. We still need to invent a more universal approach to actor shape modeling.

In current version, you can improve tracking reliability by playing with actor scale. Try adjusting actor scale based on good arms/hands modeling instead of good head modeling. Specifically, try fine-tuning actor scale based on actor pose with hands down (as opposed to T-pose). Keep in mind that good match of top-of-the-shoulders is more important then good match of top-of-the-head.

You should also consider upgrading to PlayStation Eye cameras (http://www.newegg.com/Product/ProductLi ... ye&x=0&y=0). Latest release of iPi Recorder fully supports PSEye cameras. PSEye cameras have low-distortion optics and can achieve more accurate calibration. PSEye cameras can shoot at frame rates up to 60 FPS at 640x480. (But you need a fast computer to be able to capture at high frame rates). We recently discovered that Logitech cameras introduce some strange non-linear distortions (apparently caused by internal camera video scaling rather then optics). PSEye cameras have native resolution of 640x480 and are very accurate in raw Bayer 640x480 mode. (Please note that raw Bayer videos saved to file look like grayscale videos. This is normal. Latest version of iPi Studio fully supports PSEye raw Bayer videos).

I should note that our system is designed with the goal of producing "good looking" animation rather then an accurate recording of actor motion. Our system tries to "guess" the data when it cannot accurately "measure". This is usually OK for animators but can represent a bit of a problem if you use it for robotics design. Naturally, you cannot expect sub-$100 cameras to produce data on par with expensive specialized industrial cameras. One of the biggest sources of inaccuracies is noise in camera sensors. Noise is by far a bigger problem then camera resolution; this is a complex problem, we still have to invent a way to filter it out.

One way to improve accuracy is to use more cameras. Currently, we support up to 4 cameras. With PSEye cameras, we think we'll soon be able to support up to 6 cameras on a single PC. (Sony's PSEye cameras are much more bandwidth-efficient then all other existing USB cameras due to some clever engineering tricks.)

Thanks for the insight on what things influence more capture performance. I would have loved to read this post when I first started playing with DMC. Maybe some comments on these lines could be added to the documentation pages. Will try to tweak the actor as best as possible.

Cheers,

Adolfo

Oh, and about this:


We're not using mocap to achieve precise motions, but rather "good looking" motions with a natural feel (think gestures), so I guess we're good here :). What I'm trying to measure and understand here is how smooth a motion can be captured with DMC. To this end, I'm trying to address as much as possible weak points in our setup (physical, callibration, actor model, ...), how to properly deal with body occlusions/limb crossing (this thread), and other issues related to filtering out residual noise (posted in another thread). Your tips are most appreciated, since they help us to better channel our efforts.

Adolfo

Usually my DMC fails to track a limb in a range of frames especially in high speeds maybe due to motion blur.

So I think if you want more accurate tracking, try the Hi-Res Tracking with Foot tracking enabled even though it will take longer to process unlike the normal tracking. I think the time you will spend manually correcting the mistracked limbs in normal tracking is just the same if you run it with Hi Res.

My method on manually correcting the limbs is by correcting it frame by frame using only the Rotate Tool on the elbow and shoulders (usually the shoulders). Using IK in correcting them is time consuming and very inaccurate.

Once done, turn the smoothness up to 6 and that should get rid of the wobbling.

In my experience, the more frequent occurrences of tracking loss are temporary and last for 10-15 frames, after which tracking is regained. This occurs even for slow arm motions, and is triggered by the arm coming in front of the body in at least one of the cameras. Captured motions during the lost-tracking period are characterized by joint errors of up to 20-30 degrees and (consequently) high wrist velocities.
Hi-res tracking (with foot tracking) does not appear to solve matters here. I always use it, since the output of regular pose tracking usually contains unacceptable levels of noise. I've always thought of hi-res tracking as something to smooth out smaller perturbations, and not big jumps such as the ones described above. Now, hi-res tracking is a not-so documented feature, so I may be using it in a non-optimal way for the lack of insight on its internals. For example, what does the coarse-smooth index stand for? (see [1]), what would be a reasonable value for it in this case?

Best,

Adolfo


[1] viewtopic.php?f=5&t=1758

As we can see lots of noise in the latest release in the non actor areas of the frame from the Logitec cameras. I am thinking less GAIN would help?more lighting and less gain?

Noise in non-actor areas should not bother you at all because it does not affect tracking. Noise in actor areas has some small effect on tracking.

Noise on video is unavoidable with current level of video camera technology.
For now, I recommend to leave it as is.

Brighter lighting should always improve your results. High contrast is more important then noise.