iPi Desktop Motion Capture

User Guide

Preparing for video capture

Recommended camera for use with iPi Desktop Motion Capture is Sony PlayStation 3 Eye.

Majority of webcams, camcorders and point-and-shoot digital photo cameras with video recording capability can also be used successfully for markerless motion capture. We recommend that you choose a camera with the following properties:

For the multi-camera video, you should use webcams (or other USB or Firewire cameras) and record a multi-camera video using iPi Recorder. Current release of iPi Desktop Motion Capture does not have other multi-camera options.

For a multi-camera system, it is highly recommended that you use cameras of the same model.

iPi Recorder is a stand-alone application and does not require a powerful video card. You may choose to install iPi Recorder on a notebook PC for portability.

We recommend that you record multi-camera video at 640x480 resolution. Higher resolution does not improve accuracy in this release. You may also get decent results at 320x240 (uncompressed) if shooting at close range.

You will need a flashlight or LED or some other bright small object for calibration in multi-camera configuration. We recommend using MAGLITE MINI flashlight in “candle mode”. http://www.maglite.com/product.asp?psc=2AACELL

Actor should be dressed in black long-sleeve shirt, solid-color pants and solid-color shoes. Reasonably tight-fitting clothes are preferable.

Light background is preferable. Minor background clutter is OK.




Multi-camera system calibration

Using multiple cameras for markerless motion capture is more complex then single camera. You should know your camera intrinsic parameters (notably, diagonal field of view) and camera system should be calibrated (to find out relative position of cameras).

Diagonal field of view for your camera should be specified in camera documentation. Some manufacturers (for example, Logitech) do not specify field of view for their cameras. You can use iPi Camera Calibration Tool to find out field of view of your cameras.

Diagonal field of view for Sony PlayStation 3 Eye camera is 75 degrees.

In this release, the method for calibrating multi-camera system is glowing marker based calibration. You can use a LED, a flashlight with reflector removed, or some other small bright object for calibration.

Once you calibrated the camera system, you should not move your cameras for subsequent video shoots. If you move at least one camera, you need to perform calibration again.

Take note of height of your first camera over the ground. You will need this parameter later. If you cannot measure this height accurately, then at least make a rough estimation.

We recommend that you record calibration sequence into a separate file.

With Sony PS Eye camera you should turn on Calibration Mode during calibration in iPi Recorder. With other cameras, you can set expose to a minimum if your cameras have configurable exposure. This should result in reduced motion blur.

First couple of seconds of calibration video should contain background only (no marker). Then move the LED marker through your entire capture volume (front-top-right-bottom-left-back-top-right-bottom-left).

(Recommended) You can mark the ground plain so that your animations can be captured in correct coordinate system. Place the marker as close to ground plain as possible and move it to at least 3 distinct locations on the ground (they should NOT be on the same line).

Open calibration video in iPi Studio. Adjust region of interest to contain your moving marker sequence.

Set Field of View for your cameras on the “Scene” tab. Be sure to set Field of View for all cameras.

Move and rotate virtual cameras so that their positions and orientations very roughly correspond to real positions and orientations.

Click “Calibrate” button on “Calibration” tab and wait while the system finishes calibration.

Calibration algorithm may occasionally fail to find correct camera position. In particular, this may happen if left and right cameras are swapped. If this happens, move virtual cameras using camera orbit control on “Scene” tab and click “Calibrate” again. This will cause calibration (but not marker tracking) to be repeated.

Now you can mark the frames that contain ground points. You need at least 3 ground points (they should not be on the same line; ideally, they should make a large triangle). Starting from 3rd point, the system will re-adjust ground plain taking into account each new point.

Adjust the scale of your camera configuration. You can use distance between cameras or height of camera#1 over the ground as a reference for adjusting scale.

Save you calibrated scene to a file.

Working with multi-camera video

Record a multi-camera video. The video should have the following structure:

  1. First couple of seconds: background only.

  2. Actor in T-pose, palms looking at the camera (for better skin color visibility).

  1. The motion.




Camera should not move during video recording.

Recommended frame rate is 50 or 60 frames per second.

Open multi-camera video in iPi Studio. Load scene configuration from file.

Adjust the Background Region and Region of Interest. (Region of Interest should cover the part of video that contains the motion).

Go to T-pose frame. Using character move/rotate controls, roughly align the model with the image of actor in one view. Adjust character height (on “Actor” tab) to correspond to actor height. Push the “Analyze Appearance” button on “Actor” tab. For the multi-camera video, “Analyze Appearance” button uses only one (current) camera. This makes it easier to align model with image. You can refine actor appearance by alternating “Refit Pose” and “Analyze Appearance”.

Go to the first frame of Region of Interest. Push the “Refit Pose” button on “Tracking” tab. If initial pose was recognized incorrectly, you can roughly adjust it manually and use auto-fit again.

Push the “Track Forward” button on “Tracking” tab.

You can save your motion capture project to a file. If your project is saved to a file, then the system will auto-save it after each processed frame during tracking.

Import/Export

To export animation with default iPi Studio skeleton, use “Export Animation” from “File” menu.

You can also export in 3d MAX Biped format and in iClone format using corresponding menu items.

To transfer your motion to a different rig, use “Import Target Character” menu item to import target rig.

Supported rigs in this release are:

Working with single-camera video

Single camera mode is experimental and is not officially supported.

Camera should not move during video.

The video should have the following structure:

  1. First couple of seconds: background only.

  2. Actor in T-pose, palms looking at the camera (for better skin color visibility).

  3. The motion.

Open video in iPi Studio.

Adjust the Background Region and Region of Interest. (Region of Interest should cover the part of video that contains the motion).

Go to T-pose frame. Using camera pan/tilt/zoom controls, roughly align the model with the image of actor. Push the “Analyze Appearance” button on “Actor” tab.

Go to the first frame of Region of Interest. Push the “Refit Pose” button on “Tracking” tab. If initial pose was recognized incorrectly, you can roughly adjust it manually and use auto-fit again.

Push the “Track Forward” button on “Tracking” tab.

Resulting motion will most certainly contain depth inaccuracies. You can clean them up using Rotate, ZCorrect and ZReflect tools. You can edit many frames at ones by selecting them with Strobe Range.