From NUI Group Community Wiki

Jump to: navigation, search

Simple webcams work very well for multitouch setups, but they need to be modified first. Regular webcams and cameras block out infrared light, letting only visible light in. We need just the opposite. Typically, by opening the camera up, you can just pop the filter off, but on expensive cameras this filter is usually applied directly to the lens and cannot be modified.

The performance of the multi-touch device depends on the used components. Therefore it is important to carefully select your hardware components. Before buying a camera it is important to know for what purpose you will be using it. When you are building your first (small) test multi-touch device, the requirements may be lower than when you are building one that is going to be used for demonstration purposes.

  1. Resolution: The resolution of the camera is very important. The higher the resolution the more pixels are available to detect finger or objects in the camera image. This is very important for the precision of the touch device. For small multi-touch surfaces a low resolution webcam (320 x 240 pixels) can be sufficient. Larger surfaces require cameras with a resolution of 640x480 or higher in order to maintain the precision.
  2. Frame rate: The frame rate is the number of frames a camera can take within one second. More snapshots means that we have more data of what happened in a specific time step. In order to cope with fast movements and responsiveness of the system a camera with at least a frame rate of 30 frames per second (FPS) is recommended. Higher frame rates provide a smoother and more responsive experience.
  3. Interface: Basically there are two types of interfaces that can be used to connect a camera device to a computer. Depending on the available budget one can choose between a consumer grade webcam that uses a USB interface or a professional camera that is using the IEEE 1394 interface (which is also known as FireWire). An IEEE 1394 device is recommend because it usually has less overhead and lower latency in transferring the camera image to the computer. Again, lower latency results in a more responsive system.
  4. Lens type: Most consumer webcams contain an infrared (IR) filter that prevents IR light from reaching the camera sensor. This is done to prevent image distortion. However for our purpose, we want to capture and use IR light. On some webcams it is possible to remove the IR filter. This filter is placed behind the lens and often has a red color. If it is not possible to remove the IR filter, the lens has to be replaced with a lens without coating. Webcams often use a M12 mount. Professional series cameras (IEEE 1394) often come without a lens. Depending on the type it is usualy possible to use a M12, C or CS mount to attach the lens. Choosing the right lens can be a difficult task, fortunately many manufactures provide an online lens calculator. The calculator calculates the required focal length based on two input parameters which are the distance between the lens and the object (touch surface) and the width or height of the touch surface. Be sure to check if the calculator chooses a proper lens. Lenses with a low focal length often suffer from severe image distortion (Barrel distortion / fish eye), which can complicate the calibration of the touch tracking software.
  5. Camera sensor & IR bandpass filter: Since FTIR works with IR light we need to check if our webcam is able to see IR light. Often user manuals mention the sensor name. Using the sensor name one can find the data sheets of the camera sensor. The data sheet usually contains a page with a graph similar as below. This image is belongs to the data sheet of the Sony ICX098BQ CCD sensor.

Make sure the lens on your camera can view the entire table surface area. For example a 4.3mm focal lens on Philips SPC900 views an area of about 24x27 inches when mounted 90cm away. It is recommended to use a camera with a high frame rate (30fps or higher) and little low light noise.

Before we can actually use the camera it is required to add a bandpass filter. When using the (IR sensitive) camera, it will also show all other colors of the spectrum. In order to block this light one can use a cut-off filter or a bandpass filter. The cut-off filter blocks light below a certain wave length, the bandpass filter only allows light from a specific wavelength to pass through. Bandpass filters are usually quite expensive, a cheap solution is to use overexposed developed negatives.

Security camera shops sell IR lenses, make sure you get one that is compatible with your camera, most webcams use M12x0.5 mount. Most cameras will show some infrared light without modification, but much better performance can be achieved if the filter is replaced. In general, cameras with CCD sensors should perform better than those with a CMOS chip.

Firewire cameras do have some benefits over normal USB webcams:

  • Higher framerate
  • Capture size
  • Higher Bandwidth
  • Less overhead for driver (due to less compression)

See list for an overview of the different cameras people are using inside this community.

Possible Cameras

Low End Webcams:

  • Sony Playstation Eye Camera Using AlexP's * driver, this camera can now be used on a computer (Vista, XP, Mac, Linux). One of the best cameras if on a budget as it can usually be found for retail $40USD, or under. 320x240 resolution with up to 125 frames per second, 640x480 resolution with up to 75 frames per second.
  • Microsoft Xbox 360 Camera Another popular budget camera, similarly priced to the PS3 one. 640x480 resolution at 30 frames per second.
  • Sweex WC001 Low frame rate & resolution, manages but only for playing interactions.

High End Webcams:

  • Philips SPC900NC
  • Unibrain Fire-i
  • Used Philips Vesta Pro (PCVC680K) - 30fps at VGA resolution, extremely easy to remove the IR filter, great results. No official Vista drivers - Windows 2003 included ones work well (here). Linux support - status unknown, drivers available. New linux driver support for philips cams here

Professional/Industrial Cameras:

DV Cameras: