Getting Started with CCV
From NUI Group Community Wiki
Community Core Vision - Getting Started - Full Guide
This Guide covers the majority of necessary components of getting started with CCV. A Community Core Vision (tbeta = 1.1, CCV 1.2 and CCV 1.3) diagram shows the parts and features of the tracker. The configuration and calibration sections are very important and should be followed carefully for best results.
Main Project Page:
Developers can see the code documenation here:
- Pentium 4+ (Recommended Core 2 Duo+)
- 512MB+ RAM (1024+)
- Web Camera (for live tracking)
1. Source image - Displays the raw video image from either camera or video file.
2. Use Camera Toggle - Sets the input source to camera and grabs frames from selected camera.
3. Use Video Toggle - Sets the input source to video and grabs frames from video file.
4. Previous Camera Button - Gets the previous camera device attached to computer if more than one is attached.
5. Next Camera Button - Gets the next camera device attached to computer if more than one is attached.
6. Tracked Image - Displays the final image after image filtering that is used for blob detection and tracking.
7. Inverse - Track black blobs instead white blobs.
8. Threshold Slider - Adjusts the level of acceptable tracked pixels. The higher the option is, the bigger the blobs have to be converted in tracked blobs.
9. Movement filtering - Adjust the level of acceptable distance (in pixels) before a movement of a blob is detected. The higher the option is, the more you have to actually move your finger for CCV to register a blob movement.
10. Min Blob Size - Adjust the level of acceptable minimum blob size. The higher the option is, the bigger a blob has to be to be assigned an ID.
11. Max Blob Size - Adjust the level of acceptable maximum blob size. The higher the option is, the bigger a blob can be before losing its ID.
12. Remove Background Button - Captures the current source image frame and uses it as the static background image to be subtracted from the current active frame. Press this button to recapture a static background image
13. Dynamic Subtract Toggle - Dynamically adjusts the background image. Turn this on if the environmental lighting changes often or false blobs keep appearing due to environmental changes. The slider will determine how fast the background will be learned..
14. Smooth Slider - Smoothes the image and filters out noise (random specs) from the image.
15. Highpass Blur Slider - Removes the blurry parts of the image and leaves the sharper brighter parts.
16. Highpass Noise - Filters out the noise (random specs) from the image after applying Highpass Blur.
17. Amplify Slider - Brightens weak pixels. If blobs are weak, this can be used to make them stronger.
18. On/Off Toggle - Used on each filters, this is used to turn each filter on or off.
19. Camera Settings Button - Opens a camera settings box. This will open more specific controls of the camera, especially when using a PS3 Eye camera.
20. Flip Vertical Toggle - Flips the source image vertically.
21. Flip Horizontal Toggle - Flips the source image horizontally.
22. GPU Mode Toggle - Turns on hardware acceleration and uses the GPU. This is best used on newer graphics cards only. Note: GPU mode is still in early development and may not work on all machines.
23. Send UDP Toggle - Turns on the sending of TUIO messages.
24. Flash XML - Turns on the sending of Flash XML messages (no need for flosc anymore).
25. Binary TCP - Turns on the sending of RAW messages (x,y coordinates).
26. Enter Calibration - Loads the calibration screen.
27. Save Settings - Saves all the current settings into the XML settings file.
In order to track fingers or objects, CCV first needs to be configured. The goal is to have a final tracked image that has white blobs coming from fingers or objects placed on the touch surface (with no background noise or false blobs visible or tracked). In order to reach this goal, various image filter sliders are adjusted. By following the steps below, a final tracked image with clear and distinct blobs should be achievable (like the one you see on the picture above). The settings that are used in the picture above are good ones to start with. I advise you to play a little bit with each slider to understand how it works a get a nice tracked image.
1. Download and extract the latest build of tbeta. Open the tbeta folder.
3. The application will launch and a camera or video image should be displayed under Source Image.
5. While no fingers are present, press the background remove button or "b" to capture the background. If in an environment where lighting changes often, turn on dynamic subtract.
6a. (FTIR and LLP users only)
7. Lastly, adjust the threshold slider in the Tracked Image until only fingers are tracked and no false blobs are present.
8. Press Save Settings to save the current settings. note: settings will also be auto-saved on exit.
9. Continue to the Calibration guide below to perform calibration.
In order to calibrate CCV for your camera and projector, you'll need to run the calibration process. Calibrating allows touch points to line up with elements on screen. This way, when touching something displayed on screen, the touch is registered in the correct place. In order to do this, CCV has to translate camera space into screen space; this is done by touching individual calibration points. Following the directions below will help explain how to setup and perform calibration.
For those displaying an image on the touch surface (projector or LCD) , you'll need to set up your computer so that the main monitor is the video projector so that CCV is displayed on the touch surface.
Now that everything is configured nicely and blobs are tracked well in the Tracked Image box, let's calibrate.
1. Press the enter calibration button or "c" to enter the calibration screen.
There is a white bounding box that surrounds the calibration points. If the sides of the bounding box are visible and aligned with the touch surface or a visual image is not being displayed on the touch surface, skip to Step 3. Otherwise, continue.
2. If the white bounding box is not fully visible or not aligned with the projected touch surface, follow the directions under Aligning Bounding Box to Projection Screen displayed on the CCV screen to align the bounding box and calibration points so they fit the touch surface.
To Align the Bounding Box:
3. If using a wide angle camera lens or want higher touch accuracy, more calibration points can be added by following the Changing Grid Size directions on screen. note: adding additional calibration points will not affect performance.
Hold "shift" with the above to add or remove points along the y-axis.
4. Now, you can begin calibration by pressing "c."
5. A red circle will highlight over the current calibration touch point. Follow the directions on screen and press each point until all targets are pressed.
If not projecting an image on the touch surface, you can guess or draw the touch points directly on the touch surface so you know where to press.
If a mistake is made, press "r" to return to the last touch point. If there are false blobs, press "b" to recapture the background.
6. After completion the calibration screen will return and accuracy can be tested by pressing on the touch area. If calibration is inaccurate, calibrate again (Step 4) or return to the main configuration window ("x") to adjust the filters better.
Changing Input Resolution, Frame Rate, Communication, Video, and Blob Settings
tbeta currently uses a config.xml file to save settings. Not all features of tbeta made it into the interface of the first tbeta release; therefore, some features need to be changed manually.
The following features can be manually changed by editing the config.xml file:
- Camera Resolution and Frame Rate
- Network Communication Address and Ports
- Test Video File
- Maximum Number or Blobs to Find
1. If tbeta is open, close tbeta (ESC key).
2. Open the tbeta data folder.
2. Locate the config.xml file.
3. Right click config.xml, and open with a Text Edit program.
4. Find the YOU CAN MANUALLY EDIT THE FEATURES BELOW' section at the top of the config.xml file.
Locate the CAMERA SETTINGS section. Edit the WIDTH and HEIGHT values to your desired resolution. tbeta will automatically adjust to use the closest resolution availabe from the camera.
Edit the FRAMERATE value. tbeta will automatically adjust to use the closest frame rate available from the camera.
Locate the NETWORK COMMUNICATION SETTINGS section. Edit LOCALHOST, TUIOPORT_OUT and TUIOPORT_IN to desired address and ports for sending TUIO data.
Locate the VIDEO SETTINGS section. Edit FILENAME to file name of the video you want loaded. Make sure the video is in the test_videos folder of tbeta.
Locate the BLOBS section. Edit MAXNUMBER to the maximum number of blobs you want tbeta to look for.
6. Save the config.xml file and launch tbeta to see the new settings.
Running Flash Demos
1. Find some flash demos.
Java 6+ is required for running FLOSC: http://java.com/en/download/
3. Windows users, make sure to click Start once FLOSC launches. On all platforms, the command line (terminal) should display created.
4. In order to run demo swf files, security settings first need to be set. Go to Flash Global Security Settings. Click on Edit Locations... and then Add Location.
5. In the Trust this location box, put in the absolute path or browse to the demo folder where the demo swf files is located. Click Confirm and you're done. Any future swfs added to the demo folder will automatically have set permissions.
6. Launch a flash demo (swf file)(get the latest flash player here : standalone flashplayer).