Building an interactive floor

From NUI Group Community Wiki

Jump to: navigation, search

By Donovan Solms

This wiki page explains how to build an interactive floor. An interactive floor differs mainly in the fact that multiple people can walk over and interact with the same floor space as opposed to single person interaction of regular interactive floors. A typical setup would only cover a part of the floor, ie. a lift lobby, or right at the entrance door, because covering an entire lobby - although possible - is unpractical.

Note that this should not be attempted as your first multitouch project. The reason for this is that when you build the interactive floor, you will then have the insight needed to identify problems before you go to far into the project. Depending on the setup of the room and people installing equipment in the roof, this project will take 4-8 hours. The money spent is extremely dependent on the size of the room and optional parts required. No fixed figure can be given.

Before beginning with the project, search for possibilities to go into the roof for the setup. This will enable you to hide the setup from its users and only have a small hole where the projector projects through. Also, the light conditions in the room is one of the first things to consider. Right next to a window that has sun most of the day will not work. One can, however, control the light in the room by covering the windows with IR blocking film.

You will notice no IR illuminators are needed. The reason for that is that 95% of lobby areas will have a lighting system already installed. They already emit IR in an evenly spread way. If the lobby does not have a lighting system, you can easily add an illuminator next to the camera.

The shininess of the floor is usually not a problem. It actually helps tracking in some situations. In short - the less shiny the floor, the better the result. But if the floor is semi-shiny – or reflective – don't break your head about it. White floors are prefered as they give the best projection.

Warning: Because the setup relies on having a projector and camera right above the floor you must ensure that all parts in the roof – or hanging from the roof – is 100% secure. If not, the parts could easily drop from the roof onto a person injuring him/her or severely damaging parts.

Contents

Different techniques

There are basically two ways that you can build the setup. The first being the ability to go into the roof for the installation, the second not being able to. We will cover the In-Roof technique first, then the Below-Roof technique. The following figure describes the two techniques.

Image:Interactive_floor_illustration.png

Illustration 1: Main techniques used in an interactive floor setup

In-roof technique

The tools required for every interactive floor differs. This list is only a guide to what tools you would generally need.

  1. Projector: Try keeping the projector as light as possible. Also use a wide-angle lens for the projector – or a short-throw projector to ensure the entire floor gets covered.
  2. Camera: Use a camera with a wide-angle lens to ensure the entire projected image gets covered.
  3. Hammer and nails or screwdriver and screws: Depending on how you plan to install the camera into the roof.

Step by step construction

Warning: Use a stable ladder. Have an extra person with you to help as a projector falling onto you will result in injury.

Step 1: Mount the projector

First, you need to make a hole in the roof for the projector lens to fit through. Detailed intructions on this cannot be given because of the vast number of roof types. The most important point for this step is that the projector should be secured 100% in the middle of the roof. Also ensure that the required area is filled by the projection.

Step 2: Install the camera

A CCTV-like camera will be most preferred due to the fact that they come standard with roof mounts. If yours does have a roof mount, use nails or screws to secure it to the roof. If your camera does not have a roof mount you will need to buy an extra camera mount and install it as per the mount's instructions.

Step 3: Ensure the safety of the setup

It is advised to put the setup through a test before proceeding. Have an extra person close to help with this part.

  1. Wiggle the camera a bit to see if it is secure. If not, secure it more, otherwise continue.
  2. Wiggle the projector a bit to see if it is secure. If not, secure it more, otherwise continue.

Usually you should leave the setup overnight and check the next morning to inspect for any safety issues.

Step 4: Testing

At this point you should have the installation done and ready for testing. How you plug the projector and camera into the computer is up to you. You will most probably need extension cables for the projector and camera. Install the software of choice and test the setup, adjust as needed.

Below-roof technique

The tools required for every interactive floor differs. This list is only a guide to what tools you would generally need.

  1. Projector: Try keeping the projector as light as possible. Also use a wide-angle lens for the projector – or a short-throw projector – to ensure the entire floor gets covered.
  2. Camera: Use a camera with a wide-angle lens to ensure the entire projected image gets covered.
  3. Hammer and nails or screwdriver and screws: Depending on how you plan to install the camera into the roof.
  4. Projector mount: As light as possible, but strong as well.
  5. Mirror: Any will do.
  6. Mirror mount: Usually a metal arm that can be attached to any surface.

Step by step construction

Warning: Use a stable ladder. Have an extra person with you to help as a projector falling onto you will result in injury.

Step 1: Mount the projector mount

Firstly mount the projector mount as per manufacturer's instructions.

Step 2: Mount the projector

Mount the projector into the projector mount as per manufacturer's intructions.

Step 3: Mount the mirror mount

Mount the mirror mount as per manufacturer's instructions.

Step 4: Mount the mirror

Mount the mirror into the mirror mount as per manufacturer's intructions.

Step 5: Install the camera

A CCTV-like camera will be most preferred due to the fact that they come standard with roof mounts. If yours does have a roof mount, use nails or screws to secure it to the roof. If your camera does not have a roof mount you will need to buy an extra camera mount and install it as per the mount's instructions.

Step 6: Ensure the safety of the setup

It is advised to put the setup through a test before proceeding. Have an extra person close to help with this part.

  1. Wiggle the camera a bit to see if it is secure. If not, secure it more, otherwise continue.
  2. Wiggle the projector a bit to see if it is secure. If not, secure it more, otherwise continue.
  3. Wiggle the mirror a bit to see if it is secure. If not, secure it more, otherwise continue.

Usually you should leave the setup overnight and check the next morning to inspect for any safety issues.

Step 7: Testing

At this point you should have the installation done and ready for testing. How you plug the projector and camera into the computer is up to you. You will most probably need extension cables for the projector and camera. Install the software of choice and test the setup, adjust as needed.


Calibration

The following are the steps on how to calibrate the floor using tBeta or Touchlib:

  1. Start tBeta or Touchlib
  2. Get a flashlight
  3. Shine the flashlight on the floor and calibrate the filter of tBeta or Touchlib to only see the flashlight. In other words, set rectify to a rather high amount.
  4. Turn of the flashlight, recapture background (press 'b') in tBeta or Touchlib.
  5. Press 'c' to bring up the calibration, then hit 'c' again to start the calibration.
  6. Aim the flashlight at the green cross with the red circle around it.
  7. Turn the flashlight on and turn it off again as quick as you can. It should be as a quick flash. tBeta or touchlib should have picked it up as if it was a touch to a normal screen.
  8. Repeat step 6 until calibration is complete for all the crosses.
  9. You should now have a quite accurate calibration.

Things to keep in mind while calibrating:

  1. Try to stand clear of the camera's view area, this would interfere with the calibration.
  2. Try to get the flashlight as close as possible to the camera. In other words, try to keep the light from the flashlight as close to a circle as you can. If you stand at a very low angle compared to the camera you'll see the flashlight making an elliptical light shape and thus making the calibration less accurate and more difficult to get right.

If you can't get very close to the camera, calibration is still perfectly possible, it will just take some extra time.

Software used

In order to calibrate the system, you can use touchlib or tBeta. Any tracking software that supports the filters needed for diffused illumination will also work. You need to set your software to only detect large blobs. People moving on the floor will then be picked up optimally. A midrange rectify and midrange blur is suggested.

When setting up the software, ensure that you setup and test it in varying light conditions. This will greatly affect the setup. The recommended conditions are during the day, around 12:00, not overcast (if it can be avoided). This will give the greatest accuracy in how the floor will work on a day-to-day basis.

Conclusion

This setup is fairly simple. Literally every setup differs so you need to play around with most of the steps to find what works best for your setup. The basic concepts covered here will always apply. It is advised to have some multitouch experience before building this. That would help you identify problematic areas on the floor before you start, and will save time and money.