Google Summer of Code 2009

From NUI Group Community Wiki

Revision as of 14:57, 12 August 2009 by (Talk)
(diff) ←Older revision | Current revision (diff) | Newer revision→ (diff)
Jump to: navigation, search

Proposals guideline

The student application period begins Monday, March 23th 2009 and ends Friday, April 3rd 2009.

The ideas listed on this page are just that - ideas. They are deliberately somewhat vague, because you are meant to fill in the details. We don't want to see project proposals which are basically cut-n-pasted from our ideas page. Don’t do that! If you do, you will be tossed out very early in the application sorting process. Be creative and extend our project ideas with additional description and your own feedback.

We don't have exact template for your project proposal to use but we provide a list of information that we would like to see in it which includes: name, email, timezone, age, education, blog url, location, your project proposal, why you'd like to complete this particular project, are you an active NUI Group member, have you been working on multitouch, natural user interfaces, HCI and related topic before, the reason you're the best individual to do so, what's your academic and industry background, and of course open source development experience, explanation of your development methodology is a good idea, as well. Other details as you see fit.

Note for students: Be prepared to supplement your proposal text with links to an external site, for example full version of your proposal in pdf file. You should also plan to provide an abstract of your proposal, including a brief list of deliverables, via the GSoC web app to ensure that your work receives sufficient review. - List of 2009 Proposals - Mentor meeting notes.

Additional resources to read:

Here you can find some additional documents and webpages that could be useful when writing your project proposal. Sometimes they link to other open source projects, but contain many useful guidelines which can be used when writing your NUI Group proposal.

Make Sure you see:

Propose your own idea

You really don't have to use project ideas we covered below. These are projects that we have in mind to start working on or think that would be really useful for NUI Group community. Think of them as starters, even thought some developers think it’d be great to see them done. If you have your own idea, bring it on. We are always open for new innovative project ideas, so feel free to amaze us with yours. We’ll judge applications on their strengths, not on who wants the project done or who thought of it. However you might consider and find some of the project ideas listed below interesting. If you (as a student) have a new idea that is not listed here feel free to edit this project ideas wiki page and add your idea with your contact info.

The fastest and quickest way to get feedback is by posting your idea or questions on the forums!



Feel free to contact us whenever you have any questions or if you want to use an idea, but you need more details.

You can find us at #nuigroup FreeNode IRC channel look for nicknames: nuiman, nui, whitenoize, cerupcat

Join us at NUI Group forum at or our GSoC NUI Group discussion group at to discuss GSoC & NUI Group project ideas and related topics.

You can also drop an email to our organization administrator at Image:1177495748.Png


Project Admins:
Christian Moore (Skype: naturlaui, IRC: nuiman, nui Forum: Christian Moore) and Pawel Solyga (IRC\Forum: solydzajs)

Thomas Hansen (IRC\Forum: thansen), Martin Kaltenbrunner( ), Laurence Muller (IRC\Forum: Falcon4ever), Alex Popovich (Forum: alexp), Seth Sandler (IRC\Forum: cerupcat), Mathieu Virbel (IRC: tito)

Projects Ideas

Some ideas might not be suitable for GSoC proposal because they are too easy and it's development won't last 3 months. Remember that you can still select this idea and combine it with other ideas in order to write GSoC proposal which meets GSoC requirements. In addition, some of the projects ideas might be to hard to develop during the summer by one student developer, therefore you can limit it and just work on particular parts. If you become interested you can always work on the rest features after GSoC - that's our little open source world :-)


  • Hand Tracking (DI/DSI)
  • Finger Orientation
  • ROI Selection
  • Code Optimization
  • Implement Threading and Separation of Tracking and GUI
  • GPU Acceleration (existing but not fully implemented)
  • Dynamic Filter Addition and Adjustment (in the chain)
  • Object/Fidicual Tracking
  • Better Camera Control and Support (Multiple Cameras/Stitching and Alignment)

Here is a recent snapshot of our SVN:


  • Enhance Setup GUI - Create more detailed interface for setup and configuration of the vision engine and offer a more dynamic calibration system.
  • Wii Remote support for object/finger recognition/tracking - Wii Remote support so we could use the wii-remote as an input device, we are looking for Windows, Linux and Mac versions
  • Add multi-camera support - Implement stereo vision techniques and algorithms. Have tools for camera alignment and calibration.
  • Physical Object Tracking - Work on tag (fidicual) recognition, create a separate library which can be shared with Touchlib/OpenTouch and other project or merge existing library (reactivision) into Touchlib
  • Testing framework - write a testing framework for easy blob detection and tracking algorithms testing, write unit tests for touchlib, write automated tests
  • Windows Service support for touchlib (touchlib as a daemon) - add Windows Service support to touchlib so it can run in the background as system service instead of user desktop application, define and write communication layer between service and configuration application
  • Autostart setup possibilty for exhibits. like start osc, start flosc and start flashdemo.swf (possible with batch files, more user friendly approch)


  • develop higher level application frameworks based on the existing TUIO client implementations (C++, C#, Java ...)
  • implement TUIO client libraries for programming languages or environments that are not yet supported
  • implement TUIO server modules for existing tracker applications that are not yet TUIO enabled
  • add TUIO support to MPX (multi pointer X Server) via a xorg input driver xorg-input-tuio.
  • pick up the development of the QMTSim TUIO Simulator
  • write TUIO server for iPhone (sending TUIO events from your iPhone to Desktop computer using WiFi, record your TUIO events for further replay, timeline support); use available open source C++ TUIO server reference implementation


  • UI: More high-level events - 2 finger gestures (ex: zoom), 3 finger gestures (ex: tilts in 3D apps), etc.
  • UI: Orientation-less gestures - Gesture patterns: recognize a cross or a circle whatever the orientation.
  • UI: Web Browser - MTWebContext widget (uBrowser, awesomium)
  • UI: Widgets - more widget types
  • UI: layout containers - Opportunity here to create a PyMT UI builder tool within pyMT.
  • UI: Smarter input device - Swype (smarter keyboard) (; OCR writing; Text editing patterns (Apple patents pending here... "sweep left with 2 fingers deletes whole word" for instance).
  • Network: Web connectivity - Remote web-service access with RESTful APIs.
  • Network: Multiple TUIO clients awareness and TUIO routing - Benefits: PyMT apps that several MT surfaces can use simultaneously (distributed MT space); Avoid video-stitching on the tracker and use two trackers instead. (This multiplexing feature will be part of the next generation TUIO protocol)
  • Logging: Tasks/Sessions - Propper logging of user interaction to quantitative data and visual analysis of such data.
  • Apps: More example apps - Productivity: GIS, Web frontend tools; Play: games, soundsynth, learning tools, physical objects (fiducials), etc.

GSoC 2009 ideas for PyMT on pymt-dev

Other projects that you might work on

  • BBTouch (OpenTouch)
  • touche
  • grafiti
  • Also consider mixing custom devices with your project (open source hardware) ... big bonus points :)

NUI Enabled Clients Application Proposals

Typically developed in AS3, C/C++/C#, Java, Python or a Visual Programming Language (Max/PD)

  • Module Based Application Loading
  • Advance Menu System for Application Loading and Control
  • Media Loader (Document, PDF, Video, Image)(jpg,jpeg,tga,png,dds,gif,tif, tiff, bmp,mov,avi,wav,ogg)
  • Particle Sandbox (C/C++)
  • Physics Sandbox (C/C++)
  • Paint and Drawing Sandbox (Brushes, Filters and Tools)
  • Educational Games (Match/Math Quiz/Beat the Chimp)
  • 3D Object Viewing and Manipulation (Standard 3D files)
  • Gesture Sandbox - Gesture recognition and programming that saves to flat files.
  • On Screen Keyboard and Input Devices
  • Dynamic Loading of Files via USB/Firewire or Bluetooth
  • Image Manipulation - Use MT to bend images etc
  • Audio Creation and Games - (Bloom... Loop Sequencer)
  • GSI Mapping
  • Web Browser

iPhone/iPod Touch Development

  • Touch App - propose innovative multitouch iPhone application or educational game
  • Gesture Implementation- Write multitouch gesture recognition library for iPhone (defining your own gestures)
  • Audio Analyzer- Basic audio analyst tool for iphone (spectrum/sonogram/FFT)
  • Accelerometer- Collects and sends Acceleration Data out via OSC

Community Web Apps

Custom developed web applications typically written in Coldfusion/PHP/Ruby/Python/Java

  • Browser Real-time Chat
  • Community Research and Resource Centralization
  • Forum Data Analyst and Restructuring
  • Custom NUI Search Engine and Google SEO
  • Automatic Multitouch Creator (input goal, output parts list and specifics, etc)
  • Any Web Application that could benefit Community...

Other Project Ideas

  • Linux-based stand alone blob tracker - create a lightweight Linux-distro for ultra light pc that includes firewire-camera drivers, blobtracking software and sends the touch information trough ethernet or even better trough USB to the main computer. Also a controlling software is needed for every operating system to configure this linux-box. Also is important to determine how fast processor is really needed for tracking
  • MultiTouch mouse drivers for every OS - create mouse drivers for OsX, Windows and Linux so that they can be used without external mouse trough MT-display
  • On-Screen Keyboards - create on-screen keyboard software for OsX, Windows and Linux so no external keyboard is needed to operate regular programs trough MT-display
  • Collaborative multisite collaboration using multitouch screens - write an application/framework which will allow for multisite collaboration on multitouch screens, what it means is that you can have many multitouch screen in different locations connected via internet or internal network and mt screen works as shared desktop with gestures support and synchronization between screens. You can have a look at croqodile project (, TeaTime protocol used in Croquet project for syncing, solution for any platform will be most welcome, whether it's Flash/Flex/AIR or OpenGL or any other platform specific software
  • WiiMote multitouch framework - create a framework for writing multitouch applications such as interactive whiteboard using WiiMote under Mac OS X (DarwiinRemote project might be helpful), write some example applications
  • Work on WiiMote Whiteboard project - create new example applications, port WiiMote Whiteboard to Linux and Mac, write unit tests and create installer with binary distribution
  • Improve blob detection and tracking algorithms - write optimized library for blob detection and tracking so that it can be used later with different projects ex. tbeta, touchlib, bbtouch
  • Your innovative multitouch application - propose new innovative multitouch application you would like to work on (interaction in 3d environment, creating scenes, documents), amaze us with your ideas
  • Work on WiiMoteTUIO project - example applications, bug fixing, create a cross-platform solution, write test and documentation
  • Automatic projector calibration - propose a project for automatic projector calibration used with multitouch screens in DI and FTIR setups
  • Multitouch gesture library - write multitouch gesture recognition library with support for gestures that could consist of several strokes like /\ + - = A, simple one finger gestures like mouse gestures and multitouch gestures (zooming, pinching, rotating and many more), use fingerworks gesture dictionary for example gestures (
  • Multitouch applications on Mac OS X - propose and write multitouch application for Mac OS X, using TUIO protocol for events and new Leopard graphic frameworks like Core Animation, Core Image, Core Video, QuickTime etc.
  • Firefox TUIO extension - write a Firefox TUIO client extension which maps TUIO events to JavaScript events and adds gestures support in Firefox using same JavaScript extensions used in WebKit/Safari introduced in iPhone SDK
  • Research multicomputer use for image processing - work on distributed blob detection and tracking solution, dividing computation between many machines, this would be really useful for big installations like multitouch walls and with many cameras, test your solution and integrate with OpenTouch or touchlib
  • MB Pro and MB Air multitouch trackpad application for Mac OS X - propose and write multitouch application which supports MacBook Air and MacBook Pro multitouch trackpad gestures or extend existing open source project to use the power of intuitive trackpad gestures
  • GPU accelerated blob detection and tracking algorithms - write GPU accelerated blob detection and tracking algorithms for detecting and tracking blobs. You can use CG for NVidia or compare usage of GPUCV vs OPENCV

Related Resources

Comments or Questions

We are open for any comments and questions here...