Google Summer of Code 2010

From NUI Group Community Wiki

Jump to: navigation, search

Google Summer of Code 2010

GSoC 2010 is starting and this page is dedicated to general information and ideas that can be used for student proposals. The ideas listed on this page are just that - ideas. They are deliberately somewhat vague, because you are meant to fill in the details. We don't want to see project proposals which are basically cut-n-pasted from our ideas page. Don’t do that! If you do, you will be tossed out very early in the application sorting process. Be creative and extend our project ideas with additional description and your own feedback. Also listed below is contact information for the community and specific mentors.

Past proposals and projects:

Preparing to Apply for Google Summer of Code

The most common question we get from students is "How do I prepare for Google Summer of Code?" The simple answer is not to wait for the program to get involved in open source. Start checking out projects now: take a look at documentation, check out their source code and idle in some IRC channels to see what interests you. The projects who have been past participants in Google Summer of Code are a great place to start; use the ( categorized list of mentoring organizations  (e.g. projects listed by programming languages) from 2009 to help you get started. You can also find the list of student projects for 2009 linked from each accepted organization's page on the ( 2009 site. Google will announce the list of accepted mentoring organizations for 2010 on March 18th.

We don't have exact template for your project proposal to use but we provide a list of information that we would like to see in it which includes: name, email, timezone, age, education, blog url, location, your project proposal, why you'd like to complete this particular project, are you an active NUI Group member, have you been working on multitouch, natural user interfaces, HCI and related topic before, the reason you're the best individual to do so, what's your academic and industry background, and of course open source development experience, explanation of your development methodology is a good idea, as well. Other details as you see fit.

Note for students: Be prepared to supplement your proposal text with links to an external site, for example full version of your proposal in pdf file. You should also plan to provide an abstract of your proposal, including a brief list of deliverables, via the GSoC web app to ensure that your work receives sufficient review.

Make sure to read: Getting Started with GSoC (

Additional resources to read:

Here you can find some additional documents and webpages that could be useful when writing your project proposal. Sometimes they link to other open source projects, but contain many useful guidelines which can be used when writing your NUI Group proposal.

Propose your own idea

You really don't have to use project ideas we covered below. These are projects that we have in mind to start working on or think that would be really useful for NUI Group community. Think of them as starters, even thought some developers think it’d be great to see them done. If you have your own idea, bring it on. We are always open for new innovative project ideas, so feel free to amaze us with yours. We’ll judge applications on their strengths, not on who wants the project done or who thought of it. However you might consider and find some of the project ideas listed below interesting. If you (as a student) have a new idea that is not listed here feel free to edit this project ideas wiki page and add your idea with your contact info.

The fastest and quickest way to get feedback is by posting your idea or questions on the forums!

Here: - If accepted to GSoC 2010 we will be opening up a sub-forum strictly for student/mentor questions and proposals.


Feel free to contact us whenever you have any questions or if you want to use an idea, but you need more details.

You can find us at #nuigroup FreeNode IRC channel look for nicknames: nuiman, whitenoize, cerupcat

Join us at NUI Group forum at or our GSoC NUI Group discussion group at to discuss GSoC & NUI Group project ideas and related topics.

You can also drop an email to our organization administrator at Image:1177495748.Png


2010 Mentors:
Christian Moore (Skype: naturalui, IRC: nuiman, nui Forum: Christian Moore)
Pawel Solyga (IRC\Forum: solydzajs)
Thomas Hansen (IRC\Forum: thansen)
Mathieu Virbel (IRC: tito)

Projects Ideas

Some ideas might not be suitable for GSoC proposal because they are too easy and it's development won't last 3 months. Remember that you can still select this idea and combine it with other ideas in order to write GSoC proposal which meets GSoC requirements. In addition, some of the projects ideas might be to hard to develop during the summer by one student developer, therefore you can limit it and just work on particular parts. If you become interested you can always work on the rest features after GSoC - that's our little open source world :-)

General Idea Keywords

  • Computer vision (tracking, optical flow, image processing)
  • Touch, gesture and stylus input
  • Machine learning (AI, Neural Nets, Bayesian clustering, genetic algorithms)
  • User Interface and Experience design (Human computer interaction)
  • Industrial design for sensing and projection/LCD displays
  • Mobile interaction concepts
  • Programming Languages: Actionscript 3.0, C-Sharp, C++, Java, Python and others.
  • Also consider mixing custom devices with your software project (open source hardware) ... big bonus points :)

CCV (Community Core Vision)

  • Code refactoring and modularization of vision techniques/filters and controls.
  • ROI Selection
  • Code Optimization
  • Implement Threading and Separation of Tracking and GUI
  • GPU Acceleration (existing but not fully implemented)
  • Dynamic Filter Addition and Adjustment (in the chain)
  • Object/Fidicual Tracking
  • Multiple Camera Control and Support (Multiple Cameras/Stitching and Alignment)
  • Enhance Setup GUI - Create more detailed interface for setup and configuration of the vision engine and offer a more dynamic calibration system.
  • Wii Remote support for object/finger recognition/tracking - Wii Remote support so we could use the wii-remote as an input device, we are looking for Windows, Linux and Mac versions
  • Add multi-camera support - Implement stereo vision techniques and algorithms. Have tools for camera alignment and calibration.
  • Physical Object Tracking - Work on tag (fidicual) recognition, create a separate library which can be shared other project or merge existing library (reactivision) into CCV
  • Testing framework - write a testing framework for easy blob detection and tracking algorithms testing, write unit tests for touchlib, write automated tests
  • Windows Service support for(a daemon) - add Windows Service support to touchlib so it can run in the background as system service instead of user desktop application, define and write communication layer between service and configuration application. Autostart setup possibilty for exhibits. like start osc, start flosc and start flashdemo.swf (possible with batch files, more user friendly approch)

You can learn more and get the latest source here:


  • Develop higher level application frameworks based on the existing TUIO client implementations (C++, C# ...)
  • Implement TUIO client libraries for programming languages or environments that are not yet supported
  • Implement TUIO server modules for existing tracker applications that are not yet TUIO enabled
  • Pick up the development of the QMTSim TUIO Simulator
  • Write TUIO server for iPhone (sending TUIO events from your iPhone to Desktop computer using WiFi, record your TUIO events for further replay, timeline support); use available open source C++ TUIO server reference implementation


  • UI: Hand-orientation aware widgets - E.g., Keyboard could be splitted in half: One half for each hand, and the respective half follows the hand it belongs to and rotates with it automatically.
  • UI: More high-level events - 2 finger gestures (ex: zoom), 3 finger gestures (ex: tilts in 3D apps), etc.
  • UI: Orientation-less gestures - Gesture patterns: recognize a cross or a circle whatever the orientation.
  • UI: Web Browser - MTWebContext widget (uBrowser, awesomium)
  • UI: Widgets - more widget types
  • UI: layout containers - Opportunity here to create a PyMT UI builder tool within pyMT.
  • UI: Smarter input device - Swype (smarter keyboard) (; OCR writing; Text editing patterns (Apple patents pending here... "sweep left with 2 fingers deletes whole word" for instance).
  • Network: Web connectivity - Remote web-service access with RESTful APIs.
  • Network: Multiple TUIO clients awareness and TUIO routing - Benefits: PyMT apps that several MT surfaces can use simultaneously (distributed MT space); Avoid video-stitching on the tracker and use two trackers instead. (This multiplexing feature will be part of the next generation TUIO protocol)
  • Logging: Tasks/Sessions - Propper logging of user interaction to quantitative data and visual analysis of such data.
  • Apps: More example apps - Productivity: GIS, Web frontend tools; Play: games, soundsynth, learning tools, physical objects (fiducials), etc.

GSoC 2009 ideas for PyMT on pymt-dev


  • Shader Effect integration -> Integrate a little GLSL shader framework into MT4j to allow effects like blurring or glowing for example for any visible MT4j component.
  • Create the groundwork for zoomable user interfaces (ZUIs) using MT4j -> While zoom-in functionality is already present in MT4j there is no way to "semantically" zoom to navigate to another part of an application or to another application entirely and smoothly zoom in while streaming and loading the new content to be displayed. This would allow for "endless zooming" and to navigate huge data conveniently.
  • Complete multi-touch UI toolkit -> Create a comprehensive MT4j widget toolkit (windows, lists, tables, audioplayer etc) with features like skinnable appearance, automatic layouts using the model-view-controller paradigm
  • Create and explore new, specialized UI-Components tailored to usability and interaction on multi-touch displays -> For example: new forms of menus (e.g. circular menus - see:, lists (e.g. like in Cooliris or other components that allow to browse content in a user friendly, innovative way
  • Network serialization -> Create the possibility to send and share MT4j components across networks with other MT4j applications and implement a showcase application (e.g. document sharing, image sharing etc).
  • Test framework -> Create the possibility to have automatic testing by pre-recording user input and then playing it back while asserting the current state. This could also include stress-tools that send random input to applications or widgets to check if they break.
  • 3D Model animation -> Implement 3D model animations with bone animation and skinning to allow to animate imported 3D models.
  • 3D modeling tool -> Create a multi-touch 3D modeling tool using MT4j
  • Multi-Touch games -> Create innovative multi-touch games like a warcraft-like real time strategy game for example, using MT4j

NUI Enabled Clients Application Proposals

Typically developed in AS3, C/C++/C#, Java, Python or a Visual Programming Language (Max/PD)

  • Module Based Application Loading
  • Advance Menu System for Application Loading and Control
  • Media Loader (Document, PDF, Video, Image)(jpg,jpeg,tga,png,dds,gif,tif, tiff, bmp,mov,avi,wav,ogg)
  • Particle Sandbox (C/C++)
  • Physics Sandbox (C/C++)
  • Paint and Drawing Sandbox (Brushes, Filters and Tools)
  • Educational Games (Match/Math Quiz/Beat the Chimp)
  • 3D Object Viewing and Manipulation (Standard 3D files)
  • Gesture Sandbox - Gesture recognition and programming that saves to flat files.
  • On Screen Keyboard and Input Devices
  • Dynamic Loading of Files via USB/Firewire or Bluetooth
  • Image Manipulation - Use MT to bend images etc
  • Audio Creation and Games - (Bloom... Loop Sequencer)
  • GSI Mapping
  • Web Browser

Mobile - Android/iPad/iPhone/iPod Touch Development

  • Touch App - propose innovative multitouch iPhone application or educational game
  • Gesture Implementation- Write multitouch gesture recognition library for iPhone (defining your own gestures)
  • Audio Analyzer- Basic audio analyst tool for iphone (spectrum/sonogram/FFT)
  • Accelerometer- Collects and sends Acceleration Data out via OSC

Community Web Apps

Custom developed web applications typically written in Coldfusion/PHP/Ruby/Python/Java

  • Browser Real-time Chat
  • Community Research and Resource Centralization
  • Forum Data Analyst and Restructuring
  • Custom NUI Search Engine and Google SEO
  • Automatic Multitouch Creator (input goal, output parts list and specifics, etc)
  • Any Web Application that could benefit Community...

Other Project Ideas

  • Linux-based stand alone blob tracker - create a lightweight Linux-distro for ultra light pc that includes firewire-camera drivers, blobtracking software and sends the touch information trough ethernet or even better trough USB to the main computer. Also a controlling software is needed for every operating system to configure this linux-box. Also is important to determine how fast processor is really needed for tracking
  • MultiTouch mouse drivers for every OS - create mouse drivers for OsX, Windows and Linux so that they can be used without external mouse trough MT-display
  • On-Screen Keyboards - create on-screen keyboard software for OsX, Windows and Linux so no external keyboard is needed to operate regular programs trough MT-display
  • Collaborative multisite collaboration using multitouch screens - write an application/framework which will allow for multisite collaboration on multitouch screens, what it means is that you can have many multitouch screen in different locations connected via internet or internal network and mt screen works as shared desktop with gestures support and synchronization between screens. You can have a look at croqodile project (, TeaTime protocol used in Croquet project for syncing, solution for any platform will be most welcome, whether it's Flash/Flex/AIR or OpenGL or any other platform specific software
  • WiiMote multitouch framework - create a framework for writing multitouch applications such as interactive whiteboard using WiiMote under Mac OS X (DarwiinRemote project might be helpful), write some example applications
  • Work on WiiMote Whiteboard project - create new example applications, port WiiMote Whiteboard to Linux and Mac, write unit tests and create installer with binary distribution
  • Improve blob detection and tracking algorithms - write optimized library for blob detection and tracking so that it can be used later with different projects ex. tbeta, touchlib, bbtouch
  • Your innovative multitouch application - propose new innovative multitouch application you would like to work on (interaction in 3d environment, creating scenes, documents), amaze us with your ideas
  • Work on WiiMoteTUIO project - example applications, bug fixing, create a cross-platform solution, write test and documentation
  • Automatic projector calibration - propose a project for automatic projector calibration used with multitouch screens in DI and FTIR setups
  • Multitouch gesture library - write multitouch gesture recognition library with support for gestures that could consist of several strokes like /\ + - = A, simple one finger gestures like mouse gestures and multitouch gestures (zooming, pinching, rotating and many more), use fingerworks gesture dictionary for example gestures (
  • Multitouch applications on Mac OS X - propose and write multitouch application for Mac OS X, using TUIO protocol for events and new Leopard graphic frameworks like Core Animation, Core Image, Core Video, QuickTime etc.
  • Firefox TUIO extension - write a Firefox TUIO client extension which maps TUIO events to JavaScript events and adds gestures support in Firefox using same JavaScript extensions used in WebKit/Safari introduced in iPhone SDK
  • Research multicomputer use for image processing - work on distributed blob detection and tracking solution, dividing computation between many machines, this would be really useful for big installations like multitouch walls and with many cameras, test your solution and integrate with OpenTouch or touchlib
  • MB Pro and MB Air multitouch trackpad application for Mac OS X - propose and write multitouch application which supports MacBook Air and MacBook Pro multitouch trackpad gestures or extend existing open source project to use the power of intuitive trackpad gestures
  • GPU accelerated blob detection and tracking algorithms - write GPU accelerated blob detection and tracking algorithms for detecting and tracking blobs. You can use CG for NVidia or compare usage of GPUCV vs OPENCV

Related Resources

Comments or Questions

We are open for any comments and questions here: