To get up do date with openframeworks I made some audioreactive demos to show what you can do with this great framework.

I had a long list of very cool demos/examples from others and made some remixes of their work. It also was a good exercise to transfer older code to the latest 0071 version and to convert  code to compile with VS2010. Most of the examples you get on Github are cool but only act as a starting point for own extensions. As an old multitouch dude the first obligation is to add TUIO support so you can play in an interactive way with the system.

My first demo is a remix of the Binned Particels Example from Kyle McDonald:

The Original example has no color and only has some mouse interaction, so I decided to extend this demo with TUIO input, audio forces and some color/blend stuff for better visual appealing.

The next demo is a remix of the great MPM Fluid from Grant Kot (Golan Levin OF implementation):

You can achieve a lot of  behaviours when you play with the different parameters.

The next one is my own implementation of a 2d metablob/metaball shader in openframeworks. (Don’t know exactly where I got all the code inspiration, but it’s based on some work from Adrian Boeing) :

As you can see the shader implementation make it posible to render a lot of blobs with a high framerate.

The last one is a remix of Memo Aktens MSAOpenCL  addon. Thanks to Matt Gingold who made some WIndows addition to the code, otherwise I had stumbled right at the beginning to get the example running on my PC as the original code is only for Mac. Anyway it was hard to do because it’s based on OpenCL:

I always dreamed of some particle system where you can move 1 million particles in realtime.  It’s not so easy to get into the OpenCL development and it’s a lot of trial&error as you have just limited debugging capabilities. But now I have some feeling how to think in the “OpenCL way”.

After working on the back-end site i ‘m now dealing with the front-end for multi-touch applications. After some research i come finally to the incredible MT4J framework. As I’m a real java fan i pleased to see that this framework exists and it’s absolutely brilliant! I don’t know what this guy is doing all the time but he is a master of software architecture & openGL . It’s impressive to see what performance boost you get if you work with openGL in the right way.With this framework your are able to concentrate on the interaction design and you could integrate stuff from processing, so there are a lot of code floating around the web you could use.

It’s a big argument to stay on the java platform.I know most of the people love Flash, but java is in my opinion generally the greatest platform you can develop with. You got eclipse, processing,server&database stuff, openGL with jogl and a lot of physical computing code with wiring for the arduino board. And the language itself is a thousand times better to deal with as with C++.

If you look through the NUI Forum you always see the question about which camera should i use for multi-touch?

The quick answer is: It’s depend on your budget and how much cameras you are trying to use at the same time.

I only have experience with three cameras that most of the multi-touch developer’s use :

PS3, Unibrain Fire-I and the PointGrey FireFly .

From my experience all cameras are sufficiently for the image processing you do with multi-touch and it’s more important to make the right filter settings on your software, or get the right light situation then to deal with the questions like: what’s the right sensor (CMOS/CCD?) . The cameras differ in price so what are the pro & cons of the models?

PS3: This seems to be the camera of choice for most of the multi-touch developers out there, and if you look at the price and it’s capabilities you know why. You get no cheaper camera with the same FPS. But there are some drawbacks you have to deal with: No cs-mount for objectives, you can’t mount the camera on ball-heads for adjustments and if you want to make a multi camera set-up you have to look for some third party driver.

Unibrain Fire-I: A good Firewire camera but with the lack of a real professional housing. There are a tripod mounting kit and some cs-mount adapter, but for the price you should get some more professional stuff.

PointGrey Firefly MV: This is a marvelous camera with an incredible driver suite.  PointGrey has a long time experience with multi-cam setups so you get a professional system, but this has it’s price. The real advance comes with multi-cam set-ups, but  you end up quickly in a 1000$ system only for cameras and objectives.

So for the first steps into multi-touch i prefer the PS3 . For that price you can’t go wrong! If you want to make huge multi-cam setups i prefer a Firewire PointGrey system, but this could change if the CL-Eye Platform (I’m will test in the future) deal with the USB bandwidth problems that most of the times arise with USB systems. If this will work the price difference will be so huge, that you could live with the PS3 drawbacks (no cs-mount,housing) .

If you want start with multi-touch development you have to decide what topics you are interested.
Some prefer only the interaction design, others trying to build their own soft-/hardware from the ground. Whatever you like the nuigroup forum is THE starting point on the net. There are a lot of information spread all around the forum. Without the help of so many community members I never would have achieved the things i develop.

To get the historical background and wider context on this field you should go to the site of Bill Buxton. He is a Principal Researcher at Microsoft and if somebody has a credibility on this topic then he. There is a very interesting talk from the CES 2010 with him about NUI: CES 2010: NUI with Bill Buxton