I recently built a TUIO gaming table with a 46 inch playable surface. The table can seat six quite comfortably, handles over thirty simultaneous touches, and has a crisp, LCD display running at 1920x1080.
*NEW* The software now supports native multitouch for non-TUIO systems as well. Refer to this guide (you don't have to run the translator in that case.)
TUIO is an open framework and protocol for multitouch surfaces. You can read all about it at the tuio.org web site. TUIO defines a standard for multitouch hardware to send data to applications running on your system. The multitouch device detects touches and passes them to its driver software, and then the driver software encodes and publishes those touches using the TUIO protocol. This is typically done over a UDP socket on the host machine. Applications can connect to this socket and utilize the touch data coming from the driver.
For security reasons, browsers cannot connect directly to the UDP socket where the TUIO data is available. In fact, before HTML5 browsers didn't natively support any kind of persistent socket connection at all. Fortunately HTML5 includes something called WebSockets which let the browser connect, and stay conneted, to a TCP/IP port.
The natural solution is to create a native application to gather data from the TUIO UDP port and re-publish it on a TCP/IP WebSocket. The browser then can open up WebSocket connection and read the TUIO touch events. I call this small piece of middleware the "translator." Right now the translator only forwards touch add, remove, and update events with the event ID and the X,Y coordinates of the event.
When I started working on this, Google Chrome was the only browser with proper WebSocket support, so this whole project is based on Chrome and has not been verified in other browsers.
The default configuration of the translator is to read TUIO data at UDP port 3333 and re-publish it on a WebSocket at port 3334. Both of these settings are configurable. The translator can be minimized to the system tray where it will run in the background. The translator will only connect to your TUIO stream when a browser is connected to the translator via WebSocket. This leaves your TUIO stream available for other applications to use when browser-based apps are not active. The translator is called "TouchEventServer.exe" and can be downloaded and installed via the link below.
The translator is only available for Microsoft Windows XP and newer operating systems. If the translator won't launch you may need to install the Microsoft .NET4 Redistributable and the Microsoft Visual C++ 2010 Redistributable. Those are free downloads from Microsoft, available here:
Your firewall may prompt you!! The translator needs to talk both TCP and UDP on your network, so watch for those firewall popups and grant the appropriate permissions.
If you need to change the port number settings in the translator, or are interested in other advanced features of the program, you may be interested in reading the TouchEventServer Manual.
The program shows how to use an HTML5 WebSocket to:
The program also shows how to use the HTML5 Canvas element to:
I have written several games for use with the TUIO to WebSocket translator. Please see the games page to read about my games.
Yes. The translator is fully documented here.
Most systems already have the runtime environments from Microsoft that are required by the translator. Your system needs either Microsoft .NET4 or the Microsoft Visual C++ 2010 redistributable, or both. They are free downloads from Microsoft. Install them one at a time and try running the translator again after each installation.
Your touch table's hardware will not send TUIO events directly to Chrome. TouchEventServer.exe must be running in order for this to happen. The most likely cause of the flashing message is that you aren't running TouchEventServer.exe, or it experienced a problem of some kind. Try restarting TouchEventServer.exe as well as your TUIO driver, as often times these problems trace back to an actual driver crash. Then make sure your port numbers match. Your TUIO driver settings must match the port settings in TouchEventServer.exe, and the settings there must match those in the Chrome games themselves.
Is another application using your TUIO connection? This includes other tabs of Chrome that may be up! Make sure you don't already have a game open in another tab or instance of Chrome. Other multitouch games and apps cannot share the connection with the translator. Make sure no other multitouch games or apps are running. Make sure your driver is in TUIO mode, restarting it if necessary. Lastly press F5 in the game to reload it and see if this fixes the problem, or better yet, restart Chrome. Some Chrome plugins cause zombie Chrome processes to linger. Use the process monitor (CTRL+SHIFT+ESC) to kill off any zombie Chromes. A plugin-free installation of Chrome is highly recommended.
The zooming is caused by your system recognizing gestures and forwarding special events to the operating system and then the browser. If your driver is sending mouse or Windows Touch events you will have these problems. Your driver should be set up to be sending only TUIO events. Windows gestures or other HID events must be disabled.
The games must be run fullscreen for touches to line up correctly. Press F11 to make the game run fullscreen. If this is not the cause of the problem, your surface may be out of calibration. Run the calibration tool on your TUIO driver. Also, make sure your frame is in TUIO mode. If it's sending mouse or Windows Touch events you may experience problems. Your driver should be set up to be sending only TUIO events. Windows gestures or other HID events must be disabled.
Yes. The touch events from your system will be misinterpreted unless the games are run fullscreen. You can press F11 to put an app into fullscreen mode, or you can make it always run fullscreen. In Chrome open a new tab and view your apps. Right click on any app and select "Open Full Screen" from the popup menu. You can also use this menu to create desktop shortcuts to the apps, as well as start-menu shortcuts.
TouchEventServer.exe provides a button to launch Chrome purely as a convenience. If it isn't working then it probably simply can't find Chrome on your system. Just launch Chrome the way you normally would.
Your system is probably already sending TUIO data over port 3333. This is the default, and will work nicely. The WebSocket Translator (TouchEventServer.exe) needs port 3334. Naturally, if you changed either of these defaults your firewall will need to allow traffic on the new ports.
Detailed information about advanced settings in the Translator can be found in the manual. But the short answer is to bring up the Translator and click the "advanced configuration" button. Then (important!) uncheck the automatic mode checkbox. Now that the Translator is not in automatic mode, click the Close Server button in the Websocket settings and click the Disconnect button under the TUIO settings. This kills any existing connections. Then change your port numbers to their new settings. If you are adjusting the location of your TUIO driver change only the TUIO Client Connection port number, do not change the Websocket port. Lastly re-check "Automatic mode" and re-launch your game. If you changed the WebSocket port (unusual) then don't forget to change the settings inside the game as well.
As of version 1.1 native multitouch is supported. You only need to run the translator if you have a TUIO system. If you have a system that makes touch events available to the browser you can install the app and start playing immediately. Just touch the launch screen to make the app detect your native multitouch system. If the TUIO indicator continues to flash then your multitouch system is nonstandard, or you have to enable multitouch in Chrome. This is done on the via the chrome://flags/ (Chrome configuration page) under "Enable touch events."
No. In order for the Translator to supply multitouch data to the Chrome-based games they have to be specially written. The games connect to the translator via WebSocket to receive TUIO touch events from your system. If a game is not listed in the "games" section of this web site, then it is not supported.
As of this writing, Chrome is the only browser that both fully supports HTML5 WebSockets and has the performance to run graphics-intensive applications. Additionally, the games are packaged as Chrome Web Store apps which are exclusive to Chrome. As more browsers begin to support WebSockets this will be re-evaluated.
The Translator will minimize to the system tray, and will display notifications when it establishes or loses connections. It automatically connects to your TUIO driver when a WebSocket connection from a game is detected. This means that it won't use your TUIO port if you aren't running Chrome (keeping it free for other applications). Feel free to set up your system to launch the Translator at startup. You may want to enter the system tray customization menu and set the translator to "show icon and notifications." Double click it in the tray to open the interface.
Sure! The only provision is that (if you intend to distribute it) you must offer it for free, preferably through the Chrome Web Store. The test application described above is a good starting point for learning how to write a WebSocket/Canvas application that is capable of getting multitouch events.
Please read this entire FAQ to make sure your question hasn't been answered here! Also, please read the entire translator manual if you are having issues getting your system working. Replies may be slow, but you can email email@example.com with your questions.