The Future of Wiimote Hacks

Big article on the front door today about hacking the Wiimote. You've seen the YouTube videos, now hear from the guys who are controlling drum machines and playing Half-Life with Wii controllers.

What's so amazing about this is how quickly they got everything up and running. As you'll find out when you read the story, a lot of it hinged on coincidence: the fact that the Wiimote used standard Bluetooth signals and the fact that nearly compatible drivers had already been written for other motion-tracking PC devices.

So with all this already accomplished, what's in the future for Wiimote hacks? I've included lots of interview snippets that didn't make the final article, after the jump.

Carl Kenner, GlovePIE developer:

All the motion tracking functions are working correctly now (in
version 0.24), provided the bluetooth driver is working. That includes
the IR sensor's pointer functionality. Everything on the Wiimote
itself is working at least partially, except the Wiimote's speaker. In
the next version I am working on adding calibration so it can provide
the accelerations in real world units, filter out the acceleration due
to gravity, and provide the rotation angles and velocities directly.
Currently these have to be done with scripting.

Nuchuk, and classic controller support are the features I most want to
add, but nobody has worked out how to read from them yet.

By the way, GlovePIE also lets you add spoken commands to control
games if you have a microphone, and it supports other hardware like P5
Glove, TrackIR, eMagin Z800 VR visor, 5DT Data Glove, Intersense
trackers, Polhemus trackers, Ascension trackers, WorldViz PPT
trackers, FakeSpace Pinch Gloves, and Concept 2 Rowing Machines. All
these things can be combined (along with regular input devices) to
make any sort of input control system for games that you want.

Bob Somers, WiiDrums developer:

> - What do you think the possibilities are for practical application of
> this? Could this be used by musicians to create a more natural-sounding
> synthesized drum track?

The immediate (and most obvious) practical application for this is some
kind of Wii drumming game, maybe along the lines of Guitar Hero.
However, Carl Kenner (author of the GlovePIE program) showed me how to
use GlovePIE to directly control MIDI outputs. In other words, you could
have this script controlling any MIDI device, even real-life drum
machines or sequencers. While it's not nearly ready adoption by musicians, it's not outside the
realm of possibility that it could be used for those purposes in the
future. The disadvantage with most synthesized drum tracks is that the
computer is just playing back the same sample for each drum hit in
perfect time. In real life, each note sounds slightly different from the
last, and some notes may be minutely out of time. That's what makes a
synthesized drum track sound so machine-like - there's no human element.
No synthesized drum track will ever sound completely human, but the Wii
remote allows us to use a very sensitive input device that can capture
the minute errors that make a drum track sound more human. It's a step
in the right direction.

• What are the limitations? What can't you control?Right now the software is still buggy, after all it's more of a hack
  than a real solution. Sometimes it picks up drum beats on the upstroke
  and you have to mash bizarre button combinations to get the desired
  sound. It currently only has support for one remote (although there are
  plenty of mods out there to support two, and I'll have two remotes fully
  supported in the next version). Finally, there is a physical limitation
  to how fast the accelerometers can pick up the changes in acceleration.
  However, they can pick up some pretty fast beats so unless you've got
  lightning quick hands that probably won't be a problem. That long list of downsides probably makes the whole project look pretty
  unattractive as of now, but the current version was just hacked together
  in a few hours. I'm going to address all of these issues in future versions.

• Any plans to implement the nunchuk controller for a second drumstick?As of now we don't have a way to read input from the nunchuck using
  GlovePIE, so until it's supported, probably not. However, I do have two
  remotes working in a prototype (still buggy) and it will be fully
  supported in future versions. I even toyed with the idea of strapping a
  third remote to a real bass pedal to control your bass drum, but thats
  further down on the list of things to get done. Just to recap, these are what I'm working on for future versions: + Multiple remote support
  + Mapping drums into space instead of button combinations (i.e. swing to
  your left to get a bass kick, swing to your right to get a snare hit, etc.)
  + Either encapsulate the whole program as a GlovePIE script or as a
  separate program, rather than having to use both GlovePIE and WiiDrums.
  + Force-sensitive hits, allowing you to play loud accents or softer
  notes based on how hard you swing.
  + Possibly some kind of "Drum Hero" tech demo, or some kind of sample
  drumming game to illustrate what can really be done with this technology.
  + Once we get a TRUE Windows driver (not just a hacked solution) I'll be
  updating the program to be compatible with that.

Brandon Epperson, Wii synthesizer video:

I have personally already connected it to:

Midi Synthesizers and Drum Machines

Fantastic way to control analog synths!! Not sure how practical it is for natural sounds but that is more an issue of finding types of movements that are idiomatic to acoustic instruments and then mimicking there effect with the remote. A bow, obviously drum stick, you get the point.

The drum machine mapping was a bit different on the math end being that it dealt with threshold based programming rather than stream of data programming like the controller messages.

ETC lighting console

Awesome!! I have been able to assign fixture groups, submaster bumps, and macros to the buttons; Tilt, pan, color scroller, dimmer level and filters to the axis sensors. This is a really exciting one for me. I work in the theater business here in NY.

Software

It works great as a trigger/controller for Ableton Live as well as Max/Jitter. The buttons can be assigned to transport controls of most major audio and video softwares. I love using the axis controls for color and mixing video effects in Jitter.