Reflections on the Wii

Believe it or not but until this month my exposure to the new generation of consoles, including the Wii was minimal. The only console I have ever owned was a PS1, but that was long time ago. I simply never had the urge to own one of the newer consoles, and for some strange reason I never really had a chance to play with the Wii for an extended amount of time.

This year my cousin got a Wii for Christmas and so I was able to play around with the thing. I’m not going to comment on the social aspect of the console because it is a closed topic. Needless to say, the nature of the controller makes it immediately accessible and attractive to people who usually have little interest in console gaming. It provides a more natural and intuitive interface that people have no problem learning.

But as with many innovative interfaces, it does not work well for traditional software. For example, while playing Smash Bros I was longing for the traditional controller Playing a game that required traditional input, and did not rely on the Wii’s motion control using the Wiimote and a Nunchuk was awkward to say the least. Aiming with the Wiimote is also quite difficult making it useless for FPS games. I’m not sure how much does the Wii Zapper extension change this but in my experience even something as simple as hitting a button on the screen requires a quite bit of waving, and aiming and is far from effortless or intuitive. Or rather, it is intuitive because everyone immediately knows how to use the controller, but actually putting the pointer on a button and clicking it requires some effort.

To fully exploit the device you must write software that is designed for it from the ground up, and keep in mind it’s various quirks. For example I noticed that the Wii Sports that ships with the console has a very fuzzy motion detection system. Or rather, the emphasis of the system is to create fun and engaging game play mechanic rather than forcing the user to learn complex Wiimote motions. Thus:

1. The golf game does not really care which way you swing. I would actually always swing away from the TV for better control.
2. The baseball game does not really seem to be able to tell the difference between an underhand an overhand swing when pitching. Also, both pitching and batting could be accomplished with just a quick wrist snap.
3. The tennis game did not appear to care which way you swing when you try to hit the ball. I successfully struck the ball by swinging in the opposite direction without any negative effects
4. The boxing game seemed especially fuzzy, ignoring roughly half of the frantic Wiimote wailing

All of this made me wonder how exactly does the Wiimote works. How does it report the motion back to the console, and how hard it is to detect and interpret these signals. I’m wondering if the fuzzy controls of Wii Sports are actually conscious design choices, or are they dictated by the shortcomings of the technology itself.

It might be a little bit of both. It turns out that the Wii devices use 3-axis accelerometers (probably these) which report back the acceleration imparted on the controller via Bluetooth. When you are holding the controller perfectly horizontally with the A button facing up, you will likely get a reading of 0 on the X and Y axis and a negative reading on the Z axis due to gravity). If you stand it up on the table with the IR sensor facing down, and the expansion port facing up, you will get a read on the Y axis, and zero everywhere else. Finally, if you lay it flat on the side you will get a reading on the X axis.

Based on these 3 numbers and compensating for gravity you can extrapolate how and when the Wiimote is moved, and approximately in which direction it is pitched or rolled. However, you can’t exactly pinpoint the position of the Wiimote with respect to the screen – that’s what the infrared sensors are for. You probably can’t also detect the yaw of the controller because rotating it on the spot will not exert any acceleration on the X axis.

So while Wii sports could probably detect which way I’m swinging it is likely that the actual Wiimote API actually abstracts basic motions. And if it doesn’t, I believe that the first thing that any Wii development team writes are motion libraries. Let’s face it – it is much easier to create a basic listener that tracks the Wiimote and fires of an uniquely named event when it detects a predefined motion than to re-invent the wheel every time you want to detect whether or not the user is swinging the controller or holding it still. On the other hand, detecting complex motions with high degree of accuracy might actually not be all that easy considering your only inputs are acceleration readings. It is probably trivial to detect a swing as a noticable acceleration spike on one or more axis. It is probably more difficult to differentiate between an underhand, overhand or a backhand swing. If you throw in more complex motions into the mix, it quickly becomes a mess. Which I believe is what I experienced in the Boxing mini game. Wii was dropping half of my rapid motions because it was detecting them as continuations of previous ones. When I started throwing punches one at a time, the detection rate seemed to improve.

Keeping the motions simple, guarantees good detection and responsiveness. Thus, not caring about the direction of the swing makes for a better game play.

Detecting complex Wiimote motions is actually an interesting research area. I’ll refer you to the Wiigee project which is a Java framework which uses Hidden Markov Models for training and analysis of the controller motions. They are able to recognize complex movements such as drawing a circle, square or a Z shape in the air solely based on the accelerometer input.

But you can do much more. If you combine the accelerometer readings with the IR sensor readings you can fully track the remote in 3D space allowing you to do some pretty amazing stuff. Check out Oliver Kreylo’s Wiimote Hacking page (especially this video) or the infamous head tracking project by Johny Lee.