Look ma, I'm a Jedi!
Thursday, 13 November 2008 23:50

If you've tried games like Star Wars: The Force Unleashed and the recent Star Wars: The Clone Wars - Lightsaber Duels on the Wii, you may have been disappointed that the in-game lightsaber does not always match how you are holding the wiimote.

Introducing Fusion from Mgestyk Technologies: Fusion is a patent-pending technology which combines Mgestyk's gesture processing with other sensor data such as from Nintendo's wiimote.

We'll be demoing this at MIGS next week along with some of the games from our first video. What follows is a description of what the new video shows and how Mgestyk Fusion enhances gaming with a wiimote:

If you want to swing the wiimote around like a lightsaber, you will most likely not be pointing the wiimote at the Wii's sensor bar.

On the Wii console, the sensor bar allows the console to determine data for roll, up/down translation, left/right translation, and to a very limited degree, forward/back translation. The wiimote accelerometers also register data in these four degrees of freedom.

Pitch and yaw cannot be accurately obtained via the sensor bar, although pitch does register on the wiimote's accelerometers (yaw does not).

With Mgestyk Fusion, the sensor bar is replaced with a 3d camera, which can still determine translation data (including better forward/back translation than the sensor bar), but also determines yaw through image processing. Pitch and roll data from the wiimote's accelerometers are fused with the 3d data to determine the orientation of the wiimote in 3D space. The wiimote is connected to a PC via bluetooth in our video.

Nintendo's MotionPlus was announced to use the InvenSense IDG600 chip, which will add a dual-axis gyroscope to the wiimote. We look forward to trying MotionPlus since the two gyroscopes will improve yaw and pitch tracking, but three accelerometers and two gyroscopes are still not enough for tracking all six degrees of freedom. This means that you will still need to point the wiimote at the Wii's sensor bar for certain motions required for the full "lightsaber" experience.

Accelerometers only register data when they are accelerating. They do not provide useful data if you are moving the wiimote without making sudden changes in motion. By using a 3D camera, a better measurement of the wiimote's absolute position and speed in 3D space is obtained. In the video, we show how the saber can be moved around the 3D ragdoll with ease. Mgestyk Fusion will enhance the experience of gaming input devices such as MotionPlus.

Finally, Mgestyk allows you to use hand gestures in addition to the wiimote for controlling the environment. We show a simple "force push" in this video, although a large variety of gestures, such as the ones from our first video, are possible.

Comments (6)

Allen said:

Niiiiiice, it´s an amazing job you´re doing. Keep up the good work! :D
November 14, 2008

Fabian said:

Excelente idea, but I think you should work more in the LAG you have because of image processing (you have not this LAG with the sensor bar). Anyway, keep the good work!!!
November 14, 2008

Hugh said:

Interesting stuff! However, I'd be concerned about the lag too. Is that inherent to the system, or was it just a problem with that demo?

Are you planning on doing any motion capture stuff? I'd be interested to see that.
November 14, 2008

JM said:

This is really interesting, however I have 2 questions in addition to the lag one:

1. How many players can be tracked by a single 3D cam?, in other words, would it be possible to have multi-player on the same console?

2. Are there any requirements in terms of room lights in order for this to work?
November 14, 2008

Mgestyk said:

Hi Fabian/Hugh/JM,

We are working on the lag. It's not a problem inherent to the system - it's just a matter of optimization. From the start, we knew that the lag was solvable and decided to focus on precision and gesture interpretation; these were the harder challenges.

1. Yes, you can track more than one person with one camera. Since we have two TVs set up, network multiplayer made more sense than split-screen multiplayer.

2. There are no room lighting requirements... you can use it in the dark if you want.
November 14, 2008

Unacomn said:

This is simply amazing. Speaking as an amature game designer, this control method in theory just about solves all the control implementation problmes I've been having.
I'm looking forward to seeing the finished product, so far I'm very impressed.
November 21, 2008

Write comment
smaller | bigger

busy
 

Mgestyk Poll

What should we work on next?