Follow Slashdot blog updates by subscribing to our blog RSS feed

 


Forgot your password?
Close
typodupeerror
×
GUI Software Hardware Hacking Input Devices Wii

Wiimote Turns TV into Touchless MS Surface 104

Posted by CmdrTaco from the now-thats-what-i'm-talking-about dept.
RemyBR writes "User interface project allows you to control objects on a display using gestures, working like Microsoft's Surface but without touching the screen at all. Inspired by Johnny Chung Lee's work, the system requires you to wear Minority Report-style gloves equipped with infrared emitters on your fingertips. A Wiimote on top of the display keeps track of these IR LEDs, while the software can read the motion down to two-finger pinching gestures for image zooming."
This discussion has been archived. No new comments can be posted.

Wiimote Turns TV into Touchless MS Surface More

Wiimote Turns TV into Touchless MS Surface

Comments Filter:

  • What style gloves? (Score:5, Funny)

    by techpawn ( 969834 ) on Wednesday January 23, 2008 @11:14AM (#22153354) Journal

    the system requires you to wear Minority Report-style gloves
    come on! Don't toy with my emotions!!! Power glove man! System requires you to wear modified power gloves!

  • Stop the Wii shipments!!! (Score:3, Funny)

    by kellyb9 ( 954229 ) on Wednesday January 23, 2008 @11:18AM (#22153398)
    Nintendo turns your televisions into a Microsoft product!

  • Table (Score:5, Interesting)

    by pavon ( 30274 ) on Wednesday January 23, 2008 @11:21AM (#22153450)
    Would it be possible to shine IR light through the edge of a plexiglass surface, and then when the user touches the surface it would cause the IR to scatter at that point creating a point source for the Wiimote to track?

    • Re: (Score:2)

      by Aladrin ( 926209 )
      That's crazy. Just crazy enough to work.

      Actually, I think it sounds like a great idea... I wish I had the parts and time to try things like that.

      Potential issue: Dust, fingerprints, etc... Wouldn't they also cause the same effect?

      Maybe there's a material that causes IR light (or even all light) to reflect (or reflect differently) in an area where force is applied?

    • Re: (Score:3, Interesting)

      by account_deleted ( 4530225 )
      Comment removed based on user account deletion

    • Re: (Score:2, Informative)

      by dustbowl ( 809387 )
      You can even do it with a regular webcam and some IR lights - check out the forums at http://nuigroup.com/ [nuigroup.com] I recently built my own touch table from scratch - some IR lights point at a perspex surface, and an old projector back projects onto the perspex to provide a picture to interact with. The webcam has a small IR filter attached to the front, and this cuts out the regular lights. When my fingers touch the surface, they create hotspots that are tracked (known as Diffuse Illumination). You can also put
    • Yes. In addition to the links in the above responses, also see: http://www.instructables.com/id/Interactive-Multitouch-Display/ [instructables.com], which uses that exact method.

      Simple and effective!

  • from tfa: it kinda works (Score:4, Informative)

    by 192939495969798999 ( 58312 ) <[info] [at] [devinmoore.com]> on Wednesday January 23, 2008 @11:26AM (#22153494) Homepage Journal
    If the video crashes at some point, a quick recap: it kinda works. You can see at certain points, the images get dropped and it looks like it doesn't totally track perfectly with where the fingers vs. screen are. However, it is an awesome technology and idea... maybe with a couple of remotes you could triangulate more precisely and get that true 'minority report' feel... just what i need for my tri-monitor setup :)
    • It seemed to me like the app was just lagging a bit, its possible that the demo was just put together inefficiently on a slow computer- the other demonstrations by the original hacker were very impressive and smooth. I have no doubt that this could lead to an extremely well-polished and affordable interface in the not-so-distant future. You know what would be really amazing- if they integrated the 3-d "face" tracking hack into the system and built a 3-d/2-d hybrid no-touch interface. Moving around with the

      • Re: (Score:2)

        by jma05 ( 897351 )
        > its possible that the demo was just put together inefficiently on a slow computer

        Highly doubtful. The objects move just fine after he has successfully targeted them. I think it has more to do with human performance. The issue is simply with the low motor resolution of the arm muscles compared to the more dexterous fingers. It can also be more fatiguing to hold them up in air since larger muscle groups are needed. MS Surface and Minority Report UI use large muscle groups to roughly locate the object but

        • Highly doubtful. The objects move just fine after he has successfully targeted them. I think it has more to do with human performance.

          Taking another look at the video, and the better quality one on their website, I'll have to agree with you on that.

          They don't need efficient interfaces, just fun and cool ones... kind of like the Wiimote.

          Thats exactly the point I was trying to make with that. With a no-touch multi-touch interface with position tracking using inexpensive hardware (a $40 wiimote, mostly open-source software, a few IR-LEDs that cost pennies and a pair of $5 gloves) you could create a really cool interface for a game. Even if Nintendo or a 3rd party developer doesn't come out with a game for the Wii that takes a

      • The computer was working fine, it's the wiimote itself that is having trouble. In particular it was having trouble recognizing the pinch gesture. The wiimote can track up to 4 IR sources at once, but it's pretty low resolution and when you get two points close together (the pinch) it gets confused. I tried Johnny Lee's hacks - they're pretty amazing, but don't work as well for me as they did in his demo - the wiimote's limited resolution combined with its 45 range makes apps stutter and stop responding inte
      • Forget about small scale. Combine with a 4x4x4 meter Holographic display (stage/tank) and use it for Cad/Cam or any other 3d design work. Heck I'd be happy with a 1/2x1/2x1/x meter 3d display for some of my modeling work
  • Just give me some Newton handwriting recog code on there and I'd probably never leave home again.. And no thought magic for me please, hand is the highest interface I wanna go.

    • And no thought magic for me please, hand is the highest interface I wanna go.
      Must... Resist... Joke...
    • Get this ported to OSX then. OSX has the Newton's handwriting system under the name of Inkwell. You just need the proper hardware for it.

  • Pretty flash. (Score:2, Interesting)

    by Optio ( 1225470 )
    I'd be very interested to see this kind of thing integrated with Bumptop [slashdot.org]. If Bumptop itself was then modified to have a little PC on it, well. Things start to get a bit recursive. Like standing with a mirror in front of you and another behind.

    A desktop on a desktop on a desktop....

    • What, like Enlightenment?

      • Re: (Score:1)

        by Optio ( 1225470 )
        Not having seen Enlightenment save for a quick skim read of the wikipedia article, I cant really comment, but I assume you refer to the multiple desktops and the grids theyre arranged in. In which case, kindof, except that instead of (as an analogy) a six pack of beer, more like hundreds of beer cans with the ends cut off and jammed inside the ends of the adjacent one.
        • Yeah kinda.

          Like with Gnome and KDE, Enlightenment have desktops next to each other, like sheets of paper laid onto a desk next to each other.

          But, imagine you have the page layout stacked two, three, or more high on top of each other, much like your beer can analogy.

          I get what you try to illustrate with the mirror analogy. I just find it hard to bend my mind around how it would work on a user interface...

          Also, am I the only one who doesn't find it strange that two dudes discussing a computing tech use beer c

  • Wiimote (Score:3, Interesting)

    by MasaMuneCyrus ( 779918 ) on Wednesday January 23, 2008 @11:57AM (#22153928)
    Why not just use the Wiimote itself as a remote control? It seems more fun and practical.
    • Variety. Waving around an IR-sensitive, motion-sensitive stick is useful for some purposes that other interfaces can't handle, but the same is true of virtually any input device. This has its own niche to fill.
    • have you tried pointing that thing at the screen for more than a few minutes at a time? unless you get a pistol grip add-on it feels awfully funky on your wrist after a while.

  • Touch and Feel User Interface (Score:4, Interesting)

    by Doc Ruby ( 173196 ) on Wednesday January 23, 2008 @12:00PM (#22153964) Homepage Journal
    We're used to moving an actual thing around to do stuff. The physical reaction into our fingers is very important. The mouse gives a minimum, but the trackpad gives more. Touchless manual gestures don't keep the hands locked in a feedback loop with the virtual object, so they'll be clumsy.

    What I'm waiting for is a thin memory plastic layer over a touchscreen, that can raise bumps and edges defining onscreen GUIs. Vibrating gloves could be good for simulating textures, but there's no tech for simulating tensile or inertial force in virtual objects. Maybe some kind of eccentric gyroscope, but I've never seen one.
    • This is generally true, but we've seen lack of tactile replaced fairly effectively by clicking noises inthe ipod, and I see no reason why we can't do something similar here.

      Or you could have a string that retracts when you manipulate something, simulating a force against the finger. That would be pretty sweet. Though I kinda find the whole project annoying. I like a cursor, mouse, and keyboard. I don't know how much time I want to sift through my pics, and keyboard shortcuts are way more efficient than
      • If you want to make a glove that can pull on your fingers in various ways I would look to using 'muscle wire'. When current is applied to the metal it contracts much like a muscle. Though you need to band them together in clusters to get any kind of strength and possible heat dissipation. Heat is the enemy as if the wire doesn't cool down quick enough it wont expand back to regular size. But this type of mechanic to a feedback glove would be best in my mind. Anything with like regular mechanical parts li
        • Well, memory plastic fibers do that without as much heat inefficiency. And though "addressably rigid" fibers are appropriate to simulating shapes of objects held in the gloves, they can't simulate position or other inertial values (eg. the motion of a ball through the air resisting the momentum of a catching hand). I wonder whether three elliptical gyros could cycle in proportions that both net to a 3D vector and sum to an inertial mass effect. If lots of those devices were nanoscale at very high speed, the
          • it would be pretty awesome if such an implementation was cheap :) Oh maybe (this might be just gimmicky but whatever) you could hold like a ball, that could be squeezed. It could fill with liquid via a pump and be filled or emptied based on whatever output from the comp, essentially giving you an object in hand that it tactile and changes in weight. Sounds messy but I wonder what it would feel like.
            • Maybe a ball with a surface covered in the textural memory plastic I mentioned, but elastic, with coiled shape fibers inside that can create textured shapes and sizes on demand, both convex and concave.

              I really think there's some kind of gyroscopic way to simulate at least the inertial mass, and maybe somehow a 3D motion vector. The other option is some really dynamic electromagnet, but I never like the idea of magnetic fields sprayed around a room, especially one that can have magnetic media or CRTs.

              There'

          • Re: (Score:1)

            by nedaf7 ( 851534 )
            Tried to mod you Insightful, instead I missed and modded you Troll. I am not using the interface in the video, however, so I have no excuse.

        • Heat's not much of a problem, once you get the Clan heatsinks. Just don't get your myomer formulas from anybody named Justin.

      • The clicking sound works because it's coming from something physical in your hand. The sound waves are close enough to actual mechanical vibration of your fingertips that the illusion in your ear (and in fact a tiny vibration of the fingertips) can work. But if there's nothing in your hand, the illusion from the learned association will fail, or be so tenuous as to offer a negative learning experience. Though I'd like to see tests of a demo with really high precision stereo audio apparently originating the
        • Actually, I never had the ipod itself click, though I realize that's an option. I had the click in my headphones, and it still works.

          I didn't say it was totally convincing. It's enough to be satisfied that your finger is creating an effect.

    • We're used to moving an actual thing around to do stuff. The physical reaction into our fingers is very important. The mouse gives a minimum, but the trackpad gives more.

      Actually that's one of the things the Wiimote does well, at least when using the Wii proper. Pointing it produces a mouse cursor, and as the cursor travels over each clickable icon, the Wiimote vibrates, giving something of the sense your finger would have traveling over several large embossed buttons. Of course, the pointing device in

  • Updated video posted (Score:3, Informative)

    by cynergylabs ( 1225520 ) on Wednesday January 23, 2008 @12:02PM (#22153992)
    An updated version of this video has been posted to the Cynergy Labs Site. http://labs.cynergysystems.com/ [cynergysystems.com]
  • A better explanation of how he built this can be found here [cynergysystems.com] and a better video with a cool example of navigating a 3D object can be found at the Cynergy Labs [cynergysystems.com] site.

  • Where is the software (Score:4, Insightful)

    by brent_linux ( 460882 ) on Wednesday January 23, 2008 @12:15PM (#22154184) Homepage
    Ok I get it. You can make multitouch interfaces. I have made multitouch interfaces myself using a couple different methods.

    They aren't worth a damn though unless you have something to use them with. Where is the multitouch picture organizing software that I can display on my coffee table and let me family sort through the pictures. Where is the multitouch D&D program that will let me and my friends move our characters through a dungeon with miniatures? Where is the multitouch coloring book that I can put a bunch of kids on? Multitouch math races? Multitouch Chemical Compound manipulation?

    We need software. We have ways to interact now. We need things to interact with.

  • Two words (Score:5, Insightful)

    by Reality Master 101 ( 179095 ) <RealityMaster101@gmail. c o m> on Wednesday January 23, 2008 @12:20PM (#22154246) Homepage Journal

    Two words: tired arms.

    Unfortunately, these sort of interfaces suffer from the same problems that doomed touch screen and light pens 20 years ago ("They can just touch the screen! How easy is that??") Users liked them at first, but holding your arm up is tiring. Try reaching out to your monitor and trace your Slashdot window for five minutes and see how long you last. It's *hard*.

    There's a reason people in the Old Days wrote on flat tables, and didn't write on easels. That's also why artists who do use easels typically do "stroke and rest" (and why cartoonists use a flatter table)

    A touch table is far superior for this sort of thing for that reason.

    • Re: (Score:2)

      by slapout ( 93640 )
      That's the same reason that voice activated computers won't fully take over: people get tired of talking.

      • That's the same reason that voice activated computers won't fully take over: people get tired of talking.

        Well, at least until we get real A.I., when I can just say, "Balance the damn checkbook!" or "Send a letter to that pain-in-the-ass customer." :)

    • So put the wiimote facing up from the floor and use a glass desk.
      • I used to think that'd be a decent solution, migrate to a screen/interface hybrid and put it flat on the desk, but then I remember University and how looking down at a desk for hours can really hurt my neck. I'm not sure where the touch technologies are heading, but it'd be nice to have an interface with no drawbacks in terms of ergonomics.
        • You wouldn't need to have the monitor facing down. Put the monitor in the usual position and place your hand on the glass desk towards a bottom mounted wiimote (or similar device).

    • Re: (Score:2)

      by sootman ( 158191 )
      Two words: tired arms.

      One word: exoskeletons. [slashdot.org]

    • Re: (Score:1)

      by EB FE ( 1208132 )
      Yeah, why don't we try two mice first? Hasn't anyone thought of multi-mouse computing?

    • Learn guitar.

      I'm tired of people propagating the old "your arms get tired" meme. They get tired, at first. If the interface has any merit, though, you'll stick with it, and your muscles adapt.

      I work retail, so I have to stand for several hours at a time. At first, I used to get very tired, but years later and my stamina has increased phenomenally. It would be exactly the same with a multipoint gestural interface.

    • Re: (Score:1)

      by cjh79 ( 754103 )
      You'd get used to it. As a musician I routinely work under conductors who wave their arms around wildly for many hours at a time. They don't get tired. And believe me, they don't have any sort of super human strength or stamina.

      In fact, it'd probably be an exercise benefit for most people.

      • You'd get used to it. As a musician I routinely work under conductors who wave their arms around wildly for many hours at a time. They don't get tired. And believe me, they don't have any sort of super human strength or stamina.

        Sure, it's certainly possible, but you're also talking about professionals who have done it for years. I don't think one should have to athletically train to use a computer interface. :)

    • Re: (Score:2)

      by Bo'Bob'O ( 95398 )
      Sure, but, now we have LCDs that are half an inch thick, or less. It's juts as easy to put one on a desk as it is a mouse and keyboard. Now for large amounts of data, absolutely, a keyboard is the way to go, but there are a lot of applications that could have some very useful GUIs that are based around dynamic touchscreen controls.
  • This can make for an affordable and effective screen display. Nintendo For The Wiin!

  • Rumble (Score:3, Insightful)

    by spyrochaete ( 707033 ) on Wednesday January 23, 2008 @12:24PM (#22154296) Homepage Journal
    I hope the gloves incorporate rumble. That's my favourite aspect of the Wii OS. Feeling that little bump when you scroll the cursor over a button is so tactile and tangible. It reinforces that you should immediately pay attention because you're about to execute a command.
  • As is well known in HCI research using your hands like this for some time becomes very tiring. But for showing off it's an impressive application :-)
  • Heard the word "Microsoft" way too often, how long before they own the rights?

  • It isn't Microsoft Surface! (Score:3, Informative)

    by notanatheist ( 581086 ) on Wednesday January 23, 2008 @01:00PM (#22154770) Homepage
    Please stop referring to any multi-touch device as being like the "Microsoft Surface". MS did not come up with the idea of a multi-touch display. They steal and buy 99+ % of their technologies. Let's get PC and stop giving credit where it isn't due. It is a "multi-touch" surface. Not an MS Surface. /rant

    • Re: (Score:1, Informative)

      by Anonymous Coward
      They are using this Wiimote system to control a Microsoft Surface [microsoft.com] dumbass.

    • MS did not come up with the idea of a multi-touch display.

      Yeah, yeah, and Apple didn't invent the GUI. Who cares? The one who gets the credit is the one who delivers the *practical* product, not the first solution. We say "like Microsoft Surface" because we don't have anything to refer to.

  • Perhaps those eight days would have been better spent caring for the screaming child in the background.

  • Not. (Score:4, Interesting)

    by no_opinion ( 148098 ) on Wednesday January 23, 2008 @01:24PM (#22155094)
    I got to play with the Microsoft touch at CES. This is nothing like it. The MS table uses a camera underneath the screen, so it can do things like recognize physical objects. Imagine thumbing through artwork on the table, then putting your wireless MP3 player down on the table and dragging the artwork to it and having it wirelessly sync. Pretty cool, if you ask me. They demoed this at CES. Basically anything with a barcode can be recognized as a unique device. Without this type of physical object recognition, the Wii version is a poor substitute, besides the fact you can't actually use all 10 fingers (or 20, if there are two of you) at once.
    • If your MP3 player is wireless, you don't need a Surface to recognise it or drag to it - you just need a automatically registered representation of the player in a virtual environment, like on a computer desktop.

      If the artwork is physical rather than virtual, the resolution of the image isn't going to be great from a camera - a scanner is an optimised graphical input device.

      I'm not ruling out Surface and multi-touch as useful innovations, far from it - I just think that the "cool" demos we've seen so far ar
  • can we move all the wiimote stories to their own site
  • In case anyone wondered, the Wiimote can track up to 4 infrared sources. The Wiimote's on-board hardware does all the heavy lifting as far as processing the image and determining actual coordinates (and sizes) of infrared sources. If a project only requires tracking four objects then the Wiimote makes a fantastic piece of hardware for experimental and hobbyist use.

    So in this demo, all the manipulation is done by tracking four coordinates grouped into two pairs.

    Dan East
  • A piece of $4.99 hardware and Jonny Lee Chung's Hack [gizmodo.com] creates the #1 best selling Metroid of all time.
  • That phrase shows just how clueless and gullible, most people...including Taco...are when it comes to Microsoft.

    The MS Surface 'table' (and it is one big ass table [youtube.com]) relies on gestures/movement, with none of the functionality (save the on/off switch, I suppose), dependent on touch at all.

    There are approx. 1/2 dozen cameras below the glass that triangulate movement above the glass - thus the need for the BAT.

  • Who Cares??? 3D VR!!!! (Score:3, Interesting)

    by Seraphim_72 ( 622457 ) on Wednesday January 23, 2008 @03:02PM (#22156604)
    http://www.youtube.com/watch?v=Jd3-eiid-Uw [youtube.com] Good lord this is cool. And he is right - bring on the games!!!!

  • urgh (Score:3, Funny)

    by Tom ( 822 ) on Wednesday January 23, 2008 @04:17PM (#22157832) Homepage Journal
    Interesting, except for the constant microsoft advertisement. "I love flash, but we built this in silverlight (continue on to long rant about how great that is with no relation to the topic whatsoever)", or the "and since it's built in .net it can communicate with the Wiimote", err yes? What's that gotta do with .net? Then the "oh, look a picture of me at some microsoft meeting", and on and on. All that really got on my nerves about 3/4 through the video.

  • A much better way to do it ... no glows needed http://www.youtube.com/watch?v=0awjPUkBXOU&feature=related [youtube.com]
  • while watching that video?

Slashdot Top Deals

Beware of Programmers who carry screwdrivers. -- Leonard Brandwein

Close