What is it?

SmartRetina is a lightfast gesture-tracking platform written in Macromedia Flash 8, utilizing its flash.geom. package, flash.display package, Video class, Camera
class and their motion-tracking capabilities.


SmartRetina is used to develop Retina Based Interfaces — Interfaces which:

1. Purely use visual interaction and can fully function without using text or audio.
2. Strictly serve as translators between human gestures and machine functions.
3. Can be self-explanatory at their introductory level, since representations of the supported gestures can serve as the actual visual interface (a reflexive interface)
4. Can act as self-organizing systems (see Mossalibra for example).

Retina Based Interfaces therefore escape both the language barrier and the age barrier.

Since SmartRetina serves only as a translator between human gestures and machine functions, these machine functions can include usage of audio, text and actuators thus expanding SmartRetina’s capabilities beyond the visual.

The above video demonstrate how SmartRetina can be used as a navigating tool combined with Google earth. SmartRetina (working on the black laptop on the right) captures the user’s hand gestures, understanding them as computer actions such as up, down, left, right, Zoom In etc, then sends the commands to a computer running Google earth (the desktop on the left).

The following video presents a fast navigation from Greece to Rome.

Interfacing with GoogleEarth via GoogleEearth API by Riccardo Strobbia.

Public Implementation – Mossalibra

Motion tracking , Top-Projects


  1. jez houghton · March 7, 2006

    beautiful work guys. shape recognition via motion detection using getPixel data from the bitmap class in flash eight yes? beautiful application – think of retraining software for stroke victims etc…spatial recognition and visualisation…working on all this too…. maybe its time to start building an os project (eclipse based?) to pool work here and focus on pushing all this forward and identifying target applications for this. But, congratulations, you have built a fantastic system here.

  2. Mitja Kostomaj · March 7, 2006

    Very interesting.

    Could you tell what kind of camera are you using?

  3. Stas Oskin · March 7, 2006


    This simply awesome – seeing all the uber-complicated gesture tracking systems out there, it’s unbelievable that this can be achieved via Flash. Will the source be available for a download, as with the “Flash motion tracking”?


  4. Pingback: CrunchGear » Blog Archive » DDR For the Rest of Your Body
  5. Steiner · March 7, 2006

    Hey Mitja, right now we are using a simple Creative Webcam Notebook. after all 30% of the development time was on train :).

  6. Pingback: Prototype: Interaction design Cluster / Yaniv Steiner » SmartRetina And GoogleEarth
  7. Pingback: Prototype: Interaction design Cluster / Yaniv Steiner » Artissima 2006
  8. Pingback: Katy Lilley » Blog Archive » Project: Motion Tracking
  9. Katy · March 7, 2006

    Any chance of the code for this being realeased? Am doing something sort of similar and would love to have a look at how other people have gone about it all.

  10. ofer · March 7, 2006

    Hey Katy,
    We’re pretty swamped with work at the moment, but we’re planning to
    release a stripped-down, open source version once we have some free
    time 🙂
    Meanwhile, you can send us some info about your project and code,
    and maybe we’ll have time for some specific questions.
    Good luck!


  11. Katy · March 7, 2006

    Thanks for your offer of help – Can I have an e-mail address to contact you on?

  12. Pingback: מעבדת יצירה » ארכיון » פרוייקטים
  13. stephanie · March 7, 2006

    I am a resident artist at Eyebeam in NYC. Trying to develop a game that
    uses motion detection and tracking of player’s movements to activate
    virtual game components – very similar to the Eyetoy “Play” series of
    games on the PS2. I considered Jitter but the graphics are not
    sophisticated enough. It looks like Flash 8 new motion tracking
    capabilities might be a viable direction for my project. I’d like to
    get more info on your Smart Retina. Please contact. Thanks!

  14. Bill Goodrich · March 7, 2006

    What sort of hardware is in the device itself? Is it solely a camera and all the grunt work is done on the PC? If not, could it be connected to other devices? I.E. a small microcontroller not running Mac or Win?

  15. Steiner · March 7, 2006

    Ciao Bill.
    We recommand Creative small Ultra Notebook , simple and powerful.
    it should do the trick.
    Contact me if u need more info

  16. Pingback: Motion Detection » Darren Richardson
  17. dimitron · March 7, 2006

    gesture recognition using webcam.

    I made first engine for recognition with webcam

    small video presentation for my program

    Autor: Dimitron

Comments are closed.