Barbie. The blond plastic icon. An extremely popular toy and the main source of revenue for Mattel. Back in the 60’s, Barbies were selling like iPods on Black Friday. Marching vigorously through the 70’s and 80’s, the army of Barbie dolls generated a hefty amount of revenue for their creators.
But one day, with a flash of a pixel, the marching clogged. An army of virtual ghosts — phantoms made out of logic and light — crawled out of mysteriously dark dimensions and invaded our world through quadrilateral portals called ‘screens’. Their leaders and heroes carried bizarre and horrific names such as Half Life, Grim Fandango, Diablo and Lara Croft.
Computer games are the dominating pioneers of interaction design in today’s digital world. The industry is three times more profitable than the entire sales profit of the software industry and is by far, larger than Hollywood sales. Halo 3 had the biggest entertainment launch in history, with $170 Million in sales in the US alone in the first 24 hours!
Physical toys have always been part of humanity’s “play” activity, a major tool for understanding the world around us and a way to gain practical skills in our physical environment. By being so entertaining, toys are facilitating the task of learning through enjoyment and fun. From the prehistoric age, humans have crafted dolls and playful models, representing physical elements and concepts in the real world. For thousands of years mankind kept on crafting toys, with very little change in shape or form. Can a new generation of toys emerge, combining both the physical aspect and the diversity of the digital world?

Workshop description

This workshop will provide students with both practical and theoretical knowledge in the field of computer vision, in relation to: Play, Games and Toys. The expected results are toys or games in which a person uses a physical object to interact with a digital environment.
Students will attempt to create innovative “magical” toys, that are physical — mostly appearing as physical objects or artifacts made from different natural and synthetic materials — and at the same time serve as controllers and actuators for functions dealing with digital data. Digital data can be a wide set of elements, starting from pure text and ending in audio, videos, images, and at times even social particles. The emphasis is on creating a new hybrid of physical computer games.
Structure At first, students will learn to harness and manipulate different computer-vision tools by the use of a camera, that provides machines with the ability to ‘see’. This part of the workshop will focus on building artificial systems that obtain information from images in order to understand their surrounding environment. The camera in this case is correlated with the human eye. However, the human organ that actually decodes this information is the brain and not the eye — interpreting images as what humans grasps as ‘vision’. This first step will explore ways and techniques to craft such machine-vision. The second part of the workshop will drive students into the world of Games and Play. By investigating classical computer games, physical games and toys, students will brainstorm the topic, experimenting in original ideas that will combine both physical ‘toy’ objects, and the digital world. Conceptually speaking — at this point, new interactions and games will emerge. The third and final part of the workshop will focus on realization and crafting the above concepts. At the end of the workshop student will have a physical working prototype of their idea, the result, in the form of a new game that will be presented at the end of the course to fellow students and colleague.


This course will place students at the intersection of design, technology, art and psychology. Students will incorporate these areas with the discipline that each brings with him/her, at the moment of entry to workshop. A fine mix of students coming from different disciplines will ultimately be beneficial for the end result.
On the technical side, each student will be heavily exposed to Tangible Interfaces (Also known as TUI / Tangible User interface) – communication tools which a person uses to interact with digital environments through physical objects.

Computer Vision

the science and technology of machines that can see. As a scientific discipline, computer vision is concerned with the building of artificial systems that obtain information from images. Among many tools and computer languages such as C++ and ActionScript, students will be introduced to the reacTIVision platform. reacTIVision is an open-source, cross-platform computer-vision framework for the fast and robust tracking of fiducial markers attached onto physical objects, as well as for multi-touch finger tracking. It was mainly designed as a toolkit for the rapid development of table-based TUIs and multi-touch interactive surfaces. This framework has been developed by Martin Kaltenbrunner and Ross Bencina at the Music Technology Group at the Universitat Pompeu Fabra in Barcelona, Spain as part of the the reacTable project, a novel electronic music instrument with a table-top multi-touch tangible user interface.


There is a strong and ongoing inclination for the industry to turn the presence of technology in our surrounding to ubiquitous and transparent. The miniaturisation of electronic devices has been going on for quite some time, which enables us to hide or embed them in other, sometimes seemingly unrelated products. The Internet and cellular communication, allow for our personal information and virtual property to be practically everywhere, and accessible almost anytime. These and other trends bring about occurrences that might be revered to as magic. “Any sufficiently advanced technology is indistinguishable from magic”, and this can be further divided into categories of illusions:

1. Conjure – rendering an object into existence, apparently from thin air
2. Vanish – making an object disappear
3. Teleportation – moving an object without physical contact
4. Transformation – changing a shape of an object to be something else in terms of shape
5. Restoration – recreate an object’s shape back into its original shape after distorting it
6. Prediction- the ability to predict an action the user is about to perform.
To some extent, interfaces are actually doing the same. There is no witchcraft, but rather professional techniques that amount to mere illusions — but with just as much impact on the viewer or participant. Accordingly, it might be supposed that these different aspects of the general scheme may all serve a purpose in the field of Interaction Design, and by far in the field of Computer games.
When considering interactions, these can just as well help to define functionalities and features. Here are some examples of interaction procedures, viewed from this frame of reference, and correlating to the specifications above:

1. Conjure – one of the most ancient sorceries of technology, from light suddenly glowing from a bulb, to a TV-set broadcasting wonders; from calculators displaying, without revealing how, the correct series of digits, to applications popping-open at a click of a button.

2. Vanish – as today we swiftly close, or minimise, computer programs at our will. Once we marveled at glass-doors sliding before us, and concealed out of sight, as we approached them.
3. Teleportation – advancements and discoveries have made this more and more unbelievable, where once phone-calls ran through stretched-out wires, miraculous on its own – today, mountains of information leap invisibly in the air, from one place to another, even miles away.
4. Transformation – adjustable seats; sofa-beds; Swiss army-knives; LED colors; hand-held computers turning into mobile-phones; mobile-phones turning into audio-players; audio-players turning into disks-on-key, and in some unfortunate times, to a brick.
5. Restoration – ponder on how rebooting our laptops does not erase our personal files, but more specifically on how our preferences are kept even as power is cut off of circuits; notice how after we turn the TV back on, the same channel we selected last time, and the level of volume, is still set exactly as it was; and now imagine a web TiVo, restoring our profiles even when visiting another country; not to mention the universal Undo execution, that saves one’s life far too many time.
6. Prediction – just as a camera’s auto-focus mechanism predicts the subject of our photo, auto-complete predicts the word that was just about to be written; just as a virtual tool-bar’s composition predicts which operations will be mostly chosen, a hold switch predicts accidental and unintentional button clicks when handling mobile-phones; just as the location of a camera’s shooting button predicts the way a user will hold it.

The process of perfecting a magic trick can be viewed as parallel to the process of developing an interactive object. This is also closely related to performance art, where audience feedback is vital. Once a concept for a trick is established, or say a joke in a stand-up comedy show is drafted, training commences. Whereas a magician practices his/her actions and checks his/her instruments, a comedian tests the manner of portraying the sketch to reach the best, and funniest phrasing and timing. This is crucial for the success of each, and for improving on the initial model. Agile-usability-engineering use similar methodologies, such as backtalk of a situation, where challenges of the creative progress, themselves teach and educate, rather than just present obstacles (hinting to a conclusion that the developer must also get better, not just the product).
Students will be exposed to magic-trick techniques and will experience and learn optical and physiological deception, mostly by using simple everyday objects like coins and cards.
By: Yaniv Steiner