News
Processing and OpenGL and Black Magic | Mike Cobb | HMC | AardmanWe experimented with head-tracking and open GL recently in processing, even with a hefty graphics card, the 3d graphics seem to slow down quite a lot.
We think this is strange, unfair and suspect some kind of witchCraft based sabotage from one of the other teams
Gareth Williams - Interactive Developer
Our computer has been smoking dope. | Mike Cobb | HMC | AardmanOur computer has a memory issue, we warned it that the use of psychoactive drugs would not come without consequence but in HMC our
babieshumanoid robots
computers are free to make their own decisions and sometimes they make the wrong ones.Now, I am not about to blame our current difficulties on our
toolsbabiescomputers, so I hereby present our current challenge and our experiments into solutions:If you pre-render a 3D scene at a reasonable resolution you can save a lot of processing power, unfortunately (and there is probably a law in computer science for this) you trade it for memory consumption and we cannot store all the data required to pre-render the scene (2000+ images at ~2.5mb per image.)
The traditional solution to this problem is buffering - we only load images around the one we want to display; so, if this was a movie we would load some frames ahead and behind of the current one so that as the movie plays there are enough frames loaded to continue playing the film, while we load the ones further ahead in the background.
An initial experiment in processing revealed that it was quite easy to make a simple streaming program but there are two caveats:
- moving the playhead to a random point in the film requires rebuffering (think about when you skip ahead in a youtube movie) so we can only buffer effectively when the playhead is predictable. In the case of our pre-rendered scene we have no idea where the person will want to look so we can't buffer effectively.
- Images that are 2mb in size take approximately 80 seconds (!) to load on my machine, going back to the movie analogy, this means that the amount of buffering required to ensure that the next frame can be loaded in time is massive.
Our current experiments include using quicktime VR to display our pre-rendered scene in processing and using flash to see if the automagic memory handling for images loaded from the library is any better than the more explicit management in processing.
Tonight we will have a
father sonuser/machine talk with our computer so that it can understand the errors of it's ways.Gareth Williams - Interactive Developer
Different routes | Mike Cobb | HMC | AardmanSince we have re-started development on our
babyproject it has become apparant that we are now getting to the narrow end of our steep-learning curve - perhaps ourbabyproject isbecoming a teenagermaturing, the question we continue to ask ourselves is: Where do we go from here ?To this end we have looked into which platform we will develop on; how we are going to implement our 'magic
mirrorwindow' and how we are going to construct our beta interactive prototype.Processing is a rapid-prototyping java api and ide, which provides easy access to 3D graphics and basic visual programming whilst providing advanced functionality through the standard java packages. At the moment our development experiments favour this platform for it's extra versatility in a short time-span.
However, even with the advantages provided by Processing the implementation of our prototype poses a number of challenges to our success: Tracking the movement of a user's head (without peripherals) requires a lot of horsepower; rendering a scene in 3D requires some more horsepower and pre-rendering the scene we want to display requires a lot of donkey-work. So in short we need two or more Horses and a Donkey and our computers can only *do* Gigahertz…
Gareth Williams - Interactive Developer
Our cunning plan | Mike Cobb | HMC | AardmanSo the cunning plan is to use iris tracking to overlay images on a person's reflection in a mirror. To do this, we'll be tracking the eyes to estimate a person's location in 3D space, then place our own 3D models over their reflection using the two-way mirror technique posted below. We've worked out that we might even be able to create a false perspective image - to make it seem as if you're looking through a window into someone's living room.
To create these virtual reflections, we've been using Papervision:
The dots being dragged around are a representation of what the tracking code should give us
By manipulating a box, you could be looking into Wallace & Gromit's living room
Mirror Mirror in my Frame | Mike Cobb | HMC | AardmanThe two-way mirror arrived in our office yesterday. I've been busy with my box of tools and a load of timber, and have just finished the setup for the Magic Mirror screen. Here's a quick look at how it went:
I started with a nice wooden frame for the acrylic mirror. In even lighting the mirror is semi-transparent.
I was hoping that the foil coating on the acrylic would make a good projection surface, but instead it just reflected the projection. I had to find something more advanced…
Here's a test with a sheet of the most space-age material I could find - Tracing Paper!
Next, I fitted my steroidal webcam rig to the top of the frame. Eventually, this will be hidden behind the two-way mirror.
Behind the scenes, I'm using a great big 4000 lumen projector to give a crisp image on the mirror
I'm really pleased with the final result. Black and dark colours become reflective, while lighter colours shine brightly through the mirror.The next step is to get the tracking working at this range. So far our tests have been with close up images - the hard part is getting accurate tracking on a 'passer by'.
