It is very often that I have to do video motion tracking for interactive video campaign in my daily work.
If I’m used used to do that in Flash, I made a quick experiment to do the same in javascript.
Thanks to Olof Storm who made me a perfect corner pin motion tracking in After Effects, and I’ve been using some code from Steven Wittens to draw an image in perspective.
Click here to see the motion tracking demo (give it a bit of time to fully load).
What I’m doing in this demo is drawing a video in a canvas, and using the corner pin tracking data to draw an image on top of the video.
I made 2 different videos, you can switch between them using the “switch video” button.
In the first video, I get the current video frame number by calculation (current time by video fps). And in the second video, I get the frame number using a different technique that is know to be quite accurate when we do it in Flash, I call that a binary counter.
The idea behind the binary counter is to export the video with some white and black rectangle that describe graphically the real frame number of the video. We’ve been using this technique in Flash for a while in the advertising industry, and Zeh Fernando talked about it last year.
You can see the binary counter on the right border of the video. The code gets a pixel in each bit of the binary counter (16 bits), find out if the bit is white or black to get a 0 or 1 and calculate a number out of it.
While you might not see a difference between the two frame number calculations at first glance, seeking the video with the binary counter version (using the top-right blue control) will keep the image and the video “synchronised”. Pretty cool.
This version is not perfect compared to what we do in Flash, some frames still seem to be a bit “out”. I also couldn’t find a way to solve “pixel snapping drawing” in a canvas, in other words, being able to draw an image on half a pixel.
I did really miss some powerful drawing method from the graphic API in Flash. I would love to find in javascript a “beginBitmapFill” method, apply matrix directly to a drawing method and draw triangles in an easier way to use this plane drawing technic we use in Flash.
*update*
I just found out that what I call a binary counter is actually a VITC, which I didn’t know! Interesting!
Comments
Have you looked at the CreateJS suite of canvas tools – http://createjs.com/ EaselJS might have what you are looking for…
Yes I saw Grant Skinner released his libraries, I haven’t tried them yet. Paul Neave also gave me good directions for the pixel snapping. I need to look into this.
Pingback: Augmented Reality comes to JavaScript | CreativeJS
This is an amazing example! thank you very much for putting it together. I’ve never seen anything like it before.
Hey my pleasure, glad you like it.
Pingback: plus-me.at GOOGLE | Der Video ist klasse! Ich freue mich schon auf die Applikationen die man aus so etwas…
This is great! Out of curiosity, what did you use to burn VITC into videos prepped for HTML5 video?
An after project that we drop in another, it generates a layer and draw the VITC for the whole video. I can’t really share it as I didn’t make it and the author works in another company.
Hello I’d like to discuss about freelance job using this technology.
Please mail me
Romuald,
Are you aware of any way to track the motion of a shape coming from a webcam feed? it seems in order to do what your doing here, you have hand coded each spot where the corners happen in the video. What I am trying to do is to hold up a shape into a webcam that collides with a fader (like what a music studio has) and control the fader with the shape that I hold in my hand.