It is very often that I have to do video motion tracking for interactive video campaign in my daily work.
If I’m used used to do that in Flash, I made a quick experiment to do the same in javascript.
Thanks to Olof Storm who made me a perfect corner pin motion tracking in After Effects, and I’ve been using some code from Steven Wittens to draw an image in perspective.
Click here to see the motion tracking demo (give it a bit of time to fully load).
What I’m doing in this demo is drawing a video in a canvas, and using the corner pin tracking data to draw an image on top of the video.
I made 2 different videos, you can switch between them using the “switch video” button.
In the first video, I get the current video frame number by calculation (current time by video fps). And in the second video, I get the frame number using a different technique that is know to be quite accurate when we do it in Flash, I call that a binary counter.
The idea behind the binary counter is to export the video with some white and black rectangle that describe graphically the real frame number of the video. We’ve been using this technique in Flash for a while in the advertising industry, and Zeh Fernando talked about it last year.
You can see the binary counter on the right border of the video. The code gets a pixel in each bit of the binary counter (16 bits), find out if the bit is white or black to get a 0 or 1 and calculate a number out of it.
While you might not see a difference between the two frame number calculations at first glance, seeking the video with the binary counter version (using the top-right blue control) will keep the image and the video “synchronised”. Pretty cool.
This version is not perfect compared to what we do in Flash, some frames still seem to be a bit “out”. I also couldn’t find a way to solve “pixel snapping drawing” in a canvas, in other words, being able to draw an image on half a pixel.
I did really miss some powerful drawing method from the graphic API in Flash. I would love to find in javascript a “beginBitmapFill” method, apply matrix directly to a drawing method and draw triangles in an easier way to use this plane drawing technic we use in Flash.
*update*
I just found out that what I call a binary counter is actually a VITC, which I didn’t know! Interesting!
Comments
Have you looked at the CreateJS suite of canvas tools – http://createjs.com/ EaselJS might have what you are looking for…
Yes I saw Grant Skinner released his libraries, I haven’t tried them yet. Paul Neave also gave me good directions for the pixel snapping. I need to look into this.
Pingback: Augmented Reality comes to JavaScript | CreativeJS
This is an amazing example! thank you very much for putting it together. I’ve never seen anything like it before.
Hey my pleasure, glad you like it.
Pingback: plus-me.at GOOGLE | Der Video ist klasse! Ich freue mich schon auf die Applikationen die man aus so etwas…
This is great! Out of curiosity, what did you use to burn VITC into videos prepped for HTML5 video?
An after project that we drop in another, it generates a layer and draw the VITC for the whole video. I can’t really share it as I didn’t make it and the author works in another company.
Cool, Can i tracking any object in any video ?
How to know tracking data ?
With after after effects for example:
https://helpx.adobe.com/after-effects/using/tracking-stabilizing-motion-cs5.html#motion_tracking_overview_and_resources
Search for motion tracking, then you export the data and use it in a programming language that can drive a video.
Thank you for your response, can i use opencv or other video lib to export the tracking data ?
For example :
1: i got video
2: select i want to track object
3: export tracking data
Js can do this ?
You understand that tracking an object as I’ve done is a manual task right? Something you do in after effect like in this video:
https://www.youtube.com/watch?v=sqsfLxtFQQ8
If you are trying to dynamically get this data, you will fall in domain like “motion capture” or “augmented reality”. More info there:
http://creativejs.com/2012/03/augmented-reality/
I suppose you can check what you can with javascript and opencv (node.js + opencv):
https://github.com/peterbraden/node-opencv
But this is still server side processing.
Unless you build a server that stream back a video that has been processed with openCV in real time, I don’t think there anything to do in the client. But I could be wrong, there’s so many attempts, experiments and solutions out there for processing video in real time.
I used AE and your video (tracking.mp4) try to export tracking data , but i still can not got this . i got the data not match your provide. could your tell me how to export data like your provide it ?
BTW i success to use opencv to got a tracking data , but still not match the video play speed .
My video fps is 25.00 . can i add skype or email ?
I had a script to export the data from AE formatted to be used straight away, just string manipulation. If there is something wrong using this data, you probably made a mistake somewhere.
This is a “corner pin” data set, and it has been done manually from after effects, you can’t get the exact same data.
If the speed doesn’t, wither the data is wrong, or you made a mistake using it (JS calculation, FPS mistake).
Big Thanks Romuald , I Solved my problem .. I use opencv to track object and export tracking data. Thank for your help !!!
Oakland and Kent counties have used successfully this
method to lessen a backlog of older cases awaiting trial.
Improves image of NAR, associations and members simply because they have taken the
initiative to find and supply alternatives to
litigation. Negotiation: A form of communication, direct or indirect, whereby parties
who’ve opposing interests, discuss, without turn to arbitration or another judicial processes, are any joint action that they normally takes to handle
and ultimately resolve the dispute resolution –
Mercedes – between them.
Dance Equipment classes and recitals are a way of life for
several kids and incredibly number of them really stick it out.
There are many sorts of leotard, like scooped-neck which has wide neck opening, the crew neck or polo
necked leotard and there can also be sleeveless leotard, long-sleeved, and short-sleeved leotard.
Items for example cheap jazz shoes can be obtained from the number of retailers, although prices may vary from one
to a new according to the retailer, as well as the buying method.
This variety of cats is incredibly popular and common worldwide and
a lot with the people would delight in having such cats of their home like a pet.
Once your dog has learned in the future at your command, them call him up for you and place your hand around
the end of his back. The cage ought to be built just right
to deal with Buying a cat – https://www.ancats.Com.au, grownup rabbit with bedding spread to generate a
nice little home.
Ensure that the insurance option you’ve selected will take care of
get you started here as the outback is home to many unsealed roads, which insurance companies usually do not like.
Some motorists were stranded without food and water for the, before these folks were rescued.
Now going to Melbourne in Victoria, this Australia travel best explorers
way guide – australiasexplorersway.com.au, will advise you about the fantastic
places of stop by at Melbourne.
Work With What You’ve Got Nordic Rooms… Scandinavian Designer Brands – Homewares & Gifts Last but not least, whether you’re attempting to improve your home by basic changes, remodel the entire space or
you’ve simply moved right into a home, it may be
ideal to use what you’ve already got. Leather recliners currently less
difficult more than just chairs that will recline,
though they still do this better than another chair about
the market. Don’t be afraid to reuse or repurpose components of your own home, it is possible to reupholster,
re-stain or repaint furniture to give it a whole new new look.
Home SP3 Property management
management is about caring for the tenants needs and,
when the manager is not the owner, ensuring that they’re meeting the
owners’ expectations. When residents start to feel a disconnect using their board, apathy
can result. I see this repeatedly as well as over again where people put family
or friends with it, and almost universally it ultimately ends up being
a bad situation.
My problem was that I knew nothing about building or installing
a solar panel, but I did like the idea of reducing my energy bills,
and it also should be cheaper to create your own personal rather that developing a company put in a system for you.
The sun will always shine everyday, meaning
it’s going to be your constant source of energy. A
good simple to use DIY guide features a complete set of
supplies and components which are needed to construct your panels or another system.