This is a motion detection experiment in javascript, and before starting any explanation:
Click here to see the demo.
Source code on Github.
To see this demo with your own webcam, you will need to download a browser that enables some new features as explained in this link. To make it simple, I’ve used Google Chrome Canary, go and get it!
I used an html 5 API this is not released yet, the html 5 getUserMedia API. This API makes you able to use a webcam stream in javascript. Exiting! Paul Neave has done a very nice webcam demo.
So what is the demo about?
I use a webcam feed, I put a xylophone image on top of it and I detect the user’s movements to play the xylophone.
To detect the user’s movement I used a technic that is probably well-known by developers, designers and motion designers: the blend mode difference.
The concept is, if you put a picture on top of the same picture and blend them together using the blend mode difference, you obtain… a black screen!
If there’s a slight difference between the 2 pictures, the colors start to appear, showing you the “difference” between the two pictures. You get something like that:
Not exactly like that actually, in this demo I made an average on each pixel between the red, the green and the blue, and turn them to white above a certain amount to get a bit more accuracy.
The first step is drawing the webcam feed in a canvas, grab a picture from it at a time interval, and blend the current frame to the previous one. This means that if you don’t move, the result will be black, and when you start moving some pixels will turn white.
To be a bit more precise, I loop over the pixels of the current frame and I subtract the pixels color from the same pixel color of the previous frame. This is what the blend mode difference is doing: “Difference subtracts the top layer from the bottom layer or the other way round, to always get a positive value.”.
The next step is checking in some areas (the different parts of the xylophone) if you have some white pixels in them. If you get a certain amount of non-black pixels in this area, it means that something moved since the last frame and I play a note.
I’ve been using the Web Audio API to the play the notes, the demo is fully done in javascript.
*UPDATE*
Click here to see a more in-depth article on ADC on how to create the motion detection
Comments
do you have a demo/tutorial of how you managed this?
Pingback: On Motion Detection in Video Using JavaScript « The ADC Blog
Yes, more details there:
http://www.adobe.com/devnet/html5/articles/javascript-motion-detection.html
Pingback: The wider impact of camera integration with WebRTC | Androidian
Your JS experiments would be great in the http://meemoo.org/ “hackable web apps” concept… Inspired by this post, I ported the motion detection part to Meemoo: http://meemoo.org/iframework/#gist/3083681 (uses Flash for the webcam, but you can replace onionskin.html with stream.html (but Backbone.js and Chrome Canary are not getting along for now)).
Pingback: Google’s New Chrome Browser Can Take Over Your Webcam, Should You Be Scared? | Ready Made Social Network Site Guide
Pingback: Google’s New Chrome Browser Can Take Over Your Webcam, Should You Be Scared? | Social Media News
Pingback: Google’s New Chrome Browser Can Take Over Your Webcam, Should You Be Scared? | Ready Made Social Network Website Guide
Pingback: Google’s New Chrome Browser Can Take Over Your Webcam, Should You Be Scared? | Create Own Social Network Website
Pingback: Google’s New Chrome Browser Can Take Over Your Webcam, Should You Be Scared? | Social Network Site
Pingback: Google’s New Chrome Browser Can Take Over Your Webcam, Should You Be Scared? | Build Own Social Network
Pingback: Google’s New Chrome Browser Can Take Over Your Webcam, Should You Be Scared? | Social Web Tips
Pingback: Google’s New Chrome Browser Can Take Over Your Webcam, Should You Be Scared? | Social Network Guide
Pingback: Google’s New Chrome Browser Can Take Over Your Webcam, Should You Be Scared? | Build Own Social Network Site
Pingback: Google’s New Chrome Browser Can Take Over Your Webcam, Should You Be Scared? | Create Social Network Guide
Pingback: Google’s New Chrome Browser Can Take Over Your Webcam, Should You Be Scared? | Social Network News
Pingback: Google’s New Chrome Browser Can Take Over Your Webcam, Should You Be Scared? | How To Social Network
Pingback: Google’s New Chrome Browser Can Take Over Your Webcam, Should You Be Scared? | Best Social Website Guide
Pingback: Object Detection with HTML5 getUserMedia | TechSlides
Pingback: Multi-user enabling an app in 30 lines of code - Real time Blog
Hi, I’m developer in Brazil, see my application based in your code.
http://naturalslide.herokuapp.com.
I want developer game using natural user interface. Do you want join me for a developer a game?
Pingback: Year End Roundup: 30 Must Read Javascript Tutorials | WHSR
this is amazing. blend mode difference, I never knew it would be the performance would be fast
RGB
255 – 255 = 0
122 – 122 = 0
255 – 255 = 0
it.. makes sense I suppose 🙂