“Tech Art:” Combining Face Tracking and Projection Mapping for Real-Time Face Hacking
“Face hacking” may not sound artistic, but its execution is breathtaking. A collaboration between a multimedia artist, a makeup artist and a digital image engineer – respectively Nobumichi Asai, Horoto Kuwahara and Paul Lacroix – has resulted Omote project, which combines face tracking technology with projection mapping to change people’s faces in real time. Here is the inaugural effort, developed last year and using the Yuka Sekimizu model:
“Technically, it’s very difficult to project with real-time tracking,” Asai writes.
No one has done this before, so we had to start from scratch. The process was complicated, capturing with sensors, processing the data, creating CG renderings, and using projectors. Getting the timing right was also difficult, correcting for delays and ensuring that the projection technology had the maximum precision to match the face and the visuals.
… We used OptiTrack sensors to [their] quick response, and developed original programs [in] C ++. TO [minimize] latency (delay), we have devised ways to integrate the whole process into one program. It creates amazing effects.
This year, the team took the effects one step further, using members of the J-pop group SMAP:
You can read more about the project here.