I had read about the Leap Motion Controller, but hadn’t seen it in action until SXSW 2013. The super precise motion-sensing device can track up to ten fingers within a three-feet area. Setting aside the obvious opportunities that the Leap opens up for those of us working in the interactive/experiential field, I’d like to focus on how the Leap might enter the workflow of a motion graphics artist.
There are three main ways the Leap’s ability to interpret 3D gestural data will allow for visuals to become ubiquitous that had previously been work intensive or cost prohibitive – as a 3D motion capture device, paired with a 3D sculpting app, and paired with a 3D drawing app. But perhaps the most exciting developments are happening when you modify the Leap to create home brew touch-screens.
Motion Capture for Keyframes
After Effects’s motion sketch is an oldie but goodie that I often go back to when keyframe velocities just aren’t hitting the timing quite right. More recently, the KinectToPin app has allowed for grabbing major joints (knees, elbows, etc.) as motion capture xyz positional data.
The Leap can grab the xyz of ten fingers at once, and lets you work from your desk (as opposed to the Kinect which requires space for the entire body). This could allow for quickly grabbing motion paths using your hands – in effect, creating keyframes in a way that feels more like acting or playing an instrument than point and click.
Perhaps we will use two fingers to experiment with creating a funny character walk, or animate a darting hummingbird with a back and forth of the wrist, or have the camera shake a specific way. This can all be done from scratch, but having the option to quickly motion sketch an idea and then finesse it will be an interesting alternative that may be speedier or create more interesting variations than a blank canvas. Here’s hoping for an z-position option in a future version of AE motion sketch!
Leap Motion Processing Library from onformative
3D Sculpting
One of the demos at SXSW 2013 was clay sculpting, reminiscent of Autodesk’s 123D Sculpt iPad app. Video experiments have already popped up of Unity, Blender, and AutoCAD integration, and I imagine eventually we will be able to get these inputs into Mud Box, ZBrush, etc.
My initial concern with digital sculpting is that instead of getting feedback through your hands, you’re getting it through your eyes. Even when “sculpting” on a Wacom or iPad, you still have the pressure of the stylus to a surface. The Leap allows for multi-touch zooming and rotating while pinching and pulling, but equally important will be how to dynamically and intuitively switch between tool types and controlling the strength of your sculpting tools.
I have yet to see any 3D models built with the Leap that showcase how it can beat a tablet/mouse + keyboard in terms of detail, but I’m not going to write it off yet. Perhaps the Leap’s most helpful role in this area is not necessarily replacing previous input devices, but supplementing them so that you can reserve your stylus/mouse for your sculpting tool, while your other hand gestures through menus or modifies camera views.
The Leap Motion experience at SXSW 2013 from The Verge
Leap Motion hands-on from The Verge
3D Drawing
In the Leap Motion hands-on demo above, and the 35 second mark of the Leap Motion Processing Library at the top of the article, you see a kind of 3D drawing/sculpting hybrid. By tracking the path of a user’s fingers, a model is created that’s a drawing in 3D space, reminiscent of Kinect and iPhone-enabled experiments such as Kinect Graffiti Tool, Graffiti Analysis 2.0, Movosity, and AirPaint. If these quick sketches could be easily exported as an .obj (or maybe even .fbx with animation) they’d provide an intuitive way to create organic looking 3D shapes quickly and easily.
I think this sort of 3D sketching has the most potential for creating new visual styles. For promising precedents, see Amit Pitaru‘s Rhonda and Seok-Hyung Bae, Ravin Balakrishnan, and Karan Singh’s EverybodyLovesSketch.
Amit Pitaru, Rhonda, (2003-1010)
But what about the day to day?
The three examples above indicate visual trends that could arise from the introduction of the Leap, much the way tools like Plexus, C4D’s Mograph, or Trapcode Particular allowed for visuals that were previously labor-intensive to become ubiquitous. The trick with any technique-based visual is that you’re using them on a job-by-job basis. Not all jobs require AE’s Puppet Tool, but when you need it, you’re really glad it exists. My last thoughts on the Leap would impact everyday, regardless of the brief you’ve been handed.
Jared Deckard’s experiments mounting the Leap upside-down to create a touch screen and creating an impromptu $70 Cintiq were the first tests that screamed “killer app”. If Adobe were to get on board with this type of Leap integration, I imagine it’d be a no-brainer for most artists to pick up a Leap to supplement their mouse and keyboard. Even for non-professional artists, combining the Leap with a desktop version of FiftyThree’s Paper could build a huge user base.
Deckard has a Javascript/HTML5 demo up of his open source graphics editor at inkmotion.org. I haven’t been able to test it, as I don’t have a Leap, but I’d love to hear about the experience from other developers (especially if you’re a tablet user).
Leap Motion Draw / Brush Demo by Jared Deckard
The Leap is set to ship on May 13, and has already announced that it will be bundled with ASUS computers. Their approach of opening up their SDK to developers and creating their own app store was a smart move in not going the way of the CueCat.
Gesture-based input technology, be it the Kinect, Leap, or some even newer-fangled thing, is going to become as common as the tiny video cameras that are now housed in every phone, laptop, and desktop monitor. Whether or not it’s a better (or even appropriate) substitute for the tablet or mouse in a professional creative setting will depend on effective software integration and rigorous user beta testing. I see it as a helpful addition to our toolset, rather than a wholesale replacement.
What do you guys think? I’m mainly a Photoshop/After Effects/C4D user with a keyboard, Cintiq, and 3-button mouse at the ready. I’d love to hear more our readers (particularly CG modellers) how they see gestural inputs entering their workspace.