Faceit is a Blender Add-on that allows you to semi-automatically generate a set of facial Shape Keys for humanoid characters without using any sculpting or modelling tools.
For now Faceit is especially designed to generate the Apple ARKit Shape Keys (Blendshapes), mandatory for Performance Capture with the IPhoneX. Next to the tools for generating the facial expressions, Faceit is packed with a bunch of utilities to ease the process of Performance Capture.
With Faceit you can ready your character model for high quality Animation in minutes! Save Weeks of Work!
Ready your Character for Performance Capture in Minutes!
(Rain Rig © Blender Foundation)
Get Started
Please read on here or check out the other resources for detailed footage and workflow instructions:
Documentation - Youtube Channel - FAQ
Support
I spent a lot of time working on Faceit and I want to make it the best tool it can be! If you have any questions, feedback or requests, I will be happy to hear from you!
Simply leave a message here at Blendermarket under Ask a Question.
I will answer as soon as possible!
Arbitrary Characters
Faceit is able to process arbitrary and even stylized characters. Depending on your individual project and character you will have to invest more or less time, customizing and tweaking the expressions.
Faceit supports most humanoid morphology and topology out of the box
Shape Keys without Sculpting or Modeling
Faceit presents a collection of tools that assist you in generating a set of shape keys. For now it is focused on the 52 shape keys that are required for Apple ARKit/ IPhoneX performance capture (see here!), but in future releases other shape key sets will be added, including a detailed FACS based set for high quality animation!
The 52 ARKit Expressions on Rain (Rain Rig © Blender Foundation)
How Does it Work?
Faceit leverages standard rigging techniques to generate the deformation. A given character is rigged in an auto-rigging type of workflow and along with the rig an animation is generated, that holds the individual Expressions. A benefit of this workflow is that it allows you to tweak all expressions by using the mighty pose and animation tools Blender has to offer! Of course nothing stops you from adjusting any or all of the generated shapes in a traditional workflow. You will likely still save a lot of time.
Smart Binding includes Facial Hair
Non-Destructive
You can iterate between rigging and animating the expressions and baking the shape keys by the click of a button! This allows you to non destructively bake the expressions, check them in motion with the Face Cap App and return to rigging in case you notice any artifacts.
Iterate back and forth between Rigging and Shape Keys Motion Capture
Smart Binding Process
To save you the pain of extensive weight painting sessions Faceit features a Smart Binding algorithm that generates a weight distribution based of the defined landmarks and the registered Geometry. (Disclaimer) As with any other rigging tool, for some characters the initial distribution is sufficient and on others you will have to invest in tweaking the weights.
Weights generated based on Landmarks.
Motion Capture
You can use any Apple device equipped with the True Depth Camera to capture your facial motion and retarget it to a 3D character in realtime, provided that the character is equipped with the required Shape Keys. If you never heard of this, I recommend you to watch this video.
A multitude of apps stands available that support capturing and recording of facial motions, including:
- Live Link Face (Unreal Engine)
- iFacialMocap
- Blender Motion Capture
- ...
Any character that has been processed with Faceit (equipped with the 52 Shape Keys) will be compatible with any iOS face capturing app. For now the Faceit supplies an interface for the Face Cap App by bannaflak for live viewport recording and loading of recorded motions, but I will include more capturing apps in upcoming releases.
Face Cap App
Faceit offers helper functions for people who work with the Face Cap App, but you can of course use your favorite solution without a problem!
- Reorder the Shape Key Indices to Face Cap Order
- Load Recorded Motion from .txt format
- Go Live via AddRoutes Add-on.
... All in one click.
Note!
To use Live Capturing (OSC connection) for longer than 5 seconds you will have to unlock all features in the Face Cap App.
Price: Currently 60$
Export
After you processed your character, you will be able to use it in external software without restrictions. Simply export as FBX and use it in a GameEngine like Unity or Unreal.
Compatibility
Combine your results with Body Rigging for real Performance Capture. Shape Keys are compatible with Armatures, so you will be able to use Auto-Rig Pro, Rigify, or whatever rigging tool you want to use!
IPhoneX Facial Capture combined with Auto Rig Pro Rigging and Mocap.
Why Spend the Money?
The process of creating the 52 prescribed expressions in a traditional workflow (Sculpting or Modeling) can be tedious, as I know from experience. I created Faceit out of personal interest and to save time and money on future projects. I hope it will benefit other people too!
If you want to integrate IPhoneX performance capture into your character animation pipeline than Faceit will save you weeks of work and turn Blender into a small capturing studio!
Please feel free to contact me if you have questions or suggestions for future development!
I want to see more!
Okay, check out some of the creations that have been made with Faceit:
Also have a look at Jimmy Gunawans Youtube Channel
Workflow Overview
- Register Geometry
- Create Landmarks
- Auto Generate Rig
- Smart Bind Geo to Rig
- (Optionally) Tweak Auto-Generated Expressions
- Bake Expressions to Shape Keys
- Start Performance Capture
Check out the process videos and the documentation for detailed workflow instructions!
Modally position and scale the Landmarks for a rough match.
Project the Landmarks from Front View
Generate the Rig and Smart Bind!
Check the generated expressions
Bake Deformation to Shape Keys. Done!
Versions - Releases
Version 1.1.7
- !Experimental! Asymmetry supported
- small bug fixes
Version 1.1.6
- fixed a bug from 1.1.5 where the right eye was not bound properly
Version 1.1.5
- bugfixes, 2,90+ compatibility
Version 1.1.4
- Record Motions through AddRoutes
- exposes more control over Binding
- minor Bug fixes
- better UI overview
Version 1.1.3
- Added Transfer Shape Keys Operator (from selected to active)
- Fix: Bind Secondary works in Blender 2.91+
Hotfix Version 1.1.2
- Fixed an issue where users were not able to bind when using custom weights
- Smoothing is now optional in Bind operator - default True
Features in Version 1.1.1:
- Registration of relevant Geometry (Head/Face, Eyes, Teeth, Tongue, Facial Hair - Beards, Eyelashes, Eyebrows etc., Rigid parts)
- Landmark setup in 3 easy steps
- Generation of the Rig along with an expression
- Smart Binding process based on the registered geometry and landmarks
- Easy tweaking of generated expressions through animation
- Bake deformation to Shape Keys.
- Non-destructive
- back and forth between Rig/Expressions and Shape Keys!
- back and forth between Landmarks and Rig/Expressions
- Face Cap App Utilities
- Load Captured motion (.txt)
- Setup AddRoutes paths in one click! Go live in the Viewport!