Prototypes
Software
As a part of our initial release, we are including source code, compiled binaries, and documentation for the first of several components developed to help integrate generative animation pipelines into content creation tools:
the M-body Library.
This software component is developed in C++, and can be built for both Windows and Linux.
The M-body Library provides several key pieces of functionality:
a node/graph system enabling users to define and execute custom pipelines
network communication allowing nodes from one process/machine to seamlessly communicate with those in other processes
and utility nodes to perform common operations such as file reading/writing, or file format conversion
We have already used this library to help create generative animation plugins for both Unity and Unreal (to be released soon!), and we’re excited to see what others create with it themselves!
Dataset
We are excited to announce the release of a preview of our M-body.ai dataset: an open-source, commercially usable dataset focused on human conversational animation.
This preview will consist of a small number of samples, with the intention of helping researchers understand what the full dataset will contain and allowing them to provide feedback to the M-body.ai team.
To support broader research use, we are planning to include a variety of modalities in the dataset:
Isolated Audio of spoken performance for both actors in the scene
Timed Transcripts of Spoken Performance for both actors in the scene
Reference Videos of each actor and the entire scene containing both actors
RGB and Depth Facial Performance Video captured with a head-mounted camera
Skeletal Character Animation including hands/fingers, and provided in two versions:
- on a skeleton with proportions personalized to each actor
- and on a skeleton with proportions that are consistent for all actors
3D scanned meshes of each actor, skinned to their respective personalized skeletons
We’re excited to hear about how you’re hoping to leverage this dataset to further your research work, and we’re happy to answer questions and hear feedback about how our work can better support you.
Join the community around the project !
À quoi s’attendre en rejoingnant le discord?
You must be logged in to post a comment.