AutoAvatar: Autoregressive Neural Fields for Dynamic Avatar Modeling

In their new work, AutoAvatar, researchers have made implicit avatar modeling possible for the first time. It is an autoregressive approach for modeling dynamically deforming human bodies directly from raw scans.

Animated 3D models of the human body are a key tool for applications ranging from virtual fitting to social telepresence. AutoAvatar models body geometry implicitly -- using a signed distance field (SDF) -- and can learn directly from raw scans without requiring temporal matches for control. In addition, like physics-based modeling, AutoAvatar determines the full shape of the avatar, given the history of shapes and movements.

The approach is an autoregressive model that takes a person's pose and shape history as input, and then creates an implicit surface for the future frame. The model is supervised directly with raw body scans, and requires a training dataset of 4D scans (a sequence of 3D scans) along with fitted SMPL body models.

Because the explicit representation of shapes as implicit surface level sets is prohibitively expensive for end-to-end training, the researchers introduced the notion of hinged observation points - vertex locations on the base articulation model - that are used as a local point to define the entire body geometry.

The base implicit surface is encoded as a field of elevations relative to the articulated observer points. These are then converted into dynamic latent feature maps in UV space and the resulting features are mapped to SDF, linking a continuous three-dimensional space with the learned features on the SMP vertices.

You can read all the details by clicking on the links below.

• Project - https://zqbai-jeremy.github.io/autoavatar/
• Paper - https://arxiv.org/pdf/2203.13817.pdf
• Code - https://github.com/facebookresearch/AutoAvatar

We also recall the equally interesting Fundamental AI Research (FAIR) study on audio data hypercompression, which shows how AI can be used to ensure that audio messages do not glitch or slow down when the Internet connection is poor.