BVH2SMPL

Update on July 6th, 2024

Now I recommend using rerun to visualize the 3D SMPL/SMPL-X mesh.

I provide a script /src/rendering-rerun.py as example, it inputs vertices and faces index, and using rerun to render the mesh. You can read this simple script and modify it for personal use.

Update on June 6th, 2024

After step 2, you can also try to use smplpytorch to generate vertex and face (which is .obj file) , then use pyvista.wrap to generate mesh, then use pyvista.Plotter.add_mesh to render.

Update on May 24th, 2024

After step 2, you can try to use aitviewer to render the video rather than using a lot of .obj files.

There's a reference script on Inter-X repo

Usage

Install the dependencies:

conda create -n animation python=3.10.12
pip install -r requirements.txt

Download model:

Goto SMPL

Download this one: [Download version 1.0.0 for Python 2.7 (female/male. 10 shape PCs)

Rename the neutral model into SMPL_NEUTRAL.pkl and put it into ./src/rendering_utils/smpl

Goto soruce code folder:

cd ./src

Run the script:

1. Put the bvh files into ./bvh folder, then run script to preprocessing the files.

   To get the trajectory ends at the origin, I calculate the Y rotation of the root point at the last frame, then cast the inverse Y-rotation to the root rotation and root translation of all frames.

   You'll have to find out the time window the people is about to sit (put the bvh into blender and find a proper frame), and put the frame upper and lower range into the script.

   python preprocess.py

2. Run the script to retarget the motion

   python retargeting.py

3. Run the script to render the result

   note: in this part you must have cuda.

   python rendering.py

4. run the script to generate the video

   Goto ./results-objs folder, then create a blender file, click on the bottom left, choose Text Editor , then click new to create a new script, paste ./results-objs/script-objs_to_animation-in_blender.py and run it. Like the picture.

   

   Then click the Output , choose 60fps (derived from the bvh), choose the output folder, choose the File Format as FFmpeg Video .

   

   Then click Render → Render Animation , wait for some time and you will get your video in the output folder.

   

Goal

I propose an end-to-end method to convert bvh file to SMPL mesh representation.

And you can also simply modify the retargeting.py file to fit another bvh hierarchy.

Results

Goto ./results-animation folder to see more results.

Quotation

Code of step 3 is mostly from motion-diffusion-model

Epilogue

If you want to know how I complete this work, you may go to Epilogue.md to check it out.