XRAnimator
XRAnimator is AI-based motion capture software, which you can use for Blender purposes or even for Full Body tracking 3D VTubing. (taken from Github page) It uses the machine learning (ML) solution from MediaPipe and TensorFlow.js to detect the 3D poses from a live webcam video, which is then used to drive the 3D avatar (MMD/VRM model) as if you are controlling it with your body. It can be used for VTubing and various XR/3D purposes
Things to keep in mind
Tracking quality depends on the webcam you're using
XRAnimator is heavy on the GPU load
How to Use
Get Linux version of XRAnimator on the Github
Extract the zip file, and inside, you should see a binary called "electron"
If you use Nautilus (GNOME Files), you can right click, hit properties and check if the binary is going to be executed as a program instead. Otherwise, you need to apply execute permission via terminal using:
chmod a+x /path/to/binary
Open the binary, and the first thing you see is an empty window with a giant START button. Here, drag and drop your VRM files into the program to use it. Once done, hit the START button
From here, you can just double click the Streaming option, select your camera, and XRAnimator will handle the rest
And you're basically done with XRAnimator, just add XRAnimator window into your OBS Scene as a Window Capture and you're good to go
In case you're strugging with UI, hitting ESC will bring the bottom bar. Also you can hit the Monitor icon for the UI options
Last updated