What is Boximator?

Boximator is an AI tool that can be used to improve the quality of videos by adding more detail and motion. Boximator is a combination of two things: a box and an animator. Boximator was developed by researchers at ByteDance, the parent company of TikTok.

The box is a data structure that stores information about the video, such as the shapes and colors of the objects in it. It can be used for a variety of purposes, including video editing, animation, and gaming. The animator is a set of algorithms that can manipulate the data in the box to create new video frames.

AI Tool	Boximator AI
Category	Video Animation
Developer	Bytedance (TikTok)
Github	boximator.github.io
Launch Date	2 Feb 2024
Version	v1
Team	Jiawei Wang, Yuchen Zhang, Jiaxin Zou, Yan Zeng, Guoqiang Wei, Liping Yuan, Hang Li
Cost	Free

Boximator: Generating Rich and Controllable Motions for Video Synthesis

Generating realistic motions in videos remains a hurdle in video synthesis. We introduce Boximator, a novel approach enabling precise control over object movement. Users define “hard boxes” for exact positioning and “soft boxes” for flexible shaping and motion paths.

Boximator seamlessly integrates with existing video diffusion models, preserving original knowledge while training a dedicated control module. A self-tracking technique facilitates learning object-box correlations. Boximator surpasses state-of-the-art results in video quality and motion controllability, exceeding user preference when compared to baseline models.

How does Boximator work?

1. Select an object from an image: First, you will need to select an object from an image. This can be done by drawing a bounding box around the object, or by using a more sophisticated object segmentation tool.

2. Define its motion state: Once you have selected an object, you will need to define its motion state. This can be done in a few different ways:

2.1 Ending box: You can specify an ending box, which is a bounding box that defines where the object should end up in the final frame.

2.2 Motion path: You can specify a motion path, which is a series of points that define the path that the object should take over time.

2.3 Both: You can also use both an ending box and a motion path.

3. Control as many objects as you want: Boximator allows you to control the motion of multiple objects at the same time. This means that you can create complex animations with a lot of moving parts.

4. Under the hood: All motions are guided by hard boxes and softboxes. Hard boxes are used to define the exact position and shape of an object at a given point in time. Soft boxes are used to define the approximate position and shape of an object. They are more flexible than hard boxes, and they can be used to create more natural-looking animations.

Boximator Features:

• Motion Control:
  • Hard boxes for exact positioning and shape.
  • Soft boxes for flexible shaping and motion paths.
  • User-friendly interface for easy animation.
• Seamless Integration:
  • Plug-in architecture for existing video diffusion models.
  • Frozen weights preserve base model knowledge.
  • Targeted training for efficient motion control learning.
• Advanced Performance:
  • Self-tracking simplifies training and object-motion understanding.
  • State-of-the-art video quality (FVD scores).
  • Unmatched motion controllability (bounding box alignment).
  • User-preferred video generation in evaluations.

Examples

Boximator Comparison

---

Boximator Demo

Right Now Boximator website is under development and will be available in the next 2-3 months. If you want to use the demo, you can send the Image and Text prompt to describe the motion to the email: wangjiawei.424@bytedance.com. Soon you’ll receive the response.

FAQs: