🔍 Read the full analysis: A Step-by-Step Guide To Fine-Tuning 3D Generation Models on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Hugging Face has published scripts and containers for adapting its TRELLIS.2 3D-generation model to custom asset datasets, locally or through Hugging Face Jobs. The example uses LoRA adapters and 188 footwear objects to test shoe geometry, but the supplied source reports no completed results or measured improvement.
As described in the original fine-tuning guide, Hugging Face has published scripts and containers for fine-tuning its TRELLIS.2 3D-generation model on custom assets, including a footwear example intended to address inaccurate shoe soles. The example trains LoRA adapters on 188 shoes from Google Scanned Objects, but the accompanying material does not report whether the experiment improved generated geometry.
The workflow is designed to run either on a user’s own machine with a GPU or through Hugging Face Jobs, the company’s service for containerized cloud workloads. The post says developers can upload a mesh dataset to the Hugging Face Hub and adapt the scripts using a small number of arguments. Running the example requires a Hugging Face token and account access to gated DINOv3, which is used to encode input images.
The footwear test responds to an issue noted in initial trials: generated shoes generally looked plausible, but their soles were inaccurate. The stated goal is to test whether fine-tuning on 188 footwear objects can improve results for that category. The example uses low-rank adaptation (LoRA), which adds trainable adapter matrices while keeping the model’s original weights frozen, a parameter-efficient approach also used in embedding-model fine-tuning. Hugging Face says this reduces the number of parameters trained and the memory used for gradients and optimizer state; the scripts use the PEFT library.
The example adds adapters to TRELLIS.2’s sparse-structure and shape-generation stages, which affect form and geometry. It leaves the texture-generation stage unchanged. The post says the shoe assets include diffuse color maps but not the separate material maps needed to train that stage, and that the base model already produces satisfactory colors and textures for this task. Those details explain the experiment’s design; they do not show that the selected stages produce better results.
Fine-tuning TRELLIS.2 for custom 3D assets
Hugging Face released scripts and containers to adapt TRELLIS.2 to specific object categories. The footwear example outlines an experiment, but reports no completed results.
What the workflow targets
The example addresses shoes that look plausible but have inaccurate soles.
Coarse shape
LoRA adapters target sparse structure to adapt the model to footwear.
Detailed shape
The shape stage generates geometry; the example aims to improve shoe details such as soles.
Stage left unchanged
The dataset lacks separate material maps, and the authors say base colors and textures are satisfactory.
How TRELLIS.2 builds a mesh
The described pipeline turns an input image into a textured mesh.
BiRefNet removes the background, then DINOv3-L encodes the image.
Sparse structure guides the model’s detailed geometry generation.
A separate texture stage generates materials before the outputs are combined into a GLB mesh.
Ways to run the experiment
Teams with suitable assets can choose a local or cloud workflow.
- Use the provided Docker commands with your own GPU.
- Run preprocessing and training scripts on your machine.
- Run containerized workloads on cloud hardware.
- Upload the mesh dataset to the Hugging Face Hub.
What teams need
Access and evaluation matter alongside the training setup.
- A Hugging Face token and access to gated DINOv3.
- A comparison of pretrained and fine-tuned outputs, including held-out footwear.
- Clear geometry criteria and reported training settings, hardware, and costs.
A Route to Category-Specific 3D
The release gives developers a packaged way to test whether a general-purpose 3D model can be adapted to narrow categories or specific geometry requirements. In settings such as robotics, simulation and media production, a model can look convincing while still having a shape detail that makes it unsuitable for a task. A shoe with an inaccurate sole is a focused example of that gap between visual plausibility and usable geometry.
For teams with suitable assets, scripts and containers may make it easier to run preprocessing and training on either local or cloud hardware. But the release is a workflow and an experiment, not evidence of a verified performance gain. Its practical value will depend on dataset quality, compute requirements and whether improvements carry over to images and objects not represented in training.
Products Worth Considering
Apple AirPods Pro 3 Wireless Earbuds with Active Noise Cancellation
Bose TV Speaker Compact Soundbar with Bluetooth and HDMI-ARC, Black
Logitech MX Keys S Combo Wireless Keyboard and Mouse, Graphite
As an affiliate, we earn on qualifying purchases.
How TRELLIS.2 Produces Meshes
The source describes TRELLIS.2 as using a shared latent representation and a pipeline that turns an image into a textured 3D mesh. It says the system first removes the image background with BiRefNet and encodes the image with DINOv3-L. A sparse-structure flow model predicts a coarse shape; a shape flow model then generates more detailed geometry, while a separate texture flow model creates surface materials. The outputs are combined into a mesh that can be exported as a GLB file.
The fine-tuning example focuses on the structure and geometry stages rather than changing every part of the pipeline. That choice reflects the stated problem with shoe shape and the available training assets. The post presents this as the authors’ rationale for the setup, not as a comparison proving that this component selection is better than alternatives. It also does not provide evidence that the workflow supports every 3D representation or file format discussed in the broader TRELLIS project.
“The example targets footwear geometry because initial tests produced shoes that looked right but had inaccurate soles.”
— Hugging Face, as described in the supplied post
The Shoe Test Has No Results
The supplied material does not say whether the fine-tuning run was completed or whether sole geometry improved. It includes no before-and-after examples, quantitative benchmark or independent evaluation. It also does not specify the test set, scoring method or criteria used to judge a sole as accurate.
Other practical details remain unreported, including the hardware configuration, training time, cost and amount of dataset preparation required. There is no reported assessment of how well the resulting model performs on footwear outside the 188-object subset. The authors’ initial observations and stated aims should not be treated as proof of an improvement.
Run the Workflow and Test It
Developers can use the published repository to adapt the scripts, prepare a mesh dataset on the Hugging Face Hub, and run preprocessing and training locally or through Hugging Face Jobs. They will need an appropriate GPU for local runs, a Hugging Face token and access to gated DINOv3 for the described example.
The next meaningful evidence would be a report showing the fine-tuned model’s outputs against the pretrained model, along with the evaluation method, compute requirements and performance on held-out footwear. Until those details are available, the release confirms that the workflow has been published—not that it has solved the reported geometry problem.
Key Questions
What did Hugging Face publish?
Scripts and containers for fine-tuning TRELLIS.2 on custom 3D asset datasets, with options to run locally or through Hugging Face Jobs.
What does the footwear example try to improve?
It tests whether fine-tuning can improve shoe geometry, particularly soles, using 188 footwear objects from Google Scanned Objects.
Does the source show that fine-tuning worked?
No. The supplied material reports no completed result, measured improvement or before-and-after comparison.
What does a developer need to run the example?
The workflow requires a prepared mesh dataset, a Hugging Face token and access to gated DINOv3. Local runs also require a suitable GPU; cloud runs can use Hugging Face Jobs.
Why does the example leave texture generation unchanged?
The post says the footwear assets have diffuse color maps but lack the separate material maps needed to train the texture stage. It also says the base model’s colors and textures were already satisfactory for the task.
Primary source: Hugging Face · via ThorstenMeyerAI.com
