🔍 Read the full analysis: What Makes Inkling 2 Mini A New Option For Local AI? on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Thinking Machines Lab has released Inkling 2 Mini, a multimodal mixture-of-experts model the company describes as having about 26 billion total parameters and activating about 3 billion per token. It supports several inference runtimes and includes a research version with adjustable layer passes, but independent benchmark results and practical hardware requirements have not been established in the source material.
Thinking Machines Lab has released Inkling 2 Mini, a multimodal mixture-of-experts model intended for local and other inference setups, as described in the original analysis. The company says it has about 26 billion total parameters, activates about 3 billion per token, and supports text and image input; whether it runs efficiently on common consumer hardware remains unclear.
The model has 24 physical decoder layers, according to the release. Its first and last six layers run once, while the 12 middle layers are reused in three passes. This produces 48 logical layer visits per token without giving every visit a separate set of weights. At each layer visit, a routing system selects eight experts from a pool of 128.
The release describes a 1 million-token context window and says the model’s fixed-pass version is supported by Transformers, llama.cpp and MLX, runtimes relevant to local AI workloads. SGLang provides day-one inference support for datacenter deployments. The company also says expert weights are distributed in NVFP4 and MXFP8 mixed-precision formats and that the release includes a DSpark drafter intended to speed generation with a small memory cost.
A separate research checkpoint, TML-d24-varloop-research, allows users to change the number of passes through the repeated layers, a setting that may matter when considering local AI on a Mac mini. Thinking Machines says one or two passes may make inference faster, potentially at a cost to output quality. The material does not provide measured speed or quality results for those settings, nor a complete file size or minimum hardware specification.
Inkling 2 Mini for Local AI
Thinking Machines Lab’s new multimodal model combines reused layers with expert routing. Its practical speed, hardware needs and benchmark results still need outside validation.
A loop reuses model layers
The design reuses middle-layer weights to build capacity, while routing selects part of the expert pool at each layer visit.
The first six and last six physical decoder layers run once.
The same 12 middle layers run in three passes, for 48 logical layer visits per token.
At each visit, the model selects eight experts from a pool of 128.
Options for inference
The release offers several runtimes and a research checkpoint with adjustable loop passes.
Multiple runtimes
Transformers, llama.cpp and MLX support the fixed-pass model; SGLang supports datacenter inference.
Fewer passes
The research checkpoint allows one or two passes. The company says this can be faster, with a possible quality trade-off.
Claims need outside checks
The company reports strong benchmark comparisons, but the supplied material does not include independent replication or a full score table.
- Leads in 32 of 48 benchmarks against named models.
- Reported tie with Qwen3.8-27B on Terminal-Bench 4.0.
- Trails on AutomationBench-AA v1.0.6.
- Complete memory requirements and device-specific speed.
- Independent image and long-context performance.
- Quality at reduced loop counts.
What local users should assess
Runtime availability is a starting point; actual usefulness depends on hardware and workload.
A Different Trade-Off for Local Models
Inkling 2 Mini adds another option for people weighing model capability against the memory and compute limits of local inference. Reusing middle-layer weights and activating only part of the expert pool are design choices intended to provide capacity without requiring a distinct active expert for every token. They do not, on their own, establish how much memory the model needs or how quickly it will run on a particular machine.
Its combination of image input, a claimed million-token context and support for several established runtimes may make it relevant to users comparing local multimodal models. Actual usefulness will depend on performance in image tasks, long-context behavior, output quality and compatibility with available hardware. Those questions matter to developers and local AI users deciding whether a model can handle their workloads without relying on a hosted service.
The release also offers a testable research variation: users can change how many times the middle layers run. That creates a potential speed-versus-quality trade-off, but the supplied information does not quantify it. Independent tests will be needed before readers can compare the model’s practical performance with its design claims.
Products Worth Considering

Apple AirPods Pro (2nd Generation) Wireless Earbuds with USB-C MagSafe Charging Case

Logitech MX Mechanical Wireless Illuminated Keyboard, Tactile Quiet, Graphite

Amazon Kindle Scribe Colorsoft 64GB with Premium Pen
As an affiliate, we earn on qualifying purchases.
How the Layer Loop Works
The model’s central architectural choice is to reuse its 12 middle physical layers three times, rather than assigning separate weights to all 48 logical layer visits. The technical description says each pass begins with the same input from the first six layers, while learned messages carry information between passes. A learned gate controls how much information is carried forward. The model still executes all 48 visits in its fixed configuration, and the technical description says each visit keeps its own key-value and short-convolution state.
Thinking Machines compares Mini with its Inkling 2 Small model. Both execute 48 layer visits per token, but the company says Mini has a hidden width of 2,304, compared with 4,608 for Small, and routes among 128 experts per layer rather than 384. The release also says that other architectures, including Nanbeige, have explored looping. These details describe the design; they do not independently show how it performs in use.
The company reports strong results on several evaluations, including a claim that the model leads on 32 of 48 benchmarks against Gemma 4 31B, Muse Glimmer, Qwen3.8-27B and Nemotron 3.5 Super VL. It also reports a lead on FrontierCode v2.0, a tie with Qwen3.8-27B on Terminal-Bench 4.0 and a lower result on AutomationBench-AA v1.0.6. These are company-reported comparisons; the supplied material does not include the full score table or independent replication.
“Inkling 2 Mini is a mixture-of-experts model with 24 decoder layers, but the 12 middle layers run three times in a loop.”
— Thinking Machines Lab, in the supplied release material
Hardware and Benchmarks Still Open
The supplied material does not establish independent benchmark results or provide a full score table for the comparisons the company reports. It also leaves out complete memory requirements, measured generation speeds on consumer devices and minimum hardware configurations. A model’s total parameter count and the number activated per token do not answer those questions by themselves.
Practical performance remains unverified across the model’s image input and long-context tasks, including how well it uses the claimed million-token window. The quality impact of running the research checkpoint with one or two passes is also not fully assessed in the information provided. The full model card may supply further technical details, but it was not included with the source material for this article.
Model Card and User Testing
The next evidence to watch for is the full model card, reproducible benchmark results and hands-on reports that specify the hardware and runtime used. Tests across Transformers, llama.cpp, MLX and SGLang could clarify performance differences between supported environments and show whether the model is practical on consumer systems or mainly suited to higher-capacity setups.
Independent evaluations can also check the reported benchmark comparisons and measure the effects of changing the research checkpoint’s pass count. Until those results and hardware details are available, Inkling 2 Mini is a new model release with a distinctive architecture and broad runtime support, while its real-world speed, memory needs and quality remain open questions.
Key Questions
What is Inkling 2 Mini?
It is a multimodal mixture-of-experts model released by Thinking Machines Lab. The company says it can process text and images and has about 26 billion total parameters, with about 3 billion activated per token.
What does the repeated-layer design do?
The model runs its 12 middle physical layers three times, producing 48 logical layer visits per token while reusing weights. The fixed version uses three passes; a separate research checkpoint allows the pass count to be changed.
Can Inkling 2 Mini run locally?
The release lists support for Transformers, llama.cpp and MLX; SGLang support is also provided for datacenter inference. The source material does not specify minimum hardware or confirm performance on particular consumer devices.
Are the reported benchmark results independently verified?
Not in the material supplied. The reported results are claims by Thinking Machines Lab, and the full score table and independent replication were not provided.
What remains unknown about the model?
Key open questions include memory requirements, generation speed, image and long-context quality, and the quality trade-off when using fewer passes in the research version. Independent tests and further technical documentation could address them.
Primary source: Hugging Face · via ThorstenMeyerAI.com
