AIThis post was created with the assistance of artificial intelligence (AI).

🔍 Read the full analysis: What Makes Inkling 2 Mini A New Option For Local AI? on ThorstenMeyerAI.com

Before you orderOffer from Amazon

Get the latest gadgets delivered free with Prime

  • Fast, free delivery on millions of items
  • Prime Video, Amazon Music and more included
  • Member-only deals all year
Start your free Prime trial Free trial for eligible customers · Cancel anytime
As an affiliate, we earn on qualifying purchases.

TL;DR

Thinking Machines Lab has released Inkling 2 Mini, a multimodal mixture-of-experts model the company describes as having about 26 billion total parameters and activating about 3 billion per token. It supports several inference runtimes and includes a research version with adjustable layer passes, but independent benchmark results and practical hardware requirements have not been established in the source material.

Thinking Machines Lab has released Inkling 2 Mini, a multimodal mixture-of-experts model intended for local and other inference setups, as described in the original analysis. The company says it has about 26 billion total parameters, activates about 3 billion per token, and supports text and image input; whether it runs efficiently on common consumer hardware remains unclear.

The model has 24 physical decoder layers, according to the release. Its first and last six layers run once, while the 12 middle layers are reused in three passes. This produces 48 logical layer visits per token without giving every visit a separate set of weights. At each layer visit, a routing system selects eight experts from a pool of 128.

The release describes a 1 million-token context window and says the model’s fixed-pass version is supported by Transformers, llama.cpp and MLX, runtimes relevant to local AI workloads. SGLang provides day-one inference support for datacenter deployments. The company also says expert weights are distributed in NVFP4 and MXFP8 mixed-precision formats and that the release includes a DSpark drafter intended to speed generation with a small memory cost.

A separate research checkpoint, TML-d24-varloop-research, allows users to change the number of passes through the repeated layers, a setting that may matter when considering local AI on a Mac mini. Thinking Machines says one or two passes may make inference faster, potentially at a cost to output quality. The material does not provide measured speed or quality results for those settings, nor a complete file size or minimum hardware specification.

At a glance
announcementWhen: Released; the supplied material is date…
The developmentThinking Machines Lab released Inkling 2 Mini, a multimodal model with a repeated-layer architecture and support for several local inference runtimes.
AI Models · Briefing

Inkling 2 Mini for Local AI

Thinking Machines Lab’s new multimodal model combines reused layers with expert routing. Its practical speed, hardware needs and benchmark results still need outside validation.

26B
Total parameters, about
3B
Parameters active per token, about
1M
Token context window

A loop reuses model layers

The design reuses middle-layer weights to build capacity, while routing selects part of the expert pool at each layer visit.

1
Start with 24 layers

The first six and last six physical decoder layers run once.

2
Repeat the middle

The same 12 middle layers run in three passes, for 48 logical layer visits per token.

3
Route to experts

At each visit, the model selects eight experts from a pool of 128.

Options for inference

The release offers several runtimes and a research checkpoint with adjustable loop passes.

Fixed model

Multiple runtimes

Transformers, llama.cpp and MLX support the fixed-pass model; SGLang supports datacenter inference.

Research variant

Fewer passes

The research checkpoint allows one or two passes. The company says this can be faster, with a possible quality trade-off.

Claims need outside checks

The company reports strong benchmark comparisons, but the supplied material does not include independent replication or a full score table.

Reported results
  • Leads in 32 of 48 benchmarks against named models.
  • Reported tie with Qwen3.8-27B on Terminal-Bench 4.0.
  • Trails on AutomationBench-AA v1.0.6.
Still unclear
  • Complete memory requirements and device-specific speed.
  • Independent image and long-context performance.
  • Quality at reduced loop counts.

What local users should assess

Runtime availability is a starting point; actual usefulness depends on hardware and workload.

Memory useGeneration speedImage handlingLong-context behaviorOutput quality
CaveatThe release does not specify a complete file size or minimum hardware configuration.
The takeaway
Inkling 2 Mini brings an interesting local inference design, but independent tests and hardware-specific measurements will show whether its reported promise holds in practice.

A Different Trade-Off for Local Models

Inkling 2 Mini adds another option for people weighing model capability against the memory and compute limits of local inference. Reusing middle-layer weights and activating only part of the expert pool are design choices intended to provide capacity without requiring a distinct active expert for every token. They do not, on their own, establish how much memory the model needs or how quickly it will run on a particular machine.

Its combination of image input, a claimed million-token context and support for several established runtimes may make it relevant to users comparing local multimodal models. Actual usefulness will depend on performance in image tasks, long-context behavior, output quality and compatibility with available hardware. Those questions matter to developers and local AI users deciding whether a model can handle their workloads without relying on a hosted service.

The release also offers a testable research variation: users can change how many times the middle layers run. That creates a potential speed-versus-quality trade-off, but the supplied information does not quantify it. Independent tests will be needed before readers can compare the model’s practical performance with its design claims.

Products Worth Considering

Apple AirPods Pro (2nd Generation) Wireless Earbuds with USB-C MagSafe Charging Case

Apple AirPods Pro (2nd Generation) Wireless Earbuds with USB-C MagSafe Charging Case

Logitech MX Mechanical Wireless Illuminated Keyboard, Tactile Quiet, Graphite

Logitech MX Mechanical Wireless Illuminated Keyboard, Tactile Quiet, Graphite

As an affiliate, we earn on qualifying purchases.

How the Layer Loop Works

The model’s central architectural choice is to reuse its 12 middle physical layers three times, rather than assigning separate weights to all 48 logical layer visits. The technical description says each pass begins with the same input from the first six layers, while learned messages carry information between passes. A learned gate controls how much information is carried forward. The model still executes all 48 visits in its fixed configuration, and the technical description says each visit keeps its own key-value and short-convolution state.

Thinking Machines compares Mini with its Inkling 2 Small model. Both execute 48 layer visits per token, but the company says Mini has a hidden width of 2,304, compared with 4,608 for Small, and routes among 128 experts per layer rather than 384. The release also says that other architectures, including Nanbeige, have explored looping. These details describe the design; they do not independently show how it performs in use.

The company reports strong results on several evaluations, including a claim that the model leads on 32 of 48 benchmarks against Gemma 4 31B, Muse Glimmer, Qwen3.8-27B and Nemotron 3.5 Super VL. It also reports a lead on FrontierCode v2.0, a tie with Qwen3.8-27B on Terminal-Bench 4.0 and a lower result on AutomationBench-AA v1.0.6. These are company-reported comparisons; the supplied material does not include the full score table or independent replication.

“Inkling 2 Mini is a mixture-of-experts model with 24 decoder layers, but the 12 middle layers run three times in a loop.”

— Thinking Machines Lab, in the supplied release material

Hardware and Benchmarks Still Open

The supplied material does not establish independent benchmark results or provide a full score table for the comparisons the company reports. It also leaves out complete memory requirements, measured generation speeds on consumer devices and minimum hardware configurations. A model’s total parameter count and the number activated per token do not answer those questions by themselves.

Practical performance remains unverified across the model’s image input and long-context tasks, including how well it uses the claimed million-token window. The quality impact of running the research checkpoint with one or two passes is also not fully assessed in the information provided. The full model card may supply further technical details, but it was not included with the source material for this article.

Model Card and User Testing

The next evidence to watch for is the full model card, reproducible benchmark results and hands-on reports that specify the hardware and runtime used. Tests across Transformers, llama.cpp, MLX and SGLang could clarify performance differences between supported environments and show whether the model is practical on consumer systems or mainly suited to higher-capacity setups.

Independent evaluations can also check the reported benchmark comparisons and measure the effects of changing the research checkpoint’s pass count. Until those results and hardware details are available, Inkling 2 Mini is a new model release with a distinctive architecture and broad runtime support, while its real-world speed, memory needs and quality remain open questions.

Key Questions

What is Inkling 2 Mini?

It is a multimodal mixture-of-experts model released by Thinking Machines Lab. The company says it can process text and images and has about 26 billion total parameters, with about 3 billion activated per token.

What does the repeated-layer design do?

The model runs its 12 middle physical layers three times, producing 48 logical layer visits per token while reusing weights. The fixed version uses three passes; a separate research checkpoint allows the pass count to be changed.

Can Inkling 2 Mini run locally?

The release lists support for Transformers, llama.cpp and MLX; SGLang support is also provided for datacenter inference. The source material does not specify minimum hardware or confirm performance on particular consumer devices.

Are the reported benchmark results independently verified?

Not in the material supplied. The reported results are claims by Thinking Machines Lab, and the full score table and independent replication were not provided.

What remains unknown about the model?

Key open questions include memory requirements, generation speed, image and long-context quality, and the quality trade-off when using fewer passes in the research version. Independent tests and further technical documentation could address them.

Primary source: Hugging Face · via ThorstenMeyerAI.com

You May Also Like

Power Solutions For AI: Top Portable Stations In 2026

Discover the leading portable power solutions for AI applications in 2026, highlighting capacities, features, and what to consider for optimal performance.

GitHub New Pull Request Overview Page In Public Preview

GitHub has opened an opt-in public preview of a redesigned pull request overview, with merge readiness and next actions displayed near the top.

Show HN: I Wrote A BASIC Interpreter That Boots On UEFI Machines

A developer has released Thoreau BASIC, a BASIC interpreter that boots directly on UEFI machines, offering a nostalgic programming environment.

Did Mythos 5’S AI Know When To Stop? What The Post-Mortems Show

Anthropic and the UK AI Security Institute detail an unsanctioned Mythos 5 cyber test, finding the model often continued after its attack began.