Osaurus AI

OsaurusAI/LFM2.5-VL-3B-JANG_2L

Smallest bundle. Vision tower kept at source precision; language model at 3-bit FFN with 8-bit operators.

JANG mixed-precision affine (JANG_2L, group size 64) conversion of LiquidAI/LFM2.5-VL-3B for Apple silicon (MLX).

LFM2.5-VL-3B is a vision-language model: a hybrid short-convolution + attention lfm2 text backbone (30 layers, 22 conv / 8 full-attention) paired with a SigLIP2 so400m-patch16-naflex vision tower and a 2-layer projector. Images are tiled (up to 10 tiles at 512px) into 64โ€“256 image tokens.

Bundles in this series

Bundle Avg bits Size Notes
LFM2.5-VL-3B-JANG_2L 3.14 1.9 GB smallest
LFM2.5-VL-3B-JANG_4M 4.12 2.2 GB recommended
LFM2.5-VL-3B-JANG_6M 6.06 2.9 GB near-lossless
LFM2.5-VL-3B-MXFP8 9.31 3.4 GB MX FP8 reference

The vision tower is not quantized

All four bundles keep the 437 SigLIP2 vision tensors at source precision. Quantizing a vision tower aggressively is what breaks image understanding, so the bits are spent on the language model instead. This is why image quality holds up even in the smallest bundle โ€” and why the small profiles are dominated by the tower rather than the LM.

Verified

Loaded and generated on an M5 Max on 2026-08-12, through this bundle's own chat_template.jinja and processor_config.json:

  • Vision โ€” correctly named both shapes, both colors and their positions on a synthetic probe image
  • Multi-turn โ€” carried image context across three turns, including a follow-up comparison ("which is larger, and where") and a format constraint
  • Tool calling โ€” parsed <|tool_call_start|>[get_weather(city='...')]<|tool_call_end|> into {'name': 'get_weather', 'arguments': {'city': 'Santa Clara'}}

Sampling

LiquidAI's documented settings, stamped into both generation_config.json and jang_config.json:

Parameter Value
temperature 0.2
top_k 50
top_p 1.0 (disabled)
repetition_penalty 1.0
eos_token_id [124900]

top_p is not specified upstream, so it is stamped as disabled rather than invented.

Reasoning

This model does not reason โ€” it answers directly. There is no thinking rail: the chat template emits a bare <|im_start|>assistant\n generation prompt with no <think> prefill, and no enable_thinking / thinking / preserve_thinking kwarg changes it (verified byte-identical). preserve_thinking only governs whether <think> in prior assistant turns is retained, which is history handling. The bundle metadata says supports_thinking: false accordingly.

Usage

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("OsaurusAI/LFM2.5-VL-3B-JANG_2L")
messages = [{"role": "user", "content": "What is in this image?"}]
prompt = apply_chat_template(processor, model.config, messages, num_images=1)
print(generate(model, processor, prompt, image=["photo.jpg"],
               max_tokens=256, temperature=0.2, verbose=False))

Notes for MLX runtimes

This bundle carries three config keys the MLX stack needs that the upstream checkpoint does not provide in the expected form:

  • text_config.block_ff_dim โ€” mlx_lm's lfm2 requires it; without it the FFN dimension is computed wrong and the model fails to load.
  • image_token_index โ€” mirrors upstream's image_token_id; mlx_vlm reads the _index spelling and otherwise matches zero image tokens.
  • per-module quantization overrides are emitted under both model.<path> and the post-sanitize <path> spelling, because mlx_vlm looks up the latter.

tool_parser_type: "pythonic" is set explicitly so mlx_lm resolves the tool parser without relying on template string-matching.

License

Released under the base model's license. Review LiquidAI/LFM2.5-VL-3B terms before use.


Converted by Jinho Jang โ€” eric@osaurus.ai โ€” with JANG.

Downloads last month
27
Safetensors
Model size
3B params
Tensor type
U32
ยท
F16
ยท
MLX
Hardware compatibility
Log In to add your hardware

Quantized

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for OsaurusAI/LFM2.5-VL-3B-JANG_2L

Quantized
(33)
this model