onnx-community/Voxtral-Mini-4B-Realtime-Arabic-ONNX

Voxtral Mini 4B Realtime Arabic (ONNX)

ONNX weights for mistralai/Voxtral-Mini-4B-Realtime-Arabic, a streaming speech-to-text model for Arabic dialects and Modern Standard Arabic, fine-tuned from Voxtral-Mini-4B-Realtime-2602.

This model is a ONNX conversion of the original model to make it run in a browser. For other usage. Please use the normal model :mistralai/Voxtral-Mini-4B-Realtime-Arabic,

Usage (Transformers.js)

import { VoxtralRealtimeForConditionalGeneration, VoxtralRealtimeProcessor } from "@huggingface/transformers";

const model_id = "onnx-community/Voxtral-Mini-4B-Realtime-Arabic-ONNX";
const processor = await VoxtralRealtimeProcessor.from_pretrained(model_id);
const model = await VoxtralRealtimeForConditionalGeneration.from_pretrained(model_id, { dtype: "q4" });

// `audio` is a Float32Array of 16 kHz mono samples
async function* inputFeatures(audio) {
  const { hop_length, n_fft } = processor.feature_extractor.config;
  const winHalf = Math.floor(n_fft / 2);
  const first = await processor(audio.subarray(0, processor.num_samples_first_audio_chunk), { is_streaming: true, is_first_audio_chunk: true });
  yield first.input_features;
  let melFrame = processor.num_mel_frames_first_audio_chunk;
  let start = melFrame * hop_length - winHalf;
  while (start + processor.num_samples_per_audio_chunk < audio.length) {
    const chunk = await processor(audio.slice(start, start + processor.num_samples_per_audio_chunk), { is_streaming: true, is_first_audio_chunk: false });
    yield chunk.input_features;
    melFrame += processor.audio_length_per_tok;
    start = melFrame * hop_length - winHalf;
  }
}

const first = await processor(audio.subarray(0, processor.num_samples_first_audio_chunk), { is_streaming: true, is_first_audio_chunk: true });
const outputs = await model.generate({ input_ids: first.input_ids, input_features: inputFeatures(audio), max_new_tokens: 1024 });
console.log(processor.batch_decode(outputs, { skip_special_tokens: true })[0]);

Limitations

The model targets Arabic transcription. For other languages, use the original Voxtral Mini 4B Realtime.

License

This model is licensed under the Apache 2.0 License.

You must not use this model in a manner that infringes, misappropriates, or otherwise violates any third party’s rights, including intellectual property rights.