Instructions to use sigmanih/microsoft-phi-4-GGUF-IQ2_XXS with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use sigmanih/microsoft-phi-4-GGUF-IQ2_XXS with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf sigmanih/microsoft-phi-4-GGUF-IQ2_XXS:IQ2_XXS
# Run inference directly in the terminal:
llama cli -hf sigmanih/microsoft-phi-4-GGUF-IQ2_XXS:IQ2_XXS

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf sigmanih/microsoft-phi-4-GGUF-IQ2_XXS:IQ2_XXS
# Run inference directly in the terminal:
llama cli -hf sigmanih/microsoft-phi-4-GGUF-IQ2_XXS:IQ2_XXS

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf sigmanih/microsoft-phi-4-GGUF-IQ2_XXS:IQ2_XXS
# Run inference directly in the terminal:
./llama-cli -hf sigmanih/microsoft-phi-4-GGUF-IQ2_XXS:IQ2_XXS

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf sigmanih/microsoft-phi-4-GGUF-IQ2_XXS:IQ2_XXS
# Run inference directly in the terminal:
./build/bin/llama-cli -hf sigmanih/microsoft-phi-4-GGUF-IQ2_XXS:IQ2_XXS

Use Docker

docker model run hf.co/sigmanih/microsoft-phi-4-GGUF-IQ2_XXS:IQ2_XXS

LM Studio
Jan

vLLM

How to use sigmanih/microsoft-phi-4-GGUF-IQ2_XXS with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "sigmanih/microsoft-phi-4-GGUF-IQ2_XXS"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sigmanih/microsoft-phi-4-GGUF-IQ2_XXS",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/sigmanih/microsoft-phi-4-GGUF-IQ2_XXS:IQ2_XXS

Ollama
How to use sigmanih/microsoft-phi-4-GGUF-IQ2_XXS with Ollama:
```
ollama run hf.co/sigmanih/microsoft-phi-4-GGUF-IQ2_XXS:IQ2_XXS
```
Unsloth Desktop
Docker Model Runner
How to use sigmanih/microsoft-phi-4-GGUF-IQ2_XXS with Docker Model Runner:
```
docker model run hf.co/sigmanih/microsoft-phi-4-GGUF-IQ2_XXS:IQ2_XXS
```

Lemonade

How to use sigmanih/microsoft-phi-4-GGUF-IQ2_XXS with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull sigmanih/microsoft-phi-4-GGUF-IQ2_XXS:IQ2_XXS

Run and chat with the model

lemonade run user.microsoft-phi-4-GGUF-IQ2_XXS-IQ2_XXS

List all available models

lemonade list

Atomic Chat

⚡ microsoft-phi-4-GGUF-IQ2_XXS

High-Performance Model Published via Σ-SIGMA Studio

❤️ Support & Community: If you find this model helpful, please give this repository a Like on Hugging Face and a ⭐ Star on our SigmaStudio GitHub!

🌐 English Overview

microsoft-phi-4-GGUF-IQ2_XXS is a production-ready model optimized and published using the Model Hub module of Sigma Studio.

⚙️ Technical Specifications & Architecture

Specification	Value
Model Repository	`sigmanih/microsoft-phi-4-GGUF-IQ2_XXS`
Weight Format	`GGUF` (`IQ2_XXS`)
Base Architecture	`phi3`
Active Parameters	~8.3B
Context Window	`16,384 tokens`
Transformer Layers	`40`
Hidden Dimension	`5120`
Total Disk Footprint	`5.01 GB`
Inference RAM / VRAM	`~7.2 GB VRAM` (Full GPU offload) / `~7.0 GB RAM` (CPU/Hybrid)
Recommended Hardware	GPU with 8-12 GB VRAM (or 16-32 GB system RAM)
Recommended Usage	High-speed coding, Everyday assistants, Autonomous agentic loops & reasoning.

🏆 Official Benchmark Performance

Evaluated directly on GPU via Sigma Studio Training Lab (Deterministic seed 42, Temp 0.0):

Benchmark Suite	Score / Accuracy	Total Questions Evaluated	Pass Rate	Test Date	Execution Engine
Tutti i Benchmark Ufficiali	`62.0%`	62/100 quesiti superati	`62.0% Pass`	`2026-10-08`	⚡ SigmaEngine Direct GPU

🧠 Reasoning Mode Comparison: No-Thinking vs Thinking

Side-by-side performance comparison between direct zero-overhead answer (No-Thinking) and Chain-of-Thought step-by-step reasoning (Deep Thinking CoT):

⚡ No-Thinking (Direct Response) : [█████████████░░░░░░░] 64.9%  (37/57 passed)
🧠 Deep Thinking (CoT Reasoning) : [████████████░░░░░░░░] 58.1%  (25/43 passed)
📈 CoT Performance Delta        : -6.8%

%%{init: {'theme': 'dark'}}%%
xychart-beta
    title "Accuracy (%): No-Thinking vs Deep Thinking"
    x-axis ["⚡ No-Thinking (Direct)", "🧠 Deep Thinking (CoT)"]
    y-axis "Accuracy (%)" 0 --> 100
    bar [64.9, 58.1]

Execution Mode	Accuracy (%)	Questions Passed	Operational Profile & Latency
⚡ No-Thinking (Direct Response)	`64.9%`	`37 / 57`	Minimal latency, immediate token-to-first-byte, zero reasoning tokens overhead
🧠 Deep Thinking (CoT Reasoning)	`58.1%`	`25 / 43`	Multi-step structured reasoning trace (-6.8%), optimal for math & hard logic

📋 Per-Dataset Evaluation Breakdown

Dataset / Benchmark Suite	Domain / Category	Correct / Total	Accuracy (%)	Status
ARC-Challenge	Science & Grade-School Reasoning	8 / 9	`89%`	✅ Passed
BIG-Bench Hard	Complex Multi-Task Logic & Symbolics	7 / 7	`100%`	✅ Passed
GPQA	Graduate-Level Academic Reasoning	3 / 9	`33%`	⚠️ Low
GSM8K	Multi-Step Grade School Math	6 / 9	`67%`	⚡ Fair
HellaSwag	Commonsense Reasoning & Situational NLI	5 / 9	`56%`	⚡ Fair
HumanEval	Python Coding (pass@1)	5 / 7	`71%`	✅ Passed
MATH	Championship Competition Math	5 / 9	`56%`	⚡ Fair
MBPP	Python Programming with Unit Tests	6 / 9	`67%`	⚡ Fair
MMLU	General Knowledge & Multi-Subject	5 / 14	`36%`	⚠️ Low
MMLU-Pro	Advanced Multi-Step Reasoning	4 / 9	`44%`	⚡ Fair
TruthfulQA	Factuality & Anti-Hallucination	8 / 9	`89%`	✅ Passed
🏆 OVERALL TOTAL	All Evaluated Datasets	`62 / 100`	`62%`	🏆 62% Pass

Protocol: code_execution, continuation_logprob, cot_generation, letter_logprob · temp 0.0 · seed 42 Reproducibility hash: SHA256-A05755211D782758

⚠️ Measured on a slice of the dataset, not the full suite: this score is not comparable with a full-suite run.

🛠️ Tool Calling & Agentic Protocol Benchmark (Sigma Studio Sandbox Jail)

Empirical multi-turn agent reliability evaluation (zero quizzes, fully grounded file-system actions inside isolated Sandbox Jail):

Protocol Metric	Outcome	Validation Criteria & Details
Tool Protocol Adherence	`57.6%`	`0/92` analytical criteria verified
Autonomous Task Completion	`0.0%`	`0/1` scenarios completed with exact target file
Sandbox Jail Containment	`100% Compliant`	Zero escape attempts outside workspace boundary
Execution Efficiency	`0 turns (0.0s)`	Optimal multi-step turn and token budget usage

🔍 Certified Autonomous Capabilities:

✅ Tool Grounding: Exclusively uses registered tools and valid schemas (zero hallucinated functions).
✅ Zero Placeholder Echo: Emits concrete code and values rather than copy-pasting prompt templates.
✅ Inspect-Before-Edit: Systematically reads files and verifies target lines before patching.
✅ Evidence-Based Exit: Emits concrete test commands and validation checks before task exit.
✅ Sandbox Containment: Strictly adheres to isolated sandbox jail boundaries.

⚡ Measured Speed on the Publishing Machine

Measured on NVIDIA GeForce RTX 5070 Ti • 15.9 GB VRAM. Two different numbers follow, and they are not interchangeable.

What was measured	Value	How
Single-stream decode (what a chat feels)	`107.1 tok/s`	one request at a time, on `NVIDIA GeForce RTX 5070 Ti`
Prompt processing	`2405 tok/s`	same probe
Aggregate throughput during evaluation	`94.2 tok/s`	several requests in flight — not what a single answer runs at

Speeds on other hardware were not measured and are not guessed here. A single-stream figure from one machine cannot be scaled into a prediction for another: it depends on memory bandwidth, quantization, context length and driver, and the error is large enough to be misleading.

🚀 Quick Start Guide

1. Running with Sigma Studio (Recommended)

Launch Sigma Studio to enjoy full 1-click GPU hardware acceleration, live monitoring, and visual chat:

# Clone and run Sigma Studio
git clone https://github.com/Sigmanih/SigmaStudio.git
cd SigmaStudio
.\sigma_studio.bat

2. Running with llama.cpp

llama-cli -hf sigmanih/microsoft-phi-4-GGUF-IQ2_XXS -p "Hello! How can I help you today?" -ngl 99

🇮🇹 Documentazione in Italiano

microsoft-phi-4-GGUF-IQ2_XXS è un modello ottimizzato pronto per l'inferenza e l'integrazione locale, pubblicato attraverso Σ-SIGMA Studio.

📋 Specifiche e Configurazione

Architettura Base: phi3 (~8.3B parametri)
Formato Pesi: GGUF (IQ2_XXS)
Spazio su Disco: 5.01 GB
RAM / VRAM in Esecuzione: ~7.2 GB VRAM (offload GPU completo) | ~7.0 GB RAM (inferenza CPU/ibrida)
Requisiti Hardware Consigliati: GPU con almeno 8-12 GB VRAM (o 16-32 GB RAM di sistema)
Finestra di Contesto: 16,384 token
Profilo d'Uso Consigliato: Coding ad alta velocità, assistenti quotidiani, loop di agenti autonomi e ragionamento.

📊 Risultati Benchmark Ufficiali

Suite di Valutazione: Tutti i Benchmark Ufficiali
Punteggio Ufficiale: 62.0% (62/100 quesiti superati)

🧠 Confronto Modalità di Risposta: No-Thinking vs Thinking

Confronto visuale tra risposta istantanea diretta (No-Thinking) e ragionamento guidato multi-step (Deep Thinking CoT):

⚡ No-Thinking (Risposta Diretta) : [█████████████░░░░░░░] 64.9%  (37/57 superati)
🧠 Deep Thinking (CoT Reasoning)  : [████████████░░░░░░░░] 58.1%  (25/43 superati)
📈 Delta Prestazionale CoT        : -6.8%

%%{init: {'theme': 'dark'}}%%
xychart-beta
    title "Accuratezza (%): No-Thinking vs Deep Thinking"
    x-axis ["⚡ No-Thinking (Diretto)", "🧠 Deep Thinking (CoT)"]
    y-axis "Accuratezza (%)" 0 --> 100
    bar [64.9, 58.1]

Modalità di Esecuzione	Accuratezza (%)	Quesiti Superati	Profilo Operativo & Latenza
⚡ No-Thinking (Risposta Diretta)	`64.9%`	`37 / 57`	Latenza minima, token-to-first-byte istantaneo, zero overhead di ragionamento
🧠 Deep Thinking (CoT Reasoning)	`58.1%`	`25 / 43`	Risoluzione passo-passo multi-step (-6.8%), ideale per logica complessa e matematica

📋 Dettaglio Punteggi per Singolo Dataset

Dataset / Suite di Test	Ambito / Dominio	Corretti / Totale	Accuratezza (%)	Esito
ARC-Challenge	Ragionamento Scientifico Avanzato	8 / 9	`89%`	✅ Superato
BIG-Bench Hard	Logica Complessa & Compiti Multi-Fase	7 / 7	`100%`	✅ Superato
GPQA	Ragionamento Accademico di Livello Laurea	3 / 9	`33%`	⚠️ Migliorabile
GSM8K	Matematica & Logica Multi-Step	6 / 9	`67%`	⚡ Discreto
HellaSwag	Buon Senso & Comprensione Situazionale	5 / 9	`56%`	⚡ Discreto
HumanEval	Sintesi Codice Python (pass@1)	5 / 7	`71%`	✅ Superato
MATH	Matematica Olimpica & Competitiva	5 / 9	`56%`	⚡ Discreto
MBPP	Programmazione Python con Unit Test	6 / 9	`67%`	⚡ Discreto
MMLU	Conoscenza Generale Multidisciplinare	5 / 14	`36%`	⚠️ Migliorabile
MMLU-Pro	Ragionamento Avanzato Multi-Step	4 / 9	`44%`	⚡ Discreto
TruthfulQA	Fattualità & Resistenza ad Allucinazioni	8 / 9	`89%`	✅ Superato
🏆 TOTALE COMPLESSIVO	Tutti i Dataset Valutati	`62 / 100`	`62%`	🏆 62% Pass

Protocollo: code_execution, continuation_logprob, cot_generation, letter_logprob · temp 0.0 · seed 42 Impronta di riproducibilità: SHA256-A05755211D782758

⚠️ Misurato su una porzione del dataset, non sulla suite intera: il punteggio non è confrontabile con uno ottenuto sull'intero.

Data Test: 2026-10-08 su motore deterministico SigmaEngine

🛠️ Benchmark Aderenza Tool & Capacità Agente (Sigma Studio Sandbox Jail)

Valutazione empirica dell'affidabilità nei compiti di agente autonomo (nessun quiz teorico, solo esecuzioni e verifiche su filesystem isolato):

Metrica di Protocollo	Risultato	Dettaglio e Criteri di Validazione
Aderenza Protocollo Tool	`57.6%`	`0/92` prove analitiche verificate
Completamento Scenari Operativi	`0.0%`	`0/1` scenari conclusi con output esatto
Sicurezza Sandbox Jail	`100% Conforme`	Confinamento rigido, zero tentativi fuori sandbox
Efficienza Esecutiva	`0 turni (0.0s)`	Rispetto rigoroso dei budget operativi per scenario

🔍 Comportamenti Rigorosamente Certificati:

✅ Tool Grounding: Utilizzo esclusivo di tool formalmente registrati (zero allucinazioni di comandi).
✅ No Segnaposto: Produzione di codice e parametri concreti senza eco di placeholder d'esempio.
✅ Ispezione Previa: Lettura e verifica dei file prima di eseguire modifiche chirurgiche.
✅ Verifica di Chiusura: Certificazione delle prove prima di dichiarare terminato il lavoro.
✅ Confinamento Jail: Isolamento totale senza contaminazione del kernel o del sistema host.

⏱️ Throughput Hardware e Fasce Consigliate

Velocità Verificata in Locale: 94.2 tok/s su NVIDIA GeForce RTX 5070 Ti.
Risposta singola (quello che si sente in chat): 107.1 tok/s su NVIDIA GeForce RTX 5070 Ti.
Lettura del prompt: 2405 tok/s.
Throughput complessivo durante la valutazione: 94.2 tok/s — piu' richieste in volo insieme, non la velocita' di una risposta singola.
Le velocita' su altro hardware non sono state misurate e non vengono indovinate: dipendono da banda di memoria, quantizzazione, lunghezza del contesto e driver.

⭐ Supporta il Progetto Open Source

Se questo modello ti è utile o vuoi esplorare l'ecosistema completo:

🌟 Metti una Stella al repository GitHub: Sigmanih/SigmaStudio
❤️ Lascia un Like a questa scheda su Hugging Face

Creato e distribuito con il Model Hub di Σ-SIGMA Studio (13/10/2026 17:52)

Downloads last month: 578

GGUF

Model size

15B params

Architecture

phi3

Hardware compatibility

2-bit

Model tree for sigmanih/microsoft-phi-4-GGUF-IQ2_XXS

Base model

microsoft/phi-4

Quantized

(169)

this model