The Hallucination Probe

Ask a small model 50 factual questions split into common-knowledge and long-tail buckets, score the answers with multi-alias matching, and plot the hallucination rate against the Kalai-Vempala theoretical lower bound. Measure ~5% on common questions, ~40-60% on long-tail — exactly where the proof says you should land.

Time

60–90 min

Hardware

CPU · Colab

Act

V · Where They Break

Status

Live

Artifact

A scored question bank + a measured-vs-theoretical hallucination-rate chart.

Clone the labs repo and run this lab as a script or open it as a notebook:

git clone https://github.com/iqbal-sk/Microscale-labs.git
cd Microscale
just setup-auto      # auto-detects CPU / CUDA / Mac
just run 06
# or:  jupyter lab labs/06-hallucination-probe/lab.py

Full install options (uv, pip, or the platform-specific CUDA paths) are in the labs README.