Token Visualizer: 3th Floor Studios

■ Section 1 — Tokenizer How the model reads text

—

Tokens

—

Chars

—

Words

—

Tok/Wd

—

% Ctx

Token Visualization Hover any block to inspect

Speed:

Each colored block = one token

Each block = one token (a learned word piece). The number below it is its vocabulary ID. Hover any block for details.

What You Are Seeing

Each colored block is one token, a learned byte sequence, not necessarily a word.

English typically runs 1.3–1.5 tok/word. Jargon, acronyms, and rare terms cost more.

Fine-tuned models use the same tokenizer as the base. 3thfloor-tester and Qwen2.5 tokenize identically.

Vocabulary size: Qwen2.5 has 151,936 possible tokens. Every ID in the viz is one slot in that lookup table (0 through 151,935).

Context window: 3thfloor-2b: 2,048 tokens. Qwen2.5-3B: 32,768. Qwen2.5-7B: 131,072. The entire input + output must fit inside this limit.

Scale math, based on your input:

Your input—

Tokenize this text (CPU, 4M tok/s)—

3thfloor-2b corpus (45B tokens)—

Your input is 1 in—

■ Section 2 — Training How any model learns to predict language

Speed:

Step 0 / 1,000 (demo scale, real 2B: ~20,000 steps)

Training data stream — what the model is actually seeing

One example — guess the missing word:

—

loss

lower = model predicted correctly with more confidence

What You Are Seeing

The data stream is what training looks like at scale: billions of tokens from books, code, and the web, flowing through the model continuously.

For each example, the model must guess the missing word. It assigns a probability to every token in its vocabulary, all 151,936 of them, every step.

Step 0: weights are random. The correct word has no advantage over any other word.

Each step: guess, score (loss), adjust weights slightly toward the right answer, then the next batch streams in.

Step 1,000: the right word is now clearly dominant. This is illustrative. A real batch is 2–4 million tokens at once, not 5 words.

Real scale, pre-training from scratch:

Model	Tokens needed	Steps	Time (1×A100)	Cloud cost
3thfloor-2b	45B	86K	~111 days (Behemoth)	$0 (Behemoth)
Qwen2.5-3B	60B–150B	30K–75K	70–175 days	$2,200–$5,400
Qwen2.5-7B	150B–300B	75K–150K	1.1–2.3 yrs	$10K–$26K

Scale math, based on your input:

Train on this text, 1 pass (Behemoth)—

3thfloor-2b full run (45B tokens)—

CAST checkpoint (~12B tokens)—

Qwen2.5-3B from scratch (60B tokens)—

Qwen2.5-7B from scratch (150B, 8xA100)—

■ Section 3 — Generation Watch the model think token by token

Prompt — leave blank to use main input above

Speed:

HW:

Output appears here one token at a time. Blue flash = new token arriving.

Other tokens the model was considering at this position

Tokens: 0 Speed: — tok/sec Time: —

What You Are Seeing

Each blue flash is one token arriving. The model cannot skip ahead. Left to right, one at a time. Each token requires a full forward pass through every layer.

Switch HW: GPU / CPU above to feel the real speed difference. GPU = 20–60 tok/sec, CPU only = 2–8 tok/sec.

Why GPU wins: generating one token on a 2B model = ~4 billion multiply-adds. On a 7B model = ~14 billion. A GPU does these in parallel; a CPU does them serially.

Try the same prompt on both models using the dropdown above. Fine-tuned vs base.

No memory between runs. Every time you hit Generate, it starts completely fresh.

Time to generate this many tokens:

3thfloor-2b, GPU (~50 tok/s)—

3thfloor-2b, CPU only (~5 tok/s)—

Qwen2.5-3B, GPU (~35 tok/s)—

Qwen2.5-7B, GPU (~20 tok/s)—

Temperature — same prompt, three randomness levels

Demo: pick a temperature to see how output changes

Select an example above to see temperature comparisons.

Temperature controls how random token selection is. Low = always picks the most likely next token. High = surprising choices that may lose coherence.

Temp 0: deterministic, robotic but accurate. Temp 0.7: the Goldilocks zone for most tasks. Temp 1.5+: creative but chaotic.

■ Section 4 — Vectors What words mean to the model

How to use this section:

1. Phrase A is already filled from your paragraph above — no action needed.

2. Pick a comparison from Quick Compare below, or type your own in Phrase B.

3. Hit Compare — you'll see a similarity score from ~0.5 (unrelated) to 1 (identical).

Requires an embed model. If you see a timeout error, run: ollama pull nomic-embed-text in a terminal first.

Phrase A — auto-filled from your paragraph above

Phrase B — compare against

Quick compare — pick one

Try These Pairs

What You Are Seeing

Words become vectors, hundreds of numbers. The numbers encode meaning in the model's internal space.

Cosine similarity: 1.0 = same concept. 0.5 = unrelated. Values below 0 = conceptually opposite.

The heatmap shows 64 of ~768 dimensions. Blue = positive value. Red = negative.

Dimensions by model: nomic-embed-text = 768. Qwen2.5-3B internal hidden state = 2,048. Qwen2.5-7B = 3,584. 3thfloor-2b = 2,048. More dimensions = finer-grained meaning distinctions.

Embedding model is separate from the chat model. Needs to be downloaded separately.

Scale math, based on your input:

Embed this text (nomic, GPU)—

Build RAG index: 10K docs like this—

Search 10K index (cosine similarity)<5 ms, always

Qwen2.5-7B embed, this text (full forward pass)—

Qwen2.5-7B: same 10K docs—

■ Section 5 — Fine-Tuning Teaching the model software testing

Pick a software testing scenario:

Speed:

Step 0 / 200 (demo scale, real LoRA: 1,000–5,000 steps; pre-train: 20,000+)

Fine-tuning corpus — domain-specific text flowing in

Base Model (frozen)

Fine-Tuned (3thfloor-tester)

What You Are Seeing

Base model was trained on general text, Wikipedia, books, code. It knows English but not testing vocabulary.

Fine-tuning feeds the trained model thousands of sentences from ISTQB syllabi, testing books, and methodology docs.

Same loss mechanism as training. Far fewer steps. The model already knows language. You are teaching it a specialty, not a new skill from scratch.

This is exactly how 3thfloor-tester was built: Qwen2.5-7B base model + software testing corpus.

Real scale, LoRA fine-tuning:

Model	Dataset	Steps	Time (RTX 4090)	Cost
3thfloor-2b	5M tokens	~2,500	2–4 hours	~$0 (local)
Qwen2.5-3B	5M tokens	~2,500	4–8 hours	~$0 (local)
Qwen2.5-7B (3thfloor-tester)	5M tokens	~2,500	8–16 hours	~$3 (cloud)

16,000× cheaper per token than pre-training from scratch. That ratio is why fine-tuning exists.

Scale math, if your input is one training example:

100K examples like this—

3thfloor-2b LoRA on that corpus (Behemoth)—

Qwen2.5-7B LoRA on that corpus (RTX 4090)—

Pre-training 3thfloor-2b needs—

■ Setup Guide — Full Instructions for Ollama and LM Studio (click to expand)

▼

Option A — Ollama (Recommended)

Free, runs fully local. Supports exact tokenization, embeddings, and streaming generation.

Go to ollama.com and download the installer for your OS (Mac, Windows, Linux). Install it.

Open Terminal (Mac: press Cmd+Space and type Terminal). Type this and press Enter:

ollama serve

Leave this window open. The server runs here.

Open a second Terminal tab (Cmd+T). Download the workshop model:

ollama pull 3thfloor-tester

Also pull the embedding model (needed for Section 4 — Vectors):

ollama pull nomic-embed-text

Reload this page. The dot in the top-right turns green.

Keep the ollama serve terminal window open for the whole workshop session.

Option B — LM Studio

Has a visual interface. Token counts will be estimates since LM Studio has no tokenize API.

Go to lmstudio.ai and download LM Studio. Install and open it.

Click the robot+magnifying glass icon in the sidebar. Search for a model — try qwen2.5-3b-instruct — and download it.

Click the square icon (Developer, 2nd icon in the sidebar). This opens the Developer panel.

Click Server Settings (button at the top of the Developer panel). Find CORS and enable it. Save and close. Without CORS this page cannot reach LM Studio at all.

Click + Load Model (top right of Developer panel). Select the model you downloaded. Wait for status to say READY.

Flip the Status toggle to ON (turns green). The server starts. Look at the Reachable at: address shown in the top bar.

Come back to this page. Click LM Studio in the server picker. If the dot stays gray, copy the Reachable at: address from LM Studio and paste it into the URL field that appears next to the button (e.g. http://100.75.170.97:1234). The dot should turn green.

LM Studio has no tokenize endpoint — Section 1 counts are estimates. CORS must be ON or every request will be blocked by the browser. The "Reachable at" IP may differ from localhost if you have Tailscale or a VPN running.

Required — Chat (Sections 1+2)

The workshop fine-tune

ollama pull 3thfloor-tester

Required — Vectors (Section 4)

Embedding model, separate pull

ollama pull nomic-embed-text

Optional — Base Comparison

Compare fine-tune vs base

ollama pull qwen3:4b

Optional — Larger Model

Needs 10+ GB RAM

ollama pull qwen3:14b

Dot stays gray after following stepsMake sure ollama serve is still running in its terminal window. If you closed it, run it again and reload.

"Model not found" errorRun ollama list in a terminal to see what is installed. If the model is missing, pull it again.

Vectors gives an errorThe embedding model is a separate download from the chat model. Run ollama pull nomic-embed-text.

LM Studio: CORS error in browser consoleYou must enable CORS in the LM Studio Developer panel before starting the server. Stop the server, enable CORS, start again.

Generation is very slowYou are running on CPU. 14B models need 10+ GB RAM. Switch to 3thfloor-tester (4B) for faster output.

Token colors look wrongThe visualization is approximate — it splits on spaces and punctuation. The number at top is exact from the model API.

Token Visualizer

Enter Text — All Five Sections Update From This Input

▼ Compare Tokenization — side-by-side token counts