Tokens, context windows & the first Claude call

Today at a glance

• Recall10 min
• Concept · Tokens and context55 min
• Build · First Claude call70 min
• Drill · JSON folder summary25 min
• Job search20 min

Yesterday the repo could build and the secret scan could fire, and no model had been called. Today you learn the unit the API actually bills and limits — the token — and you send one Messages API request through a client you write by hand. The call logs its stop reason, its input and output token counts, and a cost computed from prices you looked up. That log is the first number SWIFTCLAIM is allowed to put under Measured.

Recall (10 min, no notes)

Tuesday’s recall is Python, taken from the hygiene tests you wrote yesterday. Answer aloud, then open each one.

Concept · Tokens, context windows, and what a call costs (55 min)

The model continues a sequence

A message to Claude is cut into tokens before the model sees it. A token is a chunk of text from the model’s vocabulary: sometimes a whole word, sometimes part of one, sometimes punctuation or whitespace. Words, characters and tokens are three different counts. Claude 4.7 and later use a newer tokenizer that produces more tokens for the same text than earlier Claude models (about 30% more, and the exact increase depends on the text — pricing page). A fixed “characters per token” ratio copied from an old blog post will mis-bill you. The count that matters is the one the API reports.

Given the tokens so far, the model assigns a probability to every token that could come next, one token is chosen, and that token is appended. Then it happens again. The reply is that sequence. Generation stops when the model finishes the turn, when it hits a limit you set, or when it hits a limit the API enforces. Nothing in that loop looks up a claims database. A reply that reads as finished has stopped generating. It has not been checked against a record. Thursday is when that failure gets its proper name; today, remember it when you read stop_reason.

prompt (input tokens)                         reply (output tokens)
[ system | messages | tools ]  ──prefill──▶   t → t → t → t → stop
        billed as input                         one token at a time
                                                billed as output

context window = working memory for THIS request
| system + every message + tools + this turn's output, including thinking |

The context window is this request’s working memory

The context window is every token the model can refer to while generating, including the reply it is writing. It is the request’s working memory. The training corpus is a different thing, and a knowledge cutoff (Thursday) is a different thing again.

Checked against the context windows page on 29 Oct 2026: current Claude models, including Haiku 5.5, Sonnet 5.5 and Opus 5.5, have a 1M-token window, and one synchronous Messages request can generate up to 128k output tokens. Confirm the row for the model you call; older models in that same page are at 200k. A larger window holds a longer prompt. It does not make a long prompt more accurate. The same page names the degradation as context rot: as the token count grows, accuracy and recall drop, so what you choose to include matters as much as the size of the window.

Two different overflows, from that same page:

• The input alone exceeds the window: every model returns HTTP 400 invalid_request_error (“prompt is too long”).
• On Claude 4.5 and newer, input plus the max_tokens you requested may exceed the window and the API still accepts the call. If generation then fills the window, stop_reason is model_context_window_exceeded. Treat that text as truncated.

Estimate a request before you send it with the token counting endpoint, POST /v1/messages/count_tokens. The usage object on the response is what you were billed.

Why cost and latency grow with tokens

Input tokens and output tokens are priced separately, per million tokens, and the output rate on the current price table is higher than the input rate for the same model. A longer prompt costs more because it is more input. A longer answer costs more because it is more output. max_tokens is a ceiling on how many tokens may be generated. It is a budget, and the model may stop earlier. You pay for tokens produced, which usage.output_tokens reports.

Latency follows the same split, for a mechanical reason. The reply is produced one token at a time, each conditioned on the tokens already chosen, so a longer answer is more generation steps. A longer prompt adds prefill work as well as input cost. Thinking, when the model does it, is extra output: it counts toward max_tokens, it counts toward the window, and it is billed as output. The models overview states the practical consequence: latency depends on prompt length, output length and thinking effort. There is no single milliseconds-per-token figure worth memorising.

Worked example

A smoke call on Claude Haiku 5.5 (claude-haiku-5-5), prompt under 100,000 tokens, no cache. Rates from the pricing page on 29 Oct 2026: $0.10 per million input tokens and $0.50 per million output tokens. Prompts over 100,000 tokens on this model use a higher row ($0.50 and $2.50). Other current 1M-window models stay on their standard rates. Re-read the page on the day you run the call and write that date next to the rates you multiply. If the row has moved, use the new row; the arithmetic below is here so you can check your own multiplication.

Those 120 output tokens might be a short visible reply plus thinking. You still multiply 120. If this same prompt were 150,000 tokens, Haiku 5.5’s over-100k row would apply to the whole prompt. Look up that threshold rather than assuming every 1M-window model has one.

One Messages API call

The call you will write hits POST /v1/messages through the official Python SDK. The fields this week cares about:

• model — a pinned id. For today’s smoke call use claude-haiku-5-5, the current high-volume model, after you confirm the id on the models page. Week 2 compares a small tier with a mid tier; today one model is enough.
• max_tokens — required. The ceiling on generation. Thinking tokens spend this budget before you see text, so a tiny ceiling can come back as stop_reason: "max_tokens" with little or no visible text. The official examples use 1024. Use that for the smoke call.
• system — the top-level system prompt: role and standing instructions. Pass it as its own argument.
• messages — the turns. Today that is one user message: {"role": "user", "content": "..."}.

The response carries content (a list of blocks), stop_reason, model and usage. Text you show a human comes from blocks with type == "text". Thinking blocks are a different type; leave them out of the string you store as the answer, and leave them inside the output-token count, where the API already put them.

Full list and the recommended handling: stop reasons.

Seven questions

How this shows up in SWIFTCLAIM today

The smoke call is the first measured number in the project. The client you write returns the stop reason and the token counts; a small script multiplies them by the rates you checked and appends one JSON line. The README’s Measured section stays empty of business claims — one smoke call does not say anything about triage quality — and it can record that a call was logged. The prompt contains no claim text and no personal data. Synthetic data only, and the API key stays in .env.

Build · First Claude call (70 min)

NO AI on src/swiftclaim/llm.py. Rule 2: through Week 3 the LLM client is hand-written. AI may review the module after the tests pass. It may not write the body. The test file and the smoke script below are the spec and the wiring; copy them.

1. Add the SDK inside the SWIFTCLAIM repo from yesterday.

   uv add anthropic

   The SDK reads ANTHROPIC_API_KEY from the environment. Put the real key in .env only. Leave .env.example empty.

2. Create src/swiftclaim/llm.py with this stub. You will replace the two NotImplementedError bodies. Leave the signatures and the docstrings.

   """Hand-written Messages API wrapper. Do not generate this file.
   
   AI may review it after the tests pass. It may not write the body.
   """
   
   from dataclasses import dataclass
   
   
   @dataclass(frozen=True)
   class Completion:
       """One Messages API response, reduced to what the call log needs."""
   
       text: str
       stop_reason: str
       input_tokens: int
       output_tokens: int
       model: str
   
       def cost_usd(self, input_usd_per_million: float, output_usd_per_million: float) -> float:
           """USD for this uncached call at the prices passed in.
   
           input_tokens * input_usd_per_million / 1_000_000
           + output_tokens * output_usd_per_million / 1_000_000
   
           Cache tokens are not in these two counts. Today's calls are uncached,
           so usage.input_tokens is the whole input bill.
           """
           raise NotImplementedError
   
   
   def complete(
       client,
       *,
       model: str,
       system: str,
       user: str,
       max_tokens: int,
   ) -> Completion:
       """Send one user turn. Return text, stop reason and token counts.
   
       ``client`` is an ``anthropic.Anthropic`` instance. Tests pass a fake
       with the same shape: ``client.messages.create(...)`` returns an object
       whose ``content``, ``stop_reason``, ``model`` and ``usage`` match the API.
   
       Pass ``system`` through as a string. Do not send temperature, top_p or top_k.
       ``text`` is the concatenation, in order, of content blocks whose type is
       ``"text"``. Copy stop_reason, model, usage.input_tokens and
       usage.output_tokens as returned. Do not retry: a max_tokens stop is still
       a successful return value.
       """
       raise NotImplementedError

3. Create tests/test_llm.py and run it. It must fail on NotImplementedError before you write any logic. That is the point of writing the test first.

   """Tests for the hand-written Messages API wrapper. No network."""
   
   from types import SimpleNamespace
   
   import pytest
   
   from swiftclaim.llm import Completion, complete
   
   
   class FakeMessages:
       def __init__(self, response):
           self.response = response
           self.calls = []
   
       def create(self, **kwargs):
           self.calls.append(kwargs)
           return self.response
   
   
   class FakeClient:
       def __init__(self, response):
           self.messages = FakeMessages(response)
   
   
   def message(*, text, stop_reason, input_tokens, output_tokens, model="claude-haiku-5-5", extra_blocks=()):
       return SimpleNamespace(
           model=model,
           stop_reason=stop_reason,
           content=[*extra_blocks, SimpleNamespace(type="text", text=text)],
           usage=SimpleNamespace(input_tokens=input_tokens, output_tokens=output_tokens),
       )
   
   
   def test_complete_sends_one_user_turn_and_copies_usage():
       client = FakeClient(
           message(text="ok", stop_reason="end_turn", input_tokens=20, output_tokens=5)
       )
       result = complete(
           client,
           model="claude-haiku-5-5",
           system="You are a claims-triage assistant.",
           user="Smoke test.",
           max_tokens=1024,
       )
       sent = client.messages.calls[0]
       assert sent["model"] == "claude-haiku-5-5"
       assert sent["max_tokens"] == 1024
       assert sent["system"] == "You are a claims-triage assistant."
       assert sent["messages"] == [{"role": "user", "content": "Smoke test."}]
       assert "temperature" not in sent
       assert "top_p" not in sent
       assert "top_k" not in sent
       assert result == Completion(
           text="ok",
           stop_reason="end_turn",
           input_tokens=20,
           output_tokens=5,
           model="claude-haiku-5-5",
       )
   
   
   def test_complete_keeps_text_blocks_only_in_order():
       response = message(
           text="tail",
           stop_reason="end_turn",
           input_tokens=1,
           output_tokens=2,
           extra_blocks=(
               SimpleNamespace(type="thinking", thinking="hidden"),
               SimpleNamespace(type="text", text="head "),
           ),
       )
       result = complete(FakeClient(response), model="m", system="s", user="u", max_tokens=8)
       assert result.text == "head tail"
   
   
   def test_complete_returns_empty_text_when_only_thinking_came_back():
       response = SimpleNamespace(
           model="m",
           stop_reason="max_tokens",
           content=[SimpleNamespace(type="thinking", thinking="budget spent here")],
           usage=SimpleNamespace(input_tokens=10, output_tokens=10),
       )
       result = complete(FakeClient(response), model="m", system="s", user="u", max_tokens=10)
       assert result.text == ""
       assert result.output_tokens == 10
       assert result.stop_reason == "max_tokens"
   
   
   def test_max_tokens_stop_is_returned_without_a_retry():
       client = FakeClient(
           message(text='{"summary":', stop_reason="max_tokens", input_tokens=100, output_tokens=16)
       )
       result = complete(client, model="m", system="s", user="u", max_tokens=16)
       assert result.stop_reason == "max_tokens"
       assert result.text == '{"summary":'
       assert len(client.messages.calls) == 1
   
   
   def test_cost_uses_input_and_output_rates_separately():
       completion = Completion(
           text="ok",
           stop_reason="end_turn",
           input_tokens=2_000,
           output_tokens=500,
           model="claude-haiku-5-5",
       )
       # Fixture rates, not a live price: 2000/1e6*2 + 500/1e6*10 = 0.009
       assert completion.cost_usd(2, 10) == pytest.approx(0.009)

   uv run pytest tests/test_llm.py -q

4. Make the tests pass. Open the hints only when you are stuck, and close them before you type.
   Hint · the request

   Call client.messages.create with keyword arguments model, max_tokens, system and messages. messages is a one-element list: role "user", content equal to the user string. Omit every sampling parameter, including ones you would set to a default.

   Hint · the response

   Blocks are objects, not dicts. Keep a block when block.type == "text", and read block.text. Join those strings with "" so "head " and "tail" become "head tail". A thinking block has no .text you should touch. Read response.usage.input_tokens and response.usage.output_tokens even when text is empty.

   Hint · the cost

   The docstring is the formula. Both counts are divided by one million, then multiplied by their own rate. The rates in the test (2 and 10) are fixtures so the test does not rot when the price page changes. The smoke script is where the real rates go.

5. Log one real call. Copy scripts/first_call.py. Fill the two rates and PRICE_CHECKED from the pricing page before you run it. Confirm the model id on the models page in the same sitting.

   """One real Messages API call. Run after tests/test_llm.py passes.
   
       set -a && source .env && set +a
       uv run python scripts/first_call.py
   """
   
   import json
   import os
   from datetime import datetime, timezone
   from pathlib import Path
   
   import anthropic
   
   from swiftclaim.llm import complete
   
   MODEL = "claude-haiku-5-5"  # confirm on the models page the day you run this
   MAX_TOKENS = 1024
   
   # Fill from https://platform.claude.com/docs/en/about-claude/pricing
   # Haiku 5.5 has a higher row once the prompt exceeds 100_000 tokens.
   INPUT_USD_PER_MILLION = None
   OUTPUT_USD_PER_MILLION = None
   PRICE_CHECKED = ""  # YYYY-MM-DD
   
   SYSTEM = (
       "You are a claims-triage assistant for a fictional insurer. "
       "Personal project, synthetic data. Reply with the single word ok."
   )
   USER = "Smoke test. Do not invent a claim."
   
   
   def main() -> None:
       if (
           INPUT_USD_PER_MILLION is None
           or OUTPUT_USD_PER_MILLION is None
           or not PRICE_CHECKED
       ):
           raise SystemExit("Fill the two rates and PRICE_CHECKED from the pricing page first.")
       if not os.environ.get("ANTHROPIC_API_KEY"):
           raise SystemExit("ANTHROPIC_API_KEY is missing. Source .env in this shell.")
   
       completion = complete(
           anthropic.Anthropic(),
           model=MODEL,
           system=SYSTEM,
           user=USER,
           max_tokens=MAX_TOKENS,
       )
       record = {
           "at": datetime.now(timezone.utc).isoformat(),
           "model": completion.model,
           "stop_reason": completion.stop_reason,
           "input_tokens": completion.input_tokens,
           "output_tokens": completion.output_tokens,
           "text": completion.text,
           "cost_usd": completion.cost_usd(INPUT_USD_PER_MILLION, OUTPUT_USD_PER_MILLION),
           "input_usd_per_million": INPUT_USD_PER_MILLION,
           "output_usd_per_million": OUTPUT_USD_PER_MILLION,
           "price_checked": PRICE_CHECKED,
       }
       log_path = Path("logs/calls.jsonl")
       log_path.parent.mkdir(exist_ok=True)
       with log_path.open("a") as handle:
           handle.write(json.dumps(record) + "\n")
       print(json.dumps(record, indent=2))
       if completion.stop_reason != "end_turn":
           print(f"stop_reason={completion.stop_reason}: logged, not retried.")
   
   
   if __name__ == "__main__":
       main()

   printf 'logs/\n' >> .gitignore
   set -a && source .env && set +a
   uv run python scripts/first_call.py
   uv run pytest -q
   uv run ruff check .

   What “good” looks like on the first real call

   stop_reason is end_turn and text is a short acknowledgement. output_tokens may be larger than that word, because thinking is billed as output. If you see max_tokens, the budget was spent before the reply finished: say so in the log line and leave the retry for Saturday. If you see 400, read the error. Sampling parameters and an over-long prompt are the two causes this lesson prepared you for.

6. Commit the code, not the secret and not the log. git status should show llm.py, tests/test_llm.py, scripts/first_call.py, pyproject.toml and uv.lock. It should not show .env or logs/. The prices live in the script because you set them; that is fine. The key must not.

   git add -A && git status
   git commit -m "Log a first Claude call: tokens, stop reason, cost"

   Run the secret-scan hook on the way in. If it does not run, stop and fix yesterday’s install before you commit.

Acceptance criteria

• uv run pytest -q passes, including the five tests that never touch the network.
• src/swiftclaim/llm.py was written by you. You can point at the line that builds messages and the line that reads usage.
• One real call is appended to logs/calls.jsonl with model id, stop_reason, input_tokens, output_tokens, cost_usd and the date you checked the price.
• The rates in that log match the pricing page for that model and prompt size, and you can recompute cost_usd by hand from the two token counts.
• .env and logs/ are gitignored. .env.example still has an empty API key.

Drill · Load JSON, validate, summarise (25 min)

NO AI 25 min Python, standard library only. About 20–40 lines. Narrate the invalid cases before you run them.

Problem

Write a program that reads every *.json file in a directory (not subdirectories). Each file should hold one object:

• id — non-empty string
• category — non-empty string
• amount — an int or a float, zero or greater. A JSON boolean is invalid even though bool is a subclass of int in Python.
• status — "open" or "closed"

A file that is not valid JSON, or whose object fails a rule, counts as invalid. The program keeps going. Ignore files that are not .json.

Print exactly these lines. With no valid records, the category line and the three amount lines say n/a.

files: 4
valid: 2
invalid: 2
by_category: auto=1, property=1
amount_min: 100
amount_max: 2500
amount_mean: 1300.0

Categories are sorted by name. amount_mean is the arithmetic mean of the valid amounts.

Fixtures

c1.json: {"id": "c-001", "category": "auto", "amount": 100, "status": "open"}

c2.json: {"id": "c-002", "category": "property", "amount": 2500, "status": "closed"}

bad.json: {"id": "", "category": "auto", "amount": 10, "status": "open"} — invalid, empty id.

weird.json: {"id": "c-003", "category": "auto", "amount": true, "status": "open"} — invalid, boolean amount.

notes.txt sits in the same folder and must not affect files.

import json
from pathlib import Path

REQUIRED = ("id", "category", "amount", "status")
STATUSES = {"open", "closed"}


def is_valid(record: object) -> bool:
    if not isinstance(record, dict):
        return False
    if any(key not in record for key in REQUIRED):
        return False
    if not isinstance(record["id"], str) or not record["id"].strip():
        return False
    if not isinstance(record["category"], str) or not record["category"].strip():
        return False
    amount = record["amount"]
    if isinstance(amount, bool) or not isinstance(amount, (int, float)) or amount < 0:
        return False
    return record["status"] in STATUSES


def summarise(folder: Path) -> None:
    files = sorted(folder.glob("*.json"))
    valid: list[dict] = []
    invalid = 0
    for path in files:
        try:
            record = json.loads(path.read_text())
        except json.JSONDecodeError:
            invalid += 1
            continue
        if is_valid(record):
            valid.append(record)
        else:
            invalid += 1

    counts: dict[str, int] = {}
    for record in valid:
        counts[record["category"]] = counts.get(record["category"], 0) + 1

    print(f"files: {len(files)}")
    print(f"valid: {len(valid)}")
    print(f"invalid: {invalid}")
    print(
        "by_category: "
        + (", ".join(f"{name}={counts[name]}" for name in sorted(counts)) or "n/a")
    )
    if not valid:
        print("amount_min: n/a")
        print("amount_max: n/a")
        print("amount_mean: n/a")
        return
    amounts = [record["amount"] for record in valid]
    print(f"amount_min: {min(amounts)}")
    print(f"amount_max: {max(amounts)}")
    print(f"amount_mean: {sum(amounts) / len(amounts)}")

Log it in the drill log (A5): date, problem, language, mode (NO AI), minutes, solved, the bool/int trap if it caught you.

Job search (20 min)

Yesterday’s tracker has 10 companies and an empty Applied column. Today, draft the FDE resume summary and three bullets from the claim ledger. The Applied AI variant is still owed this week; do not start it until these three bullets are done.

1. Open the ledger. Use only rows marked Keep. A Rewrite row may contribute a bullet after you have applied the rewrite. A Remove row contributes nothing.
2. Write a two- or three-line summary aimed at a forward-deployed role: who you build for, what you ship, what you can defend. Any number in it must be a number that survived the audit, with the same wording as the ledger.
3. Write three bullets. Each one is an outcome from a Keep row: what changed, what you personally did, and the measure only if the ledger has the measure. SWIFTCLAIM does not appear. It is not evidence yet.
4. Read the summary aloud. If a sentence would fail the question “show me how you measured that”, delete the number or delete the sentence.

Done means

• You can explain aloud, in under three minutes, why a longer prompt and a longer answer each add cost, and why usage.output_tokens can exceed the length of the text you see.
• You can say what end_turn guarantees (the model stopped) and what it does not guarantee (that the content is right).
• One real Claude call is logged with stop reason, both token counts and a cost you can recompute. Tests pass. The key and the log are outside git.
• The Python drill is solved without AI and logged in A5.
• The FDE resume summary and three bullets exist, and every number in them is a Keep-row number.

Resources

• Working with the Messages API: the request shape this client implements.
• Context windows: what counts, and the two ways a request runs out of room.
• Pricing: the rates you multiply. Re-read it on the day you log a cost.
• Andrej Karpathy, “Intro to Large Language Models”: next-token prediction, from the week’s resource list.