03-01 Make Context Target Examples

Why this matters

Supervised next-token training requires explicit (context, target) pairs from token streams.

Intuition first (no jargon)

Sliding windows convert a long sequence into many local prediction tasks.

Code walkthrough

js
export function makeExamples(ids, blockSize) {
  return { x: [], y: [] };
}

Your task

Implement makeExamples(ids, blockSize).

• Build x where each item is a context of length blockSize.
• Build y where each item is the token right after that context.
• Return empty arrays if there is not enough data.

Hints

• Loop index starts at blockSize.
• Context slice is ids.slice(i - blockSize, i).
• Target is ids[i].

Check your thinking

1. What changes when blockSize is larger?
2. Why must x.length === y.length?
3. How does this differ from bigram training data?

Stretch (optional)

Add optional stride to skip overlapping windows.

Likely test focus

• Correct number of examples.
• Correct x/y alignment on toy input.
• Handles short sequences safely.

What should improve

Your dataset now supports multi-token context learning instead of one-step counting only.

Bridge to next lesson

Next: map token IDs to vectors for trainable scoring.