04-01 Embedding Lookup

Why this matters

Embedding lookup replaces sparse one-hot inputs with dense trainable token representations.

Intuition first (no jargon)

Token IDs index rows in an embedding table that the model can optimize.

Code walkthrough

js
export function embeddingLookup(table, ids) {
  // table: [vocabSize][dModel]
}

Your task

Implement embeddingLookup(table, ids).

• Return an array of vectors for each ID.
• Validate ID bounds.
• Do not mutate the original table.

Hints

• Use ids.map((id) => table[id].slice()).
• Keep output shape visible in logs while debugging.
• Throw clear errors for invalid IDs.

Check your thinking

1. Why is copying rows safer than sharing references?
2. What does embedding dimension control?
3. Why might similar letters cluster?

Stretch (optional)

Add batched lookup for ids shaped [B][T].

Likely test focus

• Correct vectors returned for known IDs.
• Correct output shape.
• No table mutation side effects.

What should improve

Your model can now represent tokens with dense vectors that carry learned structure.

Bridge to next lesson

Next you combine several context embeddings into one predictive representation.