Deployed Lab

Module 2 · AI engineering foundations · Chapter 8 · Free

Embeddings and vector databases

12 min1 interactive map3 checks
In 30 seconds

The one picture to remember: a city map where similar places sit close together.

The problem Kavish works on a support team (a made-up example). A customer on the phone asks, "How do I get my money back?" The right page is titled "Refund and cancellation terms". His search says "0 results", because a word search needs the same letters.
The plan for this chapter
  1. See why a plain word search fails Kavish.
  2. Learn the fix: a map where similar meanings sit close.
  3. See how a system files sentences on that map, and finds the nearest.
  4. Try it on a bigger map, then do the maths of "near".
  5. Meet two traps that catch real teams.

After this chapter you can:

Next: a few words you will see all the way through. Each one comes with a plain meaning and a small example.

Words you'll meet

Embedding
A way to turn a piece of text into numbers that capture what it means. Example: "refund" and "money back" get similar numbers.
Vector
The list of numbers an embedding produces. Think of it as a map address, like (8, 2). Real ones have hundreds of numbers.
Chunk
A short passage cut from a longer document, about one topic. Example: one section of a help page.
Vector database
A store that keeps vectors together with their text and labels, and can quickly find the ones nearest to a new vector.
Nearest neighbour
The stored item whose address is closest to yours. "Nearest" means "most similar in meaning".
Metadata
Labels attached to a chunk: where it came from, its date, and who may read it.
Keyword search
Search that matches the exact letters you typed. It finds "refund" only if you type "refund".
Re-embed
Run every document through the embedding model again, for example after switching to a new model. It costs time and money.

Next: back to Kavish, to see exactly why his search found nothing.

Step 1: Why a word search fails

Kavish's tool uses keyword search. It looks for the exact letters you type.

People ask in their own words. A good search must match meaning, not letters.

Next: the fix. It starts with a picture of a city.

Step 2: A map where similar meanings sit close

Think of a city. Cafes cluster in one district and banks in another. If you stand in the cafe district, the nearest places are cafes.

Now picture a map where the places are sentences, and the districts are topics. That is what an embedding gives you. Click through the steps below.

A map of meaning Three districts. Money and refunds holds the pins Refund policy and Cancel subscription. Login and security holds Reset password and SSO setup. Getting started holds Quickstart and Install guide. A question pin, get my money back, sits inside the money district next to the refund pins. Money and refunds Login and security Getting started Refund policy Cancel subscription Reset password SSO setup Quickstart Install guide "get my money back"
  1. Cafes sit near cafes and banks near banks. You can find a new cafe without knowing its name: walk to the cafe district. Here the districts are topics.
  2. An embedding gives each sentence a spot on this map of meaning. Sentences about similar things get nearby spots.
  3. Someone types "get my money back". The same model places it on the map. No page uses those exact words, but its meaning is about money.
  4. The closest pins are Refund policy and Cancel subscription. That is how "money back" finds "refund" with no shared words.
MoneySecurityGetting started

Who decides where a sentence goes?

Where the picture breaks down: a real map has hundreds of directions, so you cannot draw it. But "near" and "far" mean the same thing.

Next: you know what the map is. Now: how does a system build it, and how does it use it when someone asks?

Step 3: How a search by meaning works, in two parts

Think of a library card catalogue. The librarian files every card once, in advance. Then, every time someone asks, the librarian goes straight to the right drawer.

What you are looking at: two rows of boxes. Press next to walk along them.

Filing and finding in a vector database Two lanes. In advance: documents are cut into chunks, turned into addresses and stored in a vector database with labels. For each question: the question is turned into an address, the vector database returns the nearest chunks, and a filter applies permissions. IN ADVANCE: filing the cards (once) Documents Chunks(short passages) Embedding model(embedding model) Vector databaseaddress + text + labels(source, date, who may read) EVERY QUESTION: finding the nearest cards Question Embedding model(same one) Find nearest+ check access Best matches(best few chunks)
  1. Cut each document into chunks: short passages, each on one topic. Why: a whole manual is too blurry to match a question.
  2. The embedding model turns each chunk into a vector, its spot on the map. Keep this same model for questions later.
  3. Save each vector with its text and labels: source, date, who may read it. Part 1 is now done.
  4. Someone asks in their own words. The same model puts the question on the same map.
  5. The database returns the closest few chunks that this person may read. They go to a language model that writes the answer (Chapter 9).
UserModelVector databaseYour data

Where the catalogue picture breaks down:

Next: you have seen the idea on a tiny map. Let us try it on a bigger one.

Step 4: Try it: ask the map a question

Why this exercise: to see for yourself that questions land near the right topic, even with no shared words.

The map holds 20 snippets from a made-up company help centre, in four topics. The diamond is your question. Do this:

  1. Click "How do I get my money back?". Look at which dots light up.
  2. Click "Reset my password". It lights up a different corner.
  3. Click "Who can see employee salaries?". This one lands between two groups. Remember it.
  4. Drag the slider to ask for more or fewer results.
What to notice The money question never says "refund", yet it lands next to the refund pages. The salary question sits between "security and access" and "HR and people". Hold that thought: it comes back in Trap 2.

Next: the computer cannot look at a picture. It needs a number for "near". Here is how it gets one.

Step 5: Do the maths: how "near" is measured

Let's work it out together. All numbers are made up. Each address has two numbers: (money, login).

  1. Question "get my money back" is at (8, 2).
  2. Refund page is at (9, 1). Password page is at (1, 9).
  3. Refund gaps: 9 - 8 = 1 across, and 2 - 1 = 1 up.
  4. Square each gap and add: 1 + 1 = 2. The square root of 2 is about 1.4.
  5. Password gaps: 8 - 1 = 7 across, and 9 - 2 = 7 up.
  6. Square and add: 49 + 49 = 98. The square root of 98 is about 9.9.
  7. Compare: 1.4 is far smaller than 9.9.

So the answer is: the refund page is nearest, so it matches, with no shared words.

Show the maths in one line

Distance = square root of (gap one squared + gap two squared). Real systems use hundreds of numbers, but the idea is the same.

Worked example with a two-number address A grid with money on the horizontal axis and login and security on the vertical axis. The question sits at (8, 2), very close to the refund page at (9, 1), and far from the password page at (1, 9). more about money → ↑ more about login Question (8, 2) Refund page (9, 1) Password page (1, 9)
All numbers here are illustrative. Real addresses have hundreds of numbers. Some tools score so that bigger means closer, so check which way yours points.

Next: search by meaning works. Now two traps that catch real teams. The first is about permission.

Trap 1: Closeness is not permission

Remember the salary question that landed between two groups? Here is what goes wrong.

  1. Kavish, a support agent, asks the tool: "Who can see employee salaries?"
  2. The map places the question next to the HR salary pages. They are very similar in meaning, so they are the nearest.
  3. The tool returns an HR page that Kavish is not allowed to read.

The map only knows what is similar. It has no idea who is asking, or who may read what.

The fix: store a label on every chunk saying who may read it (the metadata from the word list). Before results are shown, drop the chunks the asker may not read.

Next: the second trap: changing the model.

Trap 2: Switching the embedding model

Scenario: the manager who says just switch the model A manager reads about a newer embedding model and asks you to "just switch it on". Remember: re-embed means running every document through the new model again. The old chunks sit on map A, and the new questions would land on map B.

Each model draws its own map. An address from one map means nothing on another. Re-embed every chunk, and plan the time and cost first.

A decision you will meet at a customer

Scenario: a customer who already runs Postgres A small firm has a few thousand documents. They already run Postgres (a common database). The team wants a vector database.

Answer: start with A. It means fewer systems to approve, secure and run. Move to B only if speed or size demands it.

Mix-ups to avoid

Optional depth: chunks, k and approximate search

Recap in plain words

In the field A team tested search on a few clean documents and it looked great. On the full document set, old policy versions ranked first.

Lesson: test on your real, messy documents, not a clean sample.

Next: the model reads these chunks and writes the answer for Kavish (Chapter 9).

Chapter quiz

Four questions. Score 70% or more to mark this chapter complete.

Review