Module 2 · AI engineering foundations · Chapter 8 · Free
Embeddings and vector databases
- An embedding turns a sentence into a short list of numbers, like an address.
- Sentences with similar meaning get nearby addresses, even with no shared words.
- A vector database stores the addresses and quickly finds the nearest ones.
The one picture to remember: a city map where similar places sit close together.
- See why a plain word search fails Kavish.
- Learn the fix: a map where similar meanings sit close.
- See how a system files sentences on that map, and finds the nearest.
- Try it on a bigger map, then do the maths of "near".
- Meet two traps that catch real teams.
After this chapter you can:
- Explain in one minute why "money back" finds "refund".
- Say what a vector database stores and what it does.
- Explain why a search can show a page the user is not allowed to read, and how to stop that.
Next: a few words you will see all the way through. Each one comes with a plain meaning and a small example.
Words you'll meet
- Embedding
- A way to turn a piece of text into numbers that capture what it means. Example: "refund" and "money back" get similar numbers.
- Vector
- The list of numbers an embedding produces. Think of it as a map address, like (8, 2). Real ones have hundreds of numbers.
- Chunk
- A short passage cut from a longer document, about one topic. Example: one section of a help page.
- Vector database
- A store that keeps vectors together with their text and labels, and can quickly find the ones nearest to a new vector.
- Nearest neighbour
- The stored item whose address is closest to yours. "Nearest" means "most similar in meaning".
- Metadata
- Labels attached to a chunk: where it came from, its date, and who may read it.
- Keyword search
- Search that matches the exact letters you typed. It finds "refund" only if you type "refund".
- Re-embed
- Run every document through the embedding model again, for example after switching to a new model. It costs time and money.
Next: back to Kavish, to see exactly why his search found nothing.
Step 1: Why a word search fails
Kavish's tool uses keyword search. It looks for the exact letters you type.
- The customer's question: "How do I get my money back?"
- The right page: "Refund and cancellation terms".
- Shared words: none. So the result is "0 results".
People ask in their own words. A good search must match meaning, not letters.
Next: the fix. It starts with a picture of a city.
Step 2: A map where similar meanings sit close
Think of a city. Cafes cluster in one district and banks in another. If you stand in the cafe district, the nearest places are cafes.
Now picture a map where the places are sentences, and the districts are topics. That is what an embedding gives you. Click through the steps below.
- Cafes sit near cafes and banks near banks. You can find a new cafe without knowing its name: walk to the cafe district. Here the districts are topics.
- An embedding gives each sentence a spot on this map of meaning. Sentences about similar things get nearby spots.
- Someone types "get my money back". The same model places it on the map. No page uses those exact words, but its meaning is about money.
- The closest pins are Refund policy and Cancel subscription. That is how "money back" finds "refund" with no shared words.
Who decides where a sentence goes?
- An embedding model: a program that has read a huge amount of text and learned which sentences are used in similar situations.
- Nobody types in a list of synonyms. The model learns it from examples.
- The spot it picks is a list of numbers, called a vector. On our drawing, two numbers (left-right, up-down) are enough.
Where the picture breaks down: a real map has hundreds of directions, so you cannot draw it. But "near" and "far" mean the same thing.
Next: you know what the map is. Now: how does a system build it, and how does it use it when someone asks?
Step 3: How a search by meaning works, in two parts
Think of a library card catalogue. The librarian files every card once, in advance. Then, every time someone asks, the librarian goes straight to the right drawer.
- Part 1, in advance (top row): cut the documents into chunks, give each chunk a spot on the map, store them.
- Part 2, every question (bottom row): give the question a spot, then fetch the nearest chunks.
What you are looking at: two rows of boxes. Press next to walk along them.
- Cut each document into chunks: short passages, each on one topic. Why: a whole manual is too blurry to match a question.
- The embedding model turns each chunk into a vector, its spot on the map. Keep this same model for questions later.
- Save each vector with its text and labels: source, date, who may read it. Part 1 is now done.
- Someone asks in their own words. The same model puts the question on the same map.
- The database returns the closest few chunks that this person may read. They go to a language model that writes the answer (Chapter 9).
Where the catalogue picture breaks down:
- A paper card is found by its title. Here, cards are found by closeness of meaning.
- Real systems search "approximately" to be fast, so they may skip a close match.
- A paper catalogue does not check who may read a card. A real system must. Step 5 below comes back to this.
Next: you have seen the idea on a tiny map. Let us try it on a bigger one.
Step 4: Try it: ask the map a question
Why this exercise: to see for yourself that questions land near the right topic, even with no shared words.
The map holds 20 snippets from a made-up company help centre, in four topics. The diamond is your question. Do this:
- Click "How do I get my money back?". Look at which dots light up.
- Click "Reset my password". It lights up a different corner.
- Click "Who can see employee salaries?". This one lands between two groups. Remember it.
- Drag the slider to ask for more or fewer results.
Next: the computer cannot look at a picture. It needs a number for "near". Here is how it gets one.
Step 5: Do the maths: how "near" is measured
Let's work it out together. All numbers are made up. Each address has two numbers: (money, login).
- Question "get my money back" is at (8, 2).
- Refund page is at (9, 1). Password page is at (1, 9).
- Refund gaps: 9 - 8 = 1 across, and 2 - 1 = 1 up.
- Square each gap and add: 1 + 1 = 2. The square root of 2 is about 1.4.
- Password gaps: 8 - 1 = 7 across, and 9 - 2 = 7 up.
- Square and add: 49 + 49 = 98. The square root of 98 is about 9.9.
- Compare: 1.4 is far smaller than 9.9.
So the answer is: the refund page is nearest, so it matches, with no shared words.
Show the maths in one line
Distance = square root of (gap one squared + gap two squared). Real systems use hundreds of numbers, but the idea is the same.
Next: search by meaning works. Now two traps that catch real teams. The first is about permission.
Trap 1: Closeness is not permission
Remember the salary question that landed between two groups? Here is what goes wrong.
- Kavish, a support agent, asks the tool: "Who can see employee salaries?"
- The map places the question next to the HR salary pages. They are very similar in meaning, so they are the nearest.
- The tool returns an HR page that Kavish is not allowed to read.
The map only knows what is similar. It has no idea who is asking, or who may read what.
The fix: store a label on every chunk saying who may read it (the metadata from the word list). Before results are shown, drop the chunks the asker may not read.
Next: the second trap: changing the model.
Trap 2: Switching the embedding model
Each model draws its own map. An address from one map means nothing on another. Re-embed every chunk, and plan the time and cost first.
A decision you will meet at a customer
- Option A: add the pgvector extension to Postgres.
- Option B: buy a new dedicated vector database.
Answer: start with A. It means fewer systems to approve, secure and run. Move to B only if speed or size demands it.
Mix-ups to avoid
- "Embeddings are a dictionary of synonyms." No. A model learns them from lots of text.
- "Nearest means allowed." No. Closeness says nothing about permission. Filter on labels.
- "A bigger model fixes stale results." No. Old policy versions need date labels.
Optional depth: chunks, k and approximate search
- Chunk size: one clear topic per chunk. Too big blurs the meaning. Too small loses context.
- k: how many nearest chunks you ask for. Say k = 5 for five.
- Approximate search: an index lets the database skip most vectors. It is much faster, and may miss a close match.
Recap in plain words
- An embedding is a spot on a map of meaning. Similar meaning is nearby.
- A vector database stores vectors and labels, and finds the nearest fast.
- Closeness is not permission. Filter on labels.
- Switch the model, and you must re-embed everything.
- Old and new versions say almost the same thing, so both sit close to the question.
- Adding date and version labels, and preferring the current one, fixed it.
- A better model would not have fixed it.
Lesson: test on your real, messy documents, not a clean sample.
Next: the model reads these chunks and writes the answer for Kavish (Chapter 9).
Chapter quiz
Four questions. Score 70% or more to mark this chapter complete.