Two embedding models, one vector space
I am building a product advisor for my shop: the catalogue lives as vectors in Qdrant, a language model searches it and answers over Telegram. Going through the two workflows side by side, I noticed they do not match – in a way that is easy to miss.
The sync that writes the products in uses an embedding model from OpenAI. The chatbot that later searches them uses a local model through Ollama. Two different models for the same job.
Why this cannot work
An embedding model turns text into a list of numbers – 1536 of them for text-embedding-3-small. Those numbers are not properties of the text that exist objectively somewhere. They are coordinates in a space that this particular model built during its training.
Two models build two entirely different spaces. The value 0.42 at position 7 means something different in one model than in the other – if it means anything at all. A distance measure like cosine then compares angles between points that are not in the same coordinate system.
The loud failure and the quiet one
If the two models produce a different number of dimensions, you are lucky. Qdrant rejects the vector, you get an error, and you know exactly where to look. text-embedding-3-small returns 1536 values, nomic-embed-text returns 768 – that breaks cleanly.
The dangerous case is the other one: two models with the same dimension count. Then everything fits technically. The search runs, returns ten hits, and the hits are random.
Here is what makes it genuinely nasty: a RAG system never answers "I found nothing". It receives ten products, hands them to the language model, and the language model turns them into a fluent, confident recommendation. Asked for a black T-shirt, the bot suggests a coffee mug with total conviction. No error in the log, no exception, no red line. Just a bad answer that looks like a good one.
How to check
First look at the collection itself:
curl -s http://127.0.0.1:6333/collections/your_collection | python3 -m json.toolThat gives you size and distance. Compare the number to what your query model produces. In my case: 1536 in the collection, a local model doing the querying – that does not add up.
The second test says even more and takes ten seconds: take the exact title of a product you know is indexed and send it as the search query. If that product is not the top hit, something is fundamentally broken. Literal agreement is the easiest case there is – if that fails, harder questions are not worth thinking about.
How to prevent it
Three things that have proven worthwhile:
- One place for the model name. Not entered separately in two workflows, but as an environment variable both read. Anything maintained in two places eventually drifts apart.
- The model in the collection name.
products_openai_small_1536looks clumsy, but one glance tells you what produced the contents. - On a model change, re-index – do not top up. Old and new vectors in the same collection are worse than none: some hits are right, some are not, and you look for the fault in the wrong place.
The point behind all of this: vector databases have no schema in the classic sense. They cannot check whether your numbers came from the same model as the ones already stored – they only see that it is the right quantity of numbers. That check is yours to build.