How Embeddings Work
An embedding model converts text into a fixed-size array of floating-point numbers — typically 768 to 3,072 dimensions. "How do I fix a bug?" and "debugging techniques" produce vectors that are mathematically close to each other (high cosine similarity) because they share semantic meaning, even though they use completely different words.
This is what makes semantic search possible. Traditional keyword search matches exact words. Embedding-based search matches meaning. A search for "fix authentication error" can find documents about "resolve login issues" or "troubleshoot OAuth failures" without keyword overlap.
Common Embedding Models
OpenAI's text-embedding-3-large (3,072 dimensions) is the most popular commercial option. For open-source, BAAI/bge-large-en-v1.5 and Cohere's embed-english-v3.0 are competitive. Smaller models like text-embedding-3-small (1,536 dimensions) trade some quality for lower cost and faster inference.
Embeddings in Practice
The typical pipeline: chunk your documents → embed each chunk → store vectors in a database (Pinecone, Weaviate, pgvector, Qdrant) → at query time, embed the query → find nearest neighbors → pass retrieved chunks to the LLM. This is the RAG architecture, and embeddings are the foundation.
Key gotcha: the embedding model used for documents must match the one used for queries. You cannot embed documents with OpenAI and query with Cohere — the vector spaces are incompatible.