Best Embedding Models for RAG in 2026 (and How to Pick the Right One)
Written By:
Matt Tanner
Published on
Your support assistant has the answer in its documents, but search never returns the passage that contains it. The model cannot use that missing passage to support its answer. In a vector-based RAG system, the embedding model helps determine which passages search finds. Choosing one by leaderboard rank alone can mean paying for more storage without improving retrieval on your documents. The useful test is whether it finds better evidence for your users' questions.
This guide compares ten embedding models worth shortlisting in 2026, with dated specifications linked to the primary documentation in the table below. Then it walks through testing candidates with your own chunking, reranking, and eval set to choose the model that fits your system.
What an Embedding Model Does in a RAG Pipeline
An embedding model turns text (and, for some models, images or PDFs) into fixed-length vector representations so that passages with similar meaning land close together in vector space. In a RAG pipeline, it runs twice: once offline on every chunk you index, and once per user query. Retrieval then becomes a nearest-neighbor search over a vector database, usually by cosine similarity.
The models in this guide work as bi-encoders, embedding the query and each document separately. That's why you can precompute millions of document vectors ahead of time and only embed the query at search time. The cost is that the query never "sees" the document during encoding, so you can add a cross-encoder reranker, which reads query and candidate together but only for the top results.
Queries have to go through the same embedding model, at the same dimension, that built your index. Vectors from different models generally live in different spaces, so comparing them produces numbers that look like similarity scores and mean nothing. The exceptions are model families built to share one space, covered below. Our RAG pipeline guide covers the rest of the flow.
How to Compare Embedding Models for RAG
The MTEB leaderboard, short for Massive Text Embedding Benchmark, is a good place to build a shortlist of text embedding models, but RAG puts constraints on an embedder that a single average can't capture.
Criterion | Why it matters for RAG | What to check |
|---|---|---|
Retrieval quality | It's the closest benchmark proxy for "did the right chunk come back" | The Retrieval column on MTEB, not the overall mean |
Max input tokens (context window) | Text past the limit is truncated, often silently | Your largest chunk vs. the limit (256 word pieces for all-MiniLM-L6-v2, 128K for Cohere Embed 5) |
Embedding dimension | Storage and search cost scale with it | Default size, and whether the model supports Matryoshka (MRL) truncation |
Multilingual and cross-lingual | Queries in one language may need documents in another | Languages on the model card; MTEB's per-language leaderboards |
Modality | Charts, screenshots and scanned PDFs carry answers too | Whether images and PDFs share the text vector space |
Cost and latency | Indexing is a batch job, but the query embedding sits on the hot path | Price per 1M tokens and rate limits, or GPU cost if you self-host |
License and data terms | "Downloadable" doesn't always mean "free for commercial use" | The license on the model card; data-use terms on API free tiers |
Embedding dimension affects storage because every chunk needs a vector. At float32 precision, each dimension takes four bytes. Ten million chunks with 3,072 dimensions therefore use about 123 GB for raw vectors; at 1,024 dimensions, they use about 41 GB. The search index adds overhead on top of that.
Matryoshka Representation Learning trains embeddings to remain useful at smaller sizes, letting you test that storage tradeoff without choosing a different model. OpenAI reports that a text-embedding-3-large vector shortened to 256 dimensions still beat a full 1,536-dimension ada-002 vector on MTEB. That is a benchmark result, so verify a shorter vector on your own queries before rebuilding the index.
Dense embeddings also have a blind spot. Lexical search matches the literal words in a query, which matters when users type part numbers, SKUs, or error codes. If yours do, read our guide to hybrid search for RAG before you spend more on a bigger embedder.
The Best Embedding Models for RAG in 2026
The shortlist covers hosted APIs and self-hosted options with different language, modality, and licensing constraints. This stuff changes relatively fast, but the linked specifications below were checked in October 2026.
Model | Provider | Open or API | Max input tokens | Dimensions | Multilingual | Multimodal | Best for |
|---|---|---|---|---|---|---|---|
API | 8,192 | 128 to 3,072 (default 3,072) | Yes, 100+ | Text, image, video, audio, PDF | Mixed-media corpora | ||
text-embedding-3-large / -small | OpenAI | API | 8,192 | 3,072 / 1,536 (shortenable) | Yes (large) | No | Cheap, well-known text baseline |
Voyage AI | API (voyage-4-nano open, Apache-2.0) | 32,000 | 1,024 (256 to 2,048) | Yes | No | Long chunks, tiered cost | |
Cohere | API | 128K | 256 to 2,048 (default 2,048) | Yes, 100+ | Text, image, mixed PDFs | Long, visually rich documents | |
Jina AI | Open weights (Qwen Research License) | 32,768 | 2,048 (to 128), plus multi-vector | Yes, 30+ | Text, image, visual docs | Visually rich docs in research or licensed use | |
Qwen | Open (Apache-2.0) | 32K | 1,024 / 2,560 / 4,096 (MRL) | Yes, 100+ | No | Self-hosted multilingual and code | |
BAAI | Open (MIT) | 8,192 | 1,024, plus sparse and multi-vector | Yes, 100+ | No | Dense plus sparse from one model | |
Nomic AI | Open (Apache-2.0) | 2,048 (8,192 configurable) | 768 (MRL) | No (English) | No | Small English self-hosted RAG | |
intfloat (E5) | Open (MIT) | 512 | 1,024 | Yes, 100 | No | Short-chunk multilingual baseline | |
sentence-transformers | Open (Apache-2.0) | 256 word pieces | 384 | No (English) | No | Fast baseline and prototypes |
Gemini Embedding 2
Consider Gemini Embedding 2 when the evidence includes images, recordings, or scanned pages as well as text. Google's gemini-embedding-2 maps text, images, video, audio, and PDFs into one embedding space across 100+ languages. It accepts up to 8,192 tokens, and the paid tier costs $0.20 per million text tokens.
For text retrieval, task instructions go in the input rather than the old task_type parameter. Queries use a format such as task: question answering | query: ...; documents use title: ... | text: .... Follow the provider's input guidance when integrating it.
Its vectors are incompatible with gemini-embedding-001. An upgrade therefore requires re-embedding the corpus, not just changing the model used for queries.
OpenAI text-embedding-3-large and text-embedding-3-small
Small returns 1,536 dimensions for $0.02 per million tokens, while large returns 3,072 for $0.13, and both accept 8,192 tokens of text. The dimensions parameter shortens either vector at request time. Use small as a lower-priced baseline before testing whether large improves retrieval on your corpus. As of October 11, 2026, neither has a complete result set on MTEB's English or Multilingual v2 leaderboards, so you can't rank them against open models there.
Voyage 4 Family
Voyage's text lineup includes voyage-4-large, voyage-4, and voyage-4-lite. Each accepts 32,000 tokens and defaults to 1,024 dimensions, with 256, 512, and 2,048 also available. They cost $0.12, $0.06, and $0.02 per million tokens; each account receives the first 200 million tokens free for these models.
The 4-series models produce compatible embeddings, including the Apache-2.0 open-weight voyage-4-nano. That lets you test indexing with a larger model and embedding queries with a cheaper one. Measure the pairing rather than assuming it matches the larger model on both sides. Int8 and binary outputs offer additional storage reductions.
Cohere Embed 5
Cohere's Embed 5 is a candidate for long or mixed-format documents. Released in Pro and Fast versions on September 30, 2026, both accept text, images, and documents such as PDFs, cover 100+ languages, and support up to 128K input tokens. Output sizes range from 256 to 2,048 dimensions. Text pricing is $0.12 per million tokens for Pro and $0.08 for Fast.
Pro and Fast share an embedding space, so you can index with Pro and query with Fast. The older embed-v4.0 is still listed.
A large input window lets you embed a long section, but does not establish that one vector will retrieve every detail in it effectively. Test chunk size as well as the model. Embed 5 had no MTEB results in the dated snapshot used here.
Jina Embeddings v4
jina-embeddings-v4 is built on Qwen2.5-VL-3B-Instruct and embeds text, images, and visually rich documents in 30+ languages. It takes 32,768 tokens, outputs 2,048-dimensional vectors you can truncate to 128, and adds a multi-vector mode plus task adapters for retrieval, text matching, and code.
Read the license before you self-host it. The weights fall under the Qwen Research License, which covers research and evaluation use and requires a separate license for commercial use. Jina's newer v5 text models are CC BY-NC 4.0, again with commercial use by agreement.
Qwen3-Embedding
Qwen3-Embedding offers three open sizes: 0.6B, 4B, and 8B parameters, with 1,024, 2,560, and 4,096 dimensions respectively. All have a 32K context, Matryoshka support, coverage of 100+ natural and programming languages, and an Apache-2.0 license.
In the October 11, 2026 MTEB(Multilingual, v2) snapshot, the 8B and 4B models ranked #4 and #6. Query instructions matter too: the Qwen team reports a 1% to 5% gain from adding a one-line task instruction, while retrieval documents need no instruction.
Start by testing the 0.6B or 4B model if memory and storage are constraints. The 8B model needs more weight memory and stores larger vectors at its full dimension; its benchmark rank alone does not establish that those costs pay off for your corpus.
BGE-M3
BGE-M3 from BAAI is the open pick when you want dense and sparse retrieval from one encoder. It produces 1,024-dimensional dense vectors, sparse token weights, and multi-vector output, handles 100+ languages and 8,192 tokens, and ships under MIT. BAAI recommends pairing it with hybrid retrieval plus reranking. It has no complete score on MTEB(Multilingual, v2) as of this writing, so your own eval set carries extra weight.
nomic-embed-text-v1.5
A 137M-parameter English model under Apache-2.0 with 768-dimension Matryoshka vectors, a 2,048-token default context, and a documented path to 8,192. Start every input with a task prefix: search_document: for chunks and search_query: for questions. Preserve those prefixes when swapping models in an existing pipeline.
E5 (multilingual-e5-large-instruct)
multilingual-e5-large-instruct has 560M parameters, 1,024-dimension vectors, an MIT license, and 100 languages, though its card warns that low-resource languages may degrade. In the 2025 MMTEB paper, it was the best-performing publicly available model in the authors' evaluation. The limit is length. Anything past 512 tokens is truncated, so it only suits short chunks, and queries need an Instruct: ... Query: ... prefix.
all-MiniLM-L6-v2
This 23M-parameter, English-only model outputs 384-dimensional vectors under Apache-2.0. The real constraint is length: input beyond 256 word pieces is truncated, so longer chunks are embedded from their opening text only. Use it as a baseline. If a 3,072-dimension API model can't clearly beat it on your eval set, the bigger model may not be worth its storage and token cost, so look at your retrieval errors, chunking, and labels before you upgrade.
Best Embedding Model by Use Case
English Text RAG
For general-purpose English RAG, start with text-embedding-3-small as the cheap baseline, then test text-embedding-3-large, voyage-4, and Qwen3-Embedding-4B against it. Keep nomic-embed-text-v1.5 or all-MiniLM-L6-v2 in the same run as an open-weight floor.
Multilingual and Cross-Lingual RAG
Qwen3-Embedding, BGE-M3, Gemini Embedding 2, and Cohere Embed 5 all list 100+ languages. For cross-lingual retrieval, such as German queries against English documents, write eval queries in every language you serve, and check MTEB's per-language leaderboards.
Code and Technical Docs
voyage-code-4 is built for code retrieval, Qwen3-Embedding counts programming languages among its 100+, and Gemini Embedding 2 documents a task: code retrieval query prefix. Domain-specific docs full of identifiers often need lexical matching too, which no embedder fixes alone.
Multimodal (Text Plus Images, PDFs)
Gemini Embedding 2, Cohere Embed 5, and Jina Embeddings v4 handle images and PDFs. Check the limits: Gemini takes one PDF of up to 6 pages per request, and Jina v4's research license means commercial use needs a separate agreement.
Low-Cost or Self-Hosted
Compare the memory and inference requirements of open models such as Qwen3-Embedding-0.6B, BGE-M3, nomic-embed-text-v1.5, and all-MiniLM-L6-v2 against your available hardware. Our fully self-hosted FiQA tutorial uses all-MiniLM-L6-v2 with FAISS exact search on a GPU, and our guide to self-hosted LLMs covers serving the rest of the stack.
Why Leaderboards Are Not Enough: Test Embedding Models on Your Own Data
A leaderboard tells you how a model did on someone else's data, and three things make that a weak predictor for your RAG system.
Read the benchmark details behind the rank. MTEB covers public datasets across classification, clustering, similarity, and retrieval. Its leaderboard uses a Borda count, which combines task rankings, so overall rank rewards performance beyond retrieval alone.
Check training overlap and missing results as well. The zero-shot percentage reports the share of benchmark tasks a model was not trained on. In the October 2026 snapshot, the #2 and #3 models on MTEB(eng, v2) both reported 48%, meaning they were trained on about half the tasks. Several API models, including text-embedding-3-large, Voyage 4, and Cohere embed-v4.0, had incomplete results. Those gaps prevent a complete comparison; they don't show that a model retrieves poorly.
The embedder interacts with chunking, retrieval depth, and reranking. The embedding model and its dimensions are part of a connected set of retrieval and context engineering choices. In our two-application RAG case study, we pinned the embedder to text-embedding-3-small and changed only chunking, search type, k, and reranking. The retrieval composite score still rose from 0.439 to 0.627 on a benchmark of 76 synthesized questions. Swap in different embedding models without re-tuning those knobs, and you're comparing setups rather than models.
An embedder change can improve one outcome much more than another. In the same study's second application, the selected configuration used text-embedding-3-large. Changing only the embedder around that configuration moved faithfulness by about 0.02, but answer correctness by more than 0.2.
Faithfulness measures support from retrieved evidence, while correctness compares the answer with a reference. Watching only faithfulness would have missed most of the measured difference. This was a local comparison on a 28-question set within the 432-config experiment, not an average effect across the whole grid.
In practice, that's a five-step loop: shortlist from MTEB plus your constraints, build an eval set, sweep embedder x chunking x reranker, compare retrieval and answer metrics, and then pick and confirm on held-out queries.
Build a Labeled Eval Set From Real Queries
Pull a few hundred real-world questions from search logs, support tickets, or user interviews, and label which source documents answer each one. Label at the document or passage level rather than by chunk ID, because chunk IDs change every time you re-chunk and you'll want to compare chunk sizes too.
Hold back a slice of queries you never tune on. Pick the winner on the development set, then confirm it on the held-out set, and repeat runs when generation is stochastic.
Pick Retrieval Metrics First
Start by checking the passages each embedder retrieves. Recall@k measures how much of the labeled relevant material appears in the top k candidates. MRR focuses on the first relevant result's position; NDCG rewards placing useful results high in the list and can account for degrees of relevance.
Then check the generated answers. Correctness compares them with reference answers, while faithfulness checks their support in the retrieved context. Our guide to RAG evaluation metrics explains the choices. In RapidFire, you supply the Python metric functions; our tutorial notebooks include precision, recall, F1, NDCG, and MRR examples.
Sweep Embedders Together With Chunking and Reranking
Three embedders, two chunk sizes, and two reranker depths give you 12 configs, and every embedder and chunk-size pair needs its own index.
RapidFire, our open-source experiment execution framework, treats the embedding model as one more knob. Its LangChain RAG spec takes an embedding_cfg for HuggingFaceEmbeddings or OpenAIEmbeddings, and almost every knob can take a List() of options. Here's a sketch adapted from our SciFact tutorial, whose notebook compares text-embedding-3-small and text-embedding-3-large.
Before you use the sketch
The sketch targets rapidfireai 0.16.1 with the eval dependencies that rapidfireai init installs.
It is a configuration fragment, not a standalone script: first define a loader for your corpus and read OPENAI_API_KEY from your environment into the Python variable of that name.
The OpenAI embedding options send corpus text to OpenAI and incur API charges; the Hugging Face model runs locally after a first-use download.
Configuration sketch
What this produces
Constructing rag does not run or print an experiment. The generator config, metric functions, and run_evals() call follow the tutorial notebook.
Each embedder and splitter combination needs its own index. Those document-side settings stay fixed once the run starts, while query-side settings such as search and reranking can still be changed.
The 12 configurations use the same eval set so you can compare their running metrics. Only supported aggregation types receive projected estimates. Confidence intervals require each metric's aggregation type and value range to be configured. You can stop weak configurations, then clone promising ones to explore new query-side settings.
Note the trap built into this grid: all-MiniLM-L6-v2 truncates at 256 word pieces, so the 512-token chunks test truncation as much as the model. Our guides to semantic chunking and reranking in RAG cover those knobs in depth.
Weigh Quality Against Cost, Latency, and Re-Indexing Effort
The top config on your eval set isn't automatically the one to ship. Weigh query embedding latency, which you pay on every request, plus storage for the chosen dimension and the cost of re-embedding whenever the model changes. A winner reflects the objective you set, as the metric tradeoffs in “Why Leaderboards Are Not Enough” illustrate; it isn't a universal optimum. Write down your objective before you look at the results.
Open-Source vs. API Embedding Models
| Open-weight models | API models |
|---|---|---|
Data leaves your network | No | Yes. Read data-use terms (Gemini's free tier data is used to improve Google's products) |
Cost model | Hardware you pay for whether it's busy or idle | Per token, $0.02 to $0.20 per 1M text tokens for the API models above |
Versioning | You pin the weights | The provider ships new versions; moving from Gemini 001 to 2 means a full re-embed |
Fine-tuning | Possible with your data and GPUs | Limited to what the provider exposes |
License | Varies: Apache-2.0 and MIT, but also research and non-commercial terms | Terms of service plus rate limits |
API embedding models let you send text for embedding without hosting the model, as in our SciFact tutorial. Self-hosted open models keep inference inside your network and let you control the weights. Compare their hardware and operating costs with API charges at your expected query volume.
Common Embedding Mistakes in RAG
Embedding queries and documents with different models, or with the same model at different dimensions. Voyage 4 and Cohere Embed 5 are documented exceptions, and only within their own families.
Switching models without re-indexing. Even within one provider, Gemini's two embedding models produce incompatible vectors.
Dropping the input prefixes a model expects. nomic needs
search_query:andsearch_document:prefixes, E5 and Qwen3 want a task instruction on queries only, Gemini Embedding 2 has separate query and document formats, and Voyage uses aninput_typeflag.Chunks longer than the model's input limit. all-MiniLM-L6-v2 truncates at 256 word pieces and multilingual-e5-large-instruct at 512 tokens, without raising an error.
Storing full-size vectors without testing a smaller dimension, or truncating by hand without re-normalizing. Normalize manually shortened OpenAI embeddings, while Gemini Embedding 2 re-normalizes truncated sizes for you.
Picking by leaderboard rank alone instead of testing on your own eval set.
Conclusion
Shortlist three or four embedding models from the table that fit your languages, modalities, license, and budget. Then, using your own eval set, run with your real chunking and reranker settings, and pick the right embedding model for your RAG system.
To try the workflow, open our RAG tutorial on Colab. To run it locally, follow the RAG install guide (Python 3.12+, pip install rapidfireai, then rapidfireai init) and add a second embedder to the embedding_cfg list. If you'd rather talk through your retrieval and context engineering setup first, you can book a technical call with our team.
FAQ
Which Embedding Model Is Best for RAG?
There isn't one best embedding model for RAG, because the right embedding model depends on your documents, languages, and license constraints. Start with text-embedding-3-small as a baseline, one larger API model (text-embedding-3-large, voyage-4, or Gemini Embedding 2), and one open model (Qwen3-Embedding or BGE-M3). Run all three on your own eval set and let the metrics decide.
Are Open-Source Embedding Models Good Enough for Production RAG?
Often, yes. As of October 11, 2026, the top model on MTEB(Multilingual, v2) is Microsoft's MIT-licensed harrier-oss-v1-27b, and open-weight Qwen3-Embedding-8B sits at #4, just ahead of gemini-embedding-001. Open versus closed source matters less than operations: who hosts the model, how you version it, and whether the license allows commercial use.
Can I Use Different Embedding Models for Queries and Documents?
Generally, no. Each model has its own vector space, so you can't compare query vectors from one model with document vectors from another. The exception is a family designed for it, such as Voyage's 4 series or Cohere's Embed 5 Pro and Fast, which their providers document as sharing one space. Different query and document prefixes for the same model are fine, and often required.
Does Embedding Dimension Size Matter for RAG?
It matters most for cost. Storage and search work scale with dimension, so 3,072-dimension vectors take three times the space of 1,024. For Gemini embeddings, lower dimensions can score comparably to higher ones on MTEB. Compare a truncated size against the full size on your eval set before paying for bigger vectors.