RAG Over Internal Documents: Build or Buy?

A support lead at a 200-person B2B distributor in Gurgaon has 4,000 PDFs spread across Google Drive, a Zoho Desk export and a shared mailbox. When someone asks how a refund is processed on an export order, the answer takes twenty minutes of searching or a call to the one person who knows. That is the problem retrieval-augmented generation (RAG) is meant to solve, and it is why so many teams find themselves weighing a build against a managed tool within a month of starting.

What does a self-hosted RAG stack cost?

A working self-hosted RAG system typically costs ₹15,000–₹50,000 a month in infrastructure and four to eight weeks of engineering to reach a version your team actually trusts. The stack itself is not exotic: an ingestion job that pulls from Drive, Slack and your CRM; a chunker; an embedding model; a vector store; a retriever with a reranker; and an LLM at the end. If you already run Postgres, pgvector removes one moving part, because it is the same database your application architecture probably already depends on. The line items that move the monthly number most are the LLM API bill, driven by query volume and how much context you stuff in, and the reranker. Storage is rarely the problem.

When should you buy instead?

Buy when your corpus is under roughly 5,000 documents, your access rules are simple, and you need something live in under three weeks. Managed options range from Azure AI Search paired with an LLM, to Cloudflare AutoRAG, to per-seat products that connect to Drive and Notion out of the box; open-source options like Onyx are worth a look too. The trade-off is usually per-seat pricing. At ₹800–₹1,500 per user per month, a 300-person company is looking at ₹29 lakh to ₹54 lakh a year before anyone has asked a single useful question. That is a fine deal if adoption is broad, and a painful one if only 40 people ever open it.

When does building win?

Building wins when permissions, data shape or query volume make an off-the-shelf tool a poor fit. Three cases come up repeatedly in Indian B2B setups. First, permission-aware retrieval: a sales rep must not pull an HR letter or a director's salary sheet, and filtering has to happen at retrieval time, not inside the prompt. Second, data that no connector handles well, such as a legacy CRM, an ERP with an odd schema, or WhatsApp enquiry logs. Third, volume: past a few thousand queries a day, per-query managed pricing stops looking cheap. Data residency and DPDP Act obligations also push some teams toward keeping documents and embeddings on infrastructure they control, a factor that shapes most AI system builds.

A worked example: 40,000 documents, 300 people

For a 300-person distributor with 40,000 documents and about 900 queries a day, building came out cheaper over two years, but only because they already ran Postgres and had one engineer with spare capacity. The modelled comparison:

• Buy: 300 seats at ₹1,200 per month works out to ₹43 lakh a year, or ₹86 lakh over two years.
• Build: infrastructure at roughly ₹28,000 a month is ₹3.4 lakh a year; six weeks of one engineer at a loaded ₹2.5 lakh; about ₹3 lakh a year for maintenance and re-indexing. The two-year total lands near ₹15 lakh.

Now change one variable. If only 40 people query daily, buying 40 seats costs ₹5.8 lakh a year and the build case collapses, because your engineer's six weeks is worth more than the difference. Seat count and query volume decide this, not document count.

What do the demos leave out?

Almost every RAG demo hides the three costs that actually decide the project: chunking quality, evaluation, and refusal behaviour. Chunking is where most systems quietly fail, because a 40-page GST circular split every 500 characters retrieves fragments that answer nothing. You need an evaluation set of 50–100 real questions with known correct answers, run before and after every change, or you are tuning blind. The model also has to say the answer is not in the documents instead of inventing a plausible policy, since a confidently wrong answer on TDS rates costs more trust than no answer. Budget for re-indexing as well: documents change, and stale embeddings are worse than none.

Where to start

Start with 50 real questions and 200 documents before you compare vendors or write a line of code. Two weeks of manual retrieval testing tells you whether the corpus is clean enough to be useful at all, and in our experience roughly a third of internal document sets contain duplicate, outdated or contradictory versions of the same policy. If top-five retrieval finds the right passage for 80% of those questions, building is viable; if it does not, no framework choice will rescue the project. Maysan Labs builds these systems for Indian B2B teams, and is equally happy to argue the other side of this comparison when buying fits you better.