Build a RAG System from Scratch: A Practical Tutorial

RAG (Retrieval-Augmented Generation) is the most popular architecture for building LLM applications. This hands-on tutorial builds a complete RAG system from scratch.

What We're Building

A document Q&A system that can answer questions about your PDFs, markdown files, or any text documents.

Prerequisites

• Python 3.10+
• OpenAI API key (or any LLM API)
• Basic understanding of Python

Step 1: Setup

mkdir rag-system && cd rag-system
python -m venv venv
source venv/bin/activate
pip install openai chromadb tiktoken pypdf

Step 2: Document Ingestion

import os
from typing import List
from pypdf import PdfReader

def load_documents(folder_path: str) -> List[str]:
    documents = []
    for filename in os.listdir(folder_path):
        if filename.endswith('.pdf'):
            reader = PdfReader(os.path.join(folder_path, filename))
            text = ""
            for page in reader.pages:
                text += page.extract_text()
            documents.append(text)
        elif filename.endswith('.txt') or filename.endswith('.md'):
            with open(os.path.join(folder_path, filename), 'r') as f:
                documents.append(f.read())
    return documents

Step 3: Chunking

def chunk_text(text: str, chunk_size=500, overlap=50) -> List[str]:
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunks.append(text[start:end])
        start = end - overlap
    return chunks

Step 4: Create Embeddings and Store

import chromadb
from openai import OpenAI

client = OpenAI()
chroma_client = chromadb.Client()
collection = chroma_client.create_collection(name="documents")

def index_documents(chunks: List[str]):
    for i, chunk in enumerate(chunks):
        response = client.embeddings.create(
            model="text-embedding-3-small",
            input=chunk
        )
        embedding = response.data[0].embedding
        collection.add(
            embeddings=[embedding],
            documents=[chunk],
            ids=[str(i)]
        )

Step 5: Retrieval and Generation

def query_rag(question: str) -> str:
    # Embed the question
    response = client.embeddings.create(
        model="text-embedding-3-small",
        input=question
    )
    query_embedding = response.data[0].embedding
    
    # Retrieve relevant chunks
    results = collection.query(
        query_embeddings=[query_embedding],
        n_results=3
    )
    
    context = "

".join(results['documents'][0])
    
    # Generate answer
    completion = client.chat.completions.create(
        model="gpt-4",
        messages=[
            {"role": "system", "content": "Answer the question based on the context provided."},
            {"role": "user", "content": f"Context:
{context}

Question: {question}"}
        ]
    )
    return completion.choices[0].message.content

Step 6: Deploy as API

from flask import Flask, request, jsonify

app = Flask(__name__)

@app.route('/query', methods=['POST'])
def query():
    data = request.json
    answer = query_rag(data['question'])
    return jsonify({'answer': answer})

if __name__ == '__main__':
    app.run(port=5000)

Next Steps

• Add re-ranking for better retrieval
• Implement hybrid search (keyword + semantic)
• Add source citations to answers
• Try different chunking strategies

Our "Build with LLMs" programme covers advanced RAG techniques and production deployment.