norag-rag project

This workspace contains 2 Python applications to compare NoRAG (full-context prompting) versus RAG (retrieval-augmented generation) using Google Gemini via the OpenAI SDK.

Applications

1. ask_norag_google.py

• NoRAG with Google Gemini via OpenAI SDK (gemini-3.5-flash-lite).
• Loads docs/CrimeAndPunishment.txt, injects a "needle" passage, and runs two direct large-context questions:
  • a baseline question about Marmeladov (present in the original book)
  • a needle question about Denis (the injected passage)

2. ask_rag_google.py

• RAG with Google Gemini for generation via OpenAI SDK.
• Uses local embeddings (all-MiniLM-L6-v2) + local ChromaDB (chroma_db/) to retrieve relevant chunks before asking the model.
• Default retrieval uses n_results=5 for the question: What does this say about Denis?

Project Structure

• docs/CrimeAndPunishment.txt: main source text used by the scripts.
• docs/CrimeAndPunishment-shorter.txt: shortened version of the main source text for quicker testing and rapid experimentation in Gemini online at https://gemini.google.com/app (See note below)
• chroma_db/: persistent local vector database (used by RAG scripts).
• requirements.txt: Python dependencies.

Prerequisites

• Python 3.10+ (recommended)
• API key:
  • GOOGLE_API_KEY from Google AI Studio

Setup

1. Create and activate a virtual environment:

python3 -m venv .venv
source .venv/bin/activate

2. Install dependencies:

pip install -r requirements.txt

3. Configure environment variables:

cp .env.example .env

Then edit .env and set your real keys.

Run Each Application

With your virtual environment activated:

1) NoRAG + Google

python ask_norag_google.py

2) RAG + Google

python ask_rag_google.py

Notes

• Both applications use the OpenAI SDK to interact with Google Gemini via Google's OpenAI-compatible endpoint.

• The RAG app splits the text into fixed-size chunks with an overlap (CHUNK_SIZE, OVERLAP_PERCENT and MIN_PARAGRAPH_LENGTH at the top of the script), indexes them into chroma_db/ and reuses them across runs.

• There is one index per document (DOCUMENT_FILE) and chunking configuration. When no index exists for the current document and settings, the script asks before building one (this takes a little while).

• Debug prompt printing can be enabled by passing debug=True in the NoRAG app.

• RAG app prints retrieved context for verification.

• Streaming output is enabled by default in both applications (controlled by STREAM_ON variable).

• Using the shortened version of the text (docs/CrimeAndPunishment-shorter.txt)

  • Go online at https://gemini.google.com/app
  • At the prompt, start writing "Here is a story: ",
  • Then paste the content of docs/CrimeAndPunishment-shorter.txt into the Gemini chat app and then, at the end of the prompt, ask "What can you tell me about Denis Amselemovitch?"
  • Observe the response from Gemini.

Free books from Project Gutenberg

• https://www.gutenberg.org/

Crime and Punishment by Fyodor Dostoevsky

• https://www.gutenberg.org/ebooks/2554

Huckleberry Finn by Mark Twain

• https://www.gutenberg.org/ebooks/76