MCON 368 · Biography RAG

The Life, from the Sources

Upload pages about one person. The app retrieves snippets for twenty questions and writes a story from those snippets only. Every sentence carries the ID of the chunk it came from; anything the sources don't cover is marked not in the sources.

1 The corpus

No store loaded yet.

Drop files here or

.txt · .md · .html · .pdf · .docx — any number

Test corpus: Benjamin Franklin

Six public-domain biographies from Project Gutenberg, about 1,450 pages.

    2 Ask the twenty questions

      Settings

      Your key stays in this browser and is sent only to this site's /api/generate, which passes it to Google.

        3 The story

        Click any chunk ID to read the snippet it came from. Sentences the checker flagged are underlined in red.

        Run the queries to generate the story — or .

        4 Review of the logs

        Nothing yet.

        5 Logs — every query, its chunk IDs, and the exact prompt

        How it works

        1. Extract. Files are read in your browser (PDF via pdf.js, Word via mammoth). Project Gutenberg boilerplate is stripped.
        2. Chunk. Text is split on paragraph and sentence boundaries into ~1,000-character chunks with ~200 characters of overlap. Each chunk gets an ID like D2-C0153 (document 2, chunk 153).
        3. Embed & store. Each chunk is embedded in your browser with all-MiniLM-L6-v2 (ONNX, int8), so there is no per-page API cost. The store (chunks + vectors) is saved in your browser's IndexedDB and can be downloaded.
        4. Retrieve. Each question plus two extra phrasings is searched with embeddings (mean-centered cosine with a hubness correction) and BM25 keywords, fused with reciprocal-rank fusion, then diversified with MMR → top 8 chunks.
        5. Gate. If nothing retrieved is similar enough, the answer is “not in the sources” and the LLM is not called.
        6. Answer. The LLM gets only the 8 snippets and must return JSON: sentences, each with chunk IDs, or not_in_sources.
        7. Check. Code checks every sentence: it must have an ID, the ID must exist and have been retrieved for that question, and every number and name in it must appear in the cited chunk. Failing sentences are dropped before the story.
        8. Story. The LLM weaves the checked sentences into a story, citing IDs, and skips “not in the sources” topics. The story is checked the same way, then a second LLM pass audits each sentence against its snippets.

        Source: GitHub repo. No API keys are in the code.