Built a retrieval-augmented generation pipeline over a custom document corpus, selecting chunk size and overlap from observed document structure rather than defaults and documenting the reasoning behind each value.
Compared fixed-size, semantic, and recursive chunking strategies on retrieval quality for multi-section queries, tracing every retrieved chunk back to its source file and producing function.
Ran a before-and-after evaluation assigning verdicts to generated answers, diagnosing failures as retrieval versus generation problems, and validating one targeted fix against the same query set.
Documented remaining failure modes and their causes instead of reporting only the improved runs.
The bullets above are reported in the full source resume ↗. Repository findings and any differences are identified separately below.
01 / SYSTEM ARCHITECTURE
How it fits together.
Ingestion preserves each Reddit comment as a source-labeled chunk and splits long comments into overlapping sentence-aware passages. ChromaDB attaches the same MiniLM embedding function to both indexing and querying, using cosine search over a local persistent collection. Generation discards distant chunks, refuses without an LLM call when none remain, and builds the source list from metadata rather than model-written citations.
Alternative considered: Larger hosted or multilingual embedding models
Short English comments suit inexpensive local embeddings; the documented trade-off is weaker recall on domain slang, long passages and other languages.
Five-question evaluation harness; sources and cosine distances are printed beside the generated answer for human review.
The Stat 134 question refused despite relevant material being in the corpus: top-four retrieval favored unrelated courses and a contentless comment. Human retrieval-quality and answer-accuracy columns are still unfilled, so no scored before/after gain is claimed.