Skip to project detailsSkip to navigation
← SELECTED WORK
BM25 / FAISS / SBERT / BERTopic / Streamlit2025

Scholarly Topic Navigator

Research discovery with an explanation.

View repository ↗AI Engineer resume ↗
research papers
22,522
tokens processed
2.7M
faster matched-accuracy inference
~50×
MRR · resume-reported
0.83

What I built.
How I evaluated it.

  • Processed 22,522 research papers from ArXiv and Semantic Scholar, building an NLP preprocessing pipeline with NLTK and spaCy covering tokenization, stopword removal, stemming, and language detection across 2.7M tokens.
  • Engineered a hybrid retrieval system combining BM25 lexical search with FAISS dense search over Word2Vec, SBERT, and SciBERT embeddings, achieving an MRR of 0.83 on a held-out query set.
  • Benchmarked LDA against BERTopic for topic modeling and selected on measured coherence (0.448 versus 0.420) rather than on qualitative inspection alone.
  • Applied knowledge distillation from a BART-MNLI zero-shot teacher into a lightweight student classifier, matching teacher accuracy at 67% across 12 categories while running approximately 50x faster at inference.
  • Added LIME explanations and structured logging so every classification could be inspected, traced, and reused as a regression test case.
  • Shipped the full pipeline as a Streamlit dashboard, exposing retrieval, topic assignment, and explanation in a single interface.

The bullets above are reported in the full source resume ↗. Repository findings and any differences are identified separately below.

01 / SYSTEM ARCHITECTURE

How it fits together.

Three notebooks collect and clean the paper corpus, build embeddings and retrieval indexes, then integrate the models into a Streamlit interface. BM25 scores are fused with cosine similarity from an IndexFlatIP FAISS index over normalized SBERT vectors. A BART-MNLI teacher supplies pseudo-labels to a frozen-SBERT plus logistic-regression student; topic models, summaries and LIME explanations share the resulting corpus.

README.md ↗
FROM THE DEPENDENCY FILES
NumPySciPypandasscikit-learnPyTorchHugging Face Transformerssentence-transformersBM25FAISSGensimBERTopicUMAPHDBSCANNLTKspaCyLIMEMatplotlibSeabornPlotlyStreamlit
requirements.txt ↗
02 / ENGINEERING DECISIONS

Choices with a reason.

  1. 01

    BM25-weighted score fusion at α = 0.6

    Alternative considered: The initial semantic-heavy weighting hypothesis

    A parameter search favored a larger lexical contribution; the published comparison still shows BM25 alone outperforming the hybrid on its synthetic relevance labels.

    Source ↗
  2. 02

    SBERT as the primary embedding

    Alternative considered: SciBERT / SPECTER embeddings for every downstream task

    The report describes very high SciBERT similarity as a clustering collapse; SBERT balances separation and semantic capture.

    Source ↗
  3. 03

    Teacher-to-student distillation

    Alternative considered: Run the large zero-shot teacher for every classification

    A frozen SBERT encoder and logistic-regression classifier reuse teacher pseudo-labels for substantially faster inference.

    Source ↗
03 / EVALUATION RECORD

The result and the method.

Published retrieval comparison using category / classification labels as synthetic relevance judgments.

MRR: BM25 0.832; FAISS 0.301; hybrid at α = 0.6, 0.675. Recall@10: 0.157, 0.060 and 0.147 respectively. This proxy is not a human relevance assessment.

Source ↗

Published classification results over the 12-class taxonomy.

Accuracy 67%, macro-F1 0.20, weighted-F1 0.62, with approximately 50× student-versus-teacher inference speedup. The macro/weighted gap exposes the class imbalance.

Source ↗

Topic-model comparison using c_v coherence.

TF-IDF + K-Means 0.28; LDA 0.448; BERTopic 0.420. More topics did not yield the highest measured coherence.

Source ↗
04 / DOCUMENTED LIMITS

What still needs work.

Excerpts from the repository’s own notes. These limits belong beside the results.

  • minority categories get ~0 F1; majority classes dominate.
    Source ↗
  • category-based relevance is a weak proxy for human judgment
    Source ↗
  • embeddings/index/corpus must be regenerated.
    Source ↗
05 / EXPLORE THE IMPLEMENTATION

Read the code.

PythonTest count not established

No tracked Python test_ functions found; evaluation scripts and notebooks are described separately. Source ↗

  1. notebooks/02_topic_modeling_classification_retrieval.ipynb ↗

    Follow index construction, the distilled classifier, topic models and evaluation.

  2. app/app.py ↗

    See how search, classification and explanations meet in the interface.

  3. reports/Final_Group_Report.md ↗

    Read the methodology and limitations behind the published tables.

Open repository ↗
THE EVALUATION PRACTICE
  1. 01

    Establish a baseline.

  2. 02

    Make a targeted change.

  3. 03

    Re-measure on the same set.

  4. 04

    Document what still fails.

NEXT PROJECT

Provenance Guard ↗