Skip to project detailsSkip to navigation
← SELECTED WORK
Python / Flask / REST / pytestSummer 2026

Provenance Guard

A verdict should come with a reason.

View repository ↗AI Engineer resume ↗
pytest regression tests
32

What I built.
How I evaluated it.

  • Built a Flask REST service that scores submitted text for AI provenance by combining a model signal and a style signal under an explicit weighted rule, returning a per-signal breakdown instead of a single opaque verdict.
  • Documented what each signal measures and, critically, what it cannot see, then used those blind spots to construct an adversarial attack set that deliberately evaded the scorer.
  • Hardened the service against the attack set and added request rate limiting, re-running the full suite before and after to quantify which evasions the fix actually closed.
  • Designed an appeal workflow allowing flagged submissions to be re-reviewed with recorded justification, keeping human judgment in the loop for contested decisions.
  • Wrote a 32-test pytest regression suite covering scoring edge cases, malformed payloads, and appeal state transitions, so rubric changes could not silently alter prior outcomes.
  • Structured scoring as configurable weights so evaluation criteria could be re-tuned without rewriting the service.

The bullets above are reported in the full source resume ↗. Repository findings and any differences are identified separately below.

01 / SYSTEM ARCHITECTURE

How it fits together.

The Flask submit route validates text, generates a content ID, evaluates a Groq signal and a deterministic stylometric signal, then returns the combined score with both explanations. Separate JSON stores hold mutable content decisions and append-only audit events. An appeal changes the stored status to under_review and records the creator’s reasoning for human review.

README.md ↗planning.md ↗
FROM THE DEPENDENCY FILES
FlaskFlask-LimiterGroqpython-dotenvpytest
requirements.txt ↗
02 / ENGINEERING DECISIONS

Choices with a reason.

  1. 01

    A 60 / 40 model-to-style weighted score

    Alternative considered: Let one detector determine the verdict

    Semantic cues receive slightly more weight while the deterministic style signal supplies a transparent second opinion.

    Source ↗
  2. 02

    A wide uncertain interval from 0.25 to 0.75

    Alternative considered: Force borderline submissions into an AI or human label

    The specification treats a false accusation against a human creator as the more harmful error.

    Source ↗
  3. 03

    A separate Flask-Limiter extension singleton

    Alternative considered: Initialize rate limiting directly in routes

    Binding the extension to each app keeps setup clear and makes limiter-state resets reliable between tests.

    Source ↗
03 / EVALUATION RECORD

The result and the method.

Two documented local examples without a Groq key; the model signal is fixed at its neutral 0.5 fallback.

Combined scores were 0.66 for a uniform paragraph and 0.40 for a casual paragraph, a 0.26 gap from stylometry alone. Both remained uncertain; this is not a detector-accuracy benchmark.

Source ↗

A documented burst of twelve local submission requests.

The first ten requests returned 200; requests eleven and twelve returned 429. This verifies the burst-limit behavior in that recorded run.

Source ↗
04 / DOCUMENTED LIMITS

What still needs work.

Excerpts from the repository’s own notes. These limits belong beside the results.

05 / EXPLORE THE IMPLEMENTATION

Read the code.

Python32 test definitions

Counted from tracked Python test functions. This audit did not execute the project suite. Source ↗

  1. provenance_guard/routes.py ↗

    Trace submission validation, signal outputs, audit writes and the appeal transition.

  2. provenance_guard/detection/scoring.py ↗

    Inspect the weighting rule, thresholds, input clamping and uncertain outcome.

  3. tests/test_contracts.py ↗

    Read API-shape, appeal-state and rate-limit regression checks.

Open repository ↗
THE EVALUATION PRACTICE
  1. 01

    Establish a baseline.

  2. 02

    Make a targeted change.

  3. 03

    Re-measure on the same set.

  4. 04

    Document what still fails.

NEXT PROJECT

FitFindr ↗