Team report: four walk-forward cutoff windows across 19 ports; 532 predictions.
Tier accuracy: V1 57.0% → V2 70.1%; V3 fell to 51.3%. This is a repository-reported team benchmark, not an individual contribution metric.
Source ↗LangGraph / BigQuery / FastAPI / GCP / Plotly2025 – 2026
DockWise AI
Maritime agents, grounded in the data.
- AIS vessel records
- 54.7M
- Los Angeles ingestion period
- 2019–2024
- seasonal significance
- p < 0.01
What I built.
How I evaluated it.
- Built a LangGraph multi-agent system over 54.7M AIS vessel position records in BigQuery with tool-use orchestration, letting analysts query multi-year maritime traffic in natural language instead of hand-written SQL.
- Implemented grounding checks and fallback handling that validate vessel, port, and timestamp references against the underlying tables before an answer is returned, reducing hallucinated entities in agent output.
- Wrote evaluation prompts and a scoring rubric covering factual grounding, response tone, and workflow alignment, used to compare agent versions before each release.
- Engineered a multi-year AIS ingestion pipeline (2019–2024) for the Port of Los Angeles, loading daily compressed .csv.zst files into BigQuery via GCP compute and storage with automated cleaning and deduplication.
- Computed vessel dwell time distributions, congestion indices, and seasonal arrival patterns, and applied Kruskal-Wallis hypothesis testing to identify statistically significant seasonal congestion shifts (p < 0.01).
- Served the system through a FastAPI backend deployed on GCP with interactive Plotly dashboards for exploratory review of agent findings.
The bullets above are reported in the full source resume ↗. Repository findings and any differences are identified separately below.
How it fits together.
The current team repository separates a FastAPI service, a live AIS microservice, and a React dashboard. A shared RiskState flows through weather, congestion, and vessel agents in sequence before a risk orchestrator combines their outputs and requests a Groq explanation. PortWatch history feeds the forecasting layer; PostgreSQL stores the application data.
FROM THE DEPENDENCY FILESChoices with a reason.
- 01
Port-specific residual scale
Alternative considered: Prophet prediction-interval width as the z-score denominator
Measure held-out residual variation so narrow model intervals do not force most ports into extreme risk tiers.
Source ↗ - 02
Keep the V2 ensemble
Alternative considered: Add adaptive thresholds, ARIMA and day-of-week adjustments in V3
The more complex version scored worse in the documented walk-forward comparison; retain the simpler seasonal and recent-trend model.
Source ↗ - 03
A minimum standard-deviation floor
Alternative considered: Let a near-zero baseline magnify one additional vessel
The floor prevents a single vessel at a low-volume port from creating an extreme congestion score.
Source ↗
The result and the method.
Team report: train to a cutoff, forecast the next seven days, and compare with actuals at 19 ports.
Tier accuracy 57.1% → 67.7%; congestion-score MAE 16.6 → 13.0; portcall MAE 3.1 → 2.8. The single-window result is distinct from the walk-forward result.
Source ↗What still needs work.
Excerpts from the repository’s own notes. These limits belong beside the results.
Read the code.
PythonTest count not established
No tracked Python test_ functions found; evaluation scripts and notebooks are described separately. Source ↗
- venv2/backend/agents.py ↗
Start with RiskState, the explicit graph edges, and the risk-score combination.
- venv2/backend/backtest.py ↗
Inspect the backtest implementation alongside the evaluation protocol.
- REPORT.md ↗
Compare V1, V2 and V3 on the same documented walk-forward protocol.
THE EVALUATION PRACTICE
- 01
Establish a baseline.
- 02
Make a targeted change.
- 03
Re-measure on the same set.
- 04
Document what still fails.