← back to index
01WORK

BeyondBot.

machine learning intern · oct 2025 — present · pune

Corpus
1,210 pp · IN industrial / mfg / env law
Pipeline stages
5 (ocr → router → retrieval → agent → api)
Eval queries
315
RAGAS Δ
+0.26 hybrid vs naive

Problem

The platform team needed an on-site question-answering layer over a 1,210-page corpus of Indian industrial, manufacturing and environmental law. Three constraints made naive RAG insufficient: (1) statute structure: legal text mixes definitions, sections, tables, and amendments, which a flat semantic index conflates; (2) citation precision: legal answers must cite the exact section, not paraphrase; (3) change tracking: amendments live in separate documents that supersede older sections.

Pipeline

mistral ocr → custom regex router (definitions / sections / tables / amendments) → LightRAG (KG + vector) with SQLite tabular lookups and a JSON amendment index → Agno agent on Gemini 3 Flash with 4 tools → FastAPI with SSE streaming → chat UI with PDF.js viewer and citation badges.

The regex router is the load-bearing piece. It runs before retrieval and tags each chunk with a structural type, which lets the retrieval layer route — definitions hit a small, exact-match path; tables hit SQLite; amendments hit the JSON index; everything else goes to LightRAG.

Evaluation

Built a RAGAS-style evaluation harness benchmarking five LightRAG retrieval modes across 315 queries on a 10k-excerpt FinDER subset. Metrics: faithfulness, answer relevancy, context precision, context recall, plus retrieval latency, token count, and deployment cost.

ModeRAGASΔ vs naive
naive0.51
local0.62+0.11
global0.66+0.15
graph0.71+0.20
hybrid0.77+0.26

Hybrid retrieval reached 0.77 RAGAS against 0.51 for the naive baseline — a roughly 49% lift on the same harness.

OpenCV visual chunker

Wrote a custom OpenCV visual chunker that rasterizes PDF pages and uses blob detection to extract bounding boxes for text blobs, recovering page hierarchy for downstream chunking. Pages with a clear title/body/table layout get clean structural tags; complex multi-column statute pages sometimes need a fallback to plain layout heuristics.

Handover

The PoC was handed to platform engineers for product integration. Open work includes promoting the regex router to a learned classifier and adding an evaluator-in-the-loop loop for the amendment index.