Curriculum Vitae · data engineering
Tahmid Jawad Shafi
Education
BSc CSE, AIUB — AIUB (Dhaka)
June 2027Projects
Bari-Linkfeatured
2026-11 – 2026-11Designed and shippedpipelinecompletedPython · Splink · DuckDB-Wasm · React · TypeScript · Vite · Pytest · Playwright
Deduplicates NGO beneficiary registries that share no identifier, using kinship constraints to cut costly false merges.
- Built a synthetic population engine and noise model, romanised-name normalisation, and a Splink Fellegi-Sunter model with kinship cannot-link constraints (Pipeline K).
- Added multi-channel household inference on top of the resolved person clusters.
- Committed hypotheses before their results and confirmed the champion pipeline on five fresh holdout seeds.
- Shipped a static DuckDB-Wasm demo of the results.
- Operational cost at a 5:1 false-merge cost ratio, Pipeline K vs the Splink reference (holdout seeds 5–9, lower is better): 812.2 reference 888.2; -8.56%; better on 5/5 seeds, one-sided sign test p = 0.031 (source: README.md:48–50 @ c518135)
- Cluster-level precision, Pipeline K (holdout seeds 5–9): 0.9656 ± 0.0106; Splink reference 0.9457 ± 0.0127 (source: README.md:47–48 @ c518135)
Curbstonefeatured
2026-11 – 2026-11Designed and shippedqualitycompletedPython · SQL · DuckDB-Wasm · Parquet · Pytest
Ranks survey enumerators by risk of fabricating interviews that range and skip checks let through.
- Designed peer-comparison features (interview duration, daily throughput, near-duplicate answers, answer dispersion, GPS spread, age heaping, non-response) and a frozen risk-scoring engine.
- Built a fabrication simulator with five fraud modes plus one mode held out from development, so detection performance is measured instead of asserted.
- Ran the same 12 SQL rule checks in Python and DuckDB-Wasm with a cross-engine parity check, which caught a timezone bug.
- Precision@4 gain over the rule-count baseline, 95% CI (72 configurations × 20 rounds): +0.131 interval +0.119 to +0.142 (source: README.md:19–21 @ 363c7b4)
- Average-precision gain over the rule-count baseline, 95% CI: +0.195 interval +0.178 to +0.211 (source: README.md:19, 21 @ 363c7b4)
- Held-out fraud mode (partial_section) detected, risk ranking vs rule checks: 11/15 rules: 1/15 (source: README.md:37 @ 363c7b4)
Nittoofeatured
2026-09 – 2026-11Designed and shippedanalyticscompletedTypeScript · React · Supabase · PostgreSQL · Python · Pydantic · DuckDB · dbt · Vitest · Playwright
Turns everyday purchases into container lifespans, true daily costs and calibrated run-out windows.
- Built the React + Supabase app: container-level tracking, run-out forecasting and restock reminders, with Row-Level Security tenant isolation verified in CI.
- Built the Forecast Lab: a Pydantic + JSON Schema export contract, a seeded household simulator, idempotent DuckDB ingestion with a rejects quarantine table, and a dbt warehouse.
- Backtested forecasts with rolling-origin evaluation and exported calibrated interval quantiles and parity fixtures back into the app.
- Overall holdout forecast error (MAE), size-normalised rate forecast shipped in the app vs the app’s original average-duration baseline, synthetic data: 4.53 days baseline 17.42 days (source: README.md:89–93 @ b0fb0fb · rate_mean (rule_only) vs baseline (none))
- Forecast error (MAE) after pack-size changes, size-normalised rate forecast shipped in the app vs the app’s original average-duration baseline, synthetic data: 4.88 days baseline 21.59 days (source: README.md:89–93 @ b0fb0fb · rate_mean (rule_only) vs baseline (none))
- P10–P90 run-out window coverage on holdout, size-normalised rate forecast shipped in the app, synthetic data: 83.7% nominal 80%; mean width 12.8 days (source: README.md:95 @ b0fb0fb · rate_mean (rule_only))
ProtoMacrofeatured
2026-09 – 2026-11Designed and shippedappcompletedJavaScript · Python · DuckDB · SQL · Pandas · Vite · Chart.js · Vitest · Pytest · Playwright
Grounds a nutrition tracker's 567 foods in Bangladeshi and USDA reference tables instead of unsourced hand-entered values.
- Built a tested DuckDB pipeline that extracts, SHA-256-verifies, normalises and validates FCTB 2013 and USDA FoodData Central data into a catalog with provenance on every row.
- Audited the legacy hand-entered values against exact primary-source matches and migrated saved v2 IDs to v3 without altering logged history.
- Shipped an offline-first PWA (Vanilla JS + Workbox) with Vitest, pytest and Playwright checks in CI.
- Foods in the sourced catalog, each with row-level provenance: 567 381 FCTB 2013 · 69 USDA FDC SR Legacy · 117 labelled estimates (source: README.md:11, 63 @ e635153)
- Calorie error found in the legacy hand-entered data (n = 74): 2.5% / 34.9% median / 90th percentile, absolute (source: README.md:21 @ e635153)
DGHS CleanPulse
2026-10 – presentDesigned and builtpipelinein-progressPython · Pandas · Pandera · DuckDB · Parquet · FastAPI · Next.js · Pytest
Validates Bangladesh dengue and district climate data through a Medallion-architecture pipeline with no silent data drops.
- Built a Medallion pipeline whose Bronze layer stores immutable raw responses, each paired with a SHA-256 provenance sidecar.
- Built a Pandera validation firewall that records anomalies instead of dropping rows silently, tested against a synthetic fault-injection fixture.
- Built a FastAPI + DuckDB analytics API and a Next.js console.
Skills
- Languages
- Python, SQL
- Data & Storage
- DuckDB, DuckDB-Wasm, Parquet, PostgreSQL
- Transform
- dbt, Pandas, Splink
- Validation
- Pydantic, Pandera
- Testing
- Pytest
Languages
- Bangla
- English