/ The Scope
Replaced a completely manual process with a fully automated pipeline — ingesting messy RTA emails from CAMS and Karvy directly into structured database records within minutes.
RTA stands for Registrar and Transfer Agent. In the mutual fund industry, CAMS and Karvy are the two major RTAs that process all SIP transactions, redemptions, and brokerage payouts. They send confirmation data via email — in inconsistent, semi-structured formats that change without notice.
Before this pipeline, someone had to manually read every email, extract the transaction data, and enter it into the system. For a platform handling multiple MFDs this was unsustainable. I built a FastAPI service that receives these emails automatically, parses the data regardless of format variations, and pushes clean records to the database — all without human intervention.
/ The Problem
CAMS and Karvy send transaction data in emails that are inconsistent, poorly formatted, and change format without warning.
Format Inconsistency
CAMS and Karvy use different email formats — and both change their formats periodically without any API versioning or changelog. A parser built for one format silently breaks when the format shifts, producing wrong data or no data.
Volume at Scale
A platform serving multiple MFDs receives hundreds of RTA emails per day. Manual data entry doesn't scale beyond a handful of MFDs. Any delay in processing means the portfolio data shown to investors is stale — a serious trust issue in fintech.
Data Integrity
SIP transactions and brokerage amounts are financial data. A parsing error that enters the wrong amount, wrong scheme code, or wrong investor into the ledger causes reconciliation failures that are painful to debug and fix days later.
/ Architecture
Email in. Clean structured data out. No human in the loop.
Karvy Email
Ingestion
FastAPI
Parser Layer
Python — Multi-format Parser
+ Dedup Check
Write to Ledger
PostgreSQL / MongoDB
Reconciled ✅
FastAPI Async
FastAPI was chosen over Node.js specifically because Python's text processing libraries — regex, email parsing, pandas — are far superior for this use case. Extracts raw body and passes to the parser layer.
Python Engine
Detects RTA sender from headers and applies the correct parsing strategy. Fallback logic catches parsing failures and routes them for manual review rather than silently producing wrong data.
Schema + Dedup
Validated against a strict schema before writing. Duplicate detection checks the transaction reference against existing records to prevent double-counting forwarded emails.
DB Records
Structured transaction records written to the ledger. What previously took hours of manual entry now completes within minutes, keeping investor views current.
/ Hard Problems Solved
Parsing unstructured financial email data that changes format without warning is harder than it sounds.
01CAMS and Karvy use completely different formats
There is no standard format for RTA mailbacks. CAMS sends HTML tables. Karvy sends a mix of plain text and inline HTML. Column names differ. Date formats differ. Amount formats differ. I had to write two separate parsers and a detector that routes each incoming email.
02Formats change silently in production
RTAs update formats without any changelog. A working parser breaks silently when the format shifts. The solution was explicit failure detection — if the parser extracts zero records from a non-empty email, it flags it for manual review rather than writing nothing.
03The same transaction arrives multiple times
RTA emails get forwarded and re-sent. Without deduplication, the same SIP transaction gets written to the ledger multiple times. Every parsed record is checked against a unique transaction reference before writing. Seen before — skip. New — write.
04Financial data has zero tolerance for silent errors
A wrong decimal place in a brokerage amount is a financial error. I added strict validation — amounts must be positive numbers within realistic ranges, scheme codes must match. Anything failing validation is quarantined for human review.
Next Case Study
ONDC Connector →