01 / What is an RTA Mailback?
RTA stands for Registrar and Transfer Agent. In India's mutual fund industry, CAMS and Karvy are the two dominant RTAs — they process every SIP transaction, redemption, dividend payout, and brokerage calculation across all fund houses.
When a transaction completes, the RTA sends a confirmation email to the registered MFD — the Mutual Fund Distributor. This email contains the transaction details: investor name, folio number, scheme code, amount, NAV, units allotted, and brokerage earned. In the industry this is called a "mailback."
The problem: these emails are the primary data source for portfolio reconciliation on our MFD platform. Without parsing them correctly, the portfolio values investors see are wrong or stale.
02 / The Manual Process (Before)
Before building this pipeline, the process was entirely manual. Someone received the RTA email, read the transaction details, and entered them into the system by hand.
For a single MFD handling a few transactions per day, this works. For a platform serving multiple MFDs with hundreds of transactions daily — it breaks immediately.
Manual entry means delayed portfolio updates, human transcription errors, and a process that doesn't scale past 2-3 MFDs without hiring people whose entire job is copying emails into a database.
The goal was zero human intervention — email arrives, pipeline runs, database is updated, investor sees accurate portfolio within 5 minutes.
03 / Why FastAPI and Not Node.js
Our primary backend is Node.js. The natural choice would have been to build the pipeline there too. I chose Python and FastAPI instead — deliberately — for three reasons.
Python's text processing ecosystem is superior
The email parsing, regex extraction, and tabular data processing libraries in Python — email, re, pandas, BeautifulSoup — are far more mature than their Node.js equivalents. RTA emails contain HTML tables, inline styles, and inconsistent whitespace. Python handles this cleanly.
FastAPI is genuinely fast for I/O-heavy workloads
FastAPI is built on Starlette and uses Python's async/await natively. For a pipeline that reads emails, runs parsing logic, and writes to a database — all I/O operations — it performs extremely well without blocking.
Isolation is a feature
Keeping the parsing pipeline as a separate FastAPI microservice means a bug in the parser never affects the main Node.js API. The two services communicate over HTTP. If parsing fails, the main platform keeps running.
04 / The Pipeline Architecture
The pipeline has four stages: ingestion, detection, parsing, and validation + write.
Stage 1 — Email Body Extraction
RTA emails arrive in multipart MIME format — both HTML and plain text versions of the same content. The HTML version is more structured but harder to parse reliably due to inline styles. The plain text version is easier to parse but sometimes truncated.
The extractor pulls both versions and passes them to the detector.
Stage 2 — RTA Detection
CAMS and Karvy use different email formats — different sender domains, different subject line patterns, different table structures. The detector reads the sender address and subject line to decide which parser to route the email to.
Stage 3 — Format-Specific Parsers
Each RTA has its own parser. CAMS sends structured HTML tables with relatively consistent column headers. Karvy mixes plain text sections with HTML tables and uses different field names for the same data.
Stage 4 — Validation and Deduplication
Every parsed record goes through schema validation before touching the database. Amounts must be positive. Folio numbers must be non-empty. Scheme codes must match known codes in our master list. Dates must be parseable.
Then deduplication — the transaction reference number is checked against existing records. RTA emails get forwarded and CC'd; the same transaction arriving twice must not be written twice.
05 / The Hardest Part: Format Changes
CAMS and Karvy update their email formats without any notification. A column gets renamed. A table structure changes. A new section appears before the transaction table, shifting all the row indices.
The naive approach — hardcoded column indices — breaks silently. The parser runs, extracts nothing, writes nothing, and the system shows no error. Portfolio data goes stale without anyone knowing.
The solution: explicit zero-record detection. If the parser returns an empty list from a non-empty email, that is flagged as a parsing failure — not treated as "no transactions today."
This means every format change gets caught immediately and routed to a review queue — where someone can inspect the email, update the parser if needed, and reprocess. No silent data loss.
06 / Results
After deploying the pipeline:
Manual data entry time: ~0 hours per day
Previously took hours per MFD per day. Now fully automated for all MFDs on the platform simultaneously.
Email to DB time: under 5 minutes
From RTA email arriving to portfolio data updated in the database. Previously same-day or next-day depending on when someone manually processed it.
Zero silent failures
Every parsing anomaly is flagged and routed to review. The pipeline either succeeds loudly or fails loudly — never silently produces wrong data.
07 / What I Would Do Differently
If I built this again, I would add a format fingerprinting system — a hash of the email structure (table count, column count, column names) stored alongside each parsed record. When a format change happens, the fingerprint mismatch triggers an immediate alert before even attempting to parse — rather than detecting the problem after an empty parse result.
I would also build a format versioning system — CAMS_v1, CAMS_v2 — so that when a format changes, the old parser keeps working for older emails in the queue while the new parser handles fresh ones. Currently a format update requires redeploying the parser.