Receipt AI Expense Tracker

Overview

A multimodal receipt-to-expense workflow with schema validation, human review, local storage, and provider guardrails.

Problem

Receipt extraction must handle multilingual text, Buddhist Era dates, inconsistent layouts, and unsafe model output without silently saving incorrect financial records.

Solution

Built a Next.js application with capability-aware provider routing, strict schema validation, Buddhist Era normalization, editable review, and IndexedDB persistence.

Technical Decisions

• Local persistence avoids a required account or cloud database.
• Mock extraction exercises the same validation and review contract as provider output.

Outcome

Users explicitly review extracted data before it enters the local ledger, while deterministic mock extraction keeps the product flow testable without provider credentials.

Key Features

• Provider capability checks prevent images from reaching text-only routes.
• Strict schemas validate extracted receipt records.
• Explicit human confirmation is required before IndexedDB persistence.

Architecture

1. 01

   Receipt upload

2. 02

   Provider router

3. 03

   Structured extraction

4. 04

   Schema validation

5. 05

   Human review

6. 06

   IndexedDB

7. 07

   Local analytics

Tech Stack

• Next.js
• TypeScript
• Zod
• IndexedDB
• AI provider routing
• Privacy guardrails

Verification

• Synthetic extraction evaluation
• Schema-validation tests
• Deterministic mock workflow

Security & Privacy

• Receipt data stays local after review.
• Provider keys remain server-side and AI output is never saved automatically.

Limitations

• Browser-local storage is device-specific and is not a synchronized backup.
• Extraction accuracy depends on image quality and provider capability.

Future Improvements

• Add encrypted export and import.
• Expose field-level confidence and duplicate detection.