# I Built a Local-First HSA Receipt Tracker with Flask, Google Drive, and AI

A tiny local python app that turns HSA receipts into structured records in Google Drive and Sheets in about five seconds.

Last year my family switched to a **High Deductible Health Plan (HDHP)** with a **Health Savings Account (HSA)**.

HSAs are unusual in the U.S. tax system because they’re **triple tax advantaged**:

*   contributions are **tax-deductible**
    
*   investments grow **tax-free**
    
*   withdrawals for qualified medical expenses are **tax-free**
    

The Bogleheads wiki has a great explanation if you're curious:

https://www.bogleheads.org/wiki/Health\_savings\_account

Because of those advantages, some people treat their HSA as a **long-term investment account** instead of reimbursing medical expenses right away. Almost everyone I tell about this has a 🤯 moment because they’ve never heard of it. So, if you're one of those lucky people to learn about it today, you're welcome!

> Your wallet loves this one weird trick!

The strategy looks like this:

1.  Contribute to the HSA
    
2.  Invest the money
    
3.  Pay medical expenses out-of-pocket
    
4.  **Save the receipts**
    
5.  Reimburse yourself years (or decades) later, or don't
    
6.  Profit
    

There’s **no deadline to reimburse yourself** as long as:

*   the expense happened after the HSA was opened
    
*   you kept documentation
    

That last part is the problem.

Here’s what the result looks like after uploading a receipt:

![HSA receipt tracker sheet](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jsez9q048vu4we210x0w.png align="center")

Every receipt automatically becomes a structured record. Each row links back to the original receipt stored in Google Drive.

| Date | Vendor | Amount | Receipt |
| --- | --- | --- | --- |
| 2026-02-14 | Quest Diagnostics | $87.43 | Drive Link |
| 2026-02-03 | Walgreens | $14.29 | Drive Link |
| 2026-01-19 | Dentist | $120.00 | Drive Link |

* * *

## The Receipt Problem

If you're saving receipts for potential reimbursement later, you need to keep track of:

*   provider
    
*   date
    
*   amount
    
*   proof of payment
    
*   the original receipt
    

Manually this usually turns into:

*   PDFs scattered across downloads folders
    
*   random email attachments
    
*   a spreadsheet you forget to update
    

I was doing this manually with Google Drive and a spreadsheet, and it wasn't *too* hard, but... I wanted something simpler.

Drop in a receipt → have everything filed automatically.

* * *

## The Idea

I built a tiny **local-first web app** that:

1.  accepts a medical receipt PDF
    
2.  extracts useful fields automatically
    
3.  stores the document in Google Drive
    
4.  logs the expense in Google Sheets
    

The whole flow takes about **5 seconds of actual work**.

* * *

## Demo

Here’s the entire workflow in real time (~5 seconds of actual work - the rest is me moving slowly for the video's sake):

{% embed https://vimeo.com/1171880979?share=copy&fl=sv&fe=ci %}

Upload a receipt → parsed → stored → logged.

* * *

## The Stack

The app is intentionally small:

*   **Flask** – local web app
    
*   **pdfplumber** – extract text from receipts
    
*   **OpenAI (optional)** – prefill receipt fields
    
*   **Google Drive API** – store receipts
    
*   **Google Sheets API** – expense log
    

The workflow looks like this:

`receipt → upload → text extraction → AI field parsing → Drive storage → Sheets entry`

* * *

## What Happens When You Upload a Receipt

When I submit a receipt, the app:

1.  saves the uploaded PDF locally
    
2.  computes a **SHA-256 hash** to detect duplicates
    
3.  extracts text using `pdfplumber`
    
4.  optionally asks the model to extract fields like:
    

*   vendor
    
*   service date
    
*   amount
    
*   payment date
    
*   payment method
    
*   notes
    

5.  creates (or reuses) a Google Drive folder for the month:
    
    `2026-03/`
    
6.  uploads the receipt there
    
7.  appends a row to Google Sheets with a link to the file
    

Example spreadsheet row:

| Date | Vendor | Amount | Receipt |
| --- | --- | --- | --- |
| 2026-02-14 | Quest Diagnostics | $87.43 | Drive Link |

Now every expense has:

*   structured data
    
*   the original document
    
*   a searchable log
    

* * *

## One Small Feature That Ended Up Being Useful

Duplicate detection.

Each uploaded file is hashed and stored in a local `receipt_hashes.json`.

If I accidentally upload the same receipt twice, the app can catch it before cluttering the spreadsheet or Drive folder.

Tiny detail, but it prevents a lot of mess.

* * *

## Why Local-First?

This could easily be a SaaS product.

But for personal admin tools, local-first is usually better:

*   no extra account to maintain
    
*   easier to hack on
    
*   my files stay in my own Google account
    
*   no database or infrastructure to run
    

I also set it up as a **macOS** `launchd` **service**, so it’s just always available on my laptop.

The goal wasn’t to build some big product. It was to remove just enough friction that I’d actually keep my receipts organized.

Now the workflow is:

Download receipt ↓ Upload ↓ Done

Which is about the amount of effort I’m realistically willing to spend on personal finance paperwork.

* * *

## Repo

If you're interested:

https://github.com/pjhoberman/hsa-tracker

* * *

If you’re using the **"save receipts and reimburse later"** HSA strategy and built something similar, I’d love to hear about it.

Or if you have a better way to automate this - please tell me before I add OCR and accidentally build an entire product.