My 10-Year Fight with a White Whale

Back in 2016, I built a price tracker for Belgium and the Netherlands called Prijzenvolger. My ultimate inspiration was CamelCamelCamel (the Amazon price tracker). Those guys make millions, and as a junior dev back then, that was exactly my goal. Just make millions. Easy, right?

Things actually started out great. I managed to get featured in HLN and even did a segment on VTM News explaining how it worked. User accounts and affiliate revenue spiked.

But as an inexperienced developer, things quickly got out of hand:

I was dealing with gigabytes of product data and price updates on a daily basis.
I relied solely on daily affiliate data feeds.
The fatal flaw: A store updates its data feed in the morning, changes the price at noon, and by the afternoon, users are looking at wrong prices on my site shouting, “This thing is broken!”
On top of that you’re never going to guess where it was hosted. Damn right it was on AWS! RDS for the DB, EC2 managed myself for the hosting.

Between the creeping server costs, the pressure of my day job, and the brutal reality of waiting for SEO to kick in, I killed the project after about a year.

The Evolution: Python, Pandas, and Postgres

This project became my white whale. I started it, stopped it, started it, and stopped it again over the years.

A few years later, armed with more experience, I rebuilt it in Python using Pandas for memory-optimized data loading and TimescaleDB for historical data. It worked way better, but the infrastructure costs still managed to creep up every time I launched a new version

Eventually, I had a realization that comes with data engineering maturity: if you have a nice, beefy server, a well-optimized Postgres instance can handle this data perfectly fine without costing a fortune.

The New Strategy: Real-Time Precision

Prijzenvolger and Waltrack have been online and running smoothly for a while now, but it’s time to step on the gas.

We solved the “stale price” problem with a new architectural requirement. Now, when a user lands on a product page, the backend instantly hits the store’s API in the background to grab the absolute latest price.

💡 How it works nowThe data feed might say a product was €10 yesterday, but the second you click on my page, our system fetches the live API data. If it dropped to €8, we tell you immediately: “Hey, it just dropped. Buy it now.” Real-time precision, no more angry users.

The Empire Roadmap

To speed things up, I’m no longer flying solo. I’ve hired some great freelancers to help with development, and we are making massive progress. The goal is to launch dedicated, localized price trackers for major stores across the globe.

Here is the official hit list for the empire: