Topic
7 articles found
Introduction Data engineering is the practice of designing and building systems for collecting, storing, transforming, and managing data so it can be safely used for reporting, analytics, machine learning, and making business decisions. Think of it as the behind-the-scenes work that makes apps, webs
The Problem We Were Actually Solving We assumed Postgres could keep up with event ordering. We were wrong. The real performance killer wasnt the database—it was our event configuration model. Veltrix tracked three things per event: user_id, event_type, and timestamp. We used a GIN index on (user_id,
This is not proof. It is early, messy evidence from my own workflow: three failures, one small comparison, and one schema bug I missed. I'd spent a week arguing, in public, that AI memory should be built on discipline before infrastructure: preserve corrections, preserve unresolved questions, decide
Speed isn't the bottleneck for most, chaotic storage is.
I wiped my entire Obsidian vault, expecting chaos. Instead, the reset showed me why I should have done it sooner, without giving up the part I needed.
Shucking sounds ridiculous until you actually do it.
If anything, it's a terrible idea to use USB-powered drives as conventional storage pools
