Distributed, high-performance blob storage on SQLite

22 points by gowthamgts 12 hours ago on lobsters | 2 comments

sebastien | an hour ago

I think SQLite's positioning is that it does not compete with other DBs, but competes with the filesystem. I realised that first hand building an event sourcing system with event batches and compaction. A month worth of simulated data would take up 5-10x the size on the FS due to block size and metadata, while becoming increasingly slow to query and process. Swapping to SQLite as a backend solved the probem, all with a simple payload agnostic schema to organise blobs.

jc00ke | 7 minutes ago

SQLite almost has this built in with it's "archive" feature: https://www.sqlite.org/sqlar.html Not the distributed part of course, but still, kinda cool that SQLite has this feature. I've always thought it would be cool to wire up Litestream to build a simple CDN.