Folks, in your launch posts remember to describe what you're launching and what it's for. Ideally in the opening paragraph.
- Intro: Explains problem but not what is Neki or what it's for.
- Why Neki: Explains alternatives and why they suck but not what is Neki or what it's for.
- How does Neki work: Describes the technical components but not what is Neki or what it's for.
- What you get beyond sharding: Explains how to run/deploy but not what is Neki or what it's for.
- What is a platform preview + Try Neki today: 140 words on the definition of "preview" and links to get started... but not what is Neki or what it's for.
Edit: The landing page has 100x more useful information up front -> https://neki.dev/
Edit #2: They've since added a "What is Neki" section. Good reaction time.
The landing page which is linked at the start of the second sentence of this page…? I don’t completely disagree with you, but this is mostly hypertext working as intended, is it not?
Yeah this meta commentary on HN is very confusing for me. The structure of the article is literally
- short paragraph saying "we launched neki"
- short paragraph saying why neki was needed
- paragraphs about what neki is (with a header)
That seems like a very reasonable structure for a blog post like this one. Not being able to get to the third paragraph of a blog post seems like a "you" problem, not a blog problem.
Why "sadly"? The person writing a link should provide enough detail for readers to figure out whether they should follow the link. I rolled my eyes the first time I saw this entry on HN because it just said "Neki" and the (generic) target domain name. A two-word description got added, and that makes it interesting instead.
I scrolled through the landing page and even the "Get started" page and I have still no idea what is this. Is this a managed db? Something I can run myself? No idea.
The first thing I want to know when looking at a distributed system is how consistency is handled, ctrl+f for "consis" on both the blog post and neki.dev has zero hits. So I can only assume it will have eventual consistency with a considerable replica lag for real-time systems.
I see no mention of foreign keys, or any other constraints, across shards. If, as I suspect, they're not implemented, it would still be useful but at the level of Spanner 10 years ago.
Are you one of the developers ? If so, you need to add a specific page explaining in detail the guarantees that this gives.
For example: point 08 says "Assign different tables or workloads to different shard groups" and point 02 says "Split hot shards as workloads grow". How do those interact ? Can a single table be split across multiple shards ? If so don't you need 2pc to enforce primary key constraints ?
Yet. But it’s built by the guy who created Vitess and cofounded PlanetScale, so while it doesn’t have sharding implemented at the moment I think it would be foolish to discount it. Plus it’s open source.
The PlanetScale team spent years improving Vitess with lessons learned from many different high scale customer workloads. We continue to invest in and improve Vitess. It's a substantial and growing part of our business.
Vitess today is not the Vitess of 2021. About 70% of the current codebase was written in the last five years, based on lessons we learned running it at scale.
Neki is built for Postgres from the ground up, by the engineers who did that work.
On the Multigres comparison, I'll admit we get a bit salty, mostly because "Vitess for Postgres" invites the comparison. But to date there's no public evidence of a working sharding implementation in Multigres. Their sharding design doc landed this week, and it lists cross-shard query planning and resharding execution as explicitly out of scope.
Neki does sharding today, including online resharding that switches traffic without a maintenance window. It isn't open source, but you can spin up a cluster and try it yourself.
I love to see this but as a heavy Vitess/MySQL user I do fear the split focus from PlanetScale. Hoping to see continued improvements on the Vitess side as well.
Selfish doubts aside, congrats to Planetscale on the launch!
don't worry, we still give a lot of love to vitess. at the end of the day they are both databases. the beauty of doing both is we can take learnings from each product and apply it to the other.
I am on a 12 Muni right now that is plastered with a Neki ad, and I looked up neki.dev literally 10 mins ago to see what this product is about. Did they time the release of this bus ad with the launch date?
The largest database to ever run on Aurora MySQL runs on PlanetScale, and the largest database to ever run on Aurora limitless also runs on PlanetScale.
We went as far as we could on GCP before PlanetScale. Spanner is amazing but also amazingly expensive. And CloudSQL, also amazing as long as you don’t need write scaling. PlanetScale (Vitess+MySQL plus their branching / deployment and monitoring tools) is just not a combination offered on GCP. And while I didn’t use Aurora, from talking to and exploring AWS it doesn’t really have this combination either.
The top concern I've gotten from dev teams when proposing HA distributed postgres (e.g through RDS Aurora global) is that eventual consistency is not suitable for many workloads.
Does Neki solve for this, and if so how? My understanding of CAP theorem is that this basically requires some compromises around availability, but I'm curious as to what that looks like in practice here.
Assuming it works the same way as their MySQL product, Vitess:
The usual way to run it is that you partition your db based on something like a user, so that single user gets a consistent DB, but anything cross-shard may not be.
I know when I worked at Block, Cashapp was using Vitess and getting cross-shard DB writes down and functioning correctly was one of the major blockers to adoption. (though I just did tls management for vitess and didn't write any workloads on top of it, so my impression might be a bit off)
Same here, first thing I need to know when considering a distributed system is how consistency is handled. If it's eventual consistency, what is the replication lag like? If it's strong consistency, can their network handle that? What happens when a node goes down?
In case you were unaware, if someone's defense against being called an asshole is to say "well nothing I said was false" they're almost certainly an asshole.
Did anyone say you said anything untruthful? No, they just called you an asshole. You can both be an asshole and be right.
PS - Just some advice that I learned in sales 101 early in my career...focusing on shit talking the competition generally isn't a strategy that works well.
There's obviously more to the story than outsiders are aware, but personally it just leaves a bit of sour taste in my mouth when the tech and blog posts you guys put out are amazing.
> he just comes off as funny on twitter once you've met him.
Never heard of the guy before today, but his dickish, edge-lord aura (on X and in this thread) makes me not want to meet him. If I had to choose between buying from him, or some other random dude, I'll hear out the random dude first.
Oh no, how am I ever going to avoid giving my custom to someone who has allegedly cornered the market on - checks notes - a db proxy? Give me a break.
I can think of at least 4 other ways of scaling horizontally without switching DB engines. I'd have to be mightily constrained to have this particular product as the only option.
Sure, I don't doubt your ability to do so yourself. I just question your ability to trust some random unknown figure with your data if you have enough scale. That's quite different, actually.
Perhaps he's quite nice over zoom, but since very few of us are going to meet him in that context, the rest of us are left to judge him by what he posts. And as they say, you never get a second chance to make a first impression.
Based on what I've read so far, with his shit-talking the competition, he seems like... a bit of an asshole.
Sam is not an asshole. His twitter is great, he's definitely opinionated, and if you haven't had the luck of meeting him you might misread his tone as grumpy. It's not. It's sarcastic/dry humour and honest.
I love how the Planetscale CEO has been shitting on multigres[0], Supabase's equivalent to Neki, and gloating about how superior Neki is, and yet Neki is not even open source!
Sam, it feels like you're just playing games. It's hard to take anything you say or do seriously. Are you going to OSS this thing? Or have you decided to backpedal?
Congrats on the checks notes closed launch, I guess?
They both are characteristics of software that people consider when deciding whether to use it.
Open source being by far the more important of the two, since it means the user is more insulated by terrible corporate decisions thus making it more suitable for long term planning.
who said open source has no value? I worked at GitHub for 6 years i clearly love open source.
We maintain Vitess using our company funds. Why are you going after a small company for defending itself against companies like Amazon but not criticizing the reason we have to do this.
I would love to open source Neki and I have not said we won't.
Sam is always shitting on others. For years he has been screaming how bad PostgreSQL is compared to MySQL. Until getting silent about this topic for a few months and then launching PostgreSQL support. Seems he had to realize than the market for PostgreSQL is too big to ignore.
No it isn’t normal, its just this one person. I don’t personally find this language particularly appealing, especially for a serious discussion on technology. But hey, its a free country, they’re a startup and its one way of grabbing the scarcest of resources: attention.
Many of the engineers who built Neki also worked on Vitess for years. But it's a ground up new implementation designed to specifically target Postgres.
That is what the “regular” PlanetScale is all about. It uses the Vitess open source project to do write scaling through shading. It is very powerful and customizable and stable. It was built to run YouTube. Former happy user here.
How are cross-shard joins and transactions handled? Are there query patterns that would require application changes despite using the same drivers and ORM?
I love the offer which is running on the "PlanetScale Metal".
They claim to provide unlimited IOPS (I/O operations Per Second).
I would like to get some of these drives myself. Sounds like a technical miracle.
On the other hand claiming such miracles makes me wonder which other aspects in this are are actually a working miracle or rather a mythos/false marketing claim.
vaibhav2614 | 17 hours ago
The pitch is compelling. I wonder how many teams will be able to operate sharded database setups in production as a result of this.
gk1 | 17 hours ago
- Intro: Explains problem but not what is Neki or what it's for.
- Why Neki: Explains alternatives and why they suck but not what is Neki or what it's for.
- How does Neki work: Describes the technical components but not what is Neki or what it's for.
- What you get beyond sharding: Explains how to run/deploy but not what is Neki or what it's for.
- What is a platform preview + Try Neki today: 140 words on the definition of "preview" and links to get started... but not what is Neki or what it's for.
Edit: The landing page has 100x more useful information up front -> https://neki.dev/
Edit #2: They've since added a "What is Neki" section. Good reaction time.
dwb | 16 hours ago
fastball | 15 hours ago
- short paragraph saying "we launched neki"
- short paragraph saying why neki was needed
- paragraphs about what neki is (with a header)
That seems like a very reasonable structure for a blog post like this one. Not being able to get to the third paragraph of a blog post seems like a "you" problem, not a blog problem.
crooked-v | 6 hours ago
simonw | 9 hours ago
entrope | 3 hours ago
DrammBA | 16 hours ago
Edit: your marketing site actually has excellent marketing information.
Edit #2: They've since replaced the blog post with a link to the marketing page. Good reaction time.
ahoka | an hour ago
tjohnell | 17 hours ago
arrowleaf | 16 hours ago
khy | 17 hours ago
imjosh-dev | 17 hours ago
1121redblackgo | 15 hours ago
ahknight | 16 hours ago
rs_rs_rs_rs_rs | 17 hours ago
samlambert | 8 hours ago
ragall | 17 hours ago
aabhay | 17 hours ago
spongeboi | 17 hours ago
ragall | 14 hours ago
For example: point 08 says "Assign different tables or workloads to different shard groups" and point 02 says "Split hot shards as workloads grow". How do those interact ? Can a single table be split across multiple shards ? If so don't you need 2pc to enforce primary key constraints ?
The intro pages just read as AI slop.
levkk | 10 hours ago
[1]: https://docs.pgdog.dev/features/sharding/2pc/
behole | 17 hours ago
samlambert | 8 hours ago
xnx | 17 hours ago
jbonatakis | 16 hours ago
kelp | 15 hours ago
jbonatakis | 15 hours ago
uberman123 | 15 hours ago
karlmush | 12 hours ago
samlambert | 9 hours ago
samlambert | 15 hours ago
kelp | 12 hours ago
Vitess today is not the Vitess of 2021. About 70% of the current codebase was written in the last five years, based on lessons we learned running it at scale.
Neki is built for Postgres from the ground up, by the engineers who did that work.
On the Multigres comparison, I'll admit we get a bit salty, mostly because "Vitess for Postgres" invites the comparison. But to date there's no public evidence of a working sharding implementation in Multigres. Their sharding design doc landed this week, and it lists cross-shard query planning and resharding execution as explicitly out of scope.
Neki does sharding today, including online resharding that switches traffic without a maintenance window. It isn't open source, but you can spin up a cluster and try it yourself.
erulabs | 17 hours ago
Selfish doubts aside, congrats to Planetscale on the launch!
imjosh-dev | 16 hours ago
kaycebasques | 17 hours ago
kelp | 17 hours ago
vira28 | 16 hours ago
samlambert | 15 hours ago
OrangeDelonge | 6 hours ago
theanirudh | 5 hours ago
tedd4u | 14 hours ago
samlambert | 9 hours ago
Atreiden | 16 hours ago
Does Neki solve for this, and if so how? My understanding of CAP theorem is that this basically requires some compromises around availability, but I'm curious as to what that looks like in practice here.
mcpherrinm | 16 hours ago
The usual way to run it is that you partition your db based on something like a user, so that single user gets a consistent DB, but anything cross-shard may not be.
I know when I worked at Block, Cashapp was using Vitess and getting cross-shard DB writes down and functioning correctly was one of the major blockers to adoption. (though I just did tls management for vitess and didn't write any workloads on top of it, so my impression might be a bit off)
ronfriedhaber | 16 hours ago
mcpherrinm | 15 hours ago
arrowleaf | 16 hours ago
Hasz | 16 hours ago
aamederen | 16 hours ago
imjosh-dev | 16 hours ago
ripe | 16 hours ago
Found this from neki.dev.
fastball | 16 hours ago
phoenix24 | 16 hours ago
dvtkrlbs | 16 hours ago
xyst | 16 hours ago
I am looking for interdimensional and interuniversal scale. Which of you trust fund babies has a startup which is working on this problem?
samlambert | 9 hours ago
parthdesai | 15 hours ago
tombert | 15 hours ago
parthdesai | 15 hours ago
https://x.com/samlambert/status/2098075451373834681?s=20
samlambert | 15 hours ago
margalabargala | 15 hours ago
No one accused you of dishonesty.
mbesto | 15 hours ago
PS - Just some advice that I learned in sales 101 early in my career...focusing on shit talking the competition generally isn't a strategy that works well.
omnicognate | 15 hours ago
"You're not wrong Walter, you're just an asshole." - The Dude
ragall | 13 hours ago
Only petty people can hold that view.
prisonguard | 13 hours ago
samlambert | 15 hours ago
parthdesai | 15 hours ago
(I'm a paying PS customer)
cwisecarver | 15 hours ago
overfeed | 14 hours ago
Never heard of the guy before today, but his dickish, edge-lord aura (on X and in this thread) makes me not want to meet him. If I had to choose between buying from him, or some other random dude, I'll hear out the random dude first.
tgma | 8 hours ago
If you have such luxuries, I bet you don't have enough scale to be really profitable for him.
overfeed | 7 hours ago
I can think of at least 4 other ways of scaling horizontally without switching DB engines. I'd have to be mightily constrained to have this particular product as the only option.
tgma | 4 hours ago
JeremyNT | 11 hours ago
Based on what I've read so far, with his shit-talking the competition, he seems like... a bit of an asshole.
tonyhb | 15 hours ago
They build good stuff and care.
mbesto | 13 hours ago
browningstreet | 11 hours ago
samlambert | 9 hours ago
buremba | 9 hours ago
nateberkopec | 9 hours ago
xyzzy_plugh | 15 hours ago
Sam, it feels like you're just playing games. It's hard to take anything you say or do seriously. Are you going to OSS this thing? Or have you decided to backpedal?
Congrats on the checks notes closed launch, I guess?
0: https://github.com/multigres/multigres
samlambert | 15 hours ago
margalabargala | 15 hours ago
Open source being by far the more important of the two, since it means the user is more insulated by terrible corporate decisions thus making it more suitable for long term planning.
samlambert | 15 hours ago
xyzzy_plugh | 15 hours ago
Commerical open source is fine. It's nobody else's fault if you raised too much to make it work sustainably.
samlambert | 15 hours ago
xyzzy_plugh | 14 hours ago
cschmatzler | 14 hours ago
> When we are ready we will release Neki as an open source project suitable for running the most demanding Postgres workloads.
Has this changed?
dkgs | 14 hours ago
Sad to see this coming from a company that was itself built on the foundation of an open-source project like Vitess.
samlambert | 13 hours ago
We maintain Vitess using our company funds. Why are you going after a small company for defending itself against companies like Amazon but not criticizing the reason we have to do this.
I would love to open source Neki and I have not said we won't.
xyzzy_plugh | 13 hours ago
margalabargala | 14 hours ago
Unlike you, probably, if we're slinging unfounded accusations.
tvink | 14 hours ago
therein | 12 hours ago
Their CEO always immediately comes in and makes a fool out of himself. He is what you get when you mix narcissism and being a midwit.
fwip | 13 hours ago
Twirrim | 12 hours ago
A remarkable own goal given that if they'd literally held their tongue, I'd be at worst indifferent.
karlmush | 12 hours ago
Just launch the product and let it speak for itself. The constant need to take shots at competitors comes across as pretty childish.
vincnetas | 5 hours ago
tpetry | 3 hours ago
pm90 | 2 hours ago
kolanos | 12 hours ago
karuppiah7890 | 15 hours ago
samlambert | 15 hours ago
overfeed | 13 hours ago
samlambert | 9 hours ago
devy | 15 hours ago
samlambert | 15 hours ago
levkk | 12 hours ago
- PgDog is open source
- It runs anywhere, not just your cloud
- It's built for Postgres, not a MySQL pivot
samlambert | 9 hours ago
hanspagel | 15 hours ago
ehfeng | 15 hours ago
hanspagel | 15 hours ago
I wonder how the performance will be, but I can just rewrite everything in Rust.
Tepix | 4 hours ago
_1tan | 14 hours ago
IceWreck | 14 hours ago
kelp | 13 hours ago
tedd4u | 14 hours ago
MrDrMcCoy | 6 hours ago
dveeden2 | 4 hours ago
_kidlike | 13 hours ago
wewewedxfgdf | 13 hours ago
gk1 | 7 hours ago
fsuts | 11 hours ago
And now planetscale have made a Vitess like equivalent for Postgres which they have made proprietary
hiyer | 9 hours ago
Google is big enough to survive open-sourcing Vitess, k8s, angular, etc. Planetscale isn't.
samlambert | 8 hours ago
figmert | 4 hours ago
manlymuppet | 9 hours ago
eile23 | 5 hours ago
Tepix | 4 hours ago
Apparently not.
kamikazechaser | 3 hours ago
Yannik_Sc | an hour ago
They claim to provide unlimited IOPS (I/O operations Per Second).
I would like to get some of these drives myself. Sounds like a technical miracle.
On the other hand claiming such miracles makes me wonder which other aspects in this are are actually a working miracle or rather a mythos/false marketing claim.