Reminder that http.Server uses panics for flow control (iirc something about aborting failed responses or so), which is important to know if you're writing panic-handling middleware
As agwa points out, goroutines might be cheap, but the workload wasn't. We were mainly bounded by a combination of the number of cores available (as I mention it wasn't autoscaled) and the target latency for these requests.
We might have been able to do something with a bounded worker pool that could reject work pre-emptively when overloaded, but that would be more complex, be harder to understand and debug. And while the extra memory loads aren't going to help performance overall, we could scale it to suit.
Interesting! For in-memory data it's surprising to me that sorting time isn't totally dominated by time spent sending the result set back to the client over the wire. Can you say more about what query.compare is doing?
The engine is relatively naive, and so it'll often end up sorting 1,000,000 items and returning 50 of them. So hand waving a bit, the count of comparisons would be about N log2(N) or ~20,000,000. So if execution takes ~1s (as it almost did), that's on the order of 50ns per comparison, which doesn't seem too far off, given the dynamic dispatch for the comparators, and that'll likely need to read the attributes from main memory each time.
Oh jeez, yeah, CPU-bound services are tricky to run when clients aren't well-behaved! Fast context cancelation and autoscaling and etc. are all great and necessary, but ultimately much better to refuse new work when at capacity (block) than take it on and let the client dictate terms (drop). And in situations where latency back-pressure doesn't work, services can sometimes guard themselves without too much fuss.
I’m very confused by the choice to ellipsise the title on overflow. On my phone, I get just “That one time I used Go pan…” with no normal way of finding the rest of the title.
(Personal opinion: ellipsising text is never a good choice. I wish browser makes would get on with implementing text-overflow: fade, which is almost always a better approach if you really must hide overflow—which, incidentally, is not often. It’s been in specs for over ten years.)
minus | 22 hours ago
Reminder that http.Server uses panics for flow control (iirc something about aborting failed responses or so), which is important to know if you're writing panic-handling middleware
mxey | 9 hours ago
Not sure if that’s what you are referring to but http.Server catches panics from http.Handlers to turn them into 500s and that’s considered a mistake.
olegkovalov | a day ago
We have tried this solution but agreed on a separate goroutine for sorting (and a few other cpu intensive things).
[OP] noncrab | 23 hours ago
As agwa points out, goroutines might be cheap, but the workload wasn't. We were mainly bounded by a combination of the number of cores available (as I mention it wasn't autoscaled) and the target latency for these requests.
We might have been able to do something with a bounded worker pool that could reject work pre-emptively when overloaded, but that would be more complex, be harder to understand and debug. And while the extra memory loads aren't going to help performance overall, we could scale it to suit.
agwa | a day ago
Goroutines aren't cheap if they're executing a sorting algorithm! I don't think your solution addresses the problem in the blog post.
donio | 16 hours ago
The SortFunc() in the second example seems incomplete with broken syntax. Is it supposed to have a panic() call?
[OP] noncrab | 9 hours ago
Good spot, it's pseudo-code, but I could at least pretend it'll compile.
peterbourgon | 15 hours ago
Interesting! For in-memory data it's surprising to me that sorting time isn't totally dominated by time spent sending the result set back to the client over the wire. Can you say more about what
query.compareis doing?[OP] noncrab | 9 hours ago
The engine is relatively naive, and so it'll often end up sorting 1,000,000 items and returning 50 of them. So hand waving a bit, the count of comparisons would be about
N log2(N)or ~20,000,000. So if execution takes ~1s (as it almost did), that's on the order of 50ns per comparison, which doesn't seem too far off, given the dynamic dispatch for the comparators, and that'll likely need to read the attributes from main memory each time.peterbourgon | 5 hours ago
Oh jeez, yeah, CPU-bound services are tricky to run when clients aren't well-behaved! Fast context cancelation and autoscaling and etc. are all great and necessary, but ultimately much better to refuse new work when at capacity (block) than take it on and let the client dictate terms (drop). And in situations where latency back-pressure doesn't work, services can sometimes guard themselves without too much fuss.
chrismorgan | 10 hours ago
I’m very confused by the choice to ellipsise the title on overflow. On my phone, I get just “That one time I used Go pan…” with no normal way of finding the rest of the title.
(Personal opinion: ellipsising text is never a good choice. I wish browser makes would get on with implementing
text-overflow: fade, which is almost always a better approach if you really must hide overflow—which, incidentally, is not often. It’s been in specs for over ten years.)[OP] noncrab | 9 hours ago
Well, that was a choice, wasn't it? Thanks for pointing it out, it should just wrap normally now.
radio | 23 hours ago
Is this worth it? Maybe there is a cost to having those panic handlers there and panicking, and maybe the costs outweigh the costs of sorting?
antonmedv | 21 hours ago
We use this technic in https://expr-lang.org
Type checker panics with error. It is much easier than passing back the error from tree walking.