Interesting, I was expecting some super complex rust shenanigans when I read about the breadth of their benchmarking setup (wrapping the system allocator, large test dataset, etc.), but actually reading through the optimizations, they are pretty simple and easy to understand. A lot of them are dependent on the minute details of their exact use case, which I guess goes to show how important knowledge about your system is in optimizing it. This seems kind of obvious in retrospect, and is no doubt the reason for the complex benchmarking setup. Good read on an interesting topic
That's not what this is about. This is an optimization of a system that uses terabytes of cache memory in their own CDN system, and the first few sentences explains that just a single byte saved on each entry equates to quite a few gigabytes of saved memory across the entire system (quite a few being an understatement).
Rust does a good job with safe rust alone for a single machine but at that scale you have to resort to some weird things to squeeze out the memory across an entire system of cloudflare's size.
In this case I don’t think the problematic idiom is specific to Rust: the issue is flabby memory use due to pointer-heavy data layouts and higher-level structured representations of the data. The DNS really highlights it because DNS data has a lot of structure but the individual fields are very small, so pointers are a comparatively huge overhead. It’s difficult to represent DNS data in a manner that’s both efficient and idiomatic in any language. (And while avoiding the kinds of bugs suffered by BIND4 and dnsmasq!)
baetylboy | 15 hours ago
Interesting, I was expecting some super complex rust shenanigans when I read about the breadth of their benchmarking setup (wrapping the system allocator, large test dataset, etc.), but actually reading through the optimizations, they are pretty simple and easy to understand. A lot of them are dependent on the minute details of their exact use case, which I guess goes to show how important knowledge about your system is in optimizing it. This seems kind of obvious in retrospect, and is no doubt the reason for the complex benchmarking setup. Good read on an interesting topic
bal-e | 13 hours ago
Oh, it looks like they might be using some of the
domain::newAPI I put together!Box<Name>stands out in particular. That's really cool :)sanxiyn | 19 hours ago
Will we see more software less profligate (I dare not expect frugal, but less profligate, sure) with memory? I really hope so.
junon | 17 hours ago
That's not what this is about. This is an optimization of a system that uses terabytes of cache memory in their own CDN system, and the first few sentences explains that just a single byte saved on each entry equates to quite a few gigabytes of saved memory across the entire system (quite a few being an understatement).
Rust does a good job with safe rust alone for a single machine but at that scale you have to resort to some weird things to squeeze out the memory across an entire system of cloudflare's size.
tobin_baker | 11 hours ago
I got the impression that these sorts of simple low-level optimizations are really at odds with idiomatic Rust.
fanf | 8 hours ago
In this case I don’t think the problematic idiom is specific to Rust: the issue is flabby memory use due to pointer-heavy data layouts and higher-level structured representations of the data. The DNS really highlights it because DNS data has a lot of structure but the individual fields are very small, so pointers are a comparatively huge overhead. It’s difficult to represent DNS data in a manner that’s both efficient and idiomatic in any language. (And while avoiding the kinds of bugs suffered by BIND4 and dnsmasq!)