Since I am using Claude with parallel sessions in different workspaces I started to run out of RAM.
Instead of buying more RAM, which meant that I should replace my device, I decided to implement zram. This compresses memory. Now I can do more with the same hardware.
In many cloud environments compression is not enabled. I am curious to see if the same gains can be made in such environment.
Great article, thanks for sharing the details .. only thing I'd add is that you should put a bit of emphasis on the fact that this is not a one-size fits-all setting - users should very much apply it when appropriate to their latency budgets...
MacOS has been doing this for quite a while now. I was a bit surprised this wasn’t the default for most Linux distros, although I think Pop!_OS (what a name!) has had zram enabled for a few years already. I turned it on on one of my servers (1GB RAM) and it works fine.
> If in doubt, prefer to use zswap. Only use zram if you have a highly specific reason to.
> In terms of architecture:
> zswap sits in front of your disk swap, compresses pages in RAM, and automatically tiers cold data to disk. It integrates directly with the kernel's memory management and distributes pressure gracefully.
> zram is a compressed RAM block device with a hard capacity limit. When you put swap on it and it fills up, there is no automatic eviction, and the kernel has very little leverage to do anything about the situation. The system either OOMs or falls back to lower-priority swap, causing LRU inversion (see below). It only really makes sense for extremely memory-constrained embedded systems, diskless setups, or cases with specific security requirements around keeping private data off persistent storage. Swap on zram is also increasingly unsupported upstream.
What if I loathe swap? The I/O hit of hitting swap, even with SSD, always mean a severely degraded desktop experience. I often end up closing applications to free up RAM or would just order more RAM back when it didn't involve taking a third mortgage.
I would rather have the system tell me there's not enough memory than try to pretend there's an infinite amount of it and then slow to a crawl without apparent reason. Unix memory allocation is daft. Desktop swap to block never worked, never will.
You still want swap, The part you're missing is a good oom strategy. Your system slowing a ton has to do with needing to access swapped memory. Swap is still very useful when you have background tasks that aren't running, but are still using a bunch of RAM. The trick is that if you suddenly start reading a lot of that data repeatedly while being in that sate, you need to oom something quickly.
I just discovered zram thanks to claude after installing fedora on a new computer and man I was blown away by both zram and fedora. Fedora amongst the linux distributions for some reasons has really got everything that an end user needs and the whole experience is just way smoother than anything I have used in 25 years on linux.
I am hoping to switch my dad from ubuntu to fedora in a month or so because he has 16 GB ram and this seems like something most people dont know.
Install "needrestart", and you'll see that a reboot would be useful in Ubuntu, too. Many running programs still use the old version of a shared library even after it has been updated. With so many security updates these days, it almost always makes sense to reboot.
I have a laptop with soldered RAM, a whole whopping 16GB of it. Without zswap or zram, it’s borderline unusable with web browsing, a bunch of terminals, mail client, etc. open. Maybe it’s a skill issue on my side, but only zram makes it work properly without hickups due to the OOM killer taking too long to kick in. Since switching from zswap to zram, I basically don’t encounter OOM situations and corresponding lockups anymore at all. Just my personal anecdote :)
I guess that’s where the skill issue is coming in! That’s what I tried but probably my configuration wasn’t ideal. Maybe I should give it another shot!
Not hating on Fedora, but do you really want your dad on a distro that switches to new packages (or tries new stuff) very eagerly, and that basically requires you to update to the new version every time? Also, 16GB on Linux go a long way, unless you do mem intensive stuff.
That's a terrible statement. Perhaps it can apply to certain setups where compression (and decompression) shows decent results, in lots of applications it'd be a massive downgrade.
The distinction is in the distribution - is the Linux bootable for an end user or is it for a rack-mount industrial application - since anyway the kernel configuration has to be involved to have zswap effectively applied at kernel runtime.
It's basically in-memory swap. If you have swap then you already have the added latency (but worse) and I/O usage instead of CPU. Like swap, the downsides only come into play under high memory pressure.
They are downsides in cases where an application has to escalate a critical fault as quickly as possible - indeed where every nanosecond counts - and a process waiting for page restoration is going to be delayed even an intolerable amount.
It should be a user switch and probably will become one in most Linux desktop distributations, I would wager. However, I would not enable it - I run Ubuntu Studio for every last little bit of latency it gives me in a realtime audio-capture application, and on those systems were it to become default-on by way of distro update, I'd be kinda miffed.
RAM compression sounds very useful but the thing I really need extra RAM for is local inference.
Since the models are effectively highly compressed data already my assumption is that RAM compression won't help improve that much (I'm assuming unified memory here as per the Framework Desktop). A bit like trying to compress a JPEG in a zip file.
I haven't tried it in RAM, but your reasoning matches what I saw when I looked at compressing weights for archiving them. The weights don't compress nicely with standard, especially fast, algorithms. Some more background in this repo [1] including example compression ratios. I would also worry about decompression speed, that gives you a hard bound on the throughput. If the numbers in this repo are accurate, you likely hit the throughput limit of decompression way before you hit the memory limit, even considering the space savings and thus higher effective throughput.
I don't know at all. But I've been playing with oMlx and Splash. Using Pi there are compression extensions like the billion-context. I suspect your right.
But all seem to have massive issue boards. What I did see was oMlx release clustering so you can string 3 macs and have 3 times the memory or use one for prefilling accelerating.
Which I thought was neat and what I'll likely poke at instead of getting more ran.
Yah I think I am conflating it. I was thinking using the ram more effectively to begin to squeeze more, one of those is context size. Less VRAM the less context you can handle, so there always a lot of tuning to find that sweet spot.
> A bit like trying to compress a JPEG in a zip file
Interesting comparison, I wonder if there are loss-less compression schemes that are "weight-aware", about the statistical properties of model weights.
Back to that line - because (Huffman coded) JPEG images can be lossless recoded using arithmetic coding, which makes them smaller than if they were compressed with the best general-purpose compression algorithms/tools (e.g. `jpegtran -arithmetic < input.jpg > output.jpg` beats `xz -9 < input.jpg > output.jpg`). AC takes advantage of the properties of the image coding better than general-purpose compression tools on the output of the Huffman coding.
The library I use for archiving compressed raw LLM weights ([1] , posted in my other comment) afaik at least reorders the weights to more easily compress them. They also have a paper linked, which I admittedly didn't read, that might have more background and tradeoffs. Likely they will have discussed methods to take advantage of the structure of weights.
it interesting because usually on workstation the browser & electron apps it's that holding much of the memory hostage (it's very common to see that browser consuming 20-30gb on 64gb system), so may be if it will be significant compression win on that side it leads to more memory for model but how much performance penalty would be for inference in that case?
The max theoretical ratio of (byte-independent) compression can be approximated using the Shannon entropy.
Most quantized GGUFs have a very high Shannon entropy, so are almost perfectly compressed already. A Q4_K_M quantization is often around +7.90 bits/byte and so is virtually incompressible.
Unquantized models (F16) are often around +7bits/byte for comparison.
You are correct, compression doesn’t help reduce a model. That’s why quantization is inherently a lossy operation although there’s tricks to limit the perceived loss. That’s the only way to further compress the data.
One way to think about it is that if the weights were compressible, it’s a clear signal that better training could make the model smarter as there’s redundant information in the structure that could be used to store other meaning.
The other challenge with compressing weights of course is that you have to decompress them before you do math with them. Dequantizing weights is significantly cheaper and supports random access too (not that that matters too much but it helps in terms of leveraging SIMD techniques to do it in an embarrassingly parallel way)
> redundant information in the structure that could be used to store other meaning.
Consider the induction heads, their W_O are generally expected to be the identity matrix and it is useful to have many of them at many layers (e.g. if you only had one induction head in the whole network you'd only be able to perform one induction circuit per token emitted)
That being said the attention heads are a small portion of the total network size
Since you need to apply CPU time to compress memory, and also CPU before you can map pages back into process memory (by uncompressing), this is only useful for not frequently used memory, like swap only a bit faster.
I'm not too familiar with LLM memory access patterns, but since the model files are stored on disk, you wouldn't ever keep them in memory anyway (except caching) and just reread them on demand.
> I'm not too familiar with LLM memory access patterns
Dense architectures hit every byte of the model for every token. It's the ultimate memory bandwidth nightmare.
MoE models use subsections for each token and are more amenable to keeping parts in secondary storage, and indeed there are some local llm engines that do this.
> Memory compression is not new. If you have a Windows or Mac device you are already using it.
Yes, macOS has had it enabled by default since OS X Mavericks 10.9, released in October 2013. I’m surprised some desktop Linux distributions still require manual configuration to enable it.
For browser tabs, though, I think a better solution is automatic tab closing, using something like the Tab Wrangler extension for Chrome. Collecting tabs comes naturally; we’re much worse at closing them.
>I’m surprised some desktop Linux distributions still require manual configuration to enable it.
Its a user choice and should be a user-facing knob because there are times it is inappropriate for use, such as in safety-critical circumstances, certain realtime realities, and so on. It should probably be on by default for desktop Linux, but off by default for every other application a distribution may be targeting ...
"For browser tabs, though, I think a better solution is automatic tab closing" ... the better solution is to actually offload tabs. The issue with Chrome/Edge/... is not your tabs itself but the huge amount of memory loaded for GPU operations.
Do Shift+Esc and watch how large the GPU memory usage is. Often 2/3 to 50% of your actual tab usage.
Ironically, its not the closing of tabs that bothers me, but the fact that tab order changes, what slows my productivity/muscle memory down. And its the same reason for a lot of people why they keep tabs open.
I never understood why we do not have a basic snapshot / sleep that offloads a tabs memory on disk (and dumps the GPU memory), aka a ghost tab. And fast loads when you view the tab. Sure, there may be a small delay but it can save so much memory, while people can keep their tabs open/organized.
Some extensions try to mimic that but its often a crude solution because they do not have deeper access to the webview2 engine.
Brave on Mac (but not Linux IIRC) does this, you can choose how eagerly it will suspend background tabs, although this is somewhat erratic - I've seen it suspend a paused YT video that has been idle for maybe an hour (and not much memory pressure on the system), and then sometimes it takes its time.
In Firefox there's about:unloads where you can suspend tabs in I believe LRU order. I'd really like to see this as a default option (like: only keep N tabs active). Maybe there's an extension for that.
In many systems, adding compression makes not just storage better but increases the io you get from a system too. Here's Valkey 9.2 compression figures showing that: https://www.phoronix.com/news/Valkey-9.2
I wonder if there's any similar wins with ram? That some workloads would see net wins from compression. Some cost of cpu, but increasing net throughput significantly.
I switched to Fedora some time ago and only recently learned about zram.
I was hitting full system memory(16GB) during normal use, and after increasing my zram to 16GB, I was blown away. I had initially planned to buy additional ram.
> I've used zswap on most of my systems and generally find it beneficial
I started using zswap recently but didn't have the time yet to measure the benefit. Does it benefit machines with low amounts of RAM and wimpy CPU (like a RPi3)?
> The controller usually transparently does it as it writes to/from NAND.
In the consumer SSD space the last controllers I know did on-the-fly compression were SandForce [1], probably 15 years ago, when minimizing writes and saving the little capacity available was imperative. I don't think any other desktop SSDs do this now. A bit more common in enterprise models like ScaleFlux CSD.
I didn't think controllers generally did any compression but I remember once that on old sandforce controllers if you did something like write all zeros it wouldn't actually overwrite anything instead just mapping to another block of zeros. Which sounded more like on device dedupping than compression.
You’d hope with these amazing GPUs today, that some cycles can be used to compress RAM very efficiently. So many parallel cores, surely we can get even better compression from this latent hardware?
Windows doesn't do memory compression if the page file is not enabled. I never particularly saw the point of the page file because I don't enjoy swapping, but it appears that the memory management system disproportionately depends on it to the point where it simply refuses to work properly with it off. So make sure that's turned on if you want memory compression. (Do not let it auto-size or it will happily fill up your whole drive, set it to something reasonable)
I thought the point of memory compression was fitting more pages into the same memory, by compressing their contents when they're not being used. It doesn't make sense that you would still need to have memory for every page, as that prevents you from getting more out of the same memory, doesn't it?
SSDs make swapping less of an issue, but it's still something I'd rather not allow to ever happen, as it's difficult to get swapped-out pages back into memory without exercising every single application on the system.
I thought memory compression was a replacement for swapping entirely, because you don't need to evict from memory, you can just compress it.
Because it is a user choice, and there are applications where it is inappropriate. For desktop Linux it should probably be on, as long as its clear to the user - but yeah, for many other Linux use cases its inappropriate.
Maybe it should be generally on for desktop-focused distros and generally off for server-focused ones. The distros that focus on doing both well might want to recommend it in the installer when told it's primarily a desktop installation and explain the tradeoffs.
For a server, it really depends on what kind of service you're providing whether it would be helpful. We might want to let server admins, SREs, production engineers, or platform engineers to make an active decision about it.
In most cases you'll really want zswap rather than zram anyway.
There are plenty of scenarios in which servers are run with only nominal swap sizes or none at all and systems don't run out of RAM.
Let me size my hypervisor cluster to demand and let the system decide where to place the guests. If the hypervisors are crashing due to full RAM, that's down to planning or budget issues and not a reason to put additional unpredictable latency underneath the guests.
Let me size my SDI to UDP TS video streaming servers to the number and size of streams they'll service. There's not much in their memory pages that's going to ever be idle enough to compress, so why boot the kernel configured to do that? It's just risking extra packet IAT for buying not much benefit.
But in general, yes, zswap would be a good thing compared to swapping to a drive, unless you are mostly swapping things that are really truly idle a long time. Default swappiness on a lot of distros is around 60 by the way, and on servers it's often set closer to 10 or 15.
For a programmer, on many cases, it pays to compress the data, to improve RAM IO, because the alu may already be doing nothing meanwhile waiting for the data from ram, so you can use that time to decompress, process, compress, and store the result. A lot of code today is bottlenecked by IO. You spend a lot of effort optimizing with SIMD, processing the data super fast, and then the speed it's wssted because there is not enough bandwidth to even send the result to ram. Modern cpu should include more hardware support for memory compression
So, if it is used as swap, that means to use the data, it first needs to go to the CPU, then back to regular RAM, then back to the CPU.
Isn't that wasteful? The CPU's caches may help a bit, but I would guess first a huge block is going through the decompression, so only the last bits are still in the cache, when the actual usage starts.
I've seen a single YouTube browser process use 8GB on my desktop, so I'm not that surprised. YT eats up whatever is available if you leave some tabs open for long enough, fortunately about:processes exists.
I never want to swap to disk. If I'm ever in that situation, it means I did something wrong, so I want my userspace OOM killer to trigger and inform me of my mistake immediately instead of wasting time and SSD endurance delaying the inevitable. But zram with lz4 compression is effectively free RAM in most circumstances, so there's little reason not to enable it.
Good point, dumping to SSD isn't ideal. I think I'm gonna start trying to set memory limits on programs with systemd-run so they can self-limit themselves.
You absolutely do want to swap to disk. There are lots of non-file-backed cold pages that are pointless to keep in RAM forever. You could argue that these pages shouldn't exist, but they do, so why not shove them to a disk?
I want instant access to my rarely used pages just as much as my frequently used pages. It's only pointless to keep in RAM if it's literally never used. If we knew how to identify those pages we could just avoid allocating them in the first place.
Your system is still swapping all your file-backed pages (e.g. .so files) unless you're mlockall()-ing everything. If you pin non-file-backed pages in RAM by running without a swap file, the system has a harder time keeping your frequently-accessed file-backed pages in RAM and can't do as good a job as optimizing your usage patterns.
All you're doing by running without swap is pessimizing yourself.
What a fun loop through time. I remember downloading programs to free up ram in the early 2000's, then as RAM became more ubiqutous and programs were still small you could run them directly on RAMDisk. (I did this with Path of Exile about 10 years ago to avoid lag.) Now we are working back to getting the most out of our ram.
As others have said, macOS does this by default and it's quite impressive. I have a long running daemon with a memory leak. By its nature, the leaking memory is very repetitive and easily compressed. Right now this daemon has tried to claim 2.5GB of memory, but it's been compressed to a mere 450MB of actual memory! And because it's a memory leak, it never accesses this old stuff, so it never gets decompressed. Only 20MB of RAM is held that isn't compressed.
Of course, it would be far better to just not have the memory leak...but in the mean time, macOS memory compression is making the leak nearly a none issue.
Paging to disk has also become vastly more transparent, and less of a last resort, with M.2 SSDs. No more things slowing down 10x and endless HD noise when out of RAM.
Oh we found out about it. In the old days, a machine with a load of 57380 (probably not the highest number I have seen) means the machine has run out of memory and started swapping heavily.
Fun days.
Now, we find out when the SSD wears out, in the worst case.
If the user was feeling stochastic, the OS could theoretically just drop pages that haven't been accessed in, say, 2 days, and send the process a segfault if it tries to access them.
A new take on the OOM killer: if a long-running daemon hasn't touched some heap pages for a very long time, perhaps the OS could assume it is leaked and unmap the pages. If the daemon tries to access that memory, let it crash.
This sounds like a bad idea. No one has logged in via SSH for a while so the daemon hasn't touched any pages (or only pre-auth relevant pages) then when you log in it gets killed. Or if someone sends a rarely used request to an HTTP server then it crashes accessing some data unique to that request.
It's a good example of perverse outcomes. Rather than simply following the contract and functioning correctly, we would be:
going through extra effort to accommodate/enable buggy, poorly behaving programs, and meanwhile—
punishing those (and the programmers of those) that are well-behaved and functioning exactly as intended but that now have to contend with evading some heuristic of badness.
I wonder what the theoretical limits are of semantic compression for LLM input/output (eg. "Can you please send me the final report by Friday?" -> "Send final report by Friday." -> sr(FINAL, DEADLINE=FRIDAY)) that can still be recovered back to English
Tried porting it to swift, was hoping it would make mobile local models more useful for a project I was working on but it was hurting the accuracy too much. Can be very task dependent on how much you can compress. https://github.com/nburns/llmlingua-swift
My specific use case was using a LLM on segments of labeled data from a different faster model as a final pass.
It's funny that MS didn't integrate memory compression until 2015. From ye olden days, we used "RAM doublers" in MSDOS, which were a bit of a different beast and took a performance hit due to simpler implementation, so some of us aren't "excited" by the cost of packing things in with compression. It is a bandaid for processes that lie somewhere between being needed and not (otherwise it would goto swap right?)
If it avoids hitting the disk I get it, but to some of us, that just means it's time to buy more RAM
I remember the compression rage back on my amiga, had both ram and file compression, since you had to get as much room as you could with limited ram and a floppy.
As others have noted zswap is usually a better choice than zram, additionally depending on your situation it may be worth checking which algorithm it's using as there are a few choices (zstd, lz4, etc) and the defaults vary by distro. zstd is the right choice IMO unless you're in a situation where you know better.
In the 1990s there was a third-party add-on for Windows called SoftRAM95 [1] that doubled your RAM using compression. That one turned out to be a scam though [2].
If my understanding is correct, both ZRAM and ZSWAP write uncompressed pages to disk. And both don't pack the pages together meaning even if we'd write compressed pages (let's say they compress 5:1), we'd still write the original size to disk. This is crazy.
FWIW, DAMON (https://docs.kernel.org/mm/damon/design.html#operation-actio...) has supported explicit control over Linux working sets for a while now. For whatever reason, ISTM few people know about it, but it's a nice subsystem with a lot of policy flexibility.
Yes, but if a 4k page compresses to 1k, a 4k block on the disk still gets used: 1k + 3k of zero padding. Meaning if 100GB of data compresses 5:1, 100GB needs to be written to the disk instead of 20GB.
I think it's a great shame that kubernetes really doesn't like systems with any kind of swap on (including zswap). If memory compression were more commonly used I think many workloads could survive on smaller systems.
> Compressing memory does come with small, but not really noticeable, downsides. It takes a little bit more from your CPU and adds latency to some processes.
I recall reading an article around the time the Windows team added it (back when I still used the OS). Their compression algorithm was typically faster than the latency to the prevailing disk type at the time (SATA SSD, most likely) - at the time it had zero downsides. I'm not sure if that still holds true.
On Linux it generally doesn't: off-the-shelf compression is used, which do not outpace storage access latency.
I was load testing micro VMs with zswap and i was blown away by how much capacity the machine could have. forking thousands of processes, using 2.5x the available ram, and the server still hums along, albeit a bit sluggish. but no timeouts or lockups. still very accessible.
It's a remarkable tech that deserves more attention. Swap is back in style!
I do this and also buy old used 16GB Optane sticks to use as swap. These are $7 a pop on ebay. They're way lower latency than normal swap and you can swap data as much as you like without worrying about wear.
Doesn't this only work if you have a bunch of stuff you do not need to access frequently in RAM? i wonder what kind of algorithm optimization goes behind the scenes to make sure pages that least recently used get compressed with other pages
This guy already had 32GB RAM. I know that's "standard" or whatever but it's surprisingly difficult to afford when you already have an 8GB laptop at home that still works on Linux. Would be interesting to see someone trick out a low-powered 4GB one, I would find that very interesting.
[OP] Ecno92 | 11 hours ago
Instead of buying more RAM, which meant that I should replace my device, I decided to implement zram. This compresses memory. Now I can do more with the same hardware.
In many cloud environments compression is not enabled. I am curious to see if the same gains can be made in such environment.
XYen0n | 10 hours ago
[OP] Ecno92 | 8 hours ago
aa-jv | 8 hours ago
roosgit | 11 hours ago
kachnuv_ocasek | 10 hours ago
ChocolateGod | 4 hours ago
anthk | 3 hours ago
mrob | 2 hours ago
messe | 10 hours ago
https://linuxblog.io/zswap-better-than-zram/
https://chrisdown.name/2026/03/24/zswap-vs-zram-when-to-use-...
mzhaase | 10 hours ago
> If in doubt, prefer to use zswap. Only use zram if you have a highly specific reason to.
> In terms of architecture:
> zswap sits in front of your disk swap, compresses pages in RAM, and automatically tiers cold data to disk. It integrates directly with the kernel's memory management and distributes pressure gracefully.
> zram is a compressed RAM block device with a hard capacity limit. When you put swap on it and it fills up, there is no automatic eviction, and the kernel has very little leverage to do anything about the situation. The system either OOMs or falls back to lower-priority swap, causing LRU inversion (see below). It only really makes sense for extremely memory-constrained embedded systems, diskless setups, or cases with specific security requirements around keeping private data off persistent storage. Swap on zram is also increasingly unsupported upstream.
speed_spread | 9 hours ago
I would rather have the system tell me there's not enough memory than try to pretend there's an infinite amount of it and then slow to a crawl without apparent reason. Unix memory allocation is daft. Desktop swap to block never worked, never will.
izacus | 5 hours ago
mrob | 4 hours ago
adgjlsfhk1 | 2 hours ago
keybored | 8 hours ago
( https://news.ycombinator.com/item?id=47500746 )
It’s long and more satisfying than this In These AI Times wrapper.
sammatilda | 10 hours ago
I am hoping to switch my dad from ubuntu to fedora in a month or so because he has 16 GB ram and this seems like something most people dont know.
nico | 10 hours ago
Lio | 10 hours ago
I really like Fedora in that it seems very up to date package wise but I found it requires a reboot every time I update using the UI updater.
It's better using the dnf command but it still felt a bit lumpy compared to apt.
mritzmann | 9 hours ago
nosrepa | an hour ago
speed_spread | 9 hours ago
amelius | 8 hours ago
jrmg | 5 hours ago
swiftcoder | 4 hours ago
mkesper | 4 hours ago
anthk | 3 hours ago
kasabali | 3 hours ago
quotemstr | 17 minutes ago
hofiflo | 3 hours ago
adgjlsfhk1 | 2 hours ago
hofiflo | 39 minutes ago
hobo123 | 35 minutes ago
ChrisRR | 10 hours ago
It sounds like those are downsides
MadameMinty | 9 hours ago
xxs | 9 hours ago
Rohansi | 9 hours ago
aa-jv | 8 hours ago
Rohansi | 9 hours ago
Stitch4223 | 8 hours ago
It’s like saying that in order to breathe you have to use your nose or mouth. Which is bad if you want to retain the oxygen that is inside your lungs.
There are always scenario’s where memory compression is an outright bad idea. Just like trying to breathe under water.
[OP] Ecno92 | 8 hours ago
I did not really notice a large performance impact.
Unless you are in an environment where compute performance is critical, I think this is something interesting to try.
aa-jv | 8 hours ago
It should be a user switch and probably will become one in most Linux desktop distributations, I would wager. However, I would not enable it - I run Ubuntu Studio for every last little bit of latency it gives me in a realtime audio-capture application, and on those systems were it to become default-on by way of distro update, I'd be kinda miffed.
Lio | 9 hours ago
Since the models are effectively highly compressed data already my assumption is that RAM compression won't help improve that much (I'm assuming unified memory here as per the Framework Desktop). A bit like trying to compress a JPEG in a zip file.
Could anyone comment on that?
rft | 9 hours ago
[1] https://github.com/zipnn/zipnn#results
kalcode | 9 hours ago
But all seem to have massive issue boards. What I did see was oMlx release clustering so you can string 3 macs and have 3 times the memory or use one for prefilling accelerating.
Which I thought was neat and what I'll likely poke at instead of getting more ran.
0123456789ABCDE | 8 hours ago
kalcode | 5 hours ago
So my mind on that.
matja | 8 hours ago
Interesting comparison, I wonder if there are loss-less compression schemes that are "weight-aware", about the statistical properties of model weights.
Back to that line - because (Huffman coded) JPEG images can be lossless recoded using arithmetic coding, which makes them smaller than if they were compressed with the best general-purpose compression algorithms/tools (e.g. `jpegtran -arithmetic < input.jpg > output.jpg` beats `xz -9 < input.jpg > output.jpg`). AC takes advantage of the properties of the image coding better than general-purpose compression tools on the output of the Huffman coding.
rft | 7 hours ago
[1] https://github.com/zipnn/zipnn#results
strobe | 5 hours ago
throw-qqqqq | 4 hours ago
Most quantized GGUFs have a very high Shannon entropy, so are almost perfectly compressed already. A Q4_K_M quantization is often around +7.90 bits/byte and so is virtually incompressible.
Unquantized models (F16) are often around +7bits/byte for comparison.
https://en.wikipedia.org/wiki/Shannon%27s_source_coding_theo...
vlovich123 | 4 hours ago
One way to think about it is that if the weights were compressible, it’s a clear signal that better training could make the model smarter as there’s redundant information in the structure that could be used to store other meaning.
The other challenge with compressing weights of course is that you have to decompress them before you do math with them. Dequantizing weights is significantly cheaper and supports random access too (not that that matters too much but it helps in terms of leveraging SIMD techniques to do it in an embarrassingly parallel way)
Powdering7082 | 3 hours ago
Consider the induction heads, their W_O are generally expected to be the identity matrix and it is useful to have many of them at many layers (e.g. if you only had one induction head in the whole network you'd only be able to perform one induction circuit per token emitted)
That being said the attention heads are a small portion of the total network size
hobo123 | 48 minutes ago
I'm not too familiar with LLM memory access patterns, but since the model files are stored on disk, you wouldn't ever keep them in memory anyway (except caching) and just reread them on demand.
RossBencina | 17 minutes ago
Dense architectures hit every byte of the model for every token. It's the ultimate memory bandwidth nightmare.
MoE models use subsections for each token and are more amenable to keeping parts in secondary storage, and indeed there are some local llm engines that do this.
yummybear | 9 hours ago
Which one is it?
DeathArrow | 9 hours ago
Still perfectly usable and I won't upgrade until prices come down. Even used prices are terrible now.
theandrewbailey | 9 hours ago
Typo DDR5?
xxs | 9 hours ago
theandrewbailey | 9 hours ago
https://www.intel.com/content/www/us/en/products/sku/236783/...
DeathArrow | 8 hours ago
globular-toast | 9 hours ago
The main thing I regret is not maxing out the memory on my server. At first 32Gi seemed more than I'd ever need, but now it's feeling quite anaemic.
jakozaur | 9 hours ago
Yes, macOS has had it enabled by default since OS X Mavericks 10.9, released in October 2013. I’m surprised some desktop Linux distributions still require manual configuration to enable it.
For browser tabs, though, I think a better solution is automatic tab closing, using something like the Tab Wrangler extension for Chrome. Collecting tabs comes naturally; we’re much worse at closing them.
jve | 8 hours ago
evenfrost | 8 hours ago
jamesgeck0 | 5 hours ago
kasabali | 3 hours ago
aa-jv | 8 hours ago
Its a user choice and should be a user-facing knob because there are times it is inappropriate for use, such as in safety-critical circumstances, certain realtime realities, and so on. It should probably be on by default for desktop Linux, but off by default for every other application a distribution may be targeting ...
xnx | 5 hours ago
benjiro29 | 4 hours ago
Do Shift+Esc and watch how large the GPU memory usage is. Often 2/3 to 50% of your actual tab usage.
Ironically, its not the closing of tabs that bothers me, but the fact that tab order changes, what slows my productivity/muscle memory down. And its the same reason for a lot of people why they keep tabs open.
I never understood why we do not have a basic snapshot / sleep that offloads a tabs memory on disk (and dumps the GPU memory), aka a ghost tab. And fast loads when you view the tab. Sure, there may be a small delay but it can save so much memory, while people can keep their tabs open/organized.
Some extensions try to mimic that but its often a crude solution because they do not have deeper access to the webview2 engine.
hobo123 | 42 minutes ago
In Firefox there's about:unloads where you can suspend tabs in I believe LRU order. I'd really like to see this as a default option (like: only keep N tabs active). Maybe there's an extension for that.
kiicia | 9 hours ago
jauntywundrkind | 9 hours ago
That's for RDB persistence. https://valkey.io/topics/persistence/
I wonder if there's any similar wins with ram? That some workloads would see net wins from compression. Some cost of cpu, but increasing net throughput significantly.
certyfreak | 9 hours ago
dist-epoch | 9 hours ago
Technically yes, practically on Windows memory compression is not used.
I have Windows 11, 128 GB RAM, and my memory stats show: 42 GB in use, 380 MB compressed.
sheo | 9 hours ago
I'm a Linux user myself, but I think it would start compressing it when there would be less than 25% free
rustyhancock | 9 hours ago
What's interesting to note is that I've occasionally wondered why isn't the swap partition compressed.
If you have it on an NVMe drive it probably is as is all data! The controller usually transparently does it as it writes to/from NAND.
So the kernel sees uncompressed fixed block pages and the speed boosts happen with data arriving faster when read or written to flash.
close04 | 5 hours ago
I started using zswap recently but didn't have the time yet to measure the benefit. Does it benefit machines with low amounts of RAM and wimpy CPU (like a RPi3)?
> The controller usually transparently does it as it writes to/from NAND.
In the consumer SSD space the last controllers I know did on-the-fly compression were SandForce [1], probably 15 years ago, when minimizing writes and saving the little capacity available was imperative. I don't think any other desktop SSDs do this now. A bit more common in enterprise models like ScaleFlux CSD.
[1] https://en.wikipedia.org/wiki/SandForce#Products
mkesper | 4 hours ago
deltoidmaximus | 2 hours ago
baud9600 | 9 hours ago
BenjiWiebe | 4 hours ago
There might be gains by having the GPU recompress with a higher compression level in the background though.
LoganDark | 8 hours ago
vardump | 2 hours ago
So shouldn’t disable pagefile, unless it’s a disk space issue. Windows probably won’t use it much anyways if you have enough RAM.
LoganDark | 31 minutes ago
SSDs make swapping less of an issue, but it's still something I'd rather not allow to ever happen, as it's difficult to get swapped-out pages back into memory without exercising every single application on the system.
I thought memory compression was a replacement for swapping entirely, because you don't need to evict from memory, you can just compress it.
cubefox | 8 hours ago
aa-jv | 8 hours ago
cubefox | 8 hours ago
cestith | 5 hours ago
For a server, it really depends on what kind of service you're providing whether it would be helpful. We might want to let server admins, SREs, production engineers, or platform engineers to make an active decision about it.
cubefox | 4 hours ago
cestith | 3 hours ago
There are plenty of scenarios in which servers are run with only nominal swap sizes or none at all and systems don't run out of RAM.
Let me size my hypervisor cluster to demand and let the system decide where to place the guests. If the hypervisors are crashing due to full RAM, that's down to planning or budget issues and not a reason to put additional unpredictable latency underneath the guests.
Let me size my SDI to UDP TS video streaming servers to the number and size of streams they'll service. There's not much in their memory pages that's going to ever be idle enough to compress, so why boot the kernel configured to do that? It's just risking extra packet IAT for buying not much benefit.
But in general, yes, zswap would be a good thing compared to swapping to a drive, unless you are mostly swapping things that are really truly idle a long time. Default swappiness on a lot of distros is around 60 by the way, and on servers it's often set closer to 10 or 15.
jrmg | 5 hours ago
That’s true of everything. Default setups make those choices.
ChocolateGod | 4 hours ago
nukanse | 8 hours ago
hinkley | 8 hours ago
aktenlage | 8 hours ago
Isn't that wasteful? The CPU's caches may help a bit, but I would guess first a huge block is going through the decompression, so only the last bits are still in the cache, when the actual usage starts.
72deluxe | 7 hours ago
alpaca128 | 4 hours ago
Sopel | 4 hours ago
cogman10 | 5 hours ago
https://chrisdown.name/2026/03/24/zswap-vs-zram-when-to-use-...
ciupicri | 5 hours ago
mrob | 4 hours ago
dsp_person | 3 hours ago
01HNNWZ0MV43FF | 2 hours ago
quotemstr | 59 minutes ago
mrob | 35 minutes ago
quotemstr | 13 minutes ago
All you're doing by running without swap is pessimizing yourself.
teravor | 3 hours ago
if you don't want to swap to disk you should not use zswap.
zram is ideal in that scenario, though it may complicate OOM killer considerations (you may want a userspace one).
fernandogrd | 5 hours ago
fred_is_fred | 5 hours ago
esafak | 3 hours ago
DiskDoubler, Stacker, SuperStor, DoubleDisk...
Lerc | 5 hours ago
I find that the ratio of browserish memory use to that of other programs is enough to render the latter insignificant most of the time.
anthk | 3 hours ago
Juvination | 5 hours ago
pshirshov | 5 hours ago
tooheavy | 5 hours ago
distantsounds | 5 hours ago
nchmy | 3 hours ago
myrandomcomment | 4 hours ago
1023bytes | 4 hours ago
varenc | 4 hours ago
Of course, it would be far better to just not have the memory leak...but in the mean time, macOS memory compression is making the leak nearly a none issue.
trollbridge | 4 hours ago
Sharlin | 3 hours ago
bayindirh | 2 hours ago
Fun days.
Now, we find out when the SSD wears out, in the worst case.
dotancohen | 2 hours ago
hobo123 | 52 minutes ago
davidkwast | 3 hours ago
01HNNWZ0MV43FF | 2 hours ago
Thus solving memory leaks forever...
bayindirh | 2 hours ago
cpeterso | 2 hours ago
kevincox | 2 hours ago
Sure, they will get restarted, but not great.
cxr | an hour ago
going through extra effort to accommodate/enable buggy, poorly behaving programs, and meanwhile—
punishing those (and the programmers of those) that are well-behaved and functioning exactly as intended but that now have to contend with evading some heuristic of badness.
dotancohen | 2 hours ago
TheSoftwareGuy | an hour ago
pyaamb | 4 hours ago
NegativeLatency | 4 hours ago
Tried porting it to swift, was hoping it would make mobile local models more useful for a project I was working on but it was hurting the accuracy too much. Can be very task dependent on how much you can compress. https://github.com/nburns/llmlingua-swift
My specific use case was using a LLM on segments of labeled data from a different faster model as a final pass.
altcognito | 4 hours ago
If it avoids hitting the disk I get it, but to some of us, that just means it's time to buy more RAM
IronWolve | 4 hours ago
hedgehog | 4 hours ago
UltraSane | 3 hours ago
varispeed | 3 hours ago
As usual, working class is being told to tighten the belts whilst the rich get insanely richer.
Fuck that.
ifloop | 2 hours ago
bismanawaz | 3 hours ago
1970-01-01 | 3 hours ago
https://www.youtube.com/watch?v=4icQwducz68
https://www.youtube.com/watch?v=IXkX2ojrKZQ
grandlimelabs | 3 hours ago
flowerlad | 3 hours ago
[1] https://en.wikipedia.org/wiki/SoftRAM
[2] https://devblogs.microsoft.com/oldnewthing/20211111-00/?p=10...
jmakov | an hour ago
If my understanding is correct, both ZRAM and ZSWAP write uncompressed pages to disk. And both don't pack the pages together meaning even if we'd write compressed pages (let's say they compress 5:1), we'd still write the original size to disk. This is crazy.
quotemstr | 58 minutes ago
zamalek | 53 minutes ago
ZRAM added compressed_writeback in 7.0, it needs to be explicitly enabled.
jmakov | 48 minutes ago
vjvjvjvjghv | an hour ago
noodlesUK | 58 minutes ago
zamalek | 54 minutes ago
I recall reading an article around the time the Windows team added it (back when I still used the OS). Their compression algorithm was typically faster than the latency to the prevailing disk type at the time (SATA SSD, most likely) - at the time it had zero downsides. I'm not sure if that still holds true.
On Linux it generally doesn't: off-the-shelf compression is used, which do not outpace storage access latency.
tonymet | 53 minutes ago
It's a remarkable tech that deserves more attention. Swap is back in style!
2001zhaozhao | 35 minutes ago
bullish-bearr | 25 minutes ago
djha-skin | 18 minutes ago