Compress any LLM 3x in one click,near-lossless,in under 50 MB RAM

View article

34 points by anushka_gandhi 2 hours ago on hackernews | 31 comments

Xunjin | 2 hours ago

Is it possible to be used on bigger models with the same gains?

[OP] anushka_gandhi | an hour ago

Yes, you can use it on trillion parameter class AI models.

: tmule | an hour ago
Validated how?

ngruhn | an hour ago

No idea if the paper has merit but the UI is slop:

- gradient headline

- random vertical scrollbar on mobile

- fingernail cards

dofm | an hour ago

> - fingernail cards

I did not know that this was the name for these but I am glad I now do :-)

: bensyverson | an hour ago
omg, this is the perfect name for this. Much better than "colored left-border."
It's very curious that the models are so drawn to a fingernail (left border callout) when they must be sparse in the training data (human designers don't do this a ton).

whizzter | an hour ago

Really, started looking at the page and the slop trigger hit quickly enough for me to just skim the structure of the page and close it.

[OP] anushka_gandhi | an hour ago

Maybe it will be better if you focused more at the research rather than doing such things. Our website it not optimised for mobile users as it has more explanation to do rather than just make visitors feel good. Good luck with the things you are going.

: ngruhn | an hour ago
It's a bad smell. As a reader, if the first thing you see is slop, you risk that the rest is also slop and you're wasting your attention.
If UI is not your area or priority, that's ok. Why not just post an Arxiv link?

ElectricalUnion | an hour ago

What is the point of using a outright ancient model like DeepSeek-R1-Distill-1.5B from 2025 based on another ancient model like Qwen2.5-Math-1.5B from 2024, when a much better modern model like LFM3-1.7B exists?

[OP] anushka_gandhi | an hour ago

We have tested it on all kinds of small models including LFM3-1.7B. Our goal is to make a a system that can compress any AI. This is our first version and with your feedback we can improve.

NitpickLawyer | an hour ago

This is a well known tell for using (bad) models to do "research". They always use qwen2.5 and R1, because of the amount of "news" that the two models received. So the models "suggest" those lineages a lot. The local llama sub used to get a few of these a day. That and a zenodo "paper" link are clear tells of badly written stuff.

: dannyw | an hour ago
Although with Anthropic purposefully sandbagging their models for AI R&D, how much of it comes down to model failure, and how much of it comes to deliberate misalignment of the model maker and the customers they sell to?

mapcars | an hour ago

So this is very good for making local models more accessible, right?

: [OP] anushka_gandhi | an hour ago
Correct, you will be able to compress, load and talk with any AI models. This algorithm compressed all model weights by upto 3x.

fishfasell | an hour ago

I know a gen AI UI when I see one and this has all of the red flags. Hate to say it but I feel like that compromises the legitimacy of this

: drum55 | an hour ago
> You’re right, I didn’t actually implement any compression here! My response was misleading, I shouldn’t have claimed this was possible.
Sadly that’s just got to be assumption that there’s no human in the loop. Flashy UI, developed, a readme are no longer indicators you should believe in a project.
: [OP] anushka_gandhi | an hour ago
The website was made with Gemini. Not a big html coder. I was a bit busy on doing research on things that matter rather than focusing on UI
: twobitshifter | an hour ago
If you click through to the home page of the site you are greeting with the perfect genAI tagline:
“We didn't just build an AI voice bot. We engineered a full-stack autonomous AI call center platform with quick call center setup in 3 minutes.”

crunkmcgoop | an hour ago

I compress everything using the Sloot Digital Coding System, only use 50k for all my data!

: k__ | an hour ago
Is it using full-bit encryption?

hedora | an hour ago

Does this compose well with quantization?

Concretely, if you start with, say, a 1.5 or 4 bit quant, then run this, you get something the size of a 0.5 - 1.5 bits without further degradation in model behavior?

[OP] anushka_gandhi | an hour ago

No, you require BF16 model weights for this quantization to work. If you do run it on 4bit or 1.5bit, it will degrade the model and it will hallucinate a lot.

: hedora | 46 minutes ago
So, I should think of this as "5.3bit quantization is now roughly lossless"?
(Ignoring all the math, which I have not looked at yet.)
How does it compare to things like unsloth's work, or the ternary quants that are starting to become possible?
Also does it work well with mixture of experts models? Those tend to be hit harder by quantization artifacts than most because the parts that decide which expert to run can't tolerate error well. (5.3 bits ~ lossless on an MoE pushes the compression frontier for things like Qwen Flash Next, I think.)

tor0ugh | an hour ago

This reads like it. Given that consumer hardware is essentially capped at a certain amount (lets say 24GB for coupled mid range GPUs), this would at best triple either the ‚intelligence‘ (meaning non-degradation of heavy quantization) or the speed of locally running LLMs. There has to be a catch, this sounds too good to be true.

velox | an hour ago

I have not read the whole thing, but I'm not sure I understand the benefit of this vs. using an existing e.g. 4 bit quant of the model in question

: [OP] anushka_gandhi | an hour ago
For 4bit, it takes you 5-10 minutes to compress a model. With this, you can do that in seconds. You do not have to worry about tokenizer configuration or anything. Just one-click compress and run.
This is our first version and yes, it will have some errors that will be fixed in the upcoming versions.

antiloper | an hour ago

Yeah sure I could also quantize Qwen 3.8 27B to 0.15 bits/param. Not sure what the point of this exercise is.

Aurornis | an hour ago

If you couldn’t tell, this is LLM generated quackery.

Any claims about throwing away 2/3rds of the model weights (not just quantizing them) and maintaining “near lossless” quality are simply false.

I clicked on the benchmarks page expecting some quality benchmarks but only saw speed benchmarks before I gave up and stopped scrolling.

The claims don’t even make sense. It’s showing “100% GPU resident” for a model of a size that cannot fit into the GPU.

goodmythical | an hour ago

Would be interesting if one could verify any of these claims via the reputation of any of the research participants.

As it stands, everything credits "NeuraCryption Private Limited" which was registered very recently [0][1][2] to an adress that is shared by several other recently registered businesses [scroll down on 0], and this is their very first offering according to the zenodo link they provided [3]

Further, looking at the code, it appears to be a standard quanitizer...

Like...there is no existing method by which one could genuinely compress a tens of GB weights file in to a third the space using only 50mb of RAM.

[0]https://www.zaubacorp.com/NEURACRYPTION-PRIVATE-LIMITED-U620...

[1]https://www.registerkaro.in/company/neuracryption-private-li...

[2]https://www.falconebiz.com/company/NEURACRYPTION-PRIVATE-LIM...

[3]https://zenodo.org/search?q=metadata.creators.person_or_org....

Roark66 | 23 minutes ago

Hmm, this claims "functionally lossless" compression over 3x. But where is the starting point? Bf16 of fp16? If so, how much better is it than current quantization techniques?

This should be tested on a medium size dense model. For example Qwen3.8-27B. How much better is it t than quantization to 4.5 bits?