Crazy to see how well Qwen3.6 35b-a3b is holding up, sure it is ~20% larger but it's scores are also ~20% higher with the same number of active params (excluding the IFBench).
Hopefully Qwen follows up their 3.8 launch with a new 35b-a3b
4 months of progress. Qwen 3.8 Max is, depending on who you ask, what you're doing, standard disclaimers, really really close to Opus 4.8. That is a huge dump from 3.6 Max. Presumably improvements in their training librar(y/ies), training improvements, etc will "trickle down" to Qwen 3.8. We are using 3.6 35B internally and it is in most cases better than Sonnet 4.6 which already makes it "good enough" for most data enrichment tasks. I would be a lot more excited about a MoE Qwen 3.8 35B-A3B than the "announced" 27B dense model. I have 50 million records I need to keep up to date, an A3B MoE is waaaay more performant than a 27B dense model.
This links to the nvfp4 version of the model, so they only compare it to the bf16 in this model card. If you're looking for other similar model comparisons they are in the model card of the bf16 version here [1].
While it looks "behind" the qwen equivalent model on most benchmarks, a few personal notes:
- nemotron models feel to me a bit less benchmaxxed / "stubborn". That means that they generalise a bit better, or can be tasked to solve similar but not quite identical task types to the training data (something that's hard to do w/ qwen/ds models)
- nemotron series are also open training (w/ open training recipes and some training data public)
- nvda will have an incentive to continue this kind of releases, even if other parties slowly abandon the open release of models. Whatever other incentives 3rd party labs have (i.e. meta, goog w/ gemma, the chinese labs that IPOd, etc) nvda will always want to sell hardware so their incentive to keep pushing open models is evident and will likely continue "forever".
this "DFlash" model should be used together with one of the previous two "for lower-latency speculative decoding deployments tuned for low-concurrency data center and workstation workflows".
You might look at this and and be a bit disappointed by the performance against qwen and gemma models - but this is an entirely open source training pipeline, this is quite impressive and I don't think another model this performant exists with fully open source data and recipes alongside the weights.
Nvidia just throwing something "for peasants" to stay relevant. Where is competition spirit? More importantly why Nvidia is gatekeeping computing for everyday people?
I find these releases are bad taste.
Make 1TB DGX priced affordably, not some crap model for people to waste time on.
imo a 512GB DGX cluster for $20k is not a bad price for an enterprise that wants to do on prem LLMs. Gatekeeping is a weird way to put “running a business”
Interesting to see in the Agentic Coding Benchmarks that the codex harness is so demonstrably lower than any of the other harnesses, even Claude Code another lab harness. Does anyone know why that would be?
I tried that exact model, I get about 50 tokens/sec on an 64GB M1 Max MBP.
It writes pretty good code; it does seem to second-guess itself in thinking traces and I wonder if it just needs a reasoning budget and message.
Each time I test a model quickly in LM Studio, I ask it:
- to write a tiny little wordpress "last login" tracker plugin, asking me clarifying questions first. I ask it to use old PHP, break the WP coding guidelines to use inline anonymous functions in the hooks, avoid custom SQL, I see if it can write something useful in a singleton class, and what questions it thinks to ask. Qwen 3.6 does this very well, Nemotron has done OK, though it's a little less effective at reading between the lines, maybe.
- to offer an answer for a SQL puzzle about finding max score per category on old MySQL (5.0) without using subquery/derived tables -- it did a good job, picked up the nuances in the prompt that allow a particular solution, didn't go on a tangent about how it would be nice to have window functions or use subqueries, did a tool call to check like I asked. With this puzzle, if the model doesn't offer up an index for performance optimisation, I nudge it; this time it didn't volunteer one but when prompted about performance it offered an index and a bunch of other nice solutions, and only there did it round up the options for subqueries and derived tables, which is fair game.
(It did badly fail the car wash test, though, even on repeated nudging, where it gets more and more insane, doubling down and never getting the point, whereas Muse Glimmer solved it and well, with a thinking trace that didn't particularly suggest it had been post-trained)
I need to test it in Pi or opencode. I've been trying to motivate my brain to move to pi, but this model supports a longer context window so maybe opencode's overlong system prompt is less of an issue.
Of the 30B models in the last 24 hours (!) I think I prefer working with Muse Glimmer, which is slow but very good, and writes rather well with just a hint of being a bit of a cheeky monkey. Not tested that in an agentic coding setting yet.
gaodean | a day ago
bearjaws | a day ago
Hopefully Qwen follows up their 3.8 launch with a new 35b-a3b
WalterGR | a day ago
hadlock | a day ago
NitpickLawyer | a day ago
While it looks "behind" the qwen equivalent model on most benchmarks, a few personal notes:
- nemotron models feel to me a bit less benchmaxxed / "stubborn". That means that they generalise a bit better, or can be tasked to solve similar but not quite identical task types to the training data (something that's hard to do w/ qwen/ds models)
- nemotron series are also open training (w/ open training recipes and some training data public)
- nvda will have an incentive to continue this kind of releases, even if other parties slowly abandon the open release of models. Whatever other incentives 3rd party labs have (i.e. meta, goog w/ gemma, the chinese labs that IPOd, etc) nvda will always want to sell hardware so their incentive to keep pushing open models is evident and will likely continue "forever".
[1] - https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-...
ggcr | a day ago
This! It's literally in their best interest for open-weights models to succeed
happycube | a day ago
[OP] beklein | a day ago
I actually copied the link from NVIDIA's Technical Blog post:
- https://developer.nvidia.com/blog/nvidia-nemotron-3-5-lightn...
You can also try the model via a free API endpoint from Openrouter, would be interesting to see if it's the BF16 or NVFP4 version:
- https://openrouter.ai/nvidia/nemotron-3.5-lightning:free
adrian_b | a day ago
https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-...
to be used for further training/fine-tuning.
https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-...
main model.
https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-...
quantized version of the previous.
https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-...
this "DFlash" model should be used together with one of the previous two "for lower-latency speculative decoding deployments tuned for low-concurrency data center and workstation workflows".
https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-...
like DFlash, the previous model above, but optimized for DGX Spark.
ggcr | a day ago
kamranjon | a day ago
lysace | a day ago
https://github.com/NVIDIA-NeMo/Nemotron/blob/main/docs/nemot...
hadlock | a day ago
If you're already selling shovels, you might as well provide maps to the gold mine with every purchase.
varispeed | a day ago
I find these releases are bad taste.
Make 1TB DGX priced affordably, not some crap model for people to waste time on.
tasty_freeze | a day ago
nicman23 | a day ago
jazzyjackson | a day ago
timmmmmmay | 19 hours ago
piyh | a day ago
judem | a day ago
e2e4 | a day ago
simonw | a day ago
I used LM Studio and this model file: https://lmstudio.ai/models/nvidia/nemotron-3.5-lightning
I got a pretty solid (and reasonably fast) result for this simply coding agent activity (with simonw/llm-coding-agent):
https://gist.github.com/simonw/a8741f79280cedc86bcb8d90edfb4... - took just under a minute.EchoVoicy | 20 hours ago
>Runs fast on my Mac What specifications (CPU/RAM)?
simonw | 18 hours ago
dofm | 16 hours ago
It writes pretty good code; it does seem to second-guess itself in thinking traces and I wonder if it just needs a reasoning budget and message.
Each time I test a model quickly in LM Studio, I ask it:
- to write a tiny little wordpress "last login" tracker plugin, asking me clarifying questions first. I ask it to use old PHP, break the WP coding guidelines to use inline anonymous functions in the hooks, avoid custom SQL, I see if it can write something useful in a singleton class, and what questions it thinks to ask. Qwen 3.6 does this very well, Nemotron has done OK, though it's a little less effective at reading between the lines, maybe.
- to offer an answer for a SQL puzzle about finding max score per category on old MySQL (5.0) without using subquery/derived tables -- it did a good job, picked up the nuances in the prompt that allow a particular solution, didn't go on a tangent about how it would be nice to have window functions or use subqueries, did a tool call to check like I asked. With this puzzle, if the model doesn't offer up an index for performance optimisation, I nudge it; this time it didn't volunteer one but when prompted about performance it offered an index and a bunch of other nice solutions, and only there did it round up the options for subqueries and derived tables, which is fair game.
(It did badly fail the car wash test, though, even on repeated nudging, where it gets more and more insane, doubling down and never getting the point, whereas Muse Glimmer solved it and well, with a thinking trace that didn't particularly suggest it had been post-trained)
I need to test it in Pi or opencode. I've been trying to motivate my brain to move to pi, but this model supports a longer context window so maybe opencode's overlong system prompt is less of an issue.
Of the 30B models in the last 24 hours (!) I think I prefer working with Muse Glimmer, which is slow but very good, and writes rather well with just a hint of being a bit of a cheeky monkey. Not tested that in an agentic coding setting yet.
Interesting times.
thisisauserid | 23 hours ago
bcatanzaro | 23 hours ago
Changing the SSM backbone is a major revision, not a dot release. At least for us.
dofm | 16 hours ago
Bad, bad failure and no amount of nudging helps it see the point. Comical.
But it did OK on other mini tests.
lostmsu | 15 hours ago
Still trying to lock in, huh?