This blog post has a similar feel to what I get when I ask Claude to test an idea for me, and then I don't interrogate the result at all. It's entirely uninteresting that you can use DeltaNet for memory (that's what it's designed for!) or do logic in the feed forward networks.
The author doesn't do anything interesting with the engineered weights. The CPU is implemented using a completely separate network, then the CPU and Qwen are combined with a tensor product (so we waste time doing all the Qwen computations in "CPU mode", only to throw them away, and vice versa for "Qwen mode"). Does Qwen get anything from being able to see halfway through the CPU logic? Nope, since it's totally separate (not that I would expect that to be useful)! Is it useful to take gradients through the CPU? I wouldn't think so, but the author doesn't try.
Since the model and CPU are communicating through normal tool calls, you could get the same result by using a normal RISC-V emulator (which you could also differentiate through, if you really thought it might be useful)!
This makes me want to see a NN that's just a CPU now! And then maybe FPGA-ized, but of course, I imagine it'll be much slower, since we're describing some circuitry in less-direct ways? Anyone know anything more?
xal | 8 hours ago
This blog post has a similar feel to what I get when I ask Claude to test an idea for me, and then I don't interrogate the result at all. It's entirely uninteresting that you can use DeltaNet for memory (that's what it's designed for!) or do logic in the feed forward networks.
The author doesn't do anything interesting with the engineered weights. The CPU is implemented using a completely separate network, then the CPU and Qwen are combined with a tensor product (so we waste time doing all the Qwen computations in "CPU mode", only to throw them away, and vice versa for "Qwen mode"). Does Qwen get anything from being able to see halfway through the CPU logic? Nope, since it's totally separate (not that I would expect that to be useful)! Is it useful to take gradients through the CPU? I wouldn't think so, but the author doesn't try.
Since the model and CPU are communicating through normal tool calls, you could get the same result by using a normal RISC-V emulator (which you could also differentiate through, if you really thought it might be useful)!
pervognsen | 2 hours ago
I can't wait for "Xortron: a fully functioning XOR gate inside a multilayer perceptron".
LenFalken | 16 hours ago
Wow, some real computer science!
This makes me want to see a NN that's just a CPU now! And then maybe FPGA-ized, but of course, I imagine it'll be much slower, since we're describing some circuitry in less-direct ways? Anyone know anything more?
datboi | 16 hours ago
As much as I generally hate reading about LLM's, this is really fun!
[OP] ndr | 16 hours ago
Agreed, I wonder if there's any better tag than
aihere.schuelermine | 6 hours ago
What exactly is the benefit over just using a tool call?