I find it very amusing that the current direction of LLM optimization aimed to deliver best intelligence per Watt inevitably leads to the models becoming even bigger fachidiots then us, humans. Despite them "knowing" multiple separate fields they are unable to make the connections since those are truly rare and thus not represented well in the training set.
Sometimes when you want something smoothed, dropping "Will this moving average on the plot actually suppress aliasing or is there still too much energy above the Nyquist frequency?" unlocks the DSP databank and the model stops blindly applying leaky integrator or moving average and instead does the math and asks how many dB do you want, about the cutoff and ringing.
I imagine there are prompts that unlock other less represented fields (as opposed to web development and UNIX utilities) that would make the models apply much better knowledge to the problems we are solving, but alas, unless the driver knows the correct shibboleet (sic) they are doomed.
I'm glad you enjoyed the read! I figured at least something had to come out of all this effort.
And yeah, I guess the point of the post is that it really sucked to figure out that even if you do know the correct shibboleth, or heard it in passing, it doesn't magically load all the knowledge from the databanks into your head Matrix-style, and that discrepancy between the amount of things you actually know, vs. the amount of apparent "knowledge" you're exposed to is really overwhelming, and I'm guessing it's definitely not a technical problem, i.e. we'd just have to get better at using LLMs and not overwhelming ourselves.
But we already know that software projects are mostly hard by the virtue of being first and foremost research projects. For everyone involved.
Even if we offload some of the tedium, there is still a learning curve associated with software for the end-users. We cannot seriously expect that there would not be none for us who are way closer to the minute decisions.
What's funny to me is that the model knows better than me about some theory that is directly applicable and would help immensely, but it just won't share it because most people it has been trained on were not generalists pointing such connections out.
Yeah, I was talking with a friend today about this post and LLM-usage in general. Using Claude/Codex etc. eliminates the tedium and shortens the time writing code a lot, but that time I used to spend struggling to write code was also time I was having to spend having to think through the architecture of what I was doing, and come up with the decisions and have to write the code to make the whole thing work.
I still struggle with saying the model "knows" anything. But I agree with you re: "no generalists to point the connections out". What sucks is that there's no actual learning taking place, so even if you do manage to point out the connections, unless you stored a memory, the next session just starts blank. Until the next version of Fable comes out, I guess. But even then, that doesn't match the velocity of how humans learn/adapt. That's of course a given, but I mean to talk about how experiencing that first-hand is still really the biggest and probably first anti-koolaid you get, after that initial excitement period of feeling like you can do anything with LLMs.
I think sometimes you can reword what you don't like, in isolated sessions, until the model leaks the shibboleth. Of course, that might not be enough when describing the versions of the problem is still an open research problem itself!
mordae | 12 hours ago
This was hilarious. Thanks for sharing.
I find it very amusing that the current direction of LLM optimization aimed to deliver best intelligence per Watt inevitably leads to the models becoming even bigger fachidiots then us, humans. Despite them "knowing" multiple separate fields they are unable to make the connections since those are truly rare and thus not represented well in the training set.
Sometimes when you want something smoothed, dropping "Will this moving average on the plot actually suppress aliasing or is there still too much energy above the Nyquist frequency?" unlocks the DSP databank and the model stops blindly applying leaky integrator or moving average and instead does the math and asks how many dB do you want, about the cutoff and ringing.
I imagine there are prompts that unlock other less represented fields (as opposed to web development and UNIX utilities) that would make the models apply much better knowledge to the problems we are solving, but alas, unless the driver knows the correct shibboleet (sic) they are doomed.
[OP] luiz | 7 hours ago
I'm glad you enjoyed the read! I figured at least something had to come out of all this effort.
And yeah, I guess the point of the post is that it really sucked to figure out that even if you do know the correct shibboleth, or heard it in passing, it doesn't magically load all the knowledge from the databanks into your head Matrix-style, and that discrepancy between the amount of things you actually know, vs. the amount of apparent "knowledge" you're exposed to is really overwhelming, and I'm guessing it's definitely not a technical problem, i.e. we'd just have to get better at using LLMs and not overwhelming ourselves.
mordae | 6 hours ago
But we already know that software projects are mostly hard by the virtue of being first and foremost research projects. For everyone involved.
Even if we offload some of the tedium, there is still a learning curve associated with software for the end-users. We cannot seriously expect that there would not be none for us who are way closer to the minute decisions.
What's funny to me is that the model knows better than me about some theory that is directly applicable and would help immensely, but it just won't share it because most people it has been trained on were not generalists pointing such connections out.
A true artificial ignorance.
[OP] luiz | 6 hours ago
Yeah, I was talking with a friend today about this post and LLM-usage in general. Using Claude/Codex etc. eliminates the tedium and shortens the time writing code a lot, but that time I used to spend struggling to write code was also time I was having to spend having to think through the architecture of what I was doing, and come up with the decisions and have to write the code to make the whole thing work.
I still struggle with saying the model "knows" anything. But I agree with you re: "no generalists to point the connections out". What sucks is that there's no actual learning taking place, so even if you do manage to point out the connections, unless you stored a memory, the next session just starts blank. Until the next version of Fable comes out, I guess. But even then, that doesn't match the velocity of how humans learn/adapt. That's of course a given, but I mean to talk about how experiencing that first-hand is still really the biggest and probably first anti-koolaid you get, after that initial excitement period of feeling like you can do anything with LLMs.
k749gtnc9l3w | 9 hours ago
I think sometimes you can reword what you don't like, in isolated sessions, until the model leaks the shibboleth. Of course, that might not be enough when describing the versions of the problem is still an open research problem itself!