It is interesting that the Kimi model gets a very high “slop score” but “professional critiques” gave it a pretty high score. Do critiques have no taste or the “slop” is not slop anymore?
If slop score is mostly “looks like AI”, Kimi scoring high there while human designers rate it well isn’t that weird. Generic can still nail the brief.
I’d also guess that, considering their be massive approach to sourcing training data, it would make sense that it’s much less sloppified than anything before it
I don't understand, where are those models available in stealth mode? Usually people do this on openrouter but I couldn't find any model called "gestalt". Does anyone know or is this just black hat marketing for this contralabs platform, and the model does not even exist?
To clarify, it is not two models. Its a single model (Gestalt). The other bar on the benchmark is a human created dataset. Suffice to say we didn't expect anyone to beat it, but here we are. And no, it is not Contra model.
The name is probably a clue. I wonder if someone just bolted Fable to Astra in a loop? I do that when I'm stuck, and get great results that exceed what either can do alone... Though it is wickedly expensive.
Usually I do it when one of them is stumped on their own, or seems to be thrashing. I guess it's more like... adjacent to the main loop.
I'll spin up both TUI's in herdr, tell them both to run "hc --skill" and then talk to each other until they figure out a solution. Also, for really complex tasks I will give one of them the title "Advisor" and just leave it running in the background while the main agent runs. The main agent gets instructions to ask the Advisor when stuck. So far it's basically always worked.
Before herdr I had them use tmux and files to talk to each other. That works too, but with a little more fumbling.
I normally wouldn't even bother commenting on an insufferable site (just close the tab and move on), but this was so bad that I came to the comments looking for validation of my feelings.
Like, above the fold is a bunch of moving links to AI-generated pages that have apparently nothing to do with the page title. Perhaps it's relevant, but I find it non-obvious. The first one I clicked gave a bad gateway.
Then we get the leaderboard, then just more AI-generated projects with opaquely dense descriptions like "Real client briefs, each with its golden: the design set beside the models’ outputs as the bar.".
Just reading this makes me feel like I'm having a stroke.
interesting! any model performing better than human created outputs is very unexpected. i wonder how fast these models are gonna get better, thinking about the progress we made in a year on coding, design would be soon too
It is possible that anthropic/open ai may no longer release their best models due to their massive size and cost of running, instead releasing distilled version for public use.
Google/Gemini just went to a Flash/Standard/Pro model where Standard is $20 a month and Pro is $100 a month. I would surmise there's no way their top model can be given out for that price.
_s_a_m_ | 7 hours ago
skinfaxi | 6 hours ago
jrflo | 5 hours ago
cronin101 | 6 hours ago
[OP] cheptsov | 6 hours ago
freecodeio | 6 hours ago
akutlay | 6 hours ago
JMKH42 | 6 hours ago
danielrearden | 5 hours ago
alex7o | 6 hours ago
tmikaeld | 2 hours ago
barrenko | 6 hours ago
irthomasthomas | 6 hours ago
gorgmah | 6 hours ago
nolroz | 15 minutes ago
petesergeant | 6 hours ago
gajus | 6 hours ago
curiousgeorge10 | 5 hours ago
alfiedotwtf | 5 hours ago
mapontosevenths | 5 hours ago
metalliqaz | 3 hours ago
mapontosevenths | 3 hours ago
I'll spin up both TUI's in herdr, tell them both to run "hc --skill" and then talk to each other until they figure out a solution. Also, for really complex tasks I will give one of them the title "Advisor" and just leave it running in the background while the main agent runs. The main agent gets instructions to ask the Advisor when stuck. So far it's basically always worked.
Before herdr I had them use tmux and files to talk to each other. That works too, but with a little more fumbling.
TheAtomic | 5 hours ago
RunSet | 5 hours ago
I closed it before it could finish loading because it was so slow. Which does speak to its insufferable nature.
bwood | 3 hours ago
Like, above the fold is a bunch of moving links to AI-generated pages that have apparently nothing to do with the page title. Perhaps it's relevant, but I find it non-obvious. The first one I clicked gave a bad gateway.
Then we get the leaderboard, then just more AI-generated projects with opaquely dense descriptions like "Real client briefs, each with its golden: the design set beside the models’ outputs as the bar.".
Just reading this makes me feel like I'm having a stroke.
</rant>
ForHackernews | 5 hours ago
vonneumannstan | 5 hours ago
radzserg | 5 hours ago
lyw6 | 4 hours ago
barbacoa | 3 hours ago
cyanydeez | 2 hours ago
1. price discrimination.
2. inference discrimination
3. national security extortion racket.
basch | 2 hours ago
algoth1 | 3 hours ago