I was so incredibly tired of trying to get an actual "usable" result through a generative image model. Its always impressive results, but the fact that they're just pixel blobs, makes them entirely unpractical.
I have no idea how an agent would leave a watermark in Klad. It's got no agent inside, and every mark that an agent made would have possibly been made by a human as well. I guess they would have to really specifically target Klad to be able to make a watermark happen...
Nope, you can output an image file. Right now png, but other formats are easy to add.
The underlying engine is not just vectors: a brush stroke is saved as data, and everything is rendered on the fly into pixels. This means that things like smudging and paint blending is possible, all the while being able to edit even the very first stroke you made.
The best mental model I have right now is: it's the editibility of vectors, with the flexibility of a digital paint brush
Hmm this is interesting, perhaps you could pair it with image gen and add code to seperate the layers instead of individual brush strokes. Could be very token costly. Will try though
I have not done any benchmarks unfortunately. It really kind of depends on how complex a drawing is.
For example, but rough numbers here,.. the panel poster on the website took about 15 minutes of fable working it out, and the full scene with the creature following a traveler, was also about 10 minutes of ai work.
To me, it feels just about as fast as having a model code up a medium feature for an application. And that doesn't seem too strange, because I'm the end coding is kind of what it's doing with this app.
A benchmark I'd be really interested in, is how an agent would be able to clean up or improve a drawing made with pen tablet by a real artist. I've been focusing so much on the development side though that this is a test I have not been able to do, but I expect it to work really well. Every time I make an edit to a drawing myself, Claude just knows what I did and why.
I have been meaning to find/build a tool that would sit in between the pre-production assets of a movie clip (script, characters, locations, prop assets) and the moment you press generate.
I envisioned it as some sort of a flipbook that would allow me to *cheaply* get a low-res idea of what the 'generate' click would get me, without having to burn movie making tokens.
This may not be exactly it, but it nudged me into a direction. Thanks!
So I've done a little bit of testing. When I loaded up the editor, I expected it to communicate with the installed binaries for my large language models, or possibly use a BYOK (bring your own key) approach. But once I understood that it's driven from the command line and you see the changes happen in the editor that gets spun up, I had no problems.
I've been experimenting a fair amount with LLMs driving procedural graphics myself, although my experiments have been mostly limited to relatively low-resolution stuff, such as EGA adventure game graphics.
So I decided to put Klad through a relatively hard test: I took a reference picture of an illustrated squirrel and asked it to recreate it, then set up the rigging for a basic walk cycle. This was using Fable 5.2 high.
Then I did the same thing, but with a more traditional approach, using standard generative models to manually create a walk cycle. This was all done relatively quickly, so bear that in mind.
Here are the results. I still think these are early days, and LLM-driven graphics are only going to improve.
And there are obvious advantages to a procedural system with rigging: Much easier to maintain consistency, palette control, and the ability to make precise alterations without having to deal with any kind of conventional in-painting nonsense.
Hey thanks so much for putting Klad and Fable through this test!
I think the results are really interesting. I think it's very obvious the classic image-to-video approach yields much more natural results at this time and with the current models.
I think I'm going to do some testing of my own, and see where things can be improved. I do think that the bottleneck at this time, is the model you use itself.
But it's like you say, it's very early days with an LLM expressing its ability to draw/paint. In my own testing, it kind of feels like it did with coding 2 years ago: yeah sure the model is pretty Okay at building a function, but I wouldn't have let it build a whole feature back then.
Nowadays, I'm practically only reading code and thinking about how to refine my instructions for the next build.
I'm sort of betting that this evolution could go the same way for graphics (and possibly audio/midi as well). And I wanted to have a tool ready, to see how it evolves.
> I do think that the bottleneck at this time, is the model you use itself.
I definitely agree. For me, using an LLM to drive drawing tools for procedural graphics only really became viable in the last year with the advent of open-weight model Qwen3.8-27B and proprietary models like Opus 5x.
If you’re interested, I built something in a similar vein: a custom drawing language that renders old adventure game-style scenes and is also driven by an LLM.
It produces relatively decent results because the domain is inherently constrained to an EGA color palette and a low resolution designed to approximate old adventure games from the 1980s.
And like your tool - half the fun is watching the LLM drive the production of the final image in real time!
No browser version unfortunately. It's a pretty heavy thing to do to render thousands of brush strokes and keep them all editable. So it is written in rust, with highly optimized memory management. I don't think a browser app could deliver the necessary performance.
I am still waiting on apple for signing a build, but once I can I will release binaries as well. There's an email list you can sign up on the site, if you're interested to be notified. Once the binary is out, an MCP will be as well. Right now it's a cli and a UI app you can build yourself.
But if you want to try a headless version, just point your favorite LLM to it and ask it to build it in a sandbox
No problem, it's easy to miss. But the underlying architecture here is each brush stroke is represented as data, like a vector in SVG, but rendered as a pixel brush. That means you can also do other things like smudges, paint blending, etc... Which you can't do with SVG.
The other thing is that this is also a first class drawing app with pen support. So you don't have to use an agent, and you can still profit from the editability. Or start a drawing yourself and have an agent come in later..
I like photography, but I'm relatively poor on the photo editing. I was hoping to build an MCP server around Gimp/Darktable/RawTherapee. The idea would be that the user would describe changes he wanted to see in the image (brighten up the shirt), and the LLM would pick appropriate tools to select the region and make the edit.
At the time I looked into it, it didn't seem that Darktable/RawTherapee provided APIs to hook into. Then life got busy.
Yup, that sounds about like the same underlying idea.
I think this is much much more interesting compared to generative ai for images/photos/music/... I'm hoping more and more people will come up with alternatives to traditional software where an agent can actually really help you out, but where you stay in control
No, not yet. But maybe it's something that could be a fit for klad in the future. I think it's a whole thing though.. gotta think it through but I see no reason as to why the klad file format could not support an extra dimension as well... I'll think about it!
Klas does not contain a model itself. You steer it with your own LLM. The source code is licensed as source available, meaning you are able to build it from source and use it for your own projects. Commercial use is allowed for smaller companies up till 500k revenue in the past year.
Don't all of the ArtCraft creative apps (the Adobe clones) have first-class support for AI agents? That could be more feature-complete than rolling your own editor.
They have more features, yes, but the architecture of klad is different from the ground up.
I explicitly designed the klad file format to be git compatible, which matters for agents that can easily track changes, but it is also a lightweight format that describes every brush stroke in a drawing rather than recording pixels.
The latter means every single one of the strokes an agent or you make, remains editable even thousands in. You can move every one, change the colour, transform it like a vector... And all the pixel rendering replays for the chunks it was in, in pretty much real time.
This then enables much deeper control for any changes you or an llm want to make, and I think this fundamentally makes klad positioned much better for agents drawing than any existing adobe clone.
I could be wrong about this, of course. Time will tell. For now I have received positive reports from people especially about the editability and consistency that comes with this format
chaostheory | 20 hours ago
This is a really great tool that solves a specific pain point with image models though
[OP] ambewas | 19 hours ago
I have no idea how an agent would leave a watermark in Klad. It's got no agent inside, and every mark that an agent made would have possibly been made by a human as well. I guess they would have to really specifically target Klad to be able to make a watermark happen...
geooff_ | 19 hours ago
[OP] ambewas | 19 hours ago
tamimio | 19 hours ago
[OP] ambewas | 19 hours ago
The underlying engine is not just vectors: a brush stroke is saved as data, and everything is rendered on the fly into pixels. This means that things like smudging and paint blending is possible, all the while being able to edit even the very first stroke you made.
The best mental model I have right now is: it's the editibility of vectors, with the flexibility of a digital paint brush
arjgoel30 | 19 hours ago
[OP] ambewas | 19 hours ago
If you try, definitely report back! I'm still figuring out what the best way is for an agent to use this tool
rchrp | 19 hours ago
[OP] ambewas | 19 hours ago
For example, but rough numbers here,.. the panel poster on the website took about 15 minutes of fable working it out, and the full scene with the creature following a traveler, was also about 10 minutes of ai work.
To me, it feels just about as fast as having a model code up a medium feature for an application. And that doesn't seem too strange, because I'm the end coding is kind of what it's doing with this app.
A benchmark I'd be really interested in, is how an agent would be able to clean up or improve a drawing made with pen tablet by a real artist. I've been focusing so much on the development side though that this is a test I have not been able to do, but I expect it to work really well. Every time I make an edit to a drawing myself, Claude just knows what I did and why.
mmvvaa | 18 hours ago
I envisioned it as some sort of a flipbook that would allow me to *cheaply* get a low-res idea of what the 'generate' click would get me, without having to burn movie making tokens.
This may not be exactly it, but it nudged me into a direction. Thanks!
I will give it a try and report back.
[OP] ambewas | 18 hours ago
If there's anything I can add, do let me know.
vunderba | 18 hours ago
So I've done a little bit of testing. When I loaded up the editor, I expected it to communicate with the installed binaries for my large language models, or possibly use a BYOK (bring your own key) approach. But once I understood that it's driven from the command line and you see the changes happen in the editor that gets spun up, I had no problems.
I've been experimenting a fair amount with LLMs driving procedural graphics myself, although my experiments have been mostly limited to relatively low-resolution stuff, such as EGA adventure game graphics.
So I decided to put Klad through a relatively hard test: I took a reference picture of an illustrated squirrel and asked it to recreate it, then set up the rigging for a basic walk cycle. This was using Fable 5.2 high.
Then I did the same thing, but with a more traditional approach, using standard generative models to manually create a walk cycle. This was all done relatively quickly, so bear that in mind.
Here are the results. I still think these are early days, and LLM-driven graphics are only going to improve.
And there are obvious advantages to a procedural system with rigging: Much easier to maintain consistency, palette control, and the ability to make precise alterations without having to deal with any kind of conventional in-painting nonsense.
https://imgpb.com/vJFpP
[OP] ambewas | 18 hours ago
I think the results are really interesting. I think it's very obvious the classic image-to-video approach yields much more natural results at this time and with the current models.
I think I'm going to do some testing of my own, and see where things can be improved. I do think that the bottleneck at this time, is the model you use itself.
But it's like you say, it's very early days with an LLM expressing its ability to draw/paint. In my own testing, it kind of feels like it did with coding 2 years ago: yeah sure the model is pretty Okay at building a function, but I wouldn't have let it build a whole feature back then.
Nowadays, I'm practically only reading code and thinking about how to refine my instructions for the next build.
I'm sort of betting that this evolution could go the same way for graphics (and possibly audio/midi as well). And I wanted to have a tool ready, to see how it evolves.
vunderba | 12 hours ago
I definitely agree. For me, using an LLM to drive drawing tools for procedural graphics only really became viable in the last year with the advent of open-weight model Qwen3.8-27B and proprietary models like Opus 5x.
If you’re interested, I built something in a similar vein: a custom drawing language that renders old adventure game-style scenes and is also driven by an LLM.
It produces relatively decent results because the domain is inherently constrained to an EGA color palette and a low resolution designed to approximate old adventure games from the 1980s.
And like your tool - half the fun is watching the LLM drive the production of the final image in real time!
https://github.com/scpedicini/agi-stylist
https://kq-styles.specr.net
hvvivek | 18 hours ago
[OP] ambewas | 18 hours ago
I am still waiting on apple for signing a build, but once I can I will release binaries as well. There's an email list you can sign up on the site, if you're interested to be notified. Once the binary is out, an MCP will be as well. Right now it's a cli and a UI app you can build yourself.
But if you want to try a headless version, just point your favorite LLM to it and ask it to build it in a sandbox
theanonymousone | 18 hours ago
[OP] ambewas | 18 hours ago
The other thing is that this is also a first class drawing app with pen support. So you don't have to use an agent, and you can still profit from the editability. Or start a drawing yourself and have an agent come in later..
ImJasonH | 18 hours ago
https://github.com/imjasonh/playground/pull/715
[OP] ambewas | 17 hours ago
BeetleB | 18 hours ago
I like photography, but I'm relatively poor on the photo editing. I was hoping to build an MCP server around Gimp/Darktable/RawTherapee. The idea would be that the user would describe changes he wanted to see in the image (brighten up the shirt), and the LLM would pick appropriate tools to select the region and make the edit.
At the time I looked into it, it didn't seem that Darktable/RawTherapee provided APIs to hook into. Then life got busy.
[OP] ambewas | 17 hours ago
I think this is much much more interesting compared to generative ai for images/photos/music/... I'm hoping more and more people will come up with alternatives to traditional software where an agent can actually really help you out, but where you stay in control
ramigb | 17 hours ago
[OP] ambewas | 17 hours ago
pierreb-aiva | 17 hours ago
[OP] ambewas | 17 hours ago
tescreal | 15 hours ago
[OP] ambewas | 15 hours ago
Should all be there in a license file on GitHub!
2001zhaozhao | 9 hours ago
[OP] ambewas | 6 hours ago
I explicitly designed the klad file format to be git compatible, which matters for agents that can easily track changes, but it is also a lightweight format that describes every brush stroke in a drawing rather than recording pixels.
The latter means every single one of the strokes an agent or you make, remains editable even thousands in. You can move every one, change the colour, transform it like a vector... And all the pixel rendering replays for the chunks it was in, in pretty much real time.
This then enables much deeper control for any changes you or an llm want to make, and I think this fundamentally makes klad positioned much better for agents drawing than any existing adobe clone.
I could be wrong about this, of course. Time will tell. For now I have received positive reports from people especially about the editability and consistency that comes with this format