GIMP now has UI tests

156 points by qtqtqt 20 hours ago on hackernews | 89 comments

rafram | 19 hours ago

> We have 13 tests at this time.

Does GIMP have a very restrictive AI policy or something? I get not wanting to ship a gigantic pile of UI tests without thorough review, but basically any model released in the last six months would be able to write UI tests covering every single control in an afternoon.

samuria | 19 hours ago

It’s frustrating to create tests with models, not sure if it’s everyone’s experience, but every time I asked it to create tests, it makes the tests conform to the sometimes broken code. Rather than calling out what the logic/code is wrong, it will make the tests conform to whatever is already broken.

Waterluvian | 19 hours ago

It can’t magically know expectations. If the code is well documented or there’s decent associated documentation, I’ve found it has no issue writing tests.

t43562 | 15 hours ago

Tests are usually the best documentation and often the only documentation. If humans bother to write anything it should be the tests, not ambiguous human language...beyond the high level 1-liner overviews.

alyandon | 7 hours ago

I'm actually surprised at how well non-frontier/local models are able to infer the intent of code from the structure, method names and sparse random comments to the point that they are capable of writing reasonable tests.

forgotmypw17 | 19 hours ago

I recommend iterating on your prompts.

Prod your agent first until it can explain back to you what's wrong with the tests.

Then ask your agent to write a better prompt for writing tests (and save it to an .md file)

Then reference that .md file and iterate on the tests again. It may take a couple of tries, but I've had good results with this strategy.

Ended up with a whole prompting framework.

skydhash | 18 hours ago

Prodding an agent vs doing interesting things with my time? What should I choose?

To be fair, some people like to prod. But, for me, I'd rather take hours reading manuals than do trial and error prodding for 30 minutes (even if I knew it would be that short). Yes it's not optimal in terms of time, but it's better for my soul and well being.

0xcoffee | 18 hours ago

It's a good use of your time to ask the agent to reflect and update it's skills if you intervene to manually steer it. Why wouldn't you want it to learn from your guidance?

thisismyanon2 | 16 hours ago

Perhaps because they're more interested in learning something themselves.

HiPhish | 17 hours ago

> Yes it's not optimal in terms of time, but it's better for my soul and well being.

It is also optimal in terms of time in the long run. The prodder will have to prod over and over again because he has learned nothing. You on the other hand will actually understand what is going on and will be able to either do it yourself or give precise instructions to the machine next time. Don't let yourself become a meat proxy.

doginasuit | 18 hours ago

The problem I've had is even when it can explain back the issue, it still writes for itself awful instructions with extra fluff, strange points of focus, and borderline misinterpretations of what I said.

Maybe that's not a big deal, they can still be generally useful, but they seem important to commit with the project if they will be used going forward. I hate the idea of a massive amount of slop riding along with the code, and slop begets slop.

The idea I'm chasing now is writing the documents myself with the hope that enough examples and a document about writing documents will be enough to allow it to write clearly.

hulitu | 18 hours ago

> Rather than calling out what the logic/code is wrong, it will make the tests conform to whatever is already broken.

It is the same like humans: if they say your code is wrong, you will hire someone else who doesn't find so many bugs. It doesn't look good.

alerighi | 15 hours ago

True. It's tough useful to build non-regression tests, that is build tests that test the current behavior, assuming it correct, to possibly spot bugs in the future.

deskglass | 19 hours ago

einichi | 18 hours ago

> for contributors without Developer role

lukeify | 17 hours ago

From the blame PR:

> Of course, core developers don't need to check this monotomous checklist. This is only useful for the newcomers.

So this is saying core devs already know not to use AI, only new developers shouldn't.

It's quaint at this point. Embrace the future.

Xirdus | 17 hours ago

The future seems to be blocking external contributors and running the equivalent AI agents yourself.

desterothx | 15 hours ago

Yeah, i'm sure they want to burn their own tokens, deny others to use AI, to hog all that foss money for themselves. Definitely not related to not wanting to review 10k ai generated prs no siree

cyanydeez | 10 hours ago

thing is, we hav emuseums, well crafted historical relics.

Having projects with properly scaffolded code, et al, will be valuable when the FAA grounds all planes for a week because of <insert all the grusome things you know this administration turns a blind eye to>.

Xirdus | 3 hours ago

It absolutely is related to not wanting to review 10k AI-generated PRs. Those PRs are definitely going to be lower quality and more likely to contain backdoors than ones produced by your own AI. The back-and-forth during the review is also going to be much slower than the equivalent back-and-forth in the chat window.

If money/tokens are not an issue, never accepting external AI-made PRs makes perfect sense.

6wf9j | 9 hours ago

Weird, the author of the MR doesn't have a Developer role, he is a Contributor, and yet his commits look very much AI generated, look at this one for instance https://gitlab.gnome.org/GNOME/gimp/-/merge_requests/2909/di...

Or is it just me?

stratos123 | 15 hours ago

Huh, that's fascinating. I wonder if this policy is also meant to apply to the GIMP blog - if so, this very article is probably a violation.

jasonvorhe | 11 hours ago

RIP GIMP, Asahi, Codeberg et al

locusofself | 18 hours ago

Not using AI to at least writes tests in this day and age is crazy work. I started a desktop application 2 months ago and it has almost 3,000 tests now, MANY of them UI tests.

timcambrant | 18 hours ago

AI is a bit too excellent at writing tests. The number of tests doesn't really convey any information on test coverage. At least I've found that agents that naively improve codebases will write tests for pretty much anything, including irrelevant behavior that has little value in ensuring stability, security or correct behavior. It would probably help to run a test review stage with a strong model, where agents attempt to correct and consolidate tests to guard behavior rather than ensuring that theoretical code paths are followed.

skydhash | 18 hours ago

In my experience, designing tests cases to ensure that they're othogonal and verify your invariants (domain or technical) is very mentally taxing and not something you can truly delegate. The actual typing maybe, but testing is a lot of duplicate code and you can go far with just copy-pasting stuff around and building helpers.

aidenialislife | 18 hours ago

Designing for testing helps as well. Pure functions for core logic, coding against interfaces etc.

npteljes | 12 hours ago

I my opinion, AI only changes the ease of writing these tests, not really the quality.

I find that the human written tests on my project were the same. We were guarding not just the business behavior but also weird cases that have nothing to do with business, and also self-evident cases that basically boil down to x=x or assert(true).

Not to mention the stupid side-effect of not updating mocks when the APIs move but tests don't test that part so they are okay for a lot of time, and suddenly after a non-related change they are not.

locusofself | 6 hours ago

I agree, some of my small projects have had an explosion of tests. It usually doesn't cause my any problems other than waiting for tests to run, but that isn't great.

ares623 | 18 hours ago

Look, guys, you are literally allowed to fork and go ham on the code.

mepiethree | 17 hours ago

Yeah if people can make a better and more popular Gimp in an afternoon, go do it.

Gimp has millions of downloads this month. They are doing something right.

HiPhish | 17 hours ago

Using AI to write tests in this day and age is crazy work. The machine is great at writing tests, but the quality of the tests is awful. It will write redundant tests, it will write tests that assert too many thing in one, it will not write tests that need to be tested. It's the part where I have to clean up the most every time. Quantity is not quality, and you really don't want to skip on the quality of your quality assurance. I'm writing a React + Django full-stack application, so there should be more than enough training data out there.

byronturncoat | 10 hours ago

Some people refuse to use AI for ethical, environmental, or social reasons. It isn't crazy to want your projects to be built by humans

hulitu | 18 hours ago

> any model released in the last six months would be able to write UI tests covering every single control in an afternoon.

Microsoft also seems to use "models" to "test" its programs. Look how shitty they are.

rafram | 6 hours ago

Why "models" in quotation marks?

nomilk | 18 hours ago

This raises questions of AI ethics; namely, is it conscionable to subject AI to GIMP’s UI?

masfuerte | 7 hours ago

Ha! We'll know that LLMs are truly intelligent when they start to despair at the moronic conversations we force them to have.

joegibbs | 18 hours ago

They made a blog post to announce that they’d written thirteen unit tests? Sure, there’s value in handmade software and all that but GIMP is not really a particularly excellent piece of software - if anything was to benefit from unleashing the agents it would be GIMP.

Jolter | 18 hours ago

I’d go so far as to say that in general, adding tests for legacy software before a refactor or rewrite is the perfect time to put the agent to work.

hulitu | 18 hours ago

> if anything was to benefit from unleashing the agents

AI can write tests now ? For its own software ? Can it evaluate the results ? Correctly ? /s

aidenialislife | 18 hours ago

Writing tests never was the hard part of the job anyway.

t43562 | 15 hours ago

Lucky you. Making things testable seemed hard to me quite often and ensuring that the tests covered all the difficult bits without effectively mocking it to death so that one was testing almost nothing.

desterothx | 15 hours ago

Yup especially if you don't care if they actually show correctness or not

altmanaltman | 18 hours ago

The blog article doesn't say they have written thirteen unit tests.

> *GIMP already has unit tests*. A unit test checks one small part of the code, for example how GIMP reads an XCF file. But our unit tests were not opening a real window. So it coult not tell us if a dialog opens, or if a menu item did its job.

> Note that GIMP *already had tests of GUI code*, usually interacting directly with the core code, and sometimes even simulating input devices’ events (pointer movements, clicks, drags…). But nothing can beat interacting with the real interface. Furthermore such legacy tests directly hijacking core code come with their own problems, such as breaking all too often (and not because they detected a bug), all while not being able to detect whole categories of bugs.

> UI tests check the program as a whole, in the same way as you use it.

[OP] qtqtqt | 18 hours ago

They must have set up a reusable harness, and it's the most important part.

WD-42 | 18 hours ago

Did you read the article? These aren’t unit tests. They are UI tests which are pretty hard to do well with desktop apps.

[OP] qtqtqt | 18 hours ago

Especially given it isn't as easy as using Cypress.

slopinthebag | 18 hours ago

we forget, but before AI this would have been at least 6 months of work, at least 50 meetings, and at least 7 separate sprints.

raincole | 18 hours ago

It'd be really interesting to look back the comparison between projects with strict non-AI policies (e.g. GIMP) and ArtCraft in 2027.

My personal prediction is that ArtCraft will blow all the old non-AI projects in similar fields out of the water and GIMP will be remembered as a thing of the past. But if I got proven wrong I'd be happy too: at least we will know the value of non-AI policies.

flexagoon | 16 hours ago

GIMP is far from the best FOSS image editor for many use cases. IMO for many simple editing tasks that people usually need unless they're professional photo editors, even Inkscape, a primarily vector editor, is a better photo editor than GIMP.

raincole | 16 hours ago

Inkscape is what made me believe ArtCraft would be the clear winner, btw. It's very hard to imagine using Inkscape ever again since I tried VectorCraft. (And I'm fully aware that some Path Effects can only be done quickly in Inkscape.)

t43562 | 15 hours ago

What is the point of image editors? A fleet of AI artists should be doing it.

plant-ian | 17 hours ago

It sounds like they are nearly end to end user interface (UI) tests, not just unit tests.
Funny how GIMP spawned the Gimp Toolkit, aka GTK, which is widely used in Linux, and yet GIMP's own UI seems to have long languished as a rather outdated design. Glad that they have tests, though, and hope they continue to improve it.

alexey-salmin | 18 hours ago

Ah! I always thought it was Gnome Toolkit, didn't even bother to check. Thanks

veltas | 17 hours ago

It's GNOME Toolkit now, you're not wrong, but originally was GIMP.

yjftsjthsd-h | 17 hours ago

Well it is, regardless of the name (I think "GTK" officially doesn't stand for anything now); to quote https://www.gtk.org/ -

> GTK is a free and open-source project maintained by GNOME and an active community of contributors.

and

> GTK and the GTK logo are trademarks of the GNOME Foundation.

YuechenLi | 18 hours ago

Oh, haven't heard from GIMP in a long time, has there been significant UX improvement and have they carved out a niche like Krita yet?

ModernTimes | 18 hours ago

Timely, as I removed Gimp from my Macbook today after successfully testing PhotoCraft [1] for a week (the AI Photoshop clone written in open-source rust which has had an absured number of updates since you last tried it).

It makes me so HAPPY to remove the dinosaur Gimp, and to run "Adobe" software for free (it's good enough for me, maybe good enough for you). I expect to lose Internet points for saying that.

[1] https://getartcraft.com/apps/photocraft

Aozora7 | 18 hours ago

Regular users would use a lot more open source software if more of it was like photocraft and less of it was like GIMP.

plasticbugs | 18 hours ago

I have a long history with Gimp. I created Gimpshop... the Gimp maintainers did not appreciate that little stunt, I'll tell ya! :D

I have canceled my Adobe subscription as of last month and have fully moved over to Photocraft. Even if development of Photocraft stopped today, I would be able to do whatever I need to get done in Photoshop without paying Adobe or wrestling with Gimp. It's good enough for me as well. I had been using Adobe Photoshop for more than 30 years now and was bought in enough to spend a significant amount of effort making Gimp act like Photoshop. Bye Photoshop, bye Gimp, bye Gimpshop.

hackshack | 17 hours ago

Same with VectorCraft. I spent over a decade looking for an Illustrator alternative. Muscle memory and specific workflows won out. VectorCraft is extremely close.

TeddyDD | 15 hours ago

This and also and Affinity announced Linux version. Hopefully I'll never have to use GIMP again. The UI is soooo bad.

fuussssip | 18 hours ago

GIMP... didn't have ui tests before?

I'm surprised, tbh. I understand that UI is very much an afterthought in GIMP, but I figured it'd still at least have tests.

slopnt | 6 hours ago

afterthought? they literally invented the most popular GUI toolkit used by most linux apps (GTK)

aquafox | 18 hours ago

GIMP doesn't need UI tests, but a more intuitive UI.

fishgoesblub | 17 hours ago

Great on GIMP for continuing on and attempting to improve things, you'd think the GIMP team killed people's dogs with some of these comments..

bigstrat2003 | 17 hours ago

AI bros can't stand the thought that not everyone agrees with them about LLMs. It's like a religion to them, they feel compelled to get everyone else on board. Never mind that if they are right, it will become apparent to everyone without them having to preach.

Aozora7 | 16 hours ago

It has to be a religion, it can't just be that I prefer the quality of AI-assisted software over projects with no-AI policies.

Besides, GIMP has always been unpopular long before AI.

thisismyanon2 | 15 hours ago

When a group of people relentlessly insist on hijacking any discussion to refocus the topic onto something unrelated but more interesting to them it stops becoming a preference and starts looking more like bullying at the hands of a religion or cult. For instance:

Hey we added UI tests to GIMP

AI bros: YOU SHOULD HAVE USED AAAAAAAAAIIIIIIIIII

This is more complex than a few unit tests

AI bros: YOU SHOULD HAVE USED AAAAAAAAAIIIIIIIIII

Can we please discuss GIMP?

AI bros: STOP PERSECUTING US

Aozora7 | 15 hours ago

Yeah, sure, discussing software development process and tools in a topic about testing is "religious bullying".

thisismyanon2 | 15 hours ago

So why do you need to pivot every discussion about "software development process" to AI? Do you believe that the GIMP authors have not yet heard the good word? Do you think they need to be saved from their primitive non-AI practices?

There's a difference between discussion and proselytizing. And, yes, that rises to the level of bullying when AI bros relentlessly drag every discussion back to "WHY U NO AI???". It's absolutely bullying when you make it impossible to discuss non-AI related subjects.

npteljes | 12 hours ago

Yeah they always get the flak, even though it's not like people HAVE to use Gimp.

I also hated the UI, coming from Photoshop, but what I found is that when I go back to Photoshop, I'm just as lost as I was before switching to Gimp.

I'm glad that they hadn't lost their courage over the years. Gimp is a solid project, and the FOSS ecosystem would lose a real gem if they stopped.

WD-42 | 8 hours ago

Seriously. This comment thread is why I’m spending less time on HN and more on lobsters. All these fresh accounts losing their minds over Gimp not using ai, even when that’s not what the article is even about.

snissn | 17 hours ago

"GIMP now has UI" would be big news

iLoveOncall | 16 hours ago

The tests should simply take a screenshot and send it to an LLM with the question "Is the UI absolutely horrible?". If the answer is yes then the UI is working as expected.

zkmon | 16 hours ago

> So it coult not tell us if a dialog opens,

It feels good when I see spelling mistakes. It feels organic, human-sourced, non-AI. It feels like a good old village again.

BTW I love GIMP for everything it did to the world of graphics work. Such an awesome thing. Thank you folks.

KingOfCoders | 16 hours ago

I created all the designs for flyers, announcements, invitations etc. in Affinity for my wife. I migrated to Linux and didn't like any of the apps. I was lost.

Then Affinity announced they'll add Linux support.

But for some time now my wife just uses Codex/Claude to do her day-to-day nothing special design work.

I feel, just like ArtCraft, Gimp will not be relevant in some years (months?).

bayindirh | 16 hours ago

For some people GIMP is irrelevant for the last ~25 years. I was using it in 2000, I'm still using it in 2026.

It's a matter of getting used to things. I feel lost in Affinity tools, for example, despite having a license, I use them less and less.

JamesLeonis | 9 hours ago

Are you me?! I've also happily used GIMP for that long, just swap out Affinity for Paint Shop Pro or Adobe CS. I would probably feel just as lost going back to them now.
My son learned GIMP in school and refuses to learn anything else.

cissikatt | 9 hours ago

Maybe he will wise up with age

inbalboa | 16 hours ago

You switched to Linux, but you don't want to use Linux things. Why not just stick with Windows (or whatever you were using before)?

KingOfCoders | 12 hours ago

Because I do mainly other things for which Linux is suited better.

skrebbel | 16 hours ago

There are people who want to use Gimp (or Affinity etc), and there are people who want to have used Gimp. It’ll only lose relevance for the second group.

mamouri | 15 hours ago

Use this: https://getartcraft.com/ It is open source, built with Rust and very fast and works great and is very similar to Adobe Photoshop

gauntr | 15 hours ago

Tell me you didn't read the whole comment you replied to without telling me...

grim_io | 14 hours ago

Tests are a good first step, here's hoping it'll get a UI next :)

stuaxo | 8 hours ago

This is really good.

So much of this tech like ui tests was built for the web, and it can be a pain to get working for plain old GUI apps