Yadda 3.0.0 is out. The release modernises the JavaScript BDD library, but more interestingly, it was largely built by Claude Code and points to why executable specifications may become even more valuable in an agentic development world.
It may be an age thing. 15-ish years ago, TDD, BDD, Red/Green Testing, and a whole bunch of others were hot topics. They’re still useful approaches to know about, even if some (like, IMO, BDD) didn’t really stand the test of time.
Thanks. (That’s what it suggested for me as well. Clarifying that I meant in the context of software, it suggested a misspelling of SOAD - Service-Oriented Analysis and Design.)
This is probably HN's tech background speaking. In the enterprise dev world, everyone knows what BDD means just like every dev out there knows what OOP means.
The example of “transcibe the initial project discussion, turn that into BDD and that makes the rails to guide the AI is … either brilliant and insightful or a snake eating it’s own tail.
I've been doing that for almost a year and I feel like I'm taking crazy pills when I talk about it, because it works so well.
I transcribe most product meetings, turn them into a transcript, run that through an accuracy pass with a bunch of context (mostly vocabulary definitions etc), and use that as a starting doc for generating issues and tests. There are apps that do it fully end to end now I think.
Yeah, basically I would just start transcribing everything, if you aren't already, and get a bunch of context. It's best to start from the very beginning but the reality of enterprise is that nothing is truly greenfield.
Maybe start with a project kickoff or brainstorm, some kind of planning meeting, and select one task out of it to start. I always start with just the meeting transcript as a context, and expand context from there as needed with things like internal standards docs, libraries, references from other projects, etc, and only as little as is necessary. That generates the project plan, and then I feed that into claude code or whatever.
I think the biggest thing I've done recently is start to be extremely critical about the output, basically using promptfoo to try to figure out what the right level of context and instructions are, not just for external facing e.g. chatbots or whatever, but for my own internal development process, planning, everything. The more you can pin down and calibrate the better - it's easy to run a given prompt for a month, so spending a day (or even a week!) out of the month to refine your prompts beats just grinding ad hoc.
Edit: another thing I do is generate tests bdd-style separately from the code that they're testing (for e2e and high level at least), that derives out of the project plan too.
I also think BDD is a really powerful tool for UI acceptance tests! I've now worked on adding it to two AI codebases (one engineered, another vibecoded), and so far it has been mostly positive. Having non-engineers be able to see what effects their code changes have in the UI is nice; and I find having high level API and UI tests is forcing the first layer of cleanup (standardize access to DB through models/services, and a single API SDK).
I am also really interested in standardizing the REST interface with typespec, and the database schema with DBML. I think creating these closed-loops with high-level specs is one of the best ways to make an AI project ownable by a team.
Basically I'm designing a two layer interface, the first layer speaks app UI specific language and runs playwright directly (signin, dismiss toast, go to dashboard), and then the second speaks product language ("user opens dashboard" in Gherkin would translate to signin, wait for app to load, go to dashboard; for example). The translation from product level language to app level language is the step definition files bddgen uses. I'm building up a little skills library to try and force Claude to stick to these separate levels of abstraction; then also do deduplication and cleanup after implementing new specs.
We don't really use images for acceptance testing, it's more functionality. I don't really care where the button is, just that if I click it, it does what it says it does. To that end, we already have multi-tenancy in the application (I highly recommend every app build this in, if only for testing; then you don't need to do the Postgres template1 seeding hacks), so we spin up a tenant for a test, do all the setup with the API (all the Given statements), playwright to do the validation of a feature (the When statements), and finally validation is a mix of Playwright and API (the Then statements).
Sorry if that isn't super clear; maybe I could put together a little demo repo or something if people have any interest; I'm hoping to do that after I introduce typespec and dbml (basically spec/{acceptance,api,db}).
So, I get the idea of having a layer of abstraction between your tests and the system under test. You decouple _what_ is being done from _how_ it is done, which leads to robust, easily maintained, and even more re-usable test suites. I've seen it in action, it's a great idea, I've reaped the benefits myself. Great stuff.
What I've never really understood is why some people choose natural language for the abstraction. It just seems like such an odd and expensive choice. Most of the layers of abstraction we add in our codebase are written in the programming language of choice itself, they're functions, classes, what have you. But for this one specific flavor of abstraction, folks reach for natural language.
The one potential benefit I could think of for this over a more standard DSL is that non-technical colleagues would be able to read or even write these specs. But in practice I've only seen this pan out once in my life, and that was with a more traditional DSL rather than natural language.
Because the intent has always been that Product delivers the BDD statements not Engineering, but Product people have (for the most part in my experience) refused to deliver what BDD flows need to be tested. That’s why most of the time the test suites aren’t as resilient because the engineer is guessing at which flows to test.
In that context it makes more sense that Llms are a push to replace Product people not Engineering people.
I don't think so. Natural language is vague and repetitive.
I used hitchstory (YAML based stories) for doing this coz it's terse and typed.
I've tried using cucumber before and it reminds me of COBOL - which also had the same idea of making code more accessible via a "natural language-ish" interface.
[OP] scresswell | a day ago
ur-whale | a day ago
I'd consider explaining what BBD is at the start of the front page of your project.
cbsmith | a day ago
That's disconcerting. Software engineers ought to know.
citizenkeen | a day ago
jaggederest | a day ago
ur-whale | 21 hours ago
You evaluate candidates based on their intimate knowledge of acronyms?
LOL.
Pray tell where you work so we all collectively never apply for a job there.
ashleyn | a day ago
davepeck | a day ago
krautsauer | a day ago
I was expecting something on Binary Decision Diagrams. SAOD is worst with TLAs.
WalterGR | a day ago
contradictioned | a day ago
Funny enough, Google's overview box also suggests soad: "SAOD (or SOAD) most commonly refers to the rock band System of a Down"
WalterGR | a day ago
bossyTeacher | a day ago
Amekedl | a day ago
lifeisstillgood | a day ago
I think it has to be worth a try though…
jaggederest | a day ago
I transcribe most product meetings, turn them into a transcript, run that through an accuracy pass with a bunch of context (mostly vocabulary definitions etc), and use that as a starting doc for generating issues and tests. There are apps that do it fully end to end now I think.
lifeisstillgood | a day ago
jaggederest | a day ago
Maybe start with a project kickoff or brainstorm, some kind of planning meeting, and select one task out of it to start. I always start with just the meeting transcript as a context, and expand context from there as needed with things like internal standards docs, libraries, references from other projects, etc, and only as little as is necessary. That generates the project plan, and then I feed that into claude code or whatever.
I think the biggest thing I've done recently is start to be extremely critical about the output, basically using promptfoo to try to figure out what the right level of context and instructions are, not just for external facing e.g. chatbots or whatever, but for my own internal development process, planning, everything. The more you can pin down and calibrate the better - it's easy to run a given prompt for a month, so spending a day (or even a week!) out of the month to refine your prompts beats just grinding ad hoc.
Edit: another thing I do is generate tests bdd-style separately from the code that they're testing (for e2e and high level at least), that derives out of the project plan too.
blfr | a day ago
My experiences with Opus have been such that I always max out my Fable allotment but rarely exceed 40% of the remaining limit on Claude.
jesol | a day ago
I am also really interested in standardizing the REST interface with typespec, and the database schema with DBML. I think creating these closed-loops with high-level specs is one of the best ways to make an AI project ownable by a team.
stevefan1999 | a day ago
skybrian | a day ago
jesol | an hour ago
We don't really use images for acceptance testing, it's more functionality. I don't really care where the button is, just that if I click it, it does what it says it does. To that end, we already have multi-tenancy in the application (I highly recommend every app build this in, if only for testing; then you don't need to do the Postgres template1 seeding hacks), so we spin up a tenant for a test, do all the setup with the API (all the Given statements), playwright to do the validation of a feature (the When statements), and finally validation is a mix of Playwright and API (the Then statements).
Sorry if that isn't super clear; maybe I could put together a little demo repo or something if people have any interest; I'm hoping to do that after I introduce typespec and dbml (basically spec/{acceptance,api,db}).
wesselbindt | a day ago
What I've never really understood is why some people choose natural language for the abstraction. It just seems like such an odd and expensive choice. Most of the layers of abstraction we add in our codebase are written in the programming language of choice itself, they're functions, classes, what have you. But for this one specific flavor of abstraction, folks reach for natural language.
The one potential benefit I could think of for this over a more standard DSL is that non-technical colleagues would be able to read or even write these specs. But in practice I've only seen this pan out once in my life, and that was with a more traditional DSL rather than natural language.
Is there a hidden benefit I'm overlooking?
righthand | a day ago
In that context it makes more sense that Llms are a push to replace Product people not Engineering people.
MoreQARespect | 21 hours ago
I used hitchstory (YAML based stories) for doing this coz it's terse and typed.
I've tried using cucumber before and it reminds me of COBOL - which also had the same idea of making code more accessible via a "natural language-ish" interface.
Arubis | a day ago
sroerick | 22 hours ago