Skip to content
WANCODE
BM

Testing AI Output

18 minIntermediateHTML · CSS · JS

In this lesson

  • Turn each requirement in a prompt into an assertion
  • Recognise weak tests, including tests that just repeat the code
  • Check that your tests can fail by running them against broken versions

The big idea

A test is the requirement written as code that runs every time. Write tests from what the code should do, then make sure each one would actually fail if the code were wrong.

Reading code finds many bugs. Tests find the rest — and keep finding them every time someone (or something) changes the code later. You don't need a framework to start; an assertion is just "this input should give that output".

JavaScript
console.assert(slugify("Hello World") === "hello-world", "lowercases and joins words");
// Prints nothing when it passes, an "Assertion failed" error when it doesn't.

The built-in console.assert is the smallest possible test. In this lesson's sandbox, failures show up as errors in the console.

Test the spec, not the code

Assistants can write tests too, and that's useful — but read them. A common failure is a test that computes its expected value with the very function it's testing, or that copies whatever the current code returns. Such tests pass forever, bug or no bug.

What the assistant wrote

JavaScript
check("slugify works",
  slugify("Hello, World!"),
  slugify("Hello, World!")); // always equal

What a reviewer ships

JavaScript
check("removes punctuation",
  slugify("Hello, World!"),
  "hello-world"); // written from the requirement

The expected value must come from the requirement, written by hand. If you can't say what the right answer is, you've found a gap in the spec.

  • One test per rule. The prompt lists four rules — lowercase, trim, remove punctuation, single dashes — so you need at least four tests.
  • Choose inputs that would expose the bug. "Hello World" can't tell you whether trimming works; " Hello " can.
  • A test that can't fail is not a test. Break the code on purpose (delete .trim()) and check that some test goes red. Testing tools call this mutation testing; you're doing a manual version.

Tests surface decisions

Try slugify("Café"). The generated code returns "caf" because é isn't in a-z. Is that acceptable? It's a product decision the prompt never made — writing tests is often how you discover questions like this.

Add tests to runTests(fn) so there is at least one test for each rule in the prompt — at least 4 tests in total. Every test must call fn(...) (not slugify directly), because later we'll run your tests against other versions of the function.

All your tests must pass against the real slugify — if one fails, check whether your expected value matches the requirement.

Checks

  • runTests has at least 4 tests that call fn
  • All tests pass against the real slugify

Check understanding

Q1

What's wrong with check("works", add(2, 3), add(2, 3))?

Q2

You delete .trim() from a function on purpose and all tests still pass. What does that tell you?

What you learned

  • Write one assertion per requirement, with expected values written by hand.
  • Pick inputs that would expose each specific bug.
  • Prove tests can fail: break the code on purpose and watch them go red.
  • Review AI-written tests as carefully as AI-written code.
Preview

Console output from your code appears here.

Step 1 of 2