Test protocol
Six experiments for the first test lab
Each experiment begins with a user promise and includes a negative case. Automate the stable assertions; keep exploratory review for behavior that still needs human judgment.
Critical journey
SetupStart from a clean account and complete the product’s primary job
AssertionsThe intended outcome occurs once, persists and is visible after reload
Input boundary
SetupUse empty, long, malformed, duplicated and unexpected values
AssertionsValidation is specific, safe and consistent at the server boundary
Permission boundary
SetupCreate at least two users or organizations with different roles
AssertionsEvery read and mutation enforces the expected principal and scope
Slow and failed dependency
SetupDelay or fail data, email, payment or other external requests
AssertionsThe interface preserves state, communicates status and supports safe retry
Alternative interaction
SetupComplete the journey with keyboard, zoom and a narrow viewport
AssertionsNames, focus, reading order, status and layout remain usable
Release recovery
SetupDeploy a known harmless change to a production-like environment
AssertionsOwners can observe the release, detect the success signal and restore the prior version
A screenshot is one observation
Generated interfaces need state coverage more than visual applause
A polished default state can hide an empty list that collapses, a form that submits twice, a permission check that exists only in the client or a loading state that never recovers. Write down the states before choosing tools: initial, loading, empty, partial, success, invalid, denied, failed and recovered.
Do not make every test end-to-end. Small tests are fast and precise for pure rules; integration tests verify important boundaries; browser journeys prove that the assembled product fulfills its promises. The useful portfolio follows risk rather than a fashionable pyramid drawn without context.
- Test behavior users depend on, not implementation details
- Include at least one negative assertion per material boundary
- Use production-like configuration without production secrets
- Store commands and evidence beside the change
Production standard
A release evidence packet
A test run becomes operational evidence when another person can reproduce it and knows which failure blocks the release.
Journey
PassCritical user outcomes work from a clean state and after recovery.
FailThe demo requires hidden setup or manual database correction.
Boundary
PassInvalid input and unauthorized actions fail safely at controlled boundaries.
FailOnly the interface prevents an action or errors expose internals.
Accessibility
PassEssential flows have usable semantics, keyboard behavior and status communication.
FailA user is blocked by custom controls, focus loss or layout overflow.
Resilience
PassRetries, partial failures and timeouts preserve consistency.
FailUsers create duplicate work or cannot tell whether an action succeeded.
Release
PassThe deployed version is observable and recoverable by a named owner.
FailGreen CI is the only production signal and rollback is theoretical.
Applied example
Testing example: the form that passes every happy-path test
A contact form displays a success state in the browser immediately. The API then calls an email service, which times out. The user retries and creates three records while no email is sent.
- A visual test of the success screen passes even though the product promise fails.
- The client treats submission as complete before the authoritative result is known.
- Retry behavior is not idempotent and the partial failure is invisible to operators.
- A delayed-dependency experiment would expose all three gaps before launch.
Plain answers
Testing questions
How many tests does a small vibe-coded website need?+
There is no meaningful universal count. Cover critical promises, high-consequence boundaries and previously observed failures; avoid inflating numbers with assertions that do not protect behavior.
Can an AI agent test the website it created?+
It can generate and execute useful tests, but independent acceptance criteria and human review remain important because the same assumptions can shape both implementation and tests.
What should be tested manually?+
Exploratory behavior, content comprehension, assistive-technology use, visual hierarchy and unexpected interactions benefit from human judgment, while stable rules and journeys should be automated where practical.