07 · Testing · Every build
A replay once clicked by hand must not rely on memory at the next merge
The automation test engineer guards rules that have already been made clear. He does not record every button. First exploration, and whether the words can be read, still belong to hand testing.
- Submit Cutoff. After cutoff, the same store submits again. Assert the state stays locked and no second document is created.
- Before merge Privilege. A wrong-store account is refused. The response has none of another store's quantities.
- Overnight Write-back. Simulate a timeout. Local state stays pending write-back. Stock does not change.
Automation takes regression out of memory
A path whose rules are already stable can still break next week when an unrelated function is changed. What marks this role is choosing checks whose rules are stable and that people easily miss, then folding them into the build. Idempotency, privilege, and the state machine suit automation. A marketing page whose copy keeps changing, or a definition still under argument, does not. He writes assertions on the business result: ledger line count, document state, whether an external document number was created. Asserting only that the page shows the word “success” goes all red when copy changes, and never catches a real double post.
He maintains data setup. A case creates its own document and cleans it up at the end. It does not depend on the document someone left by hand yesterday. Automation that depends on dirty data cannot tell, when it goes red, whether it is a defect or the environment. He stops a flaky case and marks why. He does not let the team habitually rerun until it goes green.
The boundary with hand testing
First exploration, whether the words can be read, and where you stand at a station screen still belong to hand testing and the designers. Automation does not replace those. It guarantees that rules already frozen have not gone backwards. On failure he provides the request ID and the response so development can reproduce it, not only a screenshot. A screenshot cannot show which replay it was.
Three gates that enter the pipeline
Cutoff lock was once broken in a hotfix, because at the time there was only a hand record. The automation test engineer folded cutoff replay, a wrong-store account, and write-back timeout into the before-merge checks. One change made the timeout branch increment the order quantity again. The build went red. It did not enter main. That is cheaper than reconciling after go-live. Setup data for the cases is generated by script as next-day store orders. It does not borrow real members from a production copy.
Case count is not a score. Three gates that stably stop a wrong posting are more useful than three hundred button-click scripts. Every failure keeps a request ID and a database assertion. Once a week he looks at the failure split: real defect, environment, stale case. A stale case is changed or stopped that day. Test data in the pipeline is rebuilt every day. When a gate fails, the merge page must let a person see which piece of business failed: the same request ID produced two ledger lines. Seeing a title like that, development looks at the submit first, not at rerun.
The cutoff case moves the test clock itself. It does not depend on someone remembering to change the server time. The privilege case creates an account that can see only one store, and deletes it when done. The write-back case creates a pending-write-back document of its own, simulates a timeout, and asserts that local quantity does not change. Data setup for all three gates lives in the script, so another machine still goes red on the same piece of business. If setup still depends on a document left by hand, a red run can only guess at the environment. Once a week he splits failures into real defect, environment, and stale case. Stale is changed or stopped that day. A high pass rate when nobody can say which posting was guarded means there is no gate. First exploration is still done by hand. Automation only guards the three that are already frozen and that people are afraid will be broken.
A marketing page whose copy keeps changing, or a definition still under argument, does not enter the build. Assertions are written on state and line count, not on the word “success.” A screenshot cannot show how many ledger lines there are. Failure titles are written as a business sentence, so the merge page shows which posting broke first, not a red stage. A red stage tempts people to rerun first.