Lessons Learned in Software Testing
A Context-Driven Approach
by Cem Kaner, James Bach & Bret Pettichord
Buy on AmazonThere are no "best practices" in testing — only good practices in context — and these 293 lessons from 50-60 combined years of experience teach you how to think about testing, not just what to do.
Why This Book Matters
Most testing books tell you what to do: write test cases, follow templates, hit X% coverage. This book tells you how to think. It's 293 short, opinionated lessons written by three practitioners with decades of hands-on experience. They reject one-size-fits-all recipes and want you to use your brain instead.
Each lesson had to pass a filter: if almost anyone who thought about testing for 90 minutes could come up with it, it didn't make the cut. At least one author had to have successfully applied each piece of advice. At least two had to have been burned by the practice they criticize. The result is a book the authors describe as "not a collection of best practices" but rather their lessons, based on their experiences.
The Seven Principles (The Book's Foundation)
The book presents examples of context-driven thinking. In the appendix, the authors list these seven principles of the context-driven school:
The value of any practice depends on its context.
There are good practices in context, but there are no best practices.
People, working together, are the most important part of any project's context.
Projects unfold over time in ways that are often not predictable.
The product is a solution. If the problem isn't solved, the product doesn't work.
Good software testing is a challenging intellectual process.
Only through judgment and skill, exercised cooperatively throughout the entire project, are we able to do the right things at the right times to effectively test our products.
Big Idea 1: You're the Headlights, Not the Gatekeeper
The book's most important reframe: testing doesn't assure quality. You don't create quality and you don't take it away. "You may talk as if you 'break the product'; the truth is it came to you already broken." Your real job is to be the project's headlights — light the road ahead so the team can see where they are, what they're about to hit, and how close they are to the edge.
What Testing Is
- An information service that helps the team make decisions
- Finding things that "bug" someone whose opinion matters
- Focused on failure so your clients can focus on success
- A service to many clients — PMs, devs, support, marketing, users
What Testing Isn't
- A quality gate that approves or blocks releases
- A "process criticism society" (the book's words)
- "Complete" — the book lists 11 different meanings of "done testing"
- Proof that the product works (you can't prove that)
Use this tomorrow: If you have veto power over releases, share that authority immediately. When testers own the release decision, the rest of the team relaxes — quality becomes "your problem." The people who control the project should bear release responsibility.
Big Idea 2: Good Testing Happens in Your Head
The difference between a great tester and an average one is invisible from the outside — it's all in how they think. The book says: "If you see two testers working, side-by-side, you can't necessarily tell whether one is testing better than the other." The difference is in test design choices, the ability to interpret what you observe, and the ability to tell a compelling story about it.
"All tests are an attempt to answer some question." The book draws a line between a tester and a tourist: "a tester's efforts are devoted to evaluating the product, not merely witnessing it." Before any test, ask yourself what question you're trying to answer.
"All testing is based on models." Your tests come from your mental picture of the product — not the actual product. "A flawed model results in flawed tests. Learning a new way to model a product is like learning a new way to see it."
Use confusion as a tool. When you feel confused, that's useful information. Confusing spec? There might be disagreements hiding underneath. Confusing product? It might be broken. Your confusion can become "issues and questions that perhaps no one else has the courage to raise."
"Intuition is a fine beginning, but a lousy conclusion." Gut feelings are biased and easily dismissed by others. The book's advice: reframe from "this is obviously a bug" to "this is a bug because the product violates requirements X, Y, and Z, and those requirements are valued by my clients."
"You can't avoid bias, but you can manage it." The book names eight biases: assimilation, confirmation, availability, primacy, recency, framing, prominence, and representativeness. It notes that "by definition, a heuristic is also a bias" — and recommends diverse teams brainstorming together to minimize any one person's blind spots.
Use this tomorrow: When facing a complex product, try "plunge in and quit" — throw yourself at a feature for 30 minutes to an hour, then stop. Your brain processes complexity in the background. After a few rounds, better strategies emerge. The book says this method "works like magic."
Big Idea 3: Your Bug Reports Are Sales Tools
"A tester who can't report bugs well is like a refrigerator light that's only on when the door is closed." The book says bug reports are "the primary work product of most testers." They shape how everyone sees you. Every report is "a sales tool designed to convince people to give up precious resources in order to obtain a benefit that you propose."
The title is everything. In triage meetings, managers see roughly 65 characters per bug. Make it specific enough to picture the failure, and show the impact. Weak titles get dismissed without discussion.
Find the worst version, not the first version. The book says: "Uncorner your corner cases." If a field that should accept 0-999 rejects 999, don't stop there — test further and show it actually fails for everything 100-999. That gets attention.
Always report bugs you can't reproduce. They can be the most expensive bugs you ship. Developers have tools you don't — they can fix about 20% of non-reproducible bugs just from clear symptom descriptions. Some teams report rates as high as 80%.
Describe the problem, don't propose a fix. Testers who guess at solutions risk having reports rejected when the guess is wrong — and they often forget to include the actual failure data the developer needs.
Never use bug tracking to judge people. Once you use it to measure developers or testers, everyone games the system. Developers argue design bugs "aren't real." Testers file shallow bugs to inflate their numbers. The data becomes worthless.
Real result: Kaner reviewed every support call and bug report for a product. Conclusion: cheap fixes (under 4 hours each) could have prevented over half of the tech support calls for that product. After fixing them, support cost per new customer dropped by half.
Big Idea 4: Automation Is Real Software Development
The automation chapter (40 lessons) is the most myth-busting part of the book. The authors love automation — but they've seen too many teams buy a tool and expect magic.
Record-and-playback doesn't work. The book calls it "an ice-covered slope with a brick wall at the bottom." You record 100 tests that create an account. The product adds one new dialog to account creation. All 100 tests break. Now you're spending more time fixing tests than you ever spent running them manually.
Automated regression tests catch about 15% of total bugs. Even on projects with solid automation. The twist: automators usually find more bugs while building the tests than when running them later. Reusing those same tests on different platforms can push the rate to 30-80% — because then they're effectively new tests, not just regression.
Plan for 10x the effort. A well-designed GUI automated test takes about 10 times as long to build as running the same test by hand. Claims of 2-3x usually come from throwaway tests or good luck.
Test through APIs, not the GUI. "Almost any programming interface is easier to work with — more stable and more reliable." The authors see a strong link between API availability and strong automated test suites.
Manual testing isn't inferior to automated testing — they're different things. A human tester brings the "entire range of human capabilities" — they notice strange noises, odd flickers, and slowdowns without being told to look. "The prepared mind is a fantastic test tool, beyond any conceivable automation."
Use this tomorrow: Ask your developers what diagnostic tools they already built for debugging. On one project, the authors found 1,100+ diagnostic commands the testers didn't even know existed. Also: building testability inside the product (logging, assertions, test menus) is often cheaper and better than building automation outside it.
Big Idea 5: Mix Many "Good Enough" Approaches
The book's single most important strategy lesson: a less thorough but more varied testing approach is better than a more thorough but less varied one. The authors call this "diverse half-measures."
"We've seen cases where a company ran hundreds of thousands of test cases and still missed simple obvious problems, because they ran an insufficient variety of tests."
Why this works: any single testing technique finds lots of bugs at first, but then the discovery rate flattens. Switching to a different technique — one that catches different kinds of problems — makes the rate climb again. The book's "Five-fold Testing System" helps you cover all the angles:
People-based
Who tests? Beta users, domain experts, paired testers, developers eating their own dogfood.
Coverage-based
What gets tested? Functions, boundaries, states, paths, configurations, input combinations.
Problem-based
Why are you testing? To catch input constraints, output constraints, computation constraints, storage constraints, and timing issues.
Activity-based
How do you test? Regression, exploratory, scenario, load, endurance, smoke testing.
Evaluation-based
How do you judge pass/fail? Compare against saved results, specs, comparable products, user expectations, or the product's own internal consistency.
Use this tomorrow: The book says to "test to the maturity of the product." Early on, "test sympathetically" — does it basically work? In the middle, "test aggressively" — complex, demanding tests. Near the end, "test diversely" — try every technique you can think of. In the final days, "test meticulously" — pair test every change.
Big Idea 6: Be Helpful, Not Powerful
Your real influence comes from being a trusted, credible information source — not from org chart authority. The book is blunt: "Programmers are like tornadoes. Think of them as forces of nature. They will do what they do. Design your practices accordingly."
Offer direct help. Test private builds, set up test environments, review requirements for testability. The book says: "Offer to assist programmers directly. This builds trust." When you prove useful, they're more likely to share plans, early design drafts, and prototypes with you.
Report bugs, not people. Never report "Joe is a screw-up" — report the bugs you found. Testers who try to punish developers become disposable. "Some are dropped right away. Others are kept around as convenient bad cops until a Really Big Screw-up demands the sacrifice of a scapegoat."
Ask developers to draw the system on a whiteboard. Then point to a random box or arrow and ask: "What happens if this fails?" This simple question uncovers missing error handling and assumptions nobody has questioned yet.
Ask for testability features early. Logging, assertions, error simulation, test interfaces — the book calls this "visibility and control." Ask while the design is still flexible. If testability features aren't in the plan early, they won't get built.
The golden rule: "If you have integrity, you can develop your competence. If you've lost your integrity, your competence won't matter."
12 Things You Can Do at Work Tomorrow
Test in this order: changed things > core functions > basic capability > common scenarios > common threats > high-impact areas > things people asked you to test. This is the book's prioritization framework for finding important bugs fast.
Before every test, ask yourself what question you're answering. "What am I trying to learn?" If you can't answer, step back and think. The book warns: "you're more tourist than tester" if you don't know the question driving your test.
Rewrite your bug titles. Make each one specific enough to picture the failure, with scope and impact, in about 65 characters. This alone will get more bugs fixed.
Build a reusable input field test list. The book provides about 40 standard tests for any input field: nothing, empty, boundary values, non-digits, special characters, upper ASCII, nonprinting characters, interrupts, and more. Build it once, use it on every project.
Use all-pairs for combination testing. Three variables with 5 values each = 125 combinations. All-pairs can cut that to about 25 tests while still covering every pair. Add 5-10 critical combinations on top for extra safety.
Plan your exploratory sessions. Before sitting at the computer, write a charter covering what to test, what risks to look for, and what bugs you expect. The book says to protect 60-90 minutes of uninterrupted time per session.
Clean out your old test suites. The book describes the "old oak tree syndrome" — believing in the wisdom of ancestral testers who created grand (now old) test suites. Are there tests from years ago still running that check for errors that can't happen anymore? Remove or repair them.
Ask developers what diagnostics already exist. Many teams have logging, assertions, and test interfaces built for debugging that testers don't know about. Just ask.
When you stop finding bugs, switch techniques — don't stop testing. The book says "the find-rate curve will flatten out" for any given technique. That means your current approach has diminishing returns, not that the product is clean. Switch to a technique that's sensitive to a different kind of problem.
In each test cycle, test bug fixes first. The developer still remembers what they changed. Then test new features. Then everything else in order of risk. Automated regression runs last.
Rotate testers across features. When a tester feels confident about an area, that's exactly when to move them out. A fresh tester will find problems the experienced one stopped noticing.
Stop counting test cases. "Test cases are like briefcases. Counting them without regard to content tells you nothing." Talk about risks covered and problems found instead.
Lines Worth Putting on Your Wall
"You may talk as if you 'break the product'; the truth is it came to you already broken."
"If you ever find yourself testing and realize you have no questions about the product, take a break."
"The prepared mind is a fantastic test tool, beyond any conceivable automation."
"Programmers are like tornadoes. Think of them as forces of nature. They will do what they do. Design your practices accordingly."
"It's not a sin in testing to miss a bug. It's only a sin to be careless, thoughtless, or not to learn from your experience."
Who Should Read This
- SDETs and test engineers (2-5 years experience) — This is the sweet spot. It will challenge things you've been doing on autopilot and give you frameworks to think for yourself.
- Test leads and managers — The chapters on managing test projects and teams are full of practical advice: session-based testing, status report structure, why bug counts are worthless for measuring people, and how to hire a diverse team.
- Developers who test their own code — The first two chapters will change how you think about what "testing" even means.
- Anyone who thinks testing is boring — The authors' enthusiasm for testing as real intellectual work is contagious. This book makes you proud to be a tester.
§ Verdict
9 / 10
The most important book on software testing ever written. It won't teach you Selenium syntax or how to set up a CI pipeline — for that, read tool docs. What it teaches is how to think about testing, and that's worth so much more. The bug advocacy chapter (Chapter 4) and the automation chapter (Chapter 5) are worth the price alone. Some career advice in Chapter 10 is dated (it mentions faxing resumes and salary.com), but the thinking behind it is timeless. Read the whole thing, then keep it on your desk. Whenever your testing feels stale, open to a random lesson.