Software Testing
Second Edition
by Ron Patton
Buy on AmazonSoftware testing is a disciplined, methodical profession whose goal is to find bugs, find them as early as possible, and make sure they get fixed — because every bug found after release costs 10–100x more than one caught during specification.
What Problem Does This Book Solve?
Most people walk into software testing without a roadmap. They bang on keyboards hoping something breaks, or they test only to confirm things work. Patton argues both approaches are wrong. Testing is a structured discipline with defined techniques, formal processes, and measurable outcomes — and most bugs aren't caused by bad code at all.
The book's central insight is that the number one cause of software bugs is the specification — not the code. Specs that are incomplete, constantly changing, or poorly communicated create the majority of defects. This reframes the entire profession: testing isn't just about finding broken code, it's about catching problems at every stage of development.
What Counts as a Bug? (The Five Rules)
Patton provides a formal definition that goes far beyond "the software crashes." A bug exists when any of these five conditions are true:
The software doesn't do something the spec says it should do.
The software does something the spec says it shouldn't do.
The software does something the spec doesn't mention.
The software doesn't do something the spec doesn't mention but should.
The software is difficult to understand, hard to use, slow, or will be viewed by the end user as just plain not right.
Rule 5 is the catch-all that separates great testers from mechanical ones. It requires judgment, empathy, and understanding what users actually expect.
The Four Testing Approaches
Patton organizes all testing techniques along two axes: static vs. dynamic (is the software running?) and black-box vs. white-box (can you see the code?). This creates a 2×2 matrix that frames the entire book:
Static Black-Box
Reviewing the specification without running the software. Catch bugs before a single line of code is written. Use the spec attributes checklist and terminology checklist.
Dynamic Black-Box
Testing the running software without seeing the code. Equivalence partitioning, boundary conditions, state testing. "Testing the software with blinders on."
Static White-Box
Examining the code without running it. Formal reviews, walkthroughs, inspections. The book says most bugs found in reviews are discovered during preparation, not the meeting itself.
Dynamic White-Box
Testing the running software with full code access. Unit testing, integration testing, code coverage (statement, branch, condition). Use debuggers to force error conditions.
Key Testing Techniques
1 Equivalence Partitioning
"The process of methodically reducing the huge (infinite) set of possible test cases into a much smaller, but still equally effective, set." Group inputs that test the same thing into partitions, then test one representative from each. Warning: partition too aggressively and you risk missing bugs. It's science and art.
2 Boundary Conditions
"If you can safely and confidently walk along the edge of a cliff without falling off, you can almost certainly walk in the middle of a field." Always test the last valid value inside the boundary, the first invalid value outside it, and the boundary itself. Buffer overruns — the #1 cause of software security issues — are boundary condition bugs.
3 State Testing
Map every state the software can be in, every transition between states, and the conditions that trigger transitions. Then test race conditions (two events at the exact same time), repetition (do it 1,000 times — look for memory leaks), stress (starve the software of resources), and load (feed it everything it can handle). Combine all three for maximum bug exposure.
4 Code Coverage
Three levels, each more thorough: statement coverage (execute every line), branch coverage (take every path through every decision), and condition coverage (test every combination of conditions in compound expressions). Even 100% condition coverage doesn't mean complete testing — data bugs are still possible.
Famous Bug Case Studies
Patton opens the book with six real-world disasters that illustrate why testing matters:
Intel Pentium Division Bug (1994) — Engineers found the bug before release, but management decided it wasn't severe enough to fix. Intel ultimately took a $400 million charge to replace defective chips.
NASA Mars Polar Lander (1999) — A classic integration testing failure. One team tested leg deployment, another tested landing. Neither tested them together. A vibration when legs snapped open falsely triggered the "touchdown" sensor, and the lander fell 1,800 meters to its destruction.
Patriot Missile System (1991) — A tiny timing error accumulated over time. After 100+ hours of continuous operation, the tracking system was so inaccurate it failed to intercept a missile that killed 28 U.S. soldiers.
Disney's Lion King CD-ROM (1994) — Disney tested on their own PCs but not on the most common consumer configurations. On December 26, customer support was flooded with calls from parents whose kids couldn't play the game.
The Y2K Bug — A programmer named "Dave" shortened dates to 2 digits in the 1970s to save memory, thinking his program would be replaced in 25 years. It wasn't. Cost to fix worldwide: several hundred billion dollars.
The unifying lesson: the cost of fixing a bug increases logarithmically over time. A bug caught during specification costs ~$1. The same bug found after release can cost thousands or millions.
The Realities of Software Testing (8 Axioms)
1. It's impossible to test a program completely. The number of inputs, outputs, and paths is effectively infinite.
2. Software testing is a risk-based exercise. You can't test everything, so you must choose what to test based on risk.
3. Testing can't show that bugs don't exist. It can only show that they do.
4. The more bugs you find, the more bugs there are. Bugs cluster. If you find 10 in one area, there are likely more nearby.
5. The Pesticide Paradox. The more you run the same tests, the more the software becomes immune. You must continuously write new tests.
6. Not all bugs will be fixed. It comes down to a business decision based on risk, time, and workarounds.
7. Product specifications are never final. Be ready for change.
8. Software testers aren't the most popular members of the project team. You're always the bearer of bad news. Tact and diplomacy matter as much as technical skill.
Test Planning, Reporting, and Metrics
Patton dedicates the entire Part V to the documentation side of testing — and makes a critical distinction:
"The test plan is a by-product of the detailed planning process that's undertaken to create it. It's the planning process that matters, not the resulting document."
Effective Bug Reports
Must be minimal (just the facts), singular (one bug per report), obvious (simple steps to reproduce), and reproducible. "There's no such thing as a random software bug."
The Ultimate Bug Graph
Three cumulative lines: bugs opened, resolved, and closed. When all three flatten and converge, the project is approaching release readiness. A widening gap means someone is falling behind.
Severity vs. Priority
Severity = how bad is the bug. Priority = how urgently should it be fixed. Data corruption that happens rarely: Severity 1, Priority 3. A misspelling blocking users: Severity 3, Priority 2.
Quality Can't Be Tested In
"A doctor can't make someone's fever go down by taking her temperature." Adding more testers doesn't make software better — it only reveals what's already wrong. Quality must be built in from the start.
Applied Testing: Beyond the Basics
Part III covers seven specialized testing domains. Key highlights:
Configuration Testing — Hardware combinations are nearly infinite. The approach: identify all possible configurations, pare down to a manageable set based on popularity and risk, then test your software's unique features on each one.
Usability Testing — Seven traits of a good UI: follows standards, intuitive, consistent, flexible, comfortable, correct, and useful. The book introduces the term "dancing bologna" for unnecessary features.
Localization Testing — Expect 100% text expansion for individual words and 50% for sentences. Microsoft shipped Windows XP with 106 languages. A real story: Microsoft's McZee character had to get a "nose job" when a foreign office said large noses carried negative stereotypes in their culture.
Security Testing — Buffer overruns are the #1 cause of security vulnerabilities. The book uses the 1983 movie WarGames as a teaching tool and covers threat modeling with the DREAD formula: Damage, Reproducibility, Exploitability, Affected users, Discoverability.
Documentation Testing — "From the user's standpoint, it's all the same product. An incorrect step in the installation instructions is a bug just like any other software failure."
7 Actionable Takeaways
Test the spec before the code. Static black-box testing catches bugs at $1 instead of $10,000. Use the spec attributes checklist (complete, accurate, precise, consistent, relevant, feasible, code-free, testable) on every requirement.
Always test-to-pass before test-to-fail. Confirm the software fundamentally works before you try to break it. You'll be surprised how many bugs appear during normal use.
Focus on boundaries. The edges are where bugs live. Test the last valid value, the first invalid value, and the boundary itself. Don't forget sub-boundaries (powers of two, ASCII table gaps).
Always test empty/null/zero/default conditions. What happens when the user enters nothing and hits Enter? This deserves its own equivalence partition because "a different software path is followed."
Create black-box tests before white-box tests. If you look at the code first, you'll be biased into testing how the software works instead of what it's supposed to do.
A good tester finds bugs. A great tester gets them fixed. Follow up. Track every bug through its lifecycle. Write reports that are minimal, singular, obvious, and reproducible.
Fight the Pesticide Paradox. Your tests become immune over time. Continuously add new test cases, especially in areas where you've already found bugs — because the more bugs you find, the more bugs there are.
Who Should Read This
- New SDETs and QA engineers — This is the textbook. It covers the full landscape from spec review to career planning with zero assumptions about prior testing knowledge.
- Manual testers transitioning to automation — The foundational techniques (equivalence partitioning, boundary testing, state testing) apply regardless of whether you're clicking or scripting.
- Developers who want to write better tests — The code review checklist alone (7 error categories) will sharpen your unit tests immediately.
- QA leads writing their first test plan — Part V's planning framework is practical without being bureaucratic. The emphasis on process over document is refreshing.
§ Verdict
— / 10
Rating to be added after review. The book was published in 2006, so some technology examples are dated (VCRs, MS-DOS, Internet Explorer 6). But the testing principles — equivalence partitioning, boundary testing, state machines, the Pesticide Paradox, the cost curve of bugs — are timeless. If you're new to testing, read this before anything else. If you're experienced, revisit it to check whether you've drifted from the fundamentals.