Monday, 18 February 2008

Book Review: Software Testing Fundamentals by Marnie L. Hutcheson

image Driven to provide ways of providing better information to her customers, Marnie Hutcheson has identified techniques for identifying and structuring her test scope to allow her to provide estimates, negotiate and agree a prioritised scope, and report progress against that. All of which sounds like the makings of a great book.

But I have to say that it ended up as a strange little book. Unfortunately a lot of the book read like padding so I ended up skipping useful information and backtracking, and I did get confused by the book at times.
I think you can safely skip chapter 1 and just read the summary at the end of the chapter.
If you skip Chapter 2 you will miss some useful information so I suggest skipping to the middle of chapter 2 where Marnie discusses Quality as "getting the right balance between timeliness, price, features, reliability, and support to achieve customer satisfaction". While she relates this to the product under test, I think you can relate it to the test process itself and if you read the rest of the chapter in this light it becomes quite interesting.
The section in Chapter 2 relating to "picking the correct quality control tools for your environment" provides encouragement and advice on:
  1. automate your record keeping,
  2. improve your documentation techniques
  3. use pictures to describe systems and processes
  4. choose appropriate methods and metrics that help you and/or the client
Chapter 3 starts slowly but explains some useful rules:
  1. state the methods you will follow, and why
  2. state assumptions
then goes on to examine some methods of organising test teams. But I think you can probably skip the chapter and just read the summary.
The book starts to add value in Chapter 4 where it discusses the "Most Important Tests (MITs) Method".
MITs, as I understood Marnie's explanation of it:
  1. Build a test 'inventory' of all the stuff you know: assumptions, features, requirements, specs, etc.
  2. Expand the inventory into 'test's.
  3. Prioritise the inventory and related tests
  4. Estimate effort
  5. Cost the effort and negotiate the budge - as this dictates the scope of the inventory you can over
  6. Define the scope - an ongoing activity
  7. Measure progress
  8. Reflect on what happened to allow you to improve
I've paraphrase it above as Marnie does not use those exact words and the italic words are my summary keywords of the approach.
An exploration then follows of the MITs method in a plan driven, and in an Agile environment. The Agile environment does not match the Agile environments I have worked in so I found it difficult to relate exactly to what I do. Despite the useful thoughts presented here, I would have concerns if any tester in my Agile environment explained what they do in terms of the actual presentation in this book. I would have fewer concerns if they explained it in the 'spirit' of this book, or the generalised approach - perhaps using MITs lite.
The metrics chapter examines: time, cost, bug# (per attribute: component, severity, found date, etc.), some test coverage metrics, a Defect Detection Percentage  etc. If you get stuck for metrics then you'll find some in here that might work for you.
Chapter 6, 7 and 8 discuss the test inventory in more detail.
  • How to construct one through analysis of requirements, interviews, usage, system structure, data, inspiration.
  • What they can look look like, as spreadsheets, powerpoint, documents, tables
I found generally useful approach and experience documented here.
The 2 chapters on risk result in a heavily analysed inventory to identify scope and priority. I think  you should view this as a fairly typical presentation of risk and priority. The depth of coverage does highlight the importance that the MIT method places on analysis, agreement of importance and negotiation of contract, and I think you will gain some value from reading.
Two chapters cover structural path analysis. They cover one of my favourite techniques of drawing the diagrams to explain my understanding, and briefly mention using them in a dynamic way to build up a model as you 'do' something. Unfortunately, my main takeaway point was the use of path analysis as an estimation tool.
Data analysis - through paths, on boundaries, combinations - receives a quick overview and has some experience embedded within it.
So I come to the end of my reading and I found this a difficult book to read. I did not find its structure conducive to my understanding. Some of the topics that I use a lot - path analysis, data analysis - didn't lead me to believe that at the end of reading it that testers would use those techniques effectively.
I do recommend the basic principles of scoping and negotiation and if you haven't done that type of work before, and can get into the same rhythm as the book then it can probably help you in those tasks.
The basic notion of inventories and outlines seems perfectly sensible to me, but as a whole the method seemed too heavy.
I have used approaches similar to those listed in the book because I thought they were necessary for the project. But in hindsight I think they were necessary for me, at that time in my development as a tester, on those projects.
I think Marnie knows when to use her methods deeply and when to use a lite version, and how to tailor it. But I don't think her full experiences of using the approach really get communicated to the reader to allow them to do that.
I think that this book aims at the right audience of Beginner/intermediate tester. But I don't think the book communicates its underlying principles as well as it could. I think you will need to work the book to dig them out. But if you haven't used some of the techniques I've listed in this review then I think you will gain experience by reading the book and trying them.
Related Links

No comments:

Post a Comment