Examine the most current Test Evaluation Summaries
Purpose:
|
To understand the current summary assessment of product quality issues the test team have
identified.
|
Begin by examining the Test Evaluation Summaries that the test team has prepared. Compare the evaluation
information to the Test Plan for the iteration to understand the summary in the context of the planned
work. Raise any ambiguities and concerns with the test team members who prepared the summary and resolve
them.
For this step and subsequent steps that deal with gathering information and assessing the software quality,
try to obtain a balanced view incorporating both objective and subjective measures. Remember that objective
numbers only give part of the picture and need to be supported and explained by the current project
"climate". Conversely, don't rely purely on hearsay and subjective speculation about the software quality:
look for supporting objective evidence. We recommend you supplement your objective data by discussion with
either team leads or where possible individual team members to gather subjective assessments and gauge how
much confidence you can place in the objective data.
|
Examine selected Test Results for additional context
Purpose:
|
To gain a more in-depth understanding of the Test Results that support the current summary
assessment of product quality.
|
Based on the Test Evaluation Summaries, examine selected Test Results for additional context. Research the
results enough to feel confident you understand the important issues that have been identified in the Test
Evaluation Summaries.
Also review the data yourself and look for important trends evident in the Test Results data that may have
been missed. In general it's more important to identify what the relative trends in the data are indicating
rather than looking at absolute numbers. Be on the lookout for indications such as failures in one areas
that relate to failures in others.
|
Examine key Change Requests
Purpose:
|
To be gain the background to be able to effectively discuss the most important outstanding
issues and their possible solutions.
|
We recommend you limit this exercise to the most pressing Issues and associated Change Requests. You'll be
able to devote more energy to a smaller number of issues, and they are often more likely to have the most
impact on product quality. If you have a longer list of key issues, we recommend you devote appropriate
effort to them based on their relative priority: don't waste your resources by championing the least
significant issue. However, note that a significant number of outstanding lower-priority Change Requests
can make as significant a statement about the products quality as a handful of high-priority changes. Try
to group lower-priority Change Requests into logical aspects of quality based on the quality risk the
represent. This will help you articulate and advocate their combined effect on quality more clearly.
Identify important trends evident in the general Change Request data. In general it's more important to
identify what the relative trends in the data are indicating rather than looking at absolute numbers. Look
for positive signs such as a steady, continuous rate of defect resolution, or a gradual ongoing increase
and subsequent decrease in resolution rate over time. Be on the lookout for sharp peaks and troughs in
resolution rate that indicate the development team may be encountering process, environmental, political or
other problems that are reducing their productivity.
Note: you may also want to take the opportunity improve the clarity of the associated Change Requests,
eliminating ambiguity and emotive language and reasoning. If you make these changes yourself, discuss your
improvements with the individuals who originally created these work products so that they can understand why
the improvements are important.
|
Identify quality gaps and assess the associated impact and risk
Purpose:
|
To formulate your understanding of the key Issues in terms of the risk they represent to
product quality and the associated risk that poses to the success of the software development
project.
|
Identify each gap in quality and assess the associated impact and risk of each Issue that creates the gap.
Consider mitigation and contingency strategies, formulate your initial ideas for these and discuss them
with other team members.
Consider that "perfect" quality is arguably a somewhat mythical concept. Be careful to use a realistic and
attainable "quality bar" when assessing quality and identifying quality gaps. See Concept: Product Quality
|
Identify the essential actions to address quality gaps
Purpose:
|
To produce a realistic minimal list of required actions to negotiate satisfactory resolution of
the key Issues.
|
For each key gap in quality, consider potential mitigation and contingency strategies. Formulate your
initial ideas for these and discuss them informally with other team members to gain greater insight and
validate your thoughts. In the case of solutions, it's good to have options: they help you weigh up the
tradeoffs and take the best solution for the given context.
Work toward a set of useful candidate solutions and suggestions that will aid the project team in suitably
addressing each quality gap. It's important for you to do this so that the test effort is recognized as
contributing helpfully to problem resolution: not simply reporting problems. This is an important aspect of
advocating the value of the test team and gaining respect and cooperation from other team members.
|
Identify and engage with champions on major issues
Purpose:
|
To informally gather support for resolving the key issues, and gain an understanding of what
proposals are more likely to be accepted.
|
It's no fun to back a lost cause. It's usually a more effective approach to identify solutions to problems
that the project team are more likely to get behind, accept and commit to achieving. Keep a close
relationship with key decision makers, and consider starting by making these key issues visibly informally
through one-on-one discussion. Often that's a great way to win support, and find achievable solutions.
Sometimes you don't have any choice but to back a solution that is unpopular with the development team.
Where this looks likely, it's even more important to know who you can expect support form and find ways to
sell the solution that present it's value as clearly as possible or explain clearly the worse situation
that will arise by not resolving the problem.
|
Negotiate work priority
Purpose:
|
To advocate for an appropriate solution to be acted upon in an acceptable time-frame that does
not adversely effect required quality.
|
This is the crux of quality advocacy: being able to negotiate a suitable solution that both appeases the
development team and does not significantly reduce the quality of the product. Remember that in most cases
the test team primarily an advisor about quality, so you must be careful not to demand a given resolution
being taken.
However, it's important that the test team does a good job as an advocate for quality and that includes
sometimes being the bearer of news the project team would really not like to hear. This is where good test
teams provide the development effort with as much insight into the problem, its potential solutions and as
understanding as the tradeoff's for each choice as possible. You should act to some extent as an agent for
the eventual customers of the product and help negotiate solutions that will be in their best interest.
|
Monitor work progress
Purpose:
|
To remain supportive, involved and aware of the progress on the resolution of the issue.
|
Sometimes Defects and other Change Requests get lost in a sea of ongoing basic product development and
feature expansion. This is partly because it's more attractive for developers to work on "new stuff" than
it is to fix old and buggy code, and partly because business-value can be more obviously placed on adding
something new than fixing something broken. As quality advocates, the test team needs to help the project
see important defect fixes through to completion.
Successful software teams find a good balance between incremental quality improvement through defect
resolution and incremental feature expansion. The test team can assist the project team by finding ways to
encourage and support incremental quality improvement rather than taking less helpful and more adversarial
"quality police" role.
|
Confirm appropriate resolution of key Issues
Purpose:
|
To confirm that the resolutions for key issues effectively resolve the issue without
significant negative side effects.
|
Whatever solution the development team decide on to resolve a quality issue, the resolution should
ultimately improve quality. Be sure that you take time to assess the improvement in quality brought by a
given resolution and that it both addresses the original issue and does not adversely impact quality in
other ways.
For solutions that carry some level of risk themselves, it may be useful to conduct some testing of an
early release candidate before too much time and effort is devoted to following the resolution to it's
conclusion.
|
Evaluate and verify your results
Purpose:
|
To verify that the task has been completed appropriately and that the resulting work products
are acceptable.
|
Now that you have completed the work, it is beneficial to verify that the work was of sufficient value, and
that you did not simply consume vast quantities of paper. You should evaluate whether your work is of
appropriate quality, and that it is complete enough to be useful to those team members who will make
subsequent use of it as input to their work. Where possible, use the checklists provided in RUP to verify
that quality and completeness are "good enough".
Have the people performing the downstream tasks that rely on your work as input take part in reviewing
your interim work. Do this while you still have time available to take action to address their concerns.
You should also evaluate your work against the key input work products to make sure you have represented them
accurately and sufficiently. It may be useful to have the author of the input work product review your work on
this basis.
Try to remember that that RUP is an iterative delivery process and that in many cases work products evolve over time. As
such, it is not usually necessary-and is often counterproductive-to fully-form a work product that will only
be partially used or will not be used at all in immediately subsequent work. This is because there is a
high probability that the situation surrounding the work product will change-and the assumptions made when the
work product was created proven incorrect-before the work product is used, resulting in wasted effort and costly
rework. Also avoid the trap of spending too many cycles on presentation to the detriment of content value.
In project environments where presentation has importance and economic value as a project deliverable, you
might want to consider using an administrative resource to perform presentation tasks.
|
|