Examine testability needs
Purpose:
|
To gain a good understanding of the test implementation
and assessment needs to be addressed by either the software delivery process,
or the software architecture and design. |
Study the test automation architecture and test interface specifications to
gain a good understanding of the test implementation and assessment needs. In
particular, understand the constraints that these needs will place on either
the software delivery process, or the software architecture and design.
|
Assess impact and prioritize
Purpose:
|
To identify the testability needs that are most important to the test effort and advocate their
resolution before lesser needs.
|
Study the testability needs and perform basic impact analysis in terms of the impact on the test effort of
not having the need met. Also consider some basic analysis of the potential effort required by the
development team to investigate and provide a solution for the need. For each need, identify potential
alternative solutions that would have less impact the development teams.
Using this information, formulate a prioritized list that places foremost the needs that have a large
impact on the test effort if they are not met, yet have no alternative solution. Do this to both avoid
wasting valuable development resources on less essential testability needs, saving this opportunity for the
really important ones.
|
Define testability benefits
Purpose:
|
To be able to sell the value of the testability needs to the stakeholders in terms of basic
cost-benefits.
|
By asking the development team to develop software with specific provision for the test effort, you will be
adding further requirements and constraints to the development effort; that essentially equates to more
work and additional risk and complexity for the development team. Some development teams will view
designing for testability as outside the scope of their responsibility. In other cases, the testability
needs will have to compete for the development resources against customer needs and requirements that will
usually be given more priority. As such, you need to "sell" the benefits of the testability needs to the
project manager, software architect and other development team stakeholders.
Formulate an analysis of the benefits of each testability need you want to obtain commitment for. Research
papers, article and studies that support the value of your testability need, and make use of ROI statistics
where available. Think of the benefits in terms of the value provided to the development team; what useful
evaluation information will you be able to provide to them that could not be provided without this need
being met? How will this make it easier or more efficient for you to give the development team timely,
accurate, in-depth or useful feedback during each build cycle? Does this need provide the development team
with a useful feature that can be used in their own test effort or in future diagnosis of software failure?
In the case of competition against customer needs, consider ways you can show that providing a solution to
the testability need earlier will provide additional opportunities for customer requirements to be
supported in subsequent build cycles.
|
Identify and engage testability champions
Purpose:
|
To form alliances with important stakeholders who will champion the building of testable
software and support the test teams needs in this regard.
|
Given that you will be imposing potentially additional work or risk on the development team, you should
identify and engage with those influential stakeholders who have the ability to approve or mandate the
support of testability. Do this as soon as possible, before actively promoting the testability needs you
want supported.
The three most important stakeholders are the software architect, the project manager and the customer
representative. Spend time with the software architect and promote the value of creating a software
architecture that supports testability. Spend time with the project manager and promote the benefits of
testability in terms of test team productivity and fast turn around on evaluation information. Encourage
the customer to place value on a quality product being delivered.
|
Promote testability needs and benefits
Purpose:
|
To inform the relevant stakeholders of the important testability needs of the test effort, and
gain their support for testability.
|
It's important to promote testability needs in the right way. Each combination of project manager,
development team and customer stakeholders has a different social dynamic and culture, and it's important
to be sensitive to that when you promote testability needs. As general heuristics, don't mount a formal
testability "campaign" if the team is relatively laid-back and informal; and don't use an informal approach
in a high-ceremony project.
In some cases, a collaborative "brainstorming" session is a useful presentation format, where the need is
presented as a challenge to the development team, and they are encouraged to identify creative solutions to
meet the testability need(s). This encourages their ownership of the solution and fosters a feeling of
partnership in the effort.
Timing is also important for this step. As a general rule, you should try to identify and promote the most
important testability issues as early as possible, generally during the Elaboration, and where possible the
Inception phase. When testability issues are raised in these early stages of the project, the team is
typically smaller and is more receptive to change. It's also easier to include these needs in the evolving
design as minimal rework is usually required.
A good place to identify testability needs and present them in a positive and less "official" manner is to
have the test team offer their services in evaluating proof-of-concept activities and in evaluating the
selection of third-party components for use in the development effort. In particular, the involvement of
test teams during database component selection, UI control or component selection, middleware components
etc. means that testability issues can be used as one aspect of the component selection criteria. For
example, in many cases development teams will have minimal concern over which UI widget library to make use
of; if one library is more testable than another, the development team will be happy to select the more
testable widget library.
If you've had trouble identifying or engaging with testability champions, you may need to consider either
an approach that introduces the changes more incrementally making them potentially less risky and smaller
blocks of effort, or you may have to escalate the most important testability needs as critical project
issues that prevent the test effort from being successful until they are resolved. In the latter case, we
recommend you carefully consider all your options before deciding on this course of action.
|
Gain commitment to support and maintain testability
Purpose:
|
To gain an agreement that the development team will continue to support and maintain
testability features.
|
It's important to ensure the testability needs are regarded in the same way as any other requirement or
constraint placed on the development effort. You need to be assured that the testability features made
available today will not be abandoned tomorrow.
In some cases, attempting to gain this commitment may result in the development team refusing to develop or
support the testability needs. While this can be disheartening, it's better to be aware of this situation
and deal with the reality of it at as early as possible; it's much worse to have spent extensive time and
effort developing a test implementation that the development team then abandon uncommitted support for.
|
Advocate the resolution of testability issues
Purpose:
|
To monitor and champion the resolution of testability issues.
|
While the development team may agree to provide the necessary support for the testability needs of the test
effort, it's important that you take an active interest in the design, implementation and completion of
this work. Don't simply abandon concern because the development team have agreed to address the testability
needs or have begun work on a solution; you need to ensure that an appropriate solution is developed in a
timely manner.
Make yourself and the other test team staff readily available to answer the development teams questions,
and offer to evaluate the prototypes as soon as they are built. Offer constructive feedback and show
enthusiasm for the effort the development team have put into helping meet your needs. Offer to have your
key staff attend or facilitate design workshops for the more complex testability needs, but guard against
your team being overbearing and controlling the solution space of the design process for the developers.
Where issues arise and you feel they are not getting adequate attention, or being addressed with the
necessary haste, raise your concerns with the software architect and project manager. Have the project
manager log an issue on the project issue list if appropriate.
|
Evaluate and verify your results
Purpose:
|
To verify that the task has been completed appropriately and that the resulting work products
are acceptable.
|
Now that you have completed the work, it is beneficial to verify that the work was of sufficient value, and
that you did not simply consume vast quantities of paper. You should evaluate whether your work is of
appropriate quality, and that it is complete enough to be useful to those team members who will make
subsequent use of it as input to their work. Where possible, use the checklists provided in RUP to verify
that quality and completeness are "good enough".
Have the people performing the downstream tasks that rely on your work as input take part in reviewing
your interim work. Do this while you still have time available to take action to address their concerns.
You should also evaluate your work against the key input work products to make sure you have represented them
accurately and sufficiently. It may be useful to have the author of the input work product review your work on
this basis.
Try to remember that that RUP is an iterative delivery process and that in many cases work products evolve over time. As
such, it is not usually necessary-and is often counterproductive-to fully-form a work product that will only
be partially used or will not be used at all in immediately subsequent work. This is because there is a
high probability that the situation surrounding the work product will change-and the assumptions made when the
work product was created proven incorrect-before the work product is used, resulting in wasted effort and costly
rework. Also avoid the trap of spending too many cycles on presentation to the detriment of content value.
In project environments where presentation has importance and economic value as a project deliverable, you
might want to consider using an administrative resource to perform presentation tasks.
|
|