insights

Tomas’s Post: Test Process Improvement – Quick Wins

Last couple of years, I am constantly facing the similar questions from project managers, colleagues from development teams as well as clients themselves, regarding test process and outcomes of this process.

“How to increase quality of delivery, how to test more efficiently and how to find defects before they hit the production. We all would like to achieve this, but just fewer understands what is behind those improvements, how much time it costs and what has to be done.

There are several methodologies dealing with test process improvement. TMMI, ISTQB as well as TPI (as part of the TMap). All of them are good and can solve some of your problems. The challenge at the same time is that they may not be applicable strictly in every single project.

Good test manager or test consultant is able to take the best out of them and apply it in respective project environment.

Below are some of the quick wins / recommendations  to consider based on my experience divided into 3 main categories:

I just realized, that sometimes they are not as quick as needed when it comes to the fixing the problem. But even knowing the problem is the first step of moving forward.

 

A) Production defects assessment – what we can learn out of it?​

If you are a new test manager or even and existing one not doing it so far. It is always good to understand, what the production issues are about. I recommend to have a look on production defects and make an assessment out of it.

What types of defects do we have in production

  • Was the same problem reported already in the test environment?
    • If yes, was it reported properly, making sure that severity of the defect was set properly.
    • Was the problem fixed in test environment? This may lead to the problems with the release management process.
    • Is the defect still open in test environment? May point to the environment inconsistencies.
  • Did we have a chance to find and report such defect?
    • Test environment specifics
    • Test data specifics
    • Test scenario not defined properly, not updated based on the latest business changes or not detailed enough.
    • Misunderstanding of the requirement
  • Was the defect reported in a new functionality, or an existing one?
    • If the problem was in an existing functionality, we need to have a look on our regression set

Results of such assessment may be very interesting and can reveal several weak points.

B) Test team assessment – how the team itself is working internally

a. Standard processes

  • Is the test process standardized? Do we follow the standardized processed?
  • Are those processes communicated to all involved parties and organization.

b.  Tools

  • What tools are we using or need to have access to.
  • Are those tools working for testers? Are they set up properly?
  • Do we need to introduce new tools?
  • Do we need to get rid of any tool in order to make our job more easy and simple?

c.  Templates

  • Do we have reusable templates? We do not need to invent every document from the scratch each time we have a new release / sprint… This will bring standards to our work, predictable outputs that our stakeholders and receivers will like and … will make our life easier and will decrease the overhead time needed for every release/sprint. (we will have more time to actually test)

d. Defect management

  • What is the quality of the defects that testers are reporting. Are we providing details necessary to fix defects?
  • Are we using any kind of statistic regarding defect quality? How many defects were reported as duplicity, how many were finally closed as not a defect and reason why? This all can reveal quality leaks in many areas can help to identify steps to improve. E.g. data quality, test environment quality (stability/instability), design documentation quality, test team quality and necessary trainings,…

e. Test environments

  • How many environments do we have and how are they managed?
  • What is the quality of the test environment. Is it a production like environment?
  • Who is owning the environment?
  • Are we able to control, what and when is released to test environment?

f. Team sizing and capabilities

  • Is the team big enough to cover all the needs and expectations? Do we have enough time to test what we need? If not, are we able to communicate our needs.
  • Do we know how to estimate effort on testing? If the effort or other needs are underestimated and not communicated well, we are putting ourselves under a big pressure right from the beginning.
  • Do we as testers, have a capabilities to do our job? E.g. do we know how to search in DB or Logs (if this is expected from us), do we have a business knowledge, or are we testing just based on the test scenario description

g. Reporting

  • Do we report results in a clear and transparent way? Business users and / or project managers are making decisions based on reports. Do they receive all they need? Do they understand what they get, in order to make a firm decision. Pay attention on details. Sometimes even a color change can make a difference. There is nothing worst for test manager than facing hundreds of clarification questions each time a new report is issued.

h. Retrospective

  • Do we have a lessons learned sessions and are we implementing them?
  • Are we regularly reviewing regression sets. This refers also to the production section.

C) Communication flows – test team interaction with other teams

Sometimes the most tricky ones and most challenging to improve.  Once you will do the assessment of the first 2 categories, you may already find a weak points even here.

  • Do the people communicate to each other, there might be professional miscommunication but we also have to consider social aspects.
  • Are we as testers able to communicate properly both with the development teams as well as with the project management and business?
  • Do we communicate our test processes properly, to set expectations and make sure that other parties understand what it takes to test properly
  • Communication using tools. Do we use tools properly? What is the master of truth?
  • Do we have point of contact in case of any question or query?
  • Building a strong relationships with other teams is very important may be one of the most important value that good test manager can provide.

What is the company culture? Is it a learning company where asking questions and making a mistake is a way of learning, or is it a company where people are afraid of defects reported and connected to them?

Good luck

Tomas

—————-

Sources: