Quality – the constant battle to deliver products fast, yet uphold a high level of quality. Engineers have been engaged in this battle since the beginning, and the way they’d reconciled the dilemma was to introduce development process improvements. But this meant there was lesser focus on the testing standard itself. To counter this limitation, the testing community came up with TMMi.
The TMMi framework was developed by the TMMi Foundation as a guideline and reference framework for test process improvement and is positioned as a complementary model to CMMi, addressing issues important to test managers, test engineers and software quality professionals.
So, what’s the scope of TMMi?
- System Engineering & Software Engineering – Improves theses stages by improving test activities and the test process framework.
- Test Levels – There are several test process improvements out there, but what makes the TMMi process different is that it covers both high-level testing and low-level testing (e.g: Dynamic Testing, Nonfunctional testing, component integration, etc.)
- Assessments – When introducing a new concept to a company, the stakeholders will want to know where they are in terms of process at the moment and what the positives and the negatives are. The TMMi framework allows these assessments to be done easily.
- Improvement Approach – Remember, even if TMMi is introduced, and the foundations are laid out, things won’t progress further without senior management support, and that’s where this comes into play.
Now that the basics are covered, you might be wondering, why your organization need a standardized test process?
The simple answer is, no matter how big your company is, or how good the engineers are, software fails. There are several examples we can look at, of major companies going through major losses because of software failures. Let’s look at some of them:
- Airbus A380 suffers incompatible software issues in 2006
- The Patriot Missile Failure
- Nissan recalls 990,000 cars because of an airbag malfunction
TMMi Maturity Levels
The TMMi framework has 5 levels. At a given time a company can only be in one of these levels. The levels are:
- Level 1, Initial – Things are usually a bit hectic at this stage. The traits of this level are:
- Testing being a chaotic process
- Process is undefined, and is often considered a part of debugging.
- The company usually doesn’t have a stable environment to support process.
- Success is measured by competence and the heroics of people and not the use of a proven process
- Tests are developed in an ad hoc way, after coding is complete.
- Level 2, Managed – TMMi switches to being more managed process. The traits for this level are:
- The fundamental test approach is established and managed.
- Test policy and strategy is different from project to project.
- A direction for planning, monitoring and control of test design techniques are given, with each project having control over test executions.
With the introduction of Level 2, process areas at TMMi are also introduced. These are:
- Test Policy and Strategy
- Test Planning
- Test Monitoring and Control
- Test Design and Execution
- Test Environment
Things are certainly better than it was before in this stage, but the problem of late testing still exists.
- Level 3, Defined – Processes become clearly defined. The traits for this level are:
- Teams are more organized, and test training programs are implemented.
- Testing becomes integrated into the development life cycle and early development.
- Non-functional testing is planned.
- Reviews are now used in each project.
With Level 3, the process areas are now:
- Test Organization
- Test Training Program
- Test Lifecycle and Integration
- Non-functional Testing
- Peer Reviews
The main difference between Level 2 and Level 3, is that Level 2 focuses more on Functional testing, whereas in Level 3 Non- Functional testing is being focused on as well. Companies on Level 3 also puts focus into Automation.
- Level 4, Measured – It’s all about measurement and improvements in this level. The traits for this level are:
- Measurement of activities
- Outcomes are thoroughly applied early, in all projects
- Advanced reviews are in use early, in all projects
With Level 4, the process areas are now:
- Test Measurement
- Product Quality Evaluation
- Advanced Peer Reviews
The main advantage of going into this stage is that it allows the organization to build a better technical, managerial and staffing infrastructure within the organization.
Reviews are also done on current processes, where the weaknesses are identified and improved on. Product Quality Evaluations can be done as well by defining quantitative quality needs, quality matrix and quality attributes.
- Level 5, Optimization – Companies that are here, focus heavily on defect prevention. The traits for this level are:
- All activities and outcomes are assessed.
- Optimized activities are in place to ensure continuous improvement towards defect prevention and optimized quality
With Level 5, the process areas are now:
- Defect Prevention
- Quality Control
- Test Process Optimization
When companies are at this stage, they do heavy analysis on their products, to the point they know what the common issues are. Once this is recognized, measures are taken to counter this and ensure the same problems don’t repeat themselves.
TMMi when done right, could take your product to the next level. It takes the support of the senior management and commitment from the staff to implement it, but once that step is taken it could change the way quality is viewed in your company.