Effective test approaches, efficient tool support, testing certification

Knowledge, Skills and Certification

Certification path for example job roles

ISTQB®, iSQI® and TMMi® certification schemes examine software testing knowledge and skills via clearly defined learning objectives set out in their syllabi.

Whilst I agree with some of the points made in the Professional Tester’s Manifesto (which used to be published at professionaltestermanifesto.org), I disagree with the final clause about certification in the opening statement (my italics):

standards compliance is no substitute for knowledge and skills, and that possessing a certificate demonstrates neither."

Exam questions for each of the example qualifications shown above are derived from learning objectives set at a number of ‘K levels’ from K1 to K6.

Four of the above qualifications are examined with multiple choice questions.

The Certified Agile Tester (CAT) however is examined via two exams (practical and written/essay) together with the tutor’s assessment of the candidate’s capability for teamwork. The exams enable K5 and K6 learning objectives to be examined, and for candidates to demonstrate their knowledge and skills in a practical laptop based exam.

A summary of the six knowledge levels set out at http://www.istqb.org/exams-root/what-are-k-levels.html is:

  • K1 – Remember
    There are learning objectives for remembering terms, concepts and practices. They are set at Foundation level for both the ISTQB and TMMi qualifications. ISTQB Foundation has 35% of its learning objectives at this level, whereas the TMMi Professional has 15%.
  • K2 – Understand
    All of the qualifications shown above have learning objectives at this level, having between 45% and 85% of all learning objectives depending on the qualification. For example candidates need to select the best explanation for a statement related to the question topic.
  • K3 – Apply
    Mainly the ISTQB Advanced Level and iSQI CAT qualifications have learning objectives relating to selecting the correct application of a concept or technique, having between 20% and 40% of the learning objectives depending on the qualification. The ISTQB Foundation also has a small number, including relating to applying four functional testing techniques.
  • K4 – Analyse
    Again mainly the ISTQB Advanced Level and iSQI CAT qualifications have learning objectives at this level. These typically separate information related to a procedure or technique into its constituent parts for better understanding. They are often scenario based questions where the best answer depends on the testing context. They have between 10% and 30% of the learning objectives depending on the qualification. The ISTQB Foundation also has one for examining two structural testing techniques.
  • K5 – Evaluate
    These relate to making judgements based on criteria covered by the learning objective. For example a CAT candidate may need to assess the quality of example user stories and explain their deficiencies.
  • K6 – Create/Synthesis
    These relate to putting forward and justifying opinions about the learning objective, or as the iSQI CAT qualification states “(putting) information together in a unique or novel way to solve a problem”. For example a CAT candidate may need to tailor the Scrum process to cope with scenario specific constraints.

A valid point made in the Professional Tester’s Manifesto is that organisations put the quality of their testing at risk if they “use certification as a surrogate for rigorous selection processes”.

My view is that certification can have an important part to play in rigorous selection processes by demonstrating various levels of knowledge and skills. The caveats are that those setting the selection criteria for testing roles need to understand what each certificate does and doesn’t demonstrate, and to take into account all other criteria pertinent to the role.

There will be times where having a certification is mandatory. For example, the TMMi Foundation mandates that TMMi Accredited Assessors most have the TMMi Professional qualification. However, in order to undertake formal assessments of testing maturity these assessors will need to meet additional experience and training related criteria, including having had TMMi Assessor training. Accredited Lead Assessors will also need to have passed the ISTQB Advanced Level Test Manager qualification, and to have met further criteria set by the TMMi Foundation.

It’s through basing exams on clearly defined learning objectives that the ISTQB, iSQI and TMMi certificates can be used to demonstrate various levels of software testing knowledge and skills.


01 February 2015 (original)
04 February 2018 (updated)