Real QA Manifesto

Table of contents

‘Real QA’

[Real Software and System Quality Assurance]
© By Tom at Gilb.com & Kai@Gilb.com May 22 2009

Purpose of this document/manifesto:

Quality-Assurance (QA) in software worldwide has in fact degenerated into testing alone. Software/IT management has ignorantly allowed this to happen.
Of course many parts of the industry have been well-aware of more cost-effective ways of delivering required Quality in practice, but this has in fact been largely ignored; while granting very large Resources to testing alone.

It is time for a wakeup call!

This manifesto is here to tell the industry that;

  1. testing alone is 10x more costly than doing Real QA.
  2. testing alone is not good enough, this can and need not go on.
  3. we know how to do real QA much much better than testing alone, using smarter upstream Engineering practices, based on design, prevention and upstream inspections.

QA Objectives

  1. NO SURPRISES ASSURANCE: To allow management to understand release consequences fully, in relation to expectations, with the lowest costs, the lowest risks, and the lowest degree of surprises.
  2. MEET EXPECTATIONS: To make sure that the project investors and sponsors get, at least, what they expect.
  3. BUSINESS PERFORMANCE: To deliver the ‘business’ (or organisational) results envisaged and promised to project and programme supporters.
  4. TECHNICAL PERFORMANCE: To Quantify and Measure the technical performance (including all system Quality Attributes) Attributes of the system.
  5. CONSTRAINT COMPLIANCE: To ascertain that the system has not violated any specified constraints.
  6. LONG TERM ASSURANCE: To give some assurance that the long term characteristics of the system are as planned. Things like adaptability, maintainability.
  7. LEGAL COMPLIANCE: make sure the system is always compliant with legal and other compliance Policy items.

QA Strategies: valid QA Strategies include, but are never limited to:

  1. Clear and quantified project and system Level objectives (Value-Requirements), at the Level of the organization, the Product (example a software package), and the system (all related aspects such as people, support, documentation, marketing). Using ‘Planguage’* to express the objectives clearly and fully.
  2. Clear and detailed Impact and Cost estimated strategies for meeting the objectives. Rated and measured using Impact-Estimation* (Value-Decision) specification language. Strong Architecture design to meet multiple quantified objectives and constraints.
  3. Quality-Control Reviews (Agile Inspections or Spec-QC *) of all forms of specification and documentation against sufficiently high Standards (Rules, Processes, and other types of Standards *) and higher Level Source Documents.
    The reviews will operate at early stages (leading measures) to
    1. motivate and teach engineers to utilize best practices.
    2. measure the degree to which every development and maintenance output actually meets the best practice adopted Standards.
    3. give a Basis for serious numeric Process entry and Exit conditions* for all Engineering Work outputs.
    4. give a partial data-driven Basis for continuous and immediate Process improvement (like Defect-Prevention-Process (DPP) and CMMI Level 5).
  4. Rapid Evolutionary iteration, incremental value delivery to Stakeholders, data collection, feedback, analysis and change (as in Value-Delivery / Evo *).
    The purposes of this are to early, often, rapidly and throughout:
    1. be able to prioritise high value deliveries.
    2. validate that the teams are actually able to deliver value at all.
    3. learn about everything, and improve anything.
    4. enable all Quality-Assurance tactics to be tried and proven.
  5. Automated and built in Measurement and detection of problems, including very direct real-time user and Stakeholder feedback to developers and engineers. Operating Before releases and eternally After releases.
  6. Ongoing Fact Analysis: Software Engineering Accounting: capability of analysing, organisation wide and in the long term how all these things are working, so as to determine what works best and what the costs are.
  7. oh yes, and not to forget, conventional testing, at its best.

- * as detailed in Gilb: Competitive-Engineering and Evo – or any better or equivalent specification method.
This manifesto is not about Gilb methods, they are mentioned here as examples so that no one can misunderstand what we mean by 'quality', 'inspections', and other concepts.

QA Principles (general ideas that guide us in detailed decisions)

  1. Early Bird: QA must operate as early as possible to detect problems both in the current development/maintenance Process, and in the Product or system being engineered.
  2. Quantification: quantification will always be preferred as a language to express any variable Idea associated with Product, system and Process ideas. It is always possible, and it is the only sound Basis for rational thinking by management, Engineering, and academic research.
  3. Prevention: rather than tolerate eternal levels of problems to clean up later at 10x the Cost of doing it right in the first time, we will rapidly invest in a shift to defect prevention.
  4. Rapid Feedback: we will position our activities to get rapid feedback (like this same week) so we can correct bad things as soon as possible, and so they cannot fester for months and years.

QA Values

  1. Efficiency: we will always strive to find and implement the most Cost effective methods to assure long term system Quality.
  2. Confidence: our assertions of confidence in system releases can be relied on totally, and will be very explicit about caveats, Assumptions, maximum deviations, and responsibilities if accepted.
  3. Predictability: we will develop our ability to predict system Attributes based on early indicators (example: field bugs based on major defects in Requirement).
  4. Leanness: we will constantly remove and avoid all activity that does not have clear measured value contributions in relation to its real Cost.
  5. Perceptiveness: even when not directed by our Stakeholder to explicitly deal with them, we will anticipate all critical system Quality aspects.
  6. Reality: we will not be driven by ideals, fads, customs, misunderstandings or illogical arguments. We will be directed by current local realities: how things Work for us now.
  7. Delegation: avoiding micromanagement - we will practice extreme delegation for details and choice of design, and for Processes, to the people who do them daily. Management’s Role will be limited to setting clear measurable high Level objectives for products and Processes, and then enabling their staff to reach them.

The IT Management Role in Making Real QA Happen

The testers have not been leading the change to real and Complete QA. They ‘test’.
The developers have not done anything laudable either. They code.
Managers don’t seem to have a clue, but then nobody actually trained them to understand broad QA. They push deadlines, complain and pay.
The universities and their professors have no discernible honour in educating us in broad QA. They lag.

Nothing is going to change unless responsible management (CIO, CTO, if necessary CEO levels) is somehow illuminated about QA, and chooses to insist it is exploited to its fullest potential. This can be done in part through a QA Policy.


a public declaration of Policy and aims, esp. one issued Before an election by a political party or candidate.
ORIGIN mid 17th cent.; from Italian, from manifestare, from Latin, ‘make public,’ from manifestus ‘obvious’ (see manifest 1 ).
False QA
is calling your activity ‘QA’ when in fact you only do testing.


Competitive Engineering
Gilb, Tom; A Handbook For Systems-Engineering, Requirements-Engineering, and Software-Engineering Using Planguage
ISBN 0750665076, 2005, Publisher; Elsevier Butterworth-Heinemann.

Gilb, Kai; Evolutionary Project Management & Product Development.


to Add your signature, click here Real-QA Manifesto - Signatories

Three steps to sign
1. register and log in to www.gilb.com.(external link) See top right of this site.
2. Click "Edit", see bottom of this page.
3. Add your signature and click "Save"


Signatories (Advisors):

The following signatories support the ideas and Claim to be able and willing to help management achieve the Real QA Manifesto elements, in part or whole.
And they will on request, directly or publicly, give reasonable evidence of their real capability in terms of experience, results, facts, measures, references and writings (papers, books, slides) by or about their efforts to date; so that managers might fairly evaluate their potential value.

nr. Name - email - website - date

1. Tom Gilb - tom at gilb.com - www.gilb.com(external link) - 22 May 2009
2. Kai Gilb - kai at gilb.com - www.gilb.com(external link) - 22 May 2009
3. Niels Malotaux - niels at malotaux.nl - www.malotaux.nl(external link) - 27 May 2009
4. Ben Linders - http://www.benlinders.com(external link) - 27 May 2009
5. Dr. Lawrence E. Day - lawrence.e.day at boeing.com - 27 May 2009
6. Jerry E. Durant - jdurant at Int-IOM.org & jdurant at Certellus.net - 27 May 2009
7. Ron Radice - rradice at stt.com - www.stt.com(external link) - 27 May 2009
8. S M Kripanidhi - kripa at binaryessentials.com - http://www.binaryessentials.com(external link) - 27 May 2008
9. Ryan Shriver - ryanshriver at mac.com - http://theagileengineer.com(external link) - 27 May 2009
10. Rolf Goetz - rolf.goetz at gmx.de - http://ClearConceptualThinking.net(external link) & http://PlanetProject.wikidot.com(external link) - 28 May 2009
11. Lorne Mitchell - lorne at objectivedesigners.com - www.objectivedesigners.com(external link) - 28 May 2009
12. Clifford Shelley - shelley at osel.netkonect.co.uk - www.osel.co.uk(external link) 28 -05 -09
13. Ralph Young - rryoung at mitre.org - www.ralphyoung.net(external link) - 28 May 2009
14. Bob Marshall - bob.marshall at fallingblossoms.com - www.fallingblossoms.com(external link) - 28.05.2009
15. Wayne Mallinson - waynem at testdata.co.za 28.May.2009
16. Sankaran Natarajan- sankaran.natarajan at gmail.com - www.linkedin.com/in/sankaran1(external link) - 29th May 2009
17. Lars Ljungberg - lars at ljungberg-quality.com - www.ljungberg-quality.com(external link) - 29th May 2009
18. Peter Roesler - ros at reviewtechnik.de - www.reviewtechnik.de(external link) - 29 MAY 2009
19. Mary Poppendieck - mary at poppendieck.com - www.poppendieck.com(external link) - June 1, 2009.
20. Trevor Reeve - trevorreeve at mail.com - 1st June 2009
21. Peter Leeson - Peter at qpit.ltd.uk - www.qpit.net(external link) - 1st June 2009
22. Jens Egil Evensen - jens.egil.evensen at avenir.no - 1st September 2009
23. Andoni Gonzalo - andgonro at gmail.com - October 1, 2009
24. Benjamin Jurg - Benjamin.Jurg at tmc.nl - www.tmc.nl(external link) - 23 nov 2009
25. André Heijstek - andre.heijstek at improvementfocus.com - www.improvementfocus.com(external link) - 1 september 2010
26 Han Guillaume - han.guillaume at foreset.nl - July 14, 2011
27. Name - email - website - date

feel free to Add your name here

Signatories (Supporters):

The following signatories support the ideas in the Real QA Manifesto:

nr. Name - email - website - date

1. Steve Tendon - - - May 27, 2009
2. Mike Smith - msmith at testing-solutions.com - - May 27, 2009
3. Petrus Vloet - Petrus.Vloet at siemens.com - www.it-solutions.siemens.com(external link) - May 28, 2009
4. Franc Buve - franc.buve at logica.com - www.logica.com(external link) - 28-05-2009
5. John Hammink - john at johnhammink.com - www.johnhammink.com(external link) - 28.05.2009
6. Rijn Buve - rijn.buve at tomtom.com - www.tomtom.com(external link) - 28-05-2009
7. Marilyn Bush- m.w.bush at ieee.org - www.marilynbush.com(external link) - 28-05-2009
8. J. David Blaine - jblaine at san.rr.com 28 May 2009
9. Johannes Brodwall - johannes at brodwall.com - johannesbrodwall.com - June 9, 2009
10. Kurt Häusler - kurt.haeusler at gmail.com - kurthaeusler.wordpress.com - July 3, 2009
11. David Dossot - david at dossot.net - www.dossot.net(external link) - August 9, 2009
12. Bob Corrick - bobcorrick at hotmail.com - - August 28, 2009
13. Jakub Oleszkiewicz - jakub.oleszkiewicz at gmail.com - September 17, 2009
14. Andoni Gonzalo - andgonro at gmail.com - October 1, 2009
15. Stuart James Woodward - stuart at capricorn14.com - www.capricorn14.com(external link) - 31st January 2010
16. Thomas Mosel - Thomas.Mosel at landisgyr.com - www.landisgyr.com(external link) - 1st June 2010
17. Franco Martinig - www.methodsandtools.com(external link) - August 6 2010
18. Alex van de Zanden - alex.van.de.zanden at ericsson.com - www.ericsson.com(external link) - August 31 2010
19. Terry Wiegmann, CSQE, CBAP - twiegmann at pillartechnology.com - February 3, 2011
20. Carol Dekkers, PMP, CMC, CFPS - dekkers at qualityplustech.com - February 3, 2011
21. Jaap Schuttevaer - jschuttevaer at kza.nl - July 14, 2011
22. Jan de Jong - jjdejongster at gmail.com - August 8, 2011
23 Name - email - website - date
feel free to Add your name here

Created by system. Last Modification: Saturday 02 of November, 2013 23:37:11 CET by Kai Thomas Gilb323 points .