ASQ - Software Division

US TAG Meeting Notes

This report is based on the US SC7 TAG meeting at the System and Software Consortium (formerly Software Productivity Consortium) in Herndon, VA (April 4 - 6).

This was the 52nd meeting of the US SC7 TAG and membership seems to be slowly, but surely, growing. Among organizations showing an interest, by sending a representative to this meeting to view the TAG's work, were the Project Management Institute and the Department of Homeland Security.

Safety and Security

DHS was represented by Joe Jarzombek, formerly of the Department of Defense, from the Cybersecurity Division as Director of Software Assurance. In this role software safety and security are a primary concern. As noted in the IEEE notes above, there could be CSQE body of knowledge considerations based on the educational "outreach" effort DHS is funding for colleges and universities that include safety and security material in their software engineering and computer science curricula. Joe recently spoke at the 14ICSQ on a (largely aerospace) panel on software quality and he presented similar material to the TAG, including his emphasis on avoiding process "wars" and focusing on evidence (deliverables/records) of meeting requirements (esp. security) prior to receiving the software.

Given the large number of government and government contracting organizations that are members of the US TAG, this topic will likely continue to grow in importance in standards work.

Organizational Maturity

Working Group (WG) 10, which does the ISO 15504 series on Process Assessment, has a proposal before it to look into developing an organizational "maturity" determination from the process capability profiles that come from a 15504 assessment. Among the issues to be faced would be how to do this without pre-specifying the processes that have to be assessed since 15504 does not specify what processes to asses, just how to report the assessment results regardless of the processes assessed. An organizational maturity "level," at least to this point, has always been in relation to a specific set of process areas. So the key question is how to get the maturity level if you do not, in effect, link it to a Process Assessment Model. How could one, generically, define maturity apart from the domain (processes) in question?

Also, another ISO effort, not in SC7, is looking at "social responsibility" as a standards area (in light of corporate "misbehaviors" around the world).

COTS Product Evaluation

The WG6 work program includes the SQuaRE (250nn series) work combining the ISO 9126 and ISO 14598 document sets as well as ISO 12119 on Product Evaluation Requirements for COTS software. The former is covered below, but 12119 is headed toward its final stages, or so it would seem. Some concerns with this work are:

  • A rather broad definition of COTS covering use of packaged software on a personal PC as well as embedded in safety-critical systems.
  • Relying on the use of marketing (packaging) documentation and end user manuals as a basis for testing requirements and specifications.
  • Stating requirements against which a COTS product would be evaluated in a manner that would make it hard to impossible to prove whether compliance had been achieved (e.g., use of adverbs in requirements, resulting in subjective judgments of whether a requirement was met).

Mike Kress is one of the co-editors on this work and has been working very hard to move this in a direction that will result in an effective standard. There has also been a lot of support from the IEEE Society for Technical Communication looking at the user documentation impact of this work. For example, one of the requirements used to say that all features and things documented mentioned in user manuals had to have formal evidence of being tested. Of course, this could have resulted in people providing less comprehensive user information to avoid having to prove that features worked. The STC folks did not want to see something pass that would result in vendors feeling it was better to document (even) less for the user to avoid having to show test results on all of it.

SQuARE (Software Quality Requirements and Evaluation) Series

This work progresses deliberately, though there are still many concerns about it. To begin with, the original goal seemed to be to take two existing standard sets (9126 and 14598), which had some redundancy between them as well as synergy in that one described a product quality model while the other how to evaluate/test a product against a quality model. Taking 2 sets of 4-5 documents each and produce a single combined set seemed worthwhile. What has happened is that two sets of 4-5 documents have gone in and 19 documents are slated to come out.

The overview document (ISO 25000) is to be voted on as a final international standard, being the first to get to this stage after several years. The document describing the actual quality model (25010) has not yet really been started while other, more subordinate documents, are being done. Due to the many documents, the editorial effort have been divided up among many people and documents seem to be getting progressed based, not on their importance to defining the set, but on the amount of time given editors have to work on them.

Also, given the large number of documents, there is considerable redundancy between the content in many of them describing the SQuaRE series, though not identical material. That is, they talk about the same thing but not always using the same set of figures and text, creating inconsistencies and maintenance problems between documents. (The large number of documents also seems to be partly due to informative Annexes from the original 9126 and 14598 set being extracted and made separate guidance documents.)

This is a potentially very important piece of work as many people have found the "ilities" model in 9126 to be useful in focusing attention \ non-functional requirements for software products.

Software Engineer Certification Proposal

A Study Group has been working on investigation of international certification of software engineers and they have issues their final report. The SG was not to develop a certification effort as many exist (e.g., the IEEE"s CSDP, Japan's ITEE, British Computer Society exams, Germany's iSQI). Their goal was to describe principles and approaches to guide such efforts to ensure broader international acceptance of them. In doing this, the SG drew heavily on ISO 17024, which describes Conformity Assessment "requirements" for certification bodies and focused on "examination-based" certification efforts. This material might be of interest to our CSQE efforts to see how we meet what is being proposed as an international guideline for such examination efforts.

Vocabulary Proposal

I volunteered to work on this to help out the newly appointed editor proposed by the IEEE (Annette Riley from the STC, mentioned above). Basically the work is just starting with defining the requirements for the electronic repository. (A printed version of the final consolidated vocabulary will exist, but the idea is that someone, maybe IEEE, will host an online-accessible database so people can look up definitions when they need them and use them verbatim as long as the reference to the source is given.) Initial work will include consolidating IEEE's 610.12 vocabulary (10 years old and unrevised) with SC7 vocabularies from all those standards. (At the same time, there is vocabulary work at the JTC1 level being considered. JTC1 is the upper-level committee under which SC7 and 20+ other computing-related sub-committees exist, e.g., SC22 on programming language definitions.)

I think this will become useful as formal definitions at the international level will then exist for things like "(software) quality assurance" and (software) quality control" among many other terms used loosely in software or inconsistently with other domains.

Measurement Process Revision

WG13 existed some time ago and produced ISO 15939 which describes a process and model for software measurement. It was based on the Practical Software Measurement work from the DoD and is deemed to be one of the more successful software-related standards efforts. Time is approaching that its revision be considered and the work has been moved to WG7 (system and software life cycle processes), given the process definition related nature of the work and the desire not to reopen a closed WG. The proposal for the revision work has not been issued yet, but should be coming along this year. (Dave Zubrow, the US lead on the SQuaRE work, has been working to convince the international community to incorporate more and more of 15939's measurement process model as part of the SQuaRE work, including its definitions of measurement elements.)

System and Software Life Cycle Harmonization

I have left this topic until last because it is the biggest effort going on in SC7, from my perspective, given that its intent is to bring ISO 12207 (software life cycle processes) and ISO 15288 (system life cycle processes) into closer agreement on names, descriptions, purposes and outcomes where processes between the two are related. The work began some time ago, but given the significance that 12207 and 15288 play in SC7 as the framework standards, changing them while maintaining compatibility for current users is difficult.

Since this is also perhaps the most widely attended/participated in WG, there is considerable variety in the opinions and ideas about how standards within its sphere should read and work. Indeed, a typical comment response set for a ballot on a given WG7 document is often larger than the document itself, comprising hundreds of comments. When one considers that the US TAG resolves many comments within its own Technical Group (TG), the volume being produced at the national level is much larger than those that even make it to the international level.

Among the things this effort may need to address are:

  • Consistency in naming, descriptions, purposes and outcomes for essentially identical or very closely related processes;
  • process boundaries and interfaces;
  • an overall process management architecture for all processes (and projects employing them) perhaps based on steps such as: Initiate, Plan, Execute, [Measure, Assess,] Control, Close;
  • single categorization hierarchy and process architecture perhaps like the following:

    Technical Processes
    Stakeholder Requirements Definition
    Requirements Analysis
    Architecture Design
    (SW specific Detailed Design
    SW specific Coding and Unit Test)
    Verification [V&V can be apply to other than an end product]
    Validation [V&V can be apply to other than an end product]
    Maintenance ["Sustainment" is another term]

    Support Processes
    Risk Management
    Decision Analysis
    Configuration Management
    Information (document) Management [at project level]
    (Project) Quality Management
    Plan Implementation [Plan, Assess, Control - part of all processes as noted above]

    Enterprise Processes
    Infrastructure and Commitment
    Life Cycle Process Definition and Improvement
    HR Management and Training
    Financial Resource Management
    Quality Management
    Organizational Alignment
    Knowledge Management [at organization level]

    Agreement Processes
    Prepare Acquisition
    Advertise Acquisition
    Supplier Tendering
    Supplier Selection
    Contract Agreement
    Supplier Monitoring
    Contract Execution
    Acquirer Acceptance
    Product or Service Delivery and Support

One of the first documents to appear for this work may be a Definition of Concepts document which will be numbered 24748-1 since a series of harmonization related documents will appear given the breadth and depth of the subject.

Software Division (and Division Member) TAG Participation

As always, we remain involved in WG10 activities related to process (and perhaps now organizational) assessment and the WG6 work on SQuaRE and COTS product evaluation. Mike Kress, as a Boeing representative, is directly active on WG6 work where he is an international co-editor for ISO 12119. Carol Dekkers, as an IFPUG representative, is directly active on WG12 work related to Functional Size Measurement.

Joel Glazer, the US TAG alternate representative for the Division, was able to attend this meeting for the open day's morning activities and will hopefully be able to do likewise in September when the TAG meets at the NIST facility in Gaithersburg, MD.

ASQ News


Follow the
Software Division

Twitter  LinkedIn