ASQ - Software Division

IEEE Software and Systems Standards Committee (S2ESC) Meeting Report

San Diego, CA (February 13-15)
Scott Duncan, SW Division Standards Chair

This report covers the IEEE Software and Systems Engineering Standards Committee (S2ESC - Executive Committee and Management Board) meeting in San Diego, CA (February 13-15).

The Executive Committee oversees all aspects of the S2ESC work, including:

  • administrative and technical policies and procedures,
  • liaison with other organizations (such as the ASQ Software Division, the Department of Energy (DOE), the Department of Homeland Security (DHS) and the Software Engineering Institute (SEI)),
  • strategic directions,
  • the IEEE book series on standards,
  • outreach to government, industry, academia as well as individual S2ESC members,
  • liaison work with ISO/IEC JTC1/SC7 to coordinate and harmonize the S2ESC and SC7 standards sets,
  • study/planning groups in quality management, testing and systems integration,
  • the Committee’s website.

The ASQ connection to the Executive Committee comes through my position as Software Division liaison to the Committee, a separate and additional role besides our Management Board participation. In this role, I am involved with S2ESC’s policy and procedure review, the quality management study group, the IEEE book series and the website redesign work.

The Management Board along with Working Group (WG) Chairs handles the “day-to-day” business of getting standards moving through the IEEE Standards Association process. This includes identifying a WG Chair, then monitoring the progress of the WG with the Chair responsible to submit the proposal for a new, revised or reaffirmed (extended without change) standard, put together the WG, conduct meetings of the WG (face to face or telecoms or email), assemble a balloting group and respond to ballot comments.

The ASQ connection to the Management Board comes through my position as a Representative, helping to support and monitor the progress of several Working Groups (1175 on CASE tool interconnections, 829 on test documentation, 1063 on information products, 1012 on verification and validation and 828 on configuration management) as well as being the Chair of the 90003 adoption and the 1648 effort on agile methods.

Management Board Standards Status

This time, there was no separate Management Board meeting. The status of standards was covered during the Executive Committee meeting.

Below is a summary of the status of some specific standards, some of which must be acted upon this year:

730 – (software quality assurance plans (revised in 2002)) will start its revision process in April/May of this year, led by Ron Dean, and change to a process, not product, standard, addressing “planning” not “plans.”

829 – (software and system test documentation (revised in 1998)) will be going for ballot soon and the WG, led by Claire Lohr, has processed 200+ pages of comments in this new version. There has been “very strong advocacy in many directions” for this work since testing matters to everyone, no matter what methods/processes are used. However, since the approval for revision of this document started in September of 2001, a clear path to get to ballot is needed to get an extension on the existing 829 or IEEE will withdraw it. The IEEE standards process “speed of light” is 50 days (minimum critical path), but realistically, it takes several months to get the necessary reviews, action on comments, etc. This is why the ballot on 829 could not be expected to be done by the end of this year.

1008 – (software unit testing (reaffirmed in 2002)) will start up a new revision this year with Dennis Lawrence as the WG Chair. It was suggested that people from 829’s WG might want to participate on this WG.

1016 – (software design descriptions (published in 1998)) needs an extension of the time to complete the work on this Recommended Practice.

1028 – (software reviews (reaffirmed in 2002)) has formed a 50-member WG led by Dennis Lawrence. They are reviewing the comments from the re-circulation of the 2002 re-affirmation.

1044 – (classification of software anomalies (reaffirmed in 2002)) a WG formed, led by David Card, to turn this into a process standard with a focus on defect causal analysis.

1045 – (software productivity metrics (reaffirmed 2002)) is targeting a ballot early this year.

1062-1998 – (Re-affirmed in 2002) Recommended Practice for Software Acquisition. Joe Jarzombek is currently listed to prepare a PAR. It is important that this standard’s revision encompass the entire acquisition stakeholder group, not just DHS and cybersecurity, and not just OSD. Could we get someone from DAU to participate? We don’ t need to have a PAR until 2007.

1063 – (software user documentation) needs an extension, pending SC7 work on ISO 18019.

1175.2, .4, .5 – (CASE tool interconnection (new)) still have work to do before balloting can commence.

1362 – (concept of operations documentation (published in 1998)) sent to SC7 for a study on IT services (ISO/IEC 20000, fast-tracking a BSI standard which the ITIL guidance documents address) and has been extended in time to allow SC7 to complete their study.

1420.1 – (software reuse (reaffirmed 2002)) may be candidates for “stabilization” (not require 5-year reaffirmations) or withdrawal.

1471 – (architectural description of software intensive systems) has been submitted to SC7 for “fast-tracking” (as ISO 25961) but needs some action on the IEEE side this year.

1540 - will disappear and ISO 16085 will replace it, avoiding the need for a vote.

1644 – (naming conventions for application software (new)) has not made progress over the past year. There is an opportunity to cooperate with ISO TC 184.SC5/WG4 WRT ISO 16100.

1648– (adoption of agile methods (new)) next draft about to be reviewed by WG and goal is to get into ballot by the end of the 3 rd quarter of this year. The PAR name will probably have to be changed since it does address managing agile development projects (each flavor of agile has its own idea of PM). Material that is primarily informative (e.g., philosophy, etc.) will have to be removed to maintain the document focus on the kind of behaviors customers should expect of themselves and an agile development team.

2001 – (web page engineering (published 2001)) successful fast-tracking and soon to be published as an ISO standard.

2063 – (requirement engineering (new)) is a merge of IEEE 830 and 1233 to become a process standard with a template for a requirements specification.

9126-1 – (software product quality model (adoption of ISO version from 2001)) needs work to help improve the ISO-level taxonomy to cover ares in which it is weak, e.g., safety and security.

14764 – (software maintenance (merger)) is a coordinated merger of IEEE 1219 with ISO/IEC 14764 with the ISO FDIS ballot complete.

15939 – (software measurement process (adoption of ISO 15939)) will be an adoption of the revision to be done in SC7.

16326 – (project management (merger)) is a coordinated merger of IEEE 1058 with ISO/IEC 16326

90003 – (guidance in the use of ISO 9001:2001 for software (adoption of ISO 90003)) needs some more work on the cross-reference annex linking S2ESC standards to 9001, but should go into ballot by the 3 rd quarter of this year.

Some Balloting Issues

MyBallot – This is an electronic ballot voting management system that allows ballot pool members to track the voting expected of them. There were some ergonomic and usability issues raised with the first version now that there has been about a year of use: it seems hard for balloter to track what’s happened with comments submitted and to find relevant information about comment disposition. Though the system is “getting a little better,” it was noted that it actually delayed the balloting on a standard this past year.

MyProject – This is a planned electronic standards project management system that will allow Working Group Chairs to manage all project requests on-line. A pilot is being conducted and revisions are underway to the first prototype, which, among other things, provides roster, batch mailing, etc. capabilities. The pilot includes the IEEE 802 work, which manages to a strict milestone schedule and will be have an influencing role in MyProject’s evolution. Paul Croll indicated we should wait until a demo exists to make it easier for WG Chairs and members to understand its value. Roger Fujii noted the value of a central repository for book proposals, but it’s not clear that MyProject will help small WGs.

WG Training and Support

Since most WG Chairs are on the Executive Committee, this has not been an issue. But Paul Croll would like to see the development of at least a checklist of what’s expected. But others noted that there is already too much written material (e.g., policies and procedures and a handbook for Chairs). Roger Fujii pointed out that, under Fletcher Buckley, WGs were kicked off in San Francisco at an annual ISSES meeting, and WGs held their first meeting at that time. Annette Reilly noted that she had participated and found it very useful. It would also be important to educate about how ISO and IEEE standards relate.

Jim Moore suggested that WGs could be meeting in parallel, and that S2ESC members could split up and attend and provide guidance. Fujii and Walrad pointed out that their WGs were successful because of quarterly FTF meetings. Roger sees a value in dialogues between WG Chairs, because standards are often inter-related: another reason to bring them altogether at one time and place. [However, with no real budget from IEEE (or, in many cases, people’s companies, a quick check of the WG Chairs present suggested it was doubtful their WGs would make (mandatory) semi-annual meetings, besides any time they have for their specific work. This is where hardware standards have a bug advantage: companies wanting to be compatible from a product standpoint are willing to spend the money to participate in hardware standards activities just as, at the ISO level, companies participate in programming language and other product-specific software standards. The process standards work of S2ESC and SC7 does not attract such support.]

Other Management Board Business

There was some discussion of guidance that needs to be provided to Working Group Chairs, especially to ensure consistency within the S2ESC standards collection. For example, that the 12207.0 and 12207.1 material be adhered to in creating/revising standards, i.e., so traceability can be established between the standards and frameworks like 12207.

A comment, regarding the good progress made with 1074, was that the production of frequent (every 4 months) drafts helped keep people moving in the right direction by giving them a focal point on a regular basis. (Typically WG Chairs are given a good bit of latitude in running the groups, but there has been discussion about having more formal “milestones” to help the Management Board track where projects are even though less formal contact with Chairs has been acceptable. However, no formal process for interceding in a WG’s activities exists when this informality does not seem to be working.)

The IEEE Standards Association, used to working with hardware standards efforts more than software, wants budget and expense reports at the WG level. S2ESC’s response uniformly is that we have $0 expenses. SA is used to dealing with professional standards writers (in the HW world) and much vendor involvement.

There was discussion of a proposal to send plaque/certificate of appreciation to WG member’s companies/agencies regarding the person’s contributions to IEEE standards. To avoid an SA concern that this would misrepresent the company’s involvement, Carl Singer suggested just a letter to the person’s manager. Paul Croll noted the importance of recognizing the contributions of small companies, because we want their participation, and it is more costly for them. Letters, compared to plaques, etc., are easy due to budget issues – just send a draft to Kathy Land. Paul Croll will talk to IEEE VP for standards about giving a plaque for a volunteer’s contributions to standards, putting name of corp/agency Exec who sponsored the person’s activity on the plaque, as well as the names of the volunteer participants. Roger Fujii noted that such a token of appreciation helped one participant keep their company involved in IEEE standards.

Executive Committee Business

ASQ SW Division Liaison

The S2ESC chair, Paul Croll, had two suggestions for possible work between S2ESC and the Division:

  • Guidance on articles IEEE folks could do for Software Quality Professional.
  • The possibility that IEEE could get a track at the next World Congress.
IEEE-CS Liaison to ISO/IEC JTC1/SC7

Jim Moore, the IEEE-CS representative to SC7 reviewed the proposed plan for 2006-2007 liaison with SC7. His proposal (which was accepted by the Committee) included the following standards and standards areas:

  • System Engineering Standards (all IEEE - 1220:2005, 1062, 1233, 1362, 1471)
  • IEEE 12207.0, ISO 24774 (on process definition), new work in SC7 on process “tailoring” for verry small enterprizes (i.e., <=15 employees)
  • IEEE 12207.1 perhaps to be replaced by ISO 15289 (on information products, i.e., data)
  • Possible mapping of S2ESC standards to PMI, PRINCE2®, etc. (instead of adoption of any of them) [ PRINCE® stands for Projects in Controlled Environments and covers the project organization, management and control. It was developed in 1989 as a UK Government standard for IT project management.]
  • Fast-track IEEE 1012 (V&V) into SC7 [SC7 has no V&V or testing standards]
  • IEEE 1074 is a (project) process “construction” standard which is light on organizational processes, not a broad definition one like 12207.0
  • IEEE 1228 on safety and ISO 15026 on integrity levels (under revision)
  • Object management Group (OMG) work in software assurance (Clockwork - Canadian company) may be relevant
  • ISO 24773, an international framework for software engineer certifications (such as IEEE’s CSDP)

As a result of the discussion around these issues, Jim Moore asked the S2ESC for the following help related to SC7 standards work:

  1. Participate in an SC7 working group to develop a document similar to ISO/IEC 90003 but operating at the systems level. This would involve attending two week-long international meetings per year.
  2. Coordinate IEEE comments on the revision of ISO/IEC 9126-1, Quality Model, now renumbered as ISO/IEC 25010. Ideally this person would attend two international meetings a year. However, the job could be done, with less effectiveness, by attending only one meeting of by corresponding with the working group.
  3. Partner with Perry Deweese on the development of an international standardized profile for process definition in Very Small Enterprises. This document would provide guidance to very small companies in implementing 12207, 15288, 9001, etc. Perry is the editor of this standard and attends the international meetings. We need a partner who will coordinate CS balloting and commenting on the standard.
  4. Lead a revision of IEEE 828 [configuration management] that is process-oriented and suitable for fast-track into SC7.
  5. Participate in an SC7 working group responsible for CASE tool standards. Ideally this would involve attending two international meetings per year. However, the work could be done with less effectiveness by attending only one meeting or by corresponding.

We also need someone to work with PMI (and maybe IPMA) to develop a set of concepts that we can share -- in effect, an interface for our standards.”

Department of Homeland Security (DHS) Liaison

Joe Jarzombek from the DHS presented his semi-annual update on software assurance (safety and security) work at DHS. This included noting the following:

  • Two key publications from the DHS’s software assurance program can be found at https://buildsecurityin.us-cert.gov (a key website on SW Assurance), clicking on the “Additional Resources” tab and then looking for the “SW Assurance Common Body of Knowledge” and “Supplementary DHS Resources” sections:
  • Secure Software Assurance – Common Body of Knowledge (CBK) document
  • Security in the Software Lifecycle (developers’ guide with initial content from various DoD Application Security guides).
  • The Information Assurance program at the National Defense University Information Resource Management College (http://www.ndu.edu/IRMC/) is an example of some curriculum work in this area. He stated that 80 Centers of Academic Excellence for Software Assurance were expected by the end of the year, i.e., colleges and universities who are implementing curricula using the CBK (68 exist now). He also described how some early graduates are beginning to come out of such programs having used the NSA-funded scholarships. Joe also directed people to the Computing Curricula Final Report from ACM, AIS and IEEE-CS (30 September 2005).
  • The NIST Risk Management Standards for IT security (FISMA Implementation Project run by Ron Ross of NIST).
  • A DHS Speakers Bureau has been established.
  • DHS is focusing on supply-chain management for reducing security risk.

Joe stated his key position of assurance: a practice cannot be “best” if it does not address software assurance, i.e., safety and security.

Department of Energy (DoE) Liaison

Debra Sparkman described three safety software types: (1) safety control and monitoring systems, (2) safety management and administrative software and (3) software used in hazards analysis and construction of facilities. The requirements include: (1) involve facility design authority, (2) train DoE employees to perform their duties with regard to software QA (e.g., DoE std 1172); (3) software implementation must be implemented in accordance with NQA 1 2000 (or a standard which they can prove is equivalent to it) as a minimum baseline not as prescriptive as the IEEE software engineering series; (4) graded approach applied to these Software types.

Jim Moore suggested that the family of standards Sparkman described bedocumented and published in IEEE Ready Notes. Roger Fujii also urged her to do this. (Authors are paid.)

Is there an opportunity for IEEE to become more actively involved with DOE’s standards efforts? Debbie will give Paul the number of the appropriate person to contact.

Membership Outreach

Claire Lohr noted that the S2ESC membership list was about 2,000 people. (Note: TCSE is 5 - 10K.) The e-mail sent to this community seemed successful: Roger Fujii and I noted responses we had gotten to our projects. Among the other “communities” we should look at notifying about S2ESC activities were: IEEE’s Technical Committee on Software Engineering (TCSE), PMI’s ISSIG, PMI RM SIG, ITIL-users, ISACA ( Information Systems Audit and Control Association), INCOSE, FDA, FAA, verticals (e.g., financial software community, medical device software), other Standards Developing Organizations (SDO) and the CMMI community, universities and colleges (e.g., mapping SE 2000 curriculum to IEEE Standards, working to get an academic (lite?) collection that can be made available at schools, giving student members free access to the Digital Library, advertising Ready Notes to Professors), foreign governments who are major acquirers but not major standards producers (e.g., Australia, Singapore are good candidates, perhaps China -- the 802 group found that they needed to bring their standards to them, rather than inviting them to come here).

Book Series

Roger Fujii reported on the “guidebook” series, which is intended to help people apply various S2ESC standards by including informative material, sample document outlines, etc. Such detailed and prescriptive material would not be found in the standards.

Other books are addressing “historical” or seminal material to prevent loss of knowledge about where and how various aspects of computing came to be as they are today.

For short topics that need minimal refereeing and that could be available more quickly than the year process for a book, Roger mentioned “Ready Notes.” These are short, electronic documents that people could purchase and download, which would never be available in print versions.

IEEE Vocabulary Project Demonstration

The IEEE-CS, using requirements from the SC7 vocabulary project (Working Group 22), has developed a database which will house the new vocabulary which WG22 has been charged to create to consolidate SC7 (and IEEE) software and systems engineering definitions from their respective standards collections. Initially, this database will be used by volunteers to extract terms and their definitions from the standards collections to add them to the database. The first public release of the database will, therefore, describe terms and their definitions as they are which would include differing definitions from multiple sources for many terms. The next phase would be for the IEEE and SC7 to converge on agreed definitions, editing the database contents and gradually updating the standards collections as each standard comes up for review/renewal.

As the Division’s US SC7 TAG representative, I have been asked to extract definitions from the following IEEE standards:

  • IEEE 730-2002 - Standard for Software Quality Assurance Plans
  • IEEE 828-2005 - Standard for Software Configuration Management Plans
  • IEEE 829-1998 - Standard for Software Test Documentation
  • IEEE 830-1998 - Recommended Practice for Software Requirements Specifications
  • IEEE 982.1-1988 - Standard Dictionary of Measures to Produce Reliable Software
  • IEEE 1008-1987 (R2002) - Standard for Software Unit Testing
  • IEEE 1012-2004 - Standard for Software Verification and Validation
  • IEEE 1016-1998 - Recommended Practice for Software Design Descriptions
  • IEEE 1028-1997 (R2002) - Standard for Software Reviews
  • IEEE 1044-1993 (R2002) - Standard Classification for Anomalies

Standards Collection

There was some discussion of updating of the CD-ROM containing the collection, part of the responsibility for which belongs to IEEE-SA (Standards Assocition). Of more interest was discussion of proposing to IEEE that there could be subsets of the collection -- perhaps on CD, perhaps online -- targeted to different communities (e.g., for CMMI® Level 2, a student version, a lifecycle set). Jim Moore will bring this idea to the IEEE-SA.

Study Groups

Software Testing

The Software Testing Study Group has been reformed with Dennis Lawrence as the Chair. One goal will be to discuss what testing process standards S2ESC should pursue since 1008 (unit testing) is an anomaly, being the only one like that.

It was also noted that BS 7925-2, a British Standards Institute software component testing standard, could be input to this study. Jim Moore will be contacting people from BSI on this matter.

Quality management

A subgroup was formed to establish guidance for the WG for the revision of 730, which is now focused on software quality assurance plans. The guidance will address focusing the revision of 730 on an elaboration of the SQA process as described in (the revised) ISO 12207.

It was also noted that TC176 will be working with representatives of the CMMI® community to develop a cross-reference between ISO 9001:2000 and the CMMI® for implementers, auditors, and others.

John Walz noted we should influence the ISO 9004:2008 effort to reference typical Product Realization Life Cycles. He also noted that the US TAG to ISO TC 176 is working with CMMI experts to create a cross-reference table for use by implementers, auditors, and registrars.

Systems Integration

A professor from Stevens Institute of technology (NJ) ( Rashmi Jain) has submitted a proposal for a possible set of standards addressing this topic. There was enthusiasm for this because it engages the academic community as well as the system engineering one since the professor has connections with INCOSE. The work will be encouraged, but there may be a need for “mentoring” to help explain how the standards process works. T he scope of the proposed work is very broad.

Working Group Meeting Structure

An idea was submitted to hold a meeting several times a year, perhaps attached to (practitioner-related) conferences or other meetings related to software engineering, where people interested in working on standards would meet and break into subgroups to work on specific standards content. In between, email and/or conference calls could move the work along. But the key would be deliberate face-to-face gatherings to ensure work does progress and progresses quickly. Other locations suggested were the IEEE Service Center in Piscataway where there are several conference meetings, an Embassy Suites where each person has a “meeting room” as part of their room, or a college/university. Among the topics would be how common themes get addressed across the S2ESC collection.

Other Matters

S2ESC has a new secretary, Charlene (“Chuck”) Wolrad. She began her work at this meeting.

Next S2ESC Meetings

The next face-to-face meeting will be during the week of July 31-August 4, in Ft. Lauderdale. (S2ESC has monthly telecons usually the 2 nd Tuesday of each month.)

Those interested in any of the topics mentioned above, or other standards-related issues, can send email to sduncan@computer.org.

ASQ News

 

Follow the
Software Division

Twitter  LinkedIn