IEEE Software and Systems Standards Committee (S2ESC) Meeting Report
Mountain View, CA (Feb. 27 - March 1, 2007)
This report covers the IEEE Software and Systems Engineering Standards Committee (S2ESC - Executive Committee and Management Board) meeting from Feb. 27 - March 1, hosted by Microsoft at their Mountain View, CA research center.
A number of hours at this meeting were devoted to administrative issues regarding S2ESC organization, its Policy/Procedure structure, and issues regarding IEEE’s myProject/myBallot systems.
S2ESC has been simplifying its procedures and eliminating redundancy with higher-level IEEE Standards Association (IEEE-SA) policies and procedures as well as attempting to provide better oversight with regard to S2ESC Working Group (WG) formation, project plans, and review. Three topics, in particular, were mentioned:
- Initial WG (Chair) guidance and preparation of the project plan and PAR (a WG’s proposal to IEEE-SA to initiate a standards project);
- How to appropriately gain review of standards by experts in the field even if they do not have the time to be active members of the WG;
- What to do with disagreements on harmonization of individual standards with foundation standards such as 12207.
Regarding the myProject/myBallot systems, whose use has now expended to include WG rosters, there was considerable discussion. IEEE staff present at the meeting noted that there were hundreds of requested changes/enhancements to the systems. Ultimately, after much demonstration and discussion of issues, it was decided that S2ESC members -- who would be in Piscawatay, NJ at the IEEE Service Center for the US SC7 TAG meeting in two weeks -- would meet after the TAG meeting on Thursday afternoon to discuss the systems with IEEE personnel.
It was also noted during the course of the meeting that IEEE is looking for an editor for the Ready Notes series.
Angela Ortiz has been replaced with another IEEE-SA liaison to S2ESC, Malia Zaman, who will also be the US SC7 TAG administrator starting at the end of the next TAG meeting.
Standards Status Reports/Issues
610.12 (vocabulary) will be replaced by ISO/IEC 24765 in approximately 18 months.
730-2002 (software assurance plans) PAR has been submitted to replace the document standard with a process standard. Ron Dean is the revision Chair.
829-1998 (testing documentation) PAR has been extended to 12/08. A new series of international standards on testing will replace both 829 and 1008.
1008-1987 (unit testing) last revision was in 2002. The editor will likely be Ursula Parker (of Lockheed-Martin).
1012-2004 (verification and validation) is now being revised to include systems engineering. The WG is also contemplating including hardware in this revision. This could be an issue regarding integration with SC7 standards. The difficulty is that 1012 is well ahead of the SC7 effort. Roger Fujii (1012 Chair) will look at this when the current revision is complete.
1016-1998 (software design) PAR has been extended to 12/08.
1028-1987 (software reviews) was last affirmed in 2002. Dennis Lawrence has recruited about 50 people for the revision effort. 1012 will point to this standard and 730 should also be considered.
1044-1993 (1044-1993, classification for software anomalies), last re-affirmed in 2002, is now chaired by Dave Zubrow, with a change in focus to address defect causal analysis, reflected in how the approved PAR (6/9/2005) is written.
1045-1992 (software productivity metrics) was last reaffirmed in Dec. 2002. Pate Baxter is revision Chair.
1062-1998 (acquisition), last re-affirmed in 2002, will be reviewed for consistency with ISO 15288. Joe Jarzombek will draft a PAR and look at SW Acquisition area of CMMI.
1063-2001 (software user documentation) is working with SC7 as ISO 26514, but, meanwhile, 1063 is being reaffirmed.
1175.1 – 5 (set of CASE tool interconnection standards) require action on the PAR for 1175.4, which expires 12/31/2007. 1175.5 PAR expires this year (12/31/2007), but work is currently on hold.
1228-1994 (software safety plans) was last re-affirmed in 2002. Several IEEE societies have formed a group to examine safety standards. Due to liability concerns, it is difficult to get legal approval for safety standards. Note: Security standards have disclaimers to avert legal claims.
1362-1998 (concept of operations) is up for re-affirmation. RevCom has extended a 2-year extension to 12/07 for the re-affirmation. Emergence of IT governance (ITIL, etc.) in industry may make this ConOps standard irrelevant to industry. Defense still uses it, though. We need to investigate what industry needs now. Walrad will try to get someone from BMC and/or IBM PS to provide feedback on usability of this standard in IT governance, especially with regard to transition planning.
1420.1-1995, .1a, .1b were last re-affirmed in 2002 and should probably be stabilized rather than re-affirmed, thus making it still available to companies that still depend on it.
1462-1998 (IEEE adoption of ISO/IEC 14102, which has been revised and is currently undergoing its second FCD ballot in ISO/IEC JTC1/SC7). We have reaffirmed 1462. Once 14102 is published, S2ESC should initiate a project to adopt it as a replacement for 1462-1998.
1465-1998 (IEEE adoption of ISO/IEC 12119, which has been revised). We have opened a PAR for the adoption of ISO/IEC 12119 as a replacement for 1465-1998.
1471-2000 (architectural description of software intensive systems) is being worked with SC7 as ISO 25961.
P1648 (agile methods) PAR expires 12/07. Scott Duncan will initiate a PAR revision with extension to reflect change in scope (focus on customer/supplier relationships).
2001-2002 (web page engineering) has been adopted by SC7 as ISO/IEC 23026.
P2063 (requirements engineering) will be addressed by SC7, which intends to incorporate IEEE 830 and 1233.
12207.0 (software life cycle processes) has a PAR in place for the joint revision effort now underway with SC7.
12207.1 will be replaced by IEEE/ISO 15289.
12207.2 is being revised by SC7 as 24728. If that effort is not adequate for the S2ESC’s communities of practice, we will need to update this for the revised 12207.0.
ISO 15289:2006 needs a PAR to adopt it as an IEEE standard.
P15939 is the adoption of ISO/IEC 15939:2002.
16085:2006 replaces IEEE 1540, which has been withdrawn.
P16326 merges 1058 (adoption of PMI’s PMBOK standard) and ISO/IEC 16326 and will go to ballot shortly. PMI has has participated in the SC7 revision.
P90003 (adoption of ISO/IEC 90003) is in ballot.
ISO 20000.1 and 20000.2 (IT service management) have had PARs approved for their IEEE adoption.
SEVOCA Demonstration and Discussion
SEVOCA is the on-line database of software engineering vocabulary drawn from the software and systems engineering standards in IEEE’s S2ESC collection and the ISO/IEC JTC1/SC7 set. The system is in its final stage of editing and issue correction -- resolving some formatting and search issues with some entries -- before going public. The first version will not have selected a “preferred” definition yet for all the terms. The plan is to let inconsistencies in definitions be resolved through SC7 WG 22 and via revisions of definitions in standards as the standards are revised.
There will be 2 capabilities in the online system housing the vocabulary: look up and print. SEVOCAB is descriptive, not prescriptive, although a preferred definition is given first. Sources are given for each definition. There are links to the original standards, and a link to the IEEE store so that people can buy particular standards. There are now 4100 definitions. A full .pdf is 382 pages and ~13Mb long. One issue is whether, when a standard is withdrawn, its definitions should remain or somehow be marked as having come from a withdrawn standard. The intent is to have definitions in new standards reviewed against SEVOCAB to avoid conflict.
Undersecretary for Preparedness (George Foresman) oversees the National Cybersecurity Division, which includes US-CERT and Strategic Initiatives for Standards & Best Practices and Assurance. Goal is to change consumer behavior by making information about suppliers’ process capabilities available to consumers. Aim is to establish standards by which these processes could be evaluated, tools for evaluation, and provisions for built-in security of the enabling software. Standards enable communication between buyers/users and sellers/providers. They are creating a standard glossary. Jarzombek noted, as he often has, that “If a system is not secure, I can demonstrate why it is not safe.”
The SwA (software assurance) Initiative has 3 primary tenets:
Trustworthiness - No exploitable vulnerabilities exist, either maliciously or unintentionally inserted
Predictable Execution - Justifiable confidence that software, when executed, functions as intended
Conformance - Planned and systematic set of multi-disciplinary activities that ensure software processes and products conform to requirements, standards/ procedures
Jarzombek noted the importance of vendors being able to issue “bounded” statements of assurance (i.e., a system claim be claimed to be assured used as intended in an “assurance case” for the system). Developing an “Assurance Argument/Case” to convince SW managers of the value of SwA is needed. Also developing a legal definition of “malware” is becoming more critical.
Jarzombek also suggested that S2ESC co-host a non-US-centric workshop this year on Assurance as defined by SC7 scope (not just cybersecurity). UNC in Charlotte would probably be interested in co-hosting. The focus should be on strategies for achieving standardization and alignment/harmonization of safety and security practices. Paul Croll and Jim Moore will determine who should be invited (in addition to DoD and DHS).
ISO SC7 Liaison
S2ESC has adopted a strategy of harmonization with ISO standards because it enhances IEEE’s authority in the international area. In the U.S and other governments, ISO/IEC standards are often preferred to IEEE standards.
One major new effort is the SC7 4-part testing standard project, which should incorporate IEEE 829 (test documentation) and 1008 (unit testing).
ISO 15026 was originally intended to support 12207 and 15288 for software assurance. WG7 required that it become a “Process View”, meaning that it is supposed to gather process activities & tasks from existing standards, cutting across all or part of the software life cycle. Sam Redwine at James Madison U. andJim Moore are finishing a 57 page IEEE proposed draft, based on 16 standards-based practices for safety and security derived from four safety standards and four security standards and then harmonized to recognize the commonalities among the safety and security disciplines. This will be submitted to SC7 by 1 April, in time for review prior to the May plenary meeting. [The Safety and Security Extensions Project at the FAA developed the essential practices presented in this article. The safety and security practices produced from this project were required to be based on widely recognized safety and security standards1 and harmonized across the safety and security disciplines. After separate safety practices and security practices were derived capturing their respective source standards, the practices were harmonized to address commonalities, coordinate activities, align terminology, and encourage the merger of the safety and security disciplines. Additionally, the project provided a mechanism for implementing these specialty-engineering practices in the context of existing process improvement frameworks to align safety and security improvements with more general process improvement endeavors.]
Mary Beth Chrissis noted that SEI’s CMMI webpages get ~24,000 hits per day and that over 58,000 people have been trained in CMMI. For the next CMMI revision, the major themes are to reduce the complexity and size, increase coverage, and increase confidence in appraisal results. There is always tension between adding enough material to support consistency of appraisals and adding so much that the book becomes unusable. The shelf life of a certification is now 3 years. Regarding CMMI appraisals (SCAMPIs) clarifications have been added for virtual teams, unit sampling, and other topic. Appraisal disclosure statements are now produced. There are increased requirements for appraiser qualifications. High maturity appraisals must be lead by high maturity appraisers, but it was noted that low maturity organizations may not always be well served by high-maturity appraisers.
Document Editing and Production - There was a discussion of issues with graphics, especially “round trip” editing as IEEE-SA staff has primarily used FrameMaker in the past, but are adding Visio, PowerPoint and perhaps other graphics sources. There was some discussion of difficulties of using the macro-driven Word template provided by IEEE editors.
Management Tools - Issues with bad data base records and usability issues make the database not as useful to potential volunteers who may be interested in participating in standards work. It was noted that the sponsor should manage WGs, not just be informed after the fact of WG actions such as going to ballot. A meeting will be set up at IEEE to continue this discussion for those S2ESC members who will already be there in March.Working Group Rosters – It was noted that the current list of S2ESC WGs/projects under “Manage Committees” in MyProject is wrong. It contains a mixture of standards, active WGs and disbanded WGs. Also, as IEEE-SA’s use of rosters is only for indemnification, they may contain names that the chair does not consider to be members of the WG. Thus, the WG “roster” may not be the same list that the chair uses to manage the WG.
New Standards Collections CD
This was discussed in detail last summer. There still does not seem to be any new edition planned by SA. We need affordable pricing for individuals and to get them into students’ hands. Vladan Jovanovic will become our liaison to the IEEE’s Standards in Education taskforce/committee: http://www.ieee.org/portal/cms_docs/education/setf/about.html.
Vladan identified five essential standards for a student edition: software life cycle processes (12207), software project management plans, requirements specification, user's documentation and test documentation. Vladan is authorized now to create an academic advisory committee to provide input to SA. The preliminary list for the advisory board regarding academic use of IEEE SWE standards is Vladan Jovanovic, Sam Redwine, Nancy Mead, Keneth Modesitt, Dan Shoemaker, and Paul O'Neil.
Assessing Technical Merit of Standards
It was suggested that a Technical Review Board of experienced practitioners be established vet WG drafts and to be encouraged to contribute up front to help assure technical merit. People who hold CSDPs might be interested in performing this function. Paul Croll will continue this discussion in our next telecon.
We also need to make more SWE practitioners aware of the S2ESC collection. We need to devise strategies to demonstrate how standards can be helpful to practitioners by providing information that helps them in the SDLC processes.
We would like to have a mechanism for online discussion/forum on the site. The list of CSDP holders provides a group who would be likely to be active, since they would get on-going CSDP certification credits.
The next face-to-face meetings will be held in Ft. Lauderdale, FL. Two weeks were discussed as possible dates: July 16th (the main date) and July 30th.
Those interested in any of the topics mentioned above, or other standards-related issues, can send email to
Scott Duncan, the Software Division Standards Chair, at firstname.lastname@example.org.