IEEE Software and Systems Standards Committee (S2ESC) Meeting Report
Fort Lauderdale , FL (July 30-August 2)
This report covers the IEEE Software and Systems Engineering Standards Committee (S2ESC - Executive Committee and Management Board) meeting in Fort Lauderdale, FL (July 30-August 2).
S2ESC Organizational Matters
The Management Board election results were announced resulting in the following people receiving the following terms of office on the board: Scott Duncan and Ron Dean (2 years to 2008); Dave Schultz and Carl Singer (3 years to 2009).
A new Software Assurance Study Group was created, headed by Jim Moore. It will look at standards of relevance to software safety and security (i.e., "assurance" as defined by the CyberSecurity Division of the Department of Homeland Security (DHS)).
Most people probably associate the National Institute of Standards and Technology, because of its name, to be a standards development organization (SDO). But, at this meeting, it was noted that, technically, NIST develops "Federal acquisition guidance," not standards. Their relationship to S2ESC would be as an "At Large" member as compared to SEI and ASQ, for example, which are considered "Development Partners" since both organizations have de factor or de jure standards of interest to S2ESC. (In the case of ASQ, this is largely quality management through ISO 9001, which is not an ASQ standard, though ASQ administers the US TAG for TC176, involving it with the work.)
It was noted that S2ESC might approach the IT Services Qualification Center at CMU who have a model covering items beyond the CMMI ® as, for example, in he area of risk management. The ITSQC has been especially active in "Latin America" (Central/South America and the Caribbean ). Have a model that Joe has asked whether they would offer up (the "normative" part) as a standard. The ITSQC effort also tracks ISO 15504 (the international Process Assessment standard) closely. It was also noted that the ITSQC area at CMU also has a useful tool for establishing mappings between models.
In connection with this, there was also some discussion of the potentially huge impact of ISO/IEC JTC1/SC7's IT/Ops and Very Small Enterprises work. (Note: ISO/IEC JTC1/SC7 is the international standards body for which the US SC7 TAG represents the United States of America . The Software Division is a member of the US SC7 TAG.) Developing economies, especially in the Southern hemisphere and Asia , have been showing significant interest in both these topics. It was suggested that more energy should be put on the international side of IEEE-CS activities as both standards interest and IEEE membership are growing there. It was stated that the Computer Society has an outreach program going specifically to " Latin America ."
As usual, Joe Jarzombek (Director of Software Assurance within DHS), presented material on work being done in this area. In particular, he stressed the need for and role of standards so vendors can make assurance claims and provide evidence of "testing" against those claims. He noted that DHS has a 500 item dictionary of common weaknesses against which companies could view their product "testing," i.e., do they cover "x"?
Another oft heard theme was how the majority of attacks on infrastructure come through attacks on the enabling software, some of which is hosted on desktop hardware and software platforms that are quite old. In this regard, there is also a need for changing "consumer behavior" in acquisition and outsourcing activities. DHS will be putting out a Software Acquisition Guide for Software Assurance to address knowledge and awareness about this. One concern was that, too often, the word "acquisition" is viewed as "buying big stuff" when it needs to cover all size purchases as well as internal development groups supplying software to a broader organization. Along with this is the recognition that, besides outsourcing, "Reuse itself creates another whole set of vulnerability issues."
Another topic Joe often stresses is the availability of skilled professionals in the assurance area. He stated that there are far more IT jobs in the US now than in the Dot.Com era, but that there were high requirements for such jobs. (He did note that programming-only jobs are most at risk for outsourcing, saying that people need to consider what else they can bring to the table besides coding skill.) The DHS's Software Assurance Common Body of Knowledge is a critical resource for defining education and training requirements. There was some discussion about how to reach out to organizations offering accreditation and to industry to ask them to address their hiring criteria where there is a need to fill gaps to meet SwABoK coverage.
Finally, besides existing DHS industry-based work groups in various assurance areas, there will be a new work group started on malware and spyware.
The overall goal of all these efforts is to close the gap in "minimal level of responsible practice," then raise that level, moving toward "predictable execution." (Many people say the role of standards is to define that "minimal level.")
Jim Moore is the IEEE-CS liaison to SC7. Jim provided the S2ESC with a document listing areas where IEEE could use "help' in monitoring and being involved directly with SC7 efforts of interest to S2ESC standards work as well as setting up "shadow" projects within S2ESC to prepare for IEEE adoption of relevant SC7 standards.
There will be a September 6-7 workshop on the Software Project Management Process standard. This is joint work to take ISO 16326 and IEEE 1058 and combine them as a revised16326. The US SC7 TAG seems interested in this work combining system and software processes, but one question is whether such a result would not, in fact, end up being very close to the existing Project Management Institute's PMBOK.
SC7 has New Work Item proposal for a Requirements Engineering Process standard. Relevant IEEE standards would be 830, 1233 and 1362.
There is also an effort identified as Software Life Cycle Process and Guidance for Very Small Enterprises (<25 people). This effort is to produce an International Standardized Profile (ISP), which helps groups of that size configure other standards (e.g., process, assessment, life cycle models). This work has gained much interest from nations who have shown less interest in participating before with other international standards work.
The IT Services/Operations area (ISO 20000) is another very big area (again especially popular with nations who have not participated as much before).
Finally, SC7 has an assurance standard and IEEE has a safety one, so there is a need work to bring these together under the overall assurance category.
Individual IEEE Standards Statuses
610.12 (1990) - This is the only part of 610 that exists. It may contain definitions that do not link to any current standard. Indeed, many of the definitions may not point to existing standards. This is the de facto glossary used, so the lack of a definition appearing in an active standard does not matter. Indeed, existing standards may not list many definitions because they were already in 610.12 when those standards were developed.
730 (2002) - The desire is to elaborate on the IEEE/EIA 12207.0 QA process during this revision.
828 (2005) - Though not need right away, the next revision of this document needs to turn it into a configuration management process standard and incorporate the Annex addressing 12207 into the body. This could be a lot of work and S2ESC may want to initiate it as soon as possible to be ahead of the 5 year deadline for addressing each standard.
829 (1998) - The pace will be increased on this work and an extension has been requested.
1008 (1997) - This has been submitted to SC7. S2ESC had planned a new study group on testing but what really seems needed is someone at the SC7 level instead. So there will be no Study Group and S2ESC will seek someone to take on this international role.
1016 (1998) - A PAR extension will be submitted soon (if it has not already been).
1028 (1987) - A Working Group has been formed to revise it and a study group on the whole subject of reviews/inspections has been suggested.
1044 (1993) - This Working Group has been picked up by Dave Zubrow who had been an early WG member. The prior Chair's intention had been to focus on defect causal analysis. Joe Jarzombek noted how the topic of software anomalies has a direct connection to software assurance work.
1045 (1002) - No real status known other than the Chair has said they were "redlining" the current draft.
1062 (1998) - A (Project Authorization Request) PAR needs to be prepared to revise this Software Acquisition standard and the DHS acquisition working group might be one focal point for the work.
1063 (2001) - SC7 has a new standard - 26514 in CD ballot - to cover both process and product documentation. The question is whether an extension, reaffirmation ballot or a revision PAR will be used to keep the document "alive" while SC7 is working.
1175.1-5 - Part 2 is close to ballot completion. Part 4 is being edited and has an extension. Part 5 is on hold.
1228 (1994) - There is no current activity on this Software Safety Plan standard. It needs to evolve to a process standard, be aligned with ISO 12207 & 15288, and possibly be combined into ISO 15026. The standard, though, had some connection to nuclear regulatory standards but they may have moved on and then S2ESC could do as it wishes with this standard. Joe Jarzombek asked about this in relation to security and safety standards work.
1362 (1998) - The ConOps Document standard was contributed to the IT Services work in SC7. The requirements subgroup in SC7's Special Working Group 5 has identified this as a relevant document.
1420.1 (.1a & .1b) (1995) - This reuse standard has to be addressed since "stabilization" has not been pursued yet. (The standard is about the interchange of assets between reuse libraries. As the market has moved elsewhere, there are no such major libraries as envisioned when this standard was developed.)
1471 (2000) - This has been fast-tracked successfully into SC7 for a coordinated revision next year,
1644 - Working Group interest seems to have waned on this effort to address application software naming conventions.
1648 - There were a lot of comments on what this agile recommended practice standard needs to address. Three meetings have been held this summer (June, July and August) at agile conferences and this S2ESC meeting. Joe Jarzombek suggested looking into the "Federal Acquisition Regulation" documentation. It was noted that the standard could reference commercial publications within normative sections as examples of practice as Bibliography, rather than as Normative, References.
2063 - Merge of software and systems requirements specifications standards (i.e., IEEE 830 and 1233) into a process standard. There is an SC7 effort (SWG5) reviewing these two as base documents for an SC7 project under WG7. It is not yet clear what the SC7 effort may mean for S2ESC?
9126.1 - Adoption of the SC7 2001 version of ISO 9126-1. IEEE will need to use 25010 which is the new SC7 number under the SQuaRE work.
12207.0 (1996) - A PAR has been written for a coordinated revision with SC7.
12207.1 - Will adopt ISO 15289 (after its (concurrent, editorial) revision to meet revised 12207 and 15288 in 2007) to replace IEEE 12207.1.
12207.2 - This is derivative from 12207.0 and revision would be needed.
15026 - There is SC7 work on Assurance Case/Argument which was transferred to WG7 since WG9 was disbanded. One thought is to look at adding a "Specialty Engineering" concept to 15288 to raise engineering-specific attention as the current SC7 work is now being presented as a "management," rather than engineering, standard. It was suggested that perhaps SC7 would rename its effort to represent the more quality management slant SC7 is taking.
15939 - This will be an adoption of the revised SC7 document which is being changed (slightly) to make it more generically apply as a process for systems and software measurement.
15326/1058 (1998) - This is a coordinated merger with SC7's project management work. A draft PAR is under review.
90003 - The draft of this adoption will be circulated once more for comments, then submitted for balloting.
Roger Fujii edits the Software Engineering Book Series to publish "guides" to S2ESC standards. Hence, S2ESC is the review authority for these books. Topics relate very directly to existing S2ESC standards. Where there isn't a recognizable S2ESC standard, there will not be a book (at least in this series). The intention is to develop a seminar series around these topics and relate this to IEEE's Certified Software Development Professional (CSDP) program. The books and current authors in this series are:
Testing - Eva Freund and Claire Lohr
Reviews, Inspections and Audits - Linda Schaefer
Software Configuration Management - Don Schaefer
Software Lifecycle Process - Jim Moore
Verification & Validation - Roger Fujii
Risk Management - Paul Croll and Bob Charette
Maintenance - Paul Croll
Software Quality Assurance - needs an author as the prior author had to withdraw
Project Management - Mark Christiansen
There are also books series on Systems, "Perspectives" (seminal work that may be lost, scattered through materials that may be harder and harder to find), Emerging Technologies (where a book on agile methods could go), and a Security series.
There is also the Ready Notes series of "articles" or "chapters" which may be a way to test if a longer treatment would be of interest.
There was considerable discussion about S2ESC Procedures related to the level of oversight and milestone tracking needed by the Management Board and/or ExCom to know Working Groups are "on track." A good set of milestones is not present in our procedures at this time, but is needed. Management Board Chair David Schultz (along with support from Scott Duncan, Jim Moore and perhaps John Walz) will take the ideas presented, make the detailed revisions and bring them back for final approval. It was suggested that we could keep the S2ESC Procedure on project planning, rename it to a project management procedure (not just a plan preparation one), combine 5-6 other Procedures into it as a set of minimum milestones, remove all those other Procedures, and shorten the S2ESC Procedures list.
At the February meeting ( San Diego , Northrup-Grumman) the S2ESC Policies were discussed but work on them was not pursued until the Procedures were done. The S2ESC Secretary "Chuck" Walrad said she would start looking into these.
The site is currently hosted on IEEE CS servers with very good up-time. IEEE-CS has been responsive to administrative updates/issues, but is not able to support major capability enhancements (e.g., server side search, mouse-over). S2ESC will look into further support that the IEEE Portal group might provide.
S2ESC Standards Collection
It is time to reissue the S2ESC standards collection and there was some discussion of what to include (e.g., all active standards) and discussion of pricing possibilities such as student rates and a renewal/upgrade price for those who bought the prior version.
Standards Development Process
Angela Ortiz, of the IEEE Standards Association (SA), reviewed important elements of the IEEE standards development process starring with the function of Working Group roles: Chair, Vice-Chair, Technical Editor(s), Secretary, and Meeting Planner. The latter person would be the one expected to provide announcements of all Working Group meetings, giving advanced notice of at least 30 days and distributing the agenda to all anticipated attendees. The concept of a "quorum" was mentioned and there was significant discussion of this because of the wide range of Working Group membership numbers across all IEEE standards activities. For most S2ESC efforts, it did not sound like a formal quorum might always be possible. Some attendees noted that they used email distribution of drafts to keep many members of their groups informed and able to review/respond since they may not have the same funding as pothers to attend standards meetings.
Ballot response during the 30-day balloting period needs to be at least 75% of the balloting group and then approval of the standard by those returning ballots needs to be 75%. So a standard could be approved with 75% of 75%of the initial ballot pool voting Yes (e.g., ~40%). Where needed, a 30-60 day period would exist for comment resolution by a group within the Working Group - not the whole WG - and then recirculation would occur for any technical changes made to the document.
Following this overview, there was a second presentation on the electronic MyProject/MyBallot System(s). Revisions are being made to these systems and issues exist with the way different sponsor areas handle their WGs and projects. For S2ESC, a WG does a project/standard and goes away (or initiates an immediate revision effort). For other sponsor areas, WGs last and projects come and go under them such that multiple standards may be in progress at a given time. (S2ESC's 1175 series on CASE tool interconnection, which has five parts, is viewed as one standard with multiple parts rather than multiple standards. SC7's WGs work more like the latter sponsor model where multiple distinct standards are in process at once within some WGs such as WG7 which does all life cycle process standards and standards on maintenance, risk management, etc.) An interactive review of the MyBallot and MyProject Systems identified some issues with the design/"labeling" of links and workflow.
Several items were identified as worthy of strategic attention by S2ESC:
- Engaging software engineering communities in developing economies.
- Engaging academic communities to get S2ESC standards into educational institutions.
- Getting a wider representation on S2ESC of practitioners from non-defense/government domains.
- Specifying and executing a more efficient model for staffing a Working Group to develop an initial standards draft, using the ballot process to expose ideas to the larger community and targeting an 18-24 month time frame for standards creation/revision.
IEEE Professional Practices Committee
There was some discussion of the "instruments" this Committee has for software engineering such as the SWEBoK (outline of knowledge), SE2004 (curriculum requirements), and the CSDP (testing knowledge and practice). It was noted that there was a need for some "corpus of (summary of) practice" that would be freely available, coordinated with the "instruments," could explain the principle(s), and call out standards. The corpus would include terminology, process references, role descriptions, etc. Another idea was to "mine" existing sources of practice descriptions (e.g., CMMI®). A suggestion was made for a workshop to address this corpus with a broad community.
Included in the discussion on this topic was the subject of Practitioner Oriented Conference(s). One obstacle to conferences for day-to-day software development practitioners is management concern for lost labor time vs value (and cost?) of the conference. Various ideas were discussed which will likely be taken back and presented to the Committee for consideration, but they involved offering potential attendees material to describe the benefits of such a conference, a basic conference "trip report" outline that each attendee could individualize, activities/sessions related to CEUs and certification requirements, and a possible book or set of standards as part of the fee.
Various dates were discussed such as the weeks of January 22 or 29 or February 19 or 26. No locations or meeting sponsors were identified.
Those interested in any of the topics mentioned above, or other standards-related issues, can send email to Scott Duncan, the Software Division Standards Chair, at firstname.lastname@example.org .