Software Quality Professional Resource Reviews - December 1999 - ASQ

Software Quality Professional Resource Reviews - December 1999

Contents

SOFTWARE QUALITY MANAGEMENT

Leadership and the New Science: Discovering Order in a Chaotic World

Margaret J. Wheatley. 1999. San Francisco, Calif.: Berrett-Koehler. 197 pages.

Chaos and Complexity in Software: Challenging the Industry and the New Science

Robert B. Kelsey. 1999. Commack, N. Y.: Nova Science Publishers. 183 pages.

Reviewed by Richard E. Biehl

Software quality professionals face an array of changes, challenges, and opportunities in the new millennium. The developed world is undergoing a major shift in the way people think about and conduct their affairs. This shift is seen in all of the globalization, downsizing, outsourcing, and partnering going on. It requires an increased emphasis on leadership, organizational flattening, self-managing teams, and virtual organizations.

In Leadership and the New Science, Margaret Wheatley explores emerging models in science that can be applied to meeting many of these challenges. Her purpose is not to force an alignment between business and science, but to allow the science models to help anticipate actions in the real world that will be helpful while the rules of the game become clear. Wheatley has interpreted her task as “presenting material to provoke and engage,” (p. 9) knowing that each reader will evoke different ideas and actions based on the models presented. “It is not important that we agree on one expert interpretation or one best practice,” she writes.

Wheatley covers an array of scientific topics and disciplines, starting with the weakness of basing organizational ideas on outmoded models of reductionism and simple cause and effect. “We need to stop seeking after the universe of the 17th century and begin to explore what has become known to us in the 20th century.” (p. 8) “One of the first differences is a focus on holism rather than parts. Systems are understood as whole systems, and attention is given to relationships within those networks.” (p. 10)

Wheatley’s interest in these topics is not simply academic. By giving quality professionals new perspectives on natural organizations, Leadership and the New Science can provide tools that will allow them to rethink their efforts to affect the structure and processes of their organizations. These scientific models are useful to the extent that they conjure up ideas that can be tested and applied in real organizations. “It is only now, as life grows ever more turbulent and control slips away, that we are willing again to contemplate chaos.” (p. 119)

Chaos theory is an example of a science that is receiving more and more attention in the business community. Having roots in mathematics and physics and early applications in meteorology and economics, chaos theory is being used as a tool in organizational and human resource management. Describing the variability and randomness of nonlinear systems, Wheatley illustrates how chaos theory can be used to understand many of the issues seen in organizational development, explain many past failures to effectively change organizations, and imply areas for further research and action.

“From classical science, our culture has come to believe that small differences average out, slight variances converge toward a point, and approximations can give a fairly accurate picture of what might happen. But chaos theory exposes the world’s nonlinear dynamics, which in no way resemble the neat charts and figures we have drawn so skillfully. In a nonlinear system, the slightest variation can lead to catastrophic results.” (p. 121)

Side by side with chaos theory, mathematicians discuss fractals–those colorful, self-similar shapes often associated with computer graphics. The root of fractal geometry is the study of fractional dimensions (for example, an infinite length line drawn in a finite space is more than a one-dimensional line and less than a two-dimensional plane).

Consider this example: What is the length of Great Britain’s coastline? The answer varies based on the length of the measuring device used. An automobile wandering the coastal highways while keeping the coastline in sight will arrive at a different answer than the hiker who walks keeping the coastline within a few paces. The hiker determines that the coastline is quite a bit longer than the driver. A dog walking along the edge of the water would measure a longer distance still. To the ant, the coastline is many orders of magnitude longer than for the driver. The more granular the measuring device, the longer the result achieved. At the microscopic level, the coastline approaches an infinite length. It becomes an infinite line in finite space: a fractal.

The idea of self-similarity in fractals comes from the fact that the driver, hiker, dog, and ant would observe very similar geometry. Series of relatively straight stretches would be punctuated by rough-edged dips and curves, often folding back on themselves. This geometry would remain consistent whether the point of view was the driver (a very large scale view) or the ant (a very small scale view). Self-similarity in fractals raises questions about what can and cannot be objectively measured.

Quality professionals spend a lot of time talking about measurement and the ability (or inability) of information technology projects to estimate times effectively. “In organizations, we are very good at measuring activity. In fact, that is primarily what we do. Fractals suggest the futility of searching for ever finer measures that concentrate on separate parts of the system.” (p. 125) How long is a project? Even for projects already completed, the answer varies based on the granularity of the calendars and clocks used.

A project estimate in months will always differ from one attempted in days, which will always differ from one attempted in hours. Is it correct to assume that each of these estimates is getting more accurate? Regardless of the level of granularity in the estimate, each will include starts and stops, interruptions, and side-tracked activities. “There is never a satisfying end to this reductionist search, never an end point where we know everything about even one small part of the system.” (p. 125)

Chaos theory looks at entire systems, not individual components whose individual roles are not necessarily understood. It asks for the underlying order that remains hidden among the visible self-similar chaotic results. Wheatley “can’t think of an organization that isn’t deeply patterned with self-similar behaviors evident everywhere.” (p. 128) The key to fractal order is to allow a few simple rules to feedback on themselves so the general desired behaviors emerge naturally. “Organizations that display a strong commitment to their values make good use of this fractal creation process.” (p. 129) Quality professionals need to think about the contributions they are making (or not making) to such emergence. “Self-similarity is achieved not through compliance to an exhausting set of standards and rules, but from a few simple principles that everyone is accountable for, operating in a condition of individual freedom.” (p. 129)

Leadership and the New Science prescribes a significant role for chaos theory and fractals in organizational management. “These ideas speak with simple clarity to issues of effective leadership. They recall us to the power of simple governing principles: guiding visions, sincere values, organizational beliefs–the few self-referential ideas individuals can use to shape their own behavior.” (p. 130) In the terminology of chaos theory, vision and mission are the strange attractor for organizations that are otherwise in a constant state of chaos.

A strange attractor is the pattern of underlying order to be found in a chaotic system. It is contrasted with simpler attractors, such as the point at which a pendulum comes to rest or the elliptical orbit of a planet. The graphical pattern that emerges from W. Edwards Deming’s famous funnel demonstration is an example of a strange attractor showing that each observation, while apparently random, is bound by some yet-to-be-understood set of constraints.

A clearly formed and communicated vision and mission form the strange attractor for an organization in chaos. “We need leaders to understand that we are best controlled by concepts that invite our participation, not policies and procedures that curtail our contribution.” (p. 131) Chaos theory says that just a few underlying concepts or themes (such as Deming’s profound knowledge), rather than an explicit set of controls, can keep results reasonably close to some desired set of measures.

For software organizations using the Software Engineering Institute’s Capability Maturity Model for Software (CMM), chaos theory offers a rationale for measurement and the use of statistical process control (SPC). SPC is an attempt to move a system from an oscillatory state to a convergent state without allowing it to slip into a chaotic state. The continuous movement of a process through convergent states to a static state at some optimum level is an implicit goal of SPC. The goal of dropping process variability to zero, however, is elusive. SPC exhibits fractal geometry over time.

Regardless of the level of improvement seen in a process, a control chart describing that process always looks the same. Over time, the only part of a control chart that changes is the scale of variability. Variability, once measured as days or centimeters, later is measured in hours or millimeters. The scale reduces, but the inherent variability exhibits the same repeating fractal patterns over time.

For software processes at CMM levels 2 and 3, and those entering CMM level 4, thinking about SPC is dominated by the process control chart. Large-scale variations attributable to inconsistencies and noise in the software process can be observed, tracked, and eliminated through the careful use of basic statistics and the control chart. Because measuring involves selecting units and granularity, whether the length of a coastline or the duration of an activity, continuing efforts to decompose a software engineering effort into ever finer discrete components beyond CMM level 3 may actually be contrary to the objectives of the CMM.

By the time an organization has matured its software processes further, the process variances attributable to inconsistency and noise have been largely eliminated (that is, the low-hanging fruit has already been picked). Remaining variation is attributable to finer-scale measures among interdependent variables, the changing of which could drive the system into chaos. As a result, regression analysis has become the dominant SPC statistical technique, over control charts, by the time the organization emerges from CMM level 4 having attained a CMM level 5 process maturity.

Extending SPC to include various models and ideas from chaos theory provides benefit by allowing insights gained in the study of chaos to be applied almost immediately to process improvement efforts. This immediate use of new information is wholly consistent with CMM level 4 and process control charts. Institutionalizing this practice, along with introducing regression analysis, is necessary to effectively move the organization to CMM level 5.

A clear sense of purpose and delineated standards can form the strange attractor for a software engineering organization that has matured as far as CMM level 4. Remaining fluctuations and variations at the personal or activity level smooth out, or cohere, over time into definitive and predictable forms. Chaos theory says that, given a chance, just a few underlying concepts or themes can be used to explain the variety of chaotic results often perceived in organizations. The result is that an effective organization at CMM level 5 may actually require fewer standards and procedures to be in place than one at CMM level 4.

Chaos explains why reducing process variation at CMM level 4 is not enough to ensure process optimization at CMM level 5. Since in nonlinear systems even the slightest variation can have dramatic results, it is important to understand the impact and organization of any underlying factors that affect process and product outcomes. Using principles found in chaos theory, SPC can be used to identify underlying process and product issues on which the organization will eventually base its CMM level 5 strategies.

The challenge is to quantify those central themes using SPC at CMM level 4. Information technology management must allow the underlying order of each organization to emerge naturally through the day-to-day activity of its components. An inverse relationship is likely to exist between the clarity of an organization’s mission and purpose and its dependence upon formal rules and procedures. The more-is-better approach to moving from CMM level 1 to CMM level 3 gives way to a less-is-better approach to optimization by CMM level 5. Processes that are likely to be described by robustness entering CMM level 4 evolve toward characteristics such as elegance and simplicity by CMM level 5.

The challenge for software quality professionals is to allow such elegance a chance to emerge. “When chaos has banged down the door and is tossing us around the room, it is difficult to believe that clear principles are sufficient. Anytime we experience chaos, our training urges us to interfere immediately, to rush in, to stabilize, to prevent further dissolution.” (p. 131) In Leadership and the New Science, Wheatley offers guidance and tools for managers and quality professionals who hope to influence their own organizations for the better.

What Wheatley offers for organization and leadership, Robert Kelsey offers for information technology products. In Chaos and Complexity in Software, Kelsey writes: “First transpose the concepts and tools of chaos and complexity into the environment of the software industry and then analyze software production for chaotic behavior.” (p. 4) For software quality professionals to take advantage of such analysis, “we will need to understand the methodological criteria available to us for evaluating the different approaches to software development and quality assurance.” Chaos and Complexity in Software develops and establishes such criteria.

Kelsey begins by questioning the traditional approach to understanding processes that result in the definition of standards and procedural tasks. “They all assume causal relationships between performing certain tasks and getting (or avoiding) certain results.” (p. 14) Such definitions presume a certain linear determinism in process outcomes. “Tasks and events proceed in a more or less orderly manner, and surprises come from outside the development effort or from bad planning, not because the code or the design is vital or malicious.” (p. 15)

“We incrementally increase our tasks and procedures, usually in response to defects or deficiencies in the currently implemented set of tasks.” (p. 18) Kelsey points out that until one can properly understand where and when software becomes complex and behaves chaotically, such incremental tinkering with procedures and tasks is as likely to aggravate the situation as to relieve it. Software quality professionals “need to identify the structural components and conditions, peculiar to software production and execution, that enable and support nonlinear, dynamic, even emergent behavior.” (p. 25)

A chaotic system does not necessarily advertise itself. It looks perfectly normal when observed under its typical operating conditions. “It is not an unknowable, messy, incomprehensible collection of disparate components.” (p. 31) The problem is identifying the specific characteristics of software that lead to chaotic and complex behaviors, and clarify the parts of the development process that lead to such software. “What the development team intended their code to mean/ do/effect is less than what it can mean/do/effect under some circumstances.” (p. 148)

Unless software quality professionals can learn the cause-effect relationship between their development process and their software outcomes, there is no reason to believe that traditional processes would be any more or less effective when used to manage chaotic and complex software. What’s the difference between not knowing whether your approach is effective, accurate, and precise and applying that approach to an entity where effectiveness, accuracy, and precision may no longer have any meaning?” (pp. 46-47) According to Kelsey, nothing. If it is not known or cannot be learned why one’s processes can turn out chaotic and complex software, what justification is there for trying to alter and improve those processes?

“Everything we’ve learned about chaotic and complex systems indicates that they may be impenetrable to the quality profession’s deterministic tool set of reliability analysis, root cause analysis, and piece-part quality assurance.” (p. 45) In order to assure an ability to produce software that is not chaotic and that does not become complex, one needs clear definitions of what is meant by general nonchaotic and noncomplex quality characteristics. “Quality assurance for a product that was potentially a chaotic or complex system would be closer to roulette than to regimen.” (p. 46)

Kelsey offers an excellent contrasting model of effectiveness, accuracy, and precision (pp. 48-50) where he defines each concept generally, applies the concepts to software specifically, and then points out additional implications when chaos and complexity are taken into account. He uses these implications to outline three areas for his investigation: the whole system vs. component part relationship, the dynamics of systems far from equilibrium, and the role of emergence in understanding system behaviors.

In addition to the challenge of understanding the various characteristics of software that can lead to chaotic behaviors and complexity, Kelsey draws “a distinction between the unanticipated and the accidental, therewith introducing the first significant difference between chaos in software and chaos in physical systems: the role of intention and anticipation in software systems.” (p. 101) Software is encoded thought, an intentional act on the part of the developer.

“The overlap between spheres of influence and the interplay between individual horizons are just one reason why taking physical chaos theory and applying it to software development is problematic.”(p. 58) The feelings, knowledge, and motivation of the individuals on a development team, as well as for the project as a whole, are a key part of the initial conditions that need to be understood if chaos theory is to illuminate the outcomes of the development effort and resulting software products.

Without such understanding, one cannot define or understand what it means for software products or software development processes to be in equilibrium. “And if equilibrium is hard to define for software development, how is ‘oscillation’ or ‘turbulence’ to be defined? How can one claim that a system is chaotic when the attributes of chaotic behavior remain undefined and the structural components of the system are not accessible to ‘normal’ empirical analysis?” (p. 59)

To proceed with the definition of such attributes and structural components, Kelsey must clarify the breadth and depth of the system to which such analysis is to be applied. For his purposes, systems fall into two types–product and production effort–each consisting of nested systems of greater and lesser scope. Examples of product systems include the narrow view of code executing in a hardware environment and the broader view of a user using such software for a specific purpose. The definition of such systems requires the inclusion of linguistic, structural, and environmental components.

Because they primarily involve the intentions of people, production-effort systems require the modeling of a broader range of factors and attributes. Examples of such systems include the narrow view of the simple act of coding and the broader view of the entire development project. Defining such systems requires the inclusion of sociological conditions, budget constraints, departmental priorities, social environment, psychological conditions, as well as personal and linguistic conditions.

“If one defines the system too narrowly or uses a limiting strategy, it is possible to generate misleading or invalid data about the system. And since developing and using software are both intentional acts, cultural, social, and psychological influences are hard to ignore. While these influences are extremely difficult to specify, they are crucial to understanding the potential for chaotic behavior in software. On any strategy other than the coarse grained they are part of the initial conditions.” (p. 73)

Kelsey chooses to proceed using the simplest system in the nested tree, “the software in execution, in order to provide as comprehensive a description as possible.” (p. 90) Building a model of how chaos and complexity can be used to understand this narrow perspective, Kelsey warns that he is leaving the application of that model to broader and higher levels of system for subsequent research.

Kelsey then proceeds to create operational definitions for the terms and concepts needed to properly identify and discuss chaos and complexity in software. For the software practitioner still struggling to see software within the framework of chaos, the definitions on pp. 92-95 of this book are illuminating. The subsequent discussion (pp. 96-100) then uses these definitions to identify clear characteristics of systems that will, or will not, most likely exhibit chaotic behaviors. In fewer than 10 pages, Kelsey brings a general discussion of chaos down to a few characteristics that software professionals need to study and improve in software development: hardware affinity, dialogic functions, and operational complexity. Regrettably, Chaos and Complexity in Software is a difficult enough read that not all who begin will reach this rewarding point.

From chaos, Kelsey moves on to complexity, particularly the emergent properties found in software that often were not intended. Such emergent features may or may not be defects depending upon the intentions that led to them and their impact on requirements conformance. “Writing a bug is cause to slap one’s wrist; discovering a feature is cause to wonder just how it got into the code in the first place, and what to do about it now that it is there.”(p. 129)

“The potential complexity of software lies not in the difference between the expected and actual performance of the product, but in the reasons why that difference exists.” (p. 132) As developers create software they are imagining a variety of possible futures that will not become actualized until the software is used. Such imaginings must be translated into a collection of programmatic utterances that may be more or less complete for the contexts in which the software will be eventually used. “We are not aware of other possible results of our own utterance until they happen; we are not aware of features until they manifest themselves in the execution event.” (p. 139)

Kelsey contrasts chaotic and complex software, each not requiring the other. “Nonlinear, oscillatory program behavior with systemic effects is chaotic. But if the turbulence doesn’t result in a new stable systemic state, the system never becomes complex.” (p. 143) If the system exhibits one or more of the characteristics identified in his analysis, however, “the constraints of goal-directed encoded utterances are broken and the executing system is free to evolve into unanticipated states.” (p. 145)

Software can behave in unpredicted and emergent ways precisely because “a set of anticipated, linear, and deliberate behavior patterns at the microlevel are in fact combining under unanticipated conditions to produce unanticipated results at the macrolevel.” (p. 145) The complexity is exacerbated when the initial microlevel conditions are chaotic. “When a combination of actual events stimulates recovery paths in different nodes that then collide, the system’s behavior is unanticipated, but emergent in the sense that it is an extension of decision capabilities built into the application.” (p. 145)

Kelsey challenges the industry to “define its limitations with respect to chaos or complexity, drawing boundaries around itself within which it can pursue classically inclined engineering and beyond which it can make only limited claims about product quality.” (p. 149) Software quality professionals must understand the limitations and opportunities presented by Kelsey’s identification of attributes and characteristics of chaotic and complex products and processes. Concludes Kelsey, “Once a complex system is recognized as complex, our perceptions of the system are forever changed.” (p. 157)

Both Leadership and the New Science and Chaos and Complexity in Software will be valuable to software quality professionals. The combination of Wheatley’s high-level application to organizations, and Kelsey’s low-level application to software provides a powerful and useful synergy of ideas.

Kelsey’s Chaos and Complexity in Software is an advanced work. Novice readers will struggle with the academic tone and complex writing. Readers serious about understanding and applying chaos theory in information technology will want to make the effort needed to follow his reasoning as he develops his model. The effort will be rewarded by a unique exploration of how and why software products and projects can fail.

The second edition of Wheatley’s Leadership and the New Science has been revised and expanded from the earlier 1992 edition. Wheatley writes: “I am much clearer about what the science means, and I have been able to take some concepts that were difficult to grasp and explain them in more direct language.” Readers and fans of the earlier version will not want to miss this new edition. Those who struggled with the first edition will find the second edition more clearly written, easier to follow, and highly usable.

To be useful to software quality professionals, chaos theory must be more than academic. It must illuminate unexplained historical observations and it must predict behaviors that will help improve process performance in the future. Wheatley and Kelsey have provided an organizational and software framework for continued research into such applications.

Featured advertisers


ASQ is a global community of people passionate about quality, who use the tools, their ideas and expertise to make our world work better. ASQ: The Global Voice of Quality.