Gordon Skelton, Mississippi Valley Title Insurance; Annette Lerine Steenkamp, University of Detroit Mercy; and Cecil D. Burge, University of Southern Mississippi
To properly evaluate the use of the proposed analysis and testing technique one must have a system written in C++. To fulfill this requirement, a software system was created. The prototype provides for project management for a small software development group. There are four key actors in the system: 1) the executive; 2) the personnel manager; 3) the project manager; and 4) the employee. The executive serves in an oversight position with limited access to the system for viewing a specific projects or employees data. The personnel manager hires and assigns employees to projects. The project manager creates new projects and manages the status of projects as they proceed from initialization to completion.
Following the process outlined in Rationals tutorial for Rational Rose C++ (Rational 1999), the prototypes use cases were developed, followed by the creation of sequence diagrams representing each use case. Finally, classes were developed and associated with each object, thus completing the cycle from use case to class model.
From this main use-case diagram in Figure A-1 one can see that the four actors are the employee, the personnel manager, the project manager, and the executive. The project manager is responsible for maintaining project information, similar to the personnel manager who maintains employee data. The task of maintaining data includes additions, modifications, and deletions. The project manager interacts with the employee data when assigning an employee to a particular project. The executive, in the prototype, oversees both personnel and projects by examining their status but cannot make modifications to either.
Once the use-case diagram has been developed, each scenario must be modeled to provide necessary detail about the interaction of objects in support of each scenario. As discussed earlier, there are two primary UML diagrams provided for the modeling of system behavior: the sequence diagram and the collaboration diagram. Sequence diagrams provide for the major actions that can be performed by the various actors. Sequence diagrams are presented here for the create project and create employee scenarios shown in the use-case diagram. In a complete system design, sequence diagrams would be developed for each scenario.
Use cases play an important role, since they help identify the interaction of objects that were eventually translated into sequence diagrams. To help further illuminate the project under development, collaboration diagrams were created that represent a slightly different view of how the various objects interact. To fully understand the prototype, the following UML diagrams, developed using Rational Rose, provide different views of the system.
The sequence diagram, Figure A-2, illustrates adding a project by the project manager. Note that the return values are indicated as messages. One problem with this method is that Rational Rose responds as if these are messages to the object(s), not merely documentation of the return values, and thereby cannot create the proper class methods. This defect leads to faults in the generated code.
In this instance, the return values were not indicated. One will note, however, that there are two actors involved in this sequence diagram (see Figure A-3). First the employee completes and submits an employee form. The personnel manager then accepts that form and adds the employee to the employee record in the computer. The employee record sends a message to the employee list to add it to the employee list.
Once the sequence and collaboration diagrams are developed, the information contained in each diagram is used to develop the classes necessary to support the objects identified in each scenario. The functionality of each object is identified from its role in the behavioral model. This functionality leads to the development of the class methods. Class attributes are also developed from information obtained during the analysis phase where one determines the required attributes. In the case of an employee object it was determined that the following attributes were needed.
For the purposes of the prototype (see Figure A-4), the employee class is restricted since one is not concerned with any additional personal data that are outside the context of the project management system. In a more complete example, one would have additional data attributes.
Figure A-5 presents the entire class diagram for the project management system. The diagram includes all of the classes used to support the objects participating in the project management system. In addition to presenting the classes, the relationships among the classes are illustrated. This information is used by the CASE tool to generate the classes along with their inheritance and associations. The programmers can also use this diagram to verify that they are implementing the classes according to the design.
From these initial diagrams the software was developed. Then each of the scenarios discussed in Figure 6 was realized. The following sections present the results of testing each scenario with the integration testing technique.
To evaluate the effectiveness of the integration testing technique, each of the scenarios described in Figure 6 was applied to the project management prototype. The results of these scenarios are discussed in the following sections.
Scenario 1: Design is Correct, Programmers Introduce Defects
In this instance the design is complete. Under Rational Rose, the CASE tool is capable of generating code implementing the classes, including attributes and method signatures. The rest of the code is left up to the programmer. As the programmers add code to fill out the individual class methods, defects are introduced, and the individual method signatures may also be altered. Depending upon the amount of detail presented in the class diagrams, the programmers may be allowed the freedom in completing the class methods to introduce defects, even though the design properly represents the requirements.
As can be seen from the design of the project list, (see Figure A-5), the add_project method is required to handle the addition of a new project to the project_list via the provision of a pointer to the new project and the return of a boolean value indicating whether the addition was successful or not.
Using Rational Rose C++ 98i to generate the C++ code allowed for the modification of the code to simulate programmer-introduced defects. The code was then used to reverse engineer the design. The following pre- and post-implementation diagrams were then created, representing the class relationships.
The class diagram, as seen in Figure A-6, does not provide detailed information about how the actual class methods are implemented. Only if the activity diagrams are included will the flowchart for each method be presented. It is common practice to not develop such detail unless there is a need for information about a complex algorithm. Only the method signatures are provided. To access that information one must click on a particular classs method for the formal parameters to be revealed. In this instance, clicking on the add_project() would reveal that a pointer was to be used when inserting a new project to the linked list. Furthermore, unless there is sufficient documentation, the programmer might not understand that a linked list is to be used and an array may be employed instead. Still, a pointer would be indicated. The accidental use of a projects id in place of a pointer would result in a run-time error that may not have been identified until the code is executed.
As can be seen in Figure A-7, the reverse engineering tool of Rational Rose C++ 98i created a diagram similar to the one built using the design tools. Again, one cannot distinguish the differences in the two signatures for the class method - add_project(); sufficient information is not provided in the diagrams produced by the printing function of Rational Rose. Even employing the extended view of the method did not reveal the correct parameters for the method. The tool, in an attempt to understand the use of the * as indicator of a pointer, converted it to an X, which does not correspond to proper C++ syntax: add_project(projectXproject_addr:void):boolean. Here one can see that the return type for the add_project method is correct, boolean. The formal parameter, however, does not represent a pointer to a project.
Still, with this information, one can determine that the design and implementation do not correspond. Further examination of the source code reveals that the implementation was flawed. The design was correct; however, the programmers misinterpreted the design diagrams, thus introducing defects into the code.
Scenario 2: Design is Incorrect, Programmers Follow Design
This scenario is more difficult to illustrate within the context of the prototype. Defects may be introduced into a software system for a variety of reasons, ranging from miscommunication between the user and the analyst to inexperience within the programming staff. It is not uncommon for the lack of detail or incomplete system requirements to adversely affect system design. Using a CASE tool like Rational Rose will help the analyst design a system that is at least logically consistent.
Rational Rose provides a tool for checking a model that in return produces a log of all errors. The nature of Rational Rose, and its weakness in allowing for modeling return values in sequence diagrams, reduces the effectiveness of the tool in correctly identifying defects within a design. Employing the model-checking capability, along with the integration testing technique of this thesis, will improve the quality of a design. Correctness of a design is far more elusive since a complete, though flawed, design may actually pass the design-checking process.
Under this scenario, the pre- and post-implementation diagrams may correspond. As the design is implemented, however, either through the use of a CASE tool or by programmer coding, there will be continued review of the UML diagrams representing the design and implementation. Here the test group will have the opportunity to examine these diagrams and possibly identify the faults in the design and/or implementation. Because of the incorrectness of the design, there may be instances where the design will have to be modified. These modifications will be reflected in differences between the pre- and post-implementation diagrams.
As with the other scenarios that involved incorrect or incomplete designs, the technique will help highlight instances in the implementation where programmer modifications were made to the design. If the design weaknesses are not related to the operation of the software but to the logic of the design, however, it may be later in the integration and system testing that the defects are recognized. One hopes that with the additional review and inspection brought on by using the diagram comparison of the technique that the testers will recognize the faults in the design prior to completing the implementation.
Scenario 3: Design is Incorrect, Programmers Attempt to Correct It
As in Scenario 1 where the design is correct and the programmers introduce defects, the pre- and post-implementation diagrams will differ, though these differences will depend upon the type of defect present. Defects in the method signatures can be detected using detailed class diagrams that provide adequate information about the methods, their parameters, and return values. Defects in object interaction, that is, modified object sequencing and object collaboration, can be identified using sequence and collaboration diagrams.
To evaluate the testing technique, the design of the project management prototype was arbitrarily modified to allow missing elements. Here the sequence diagram for adding a new project (see Figure A-8) was modified to exclude certain elements.
This sequence diagram illustrates how a project record is added to the project list; however, exactly what data are stored in the project list remains vague. Furthermore, the fact that a boolean value should be returned indicating success or failure of the insertion into the list was accidentally left out of the sequence diagram.
In an attempt to correct the design, the programmers assumed the project record was a linked list of records, not a linked list of pointers, and implemented it as such. They correctly assumed that there should be a boolean return value indicating success or failure of the record insertion. Figure A-9 presents the sequence diagram representing the implementation.
By examining the two sequence diagrams, it is apparent that there is a difference between the diagram representing the design and the diagram representing the implementation. Highlighting these differences leads to the need to reexamine the places that indicate a fundamental difference. The issue of not returning a boolean value indicating success or failure from insertion into the project list is easily determined as a design defect. Making changes to the design and then regenerating the pre-implementation sequence diagram will bring those two diagrams into agreement. The failure of the design to correctly indicate that the project list is to be a linked list of pointers, however, was not resolved by the programmers. Therefore, the implementation remained flawed, though the sequence diagrams do correspond with one another. Only after further examination of the sequence diagrams by the programmers and the testing group was this defect detected. If this defect was not identified, it is conceivable that the linked list would have remained in the implementation as a linked list of records, not pointers. The software might have functioned correctly, however, if by chance other portions of the design, such as a reporting or query function, did have a more complete design that correctly indicated that the project list was a linked list of pointers.
From this scenario it is evident that no portion of the design can be isolated and compared with the corresponding implementation. One must take a consolidated approach to integration testing. Using the pre- and post-implementation diagrams one should examine all common elements across all the available UML diagrams to isolate potential defects.
Scenario 4: Design is Correct, However Lacking Detail, Defects Occur
This scenario correlates closely with Scenario 3 in that defects are introduced both from the design and by the programmers. Because of the lack of detail, the programmers are left with little choice except to expand the design during implementation, making decisions where the design is lacking. This scenario is perhaps the most common encountered in OO development. The design may adequately represent the systems requirements; however, details are missing that allow defects to be introduced as the code is developed. Programmers, left with the responsibility of completing design details, may make decisions that negatively impact the correctness of the implementation.
Rational Roses tool for generating C++ code from the design focuses its attention on the class diagrams. Code is generated that represents the classes and their relationships. The sequence and collaboration diagrams, however, are not converted to C++. Furthermore, the individual class methods are not produced by the tool. This lack of detail places a burden on the designer to fully document the system including all necessary information that can be used by the programmers to develop the actual code.
It is often the practice to eliminate the design of a classs methods unless there is a complex algorithm being employed. In other words, the activity diagrams of UML are not created. Furthermore, system designers often leave the detailed implementation decisions to the programmers. Areas of the design that are left undocumented can lead to flaws in the implementation.
To test the effectiveness of the technique under this scenario, the design for inquiring on a particular employee was left incomplete. Instead of stating that the container class was a linked list of employee record addresses, the design illustrated that there was a linked list of employee records. When a second programmer developed the inquiry method there was adequate information indicating the use of the linked list of addresses. Therefore, the design was correct, though incomplete. The pre- and post-implementation diagrams, both the sequence and class diagrams, provide adequate information pointing to the mismatched data types.
Using the technique of comparing pre- and post-implementation diagrams it was possible to identify the presence of a defect in the implementation. Again, as in Scenario 3, the sequence diagrams greatly assisted in indicating there was a difference between two elements of the design. Further examination of the class diagrams pointed to the fact that the query_employee and add_employee member functions used different arguments, one an employee_id and another a pointer to an employee record. Highlighting these differences lead to the discovery of the defect in the implementation. The initial design failed to indicate that the employee_list was to be a linked list of pointers allowing separate implementations by different programmers.
From this preliminary discussion one can see that reverse engineering the UML diagrams from the source code and the executing program can provide increased success in defect detection and potentially reduce the amount of time and effort required to complete integration testing of OO software.
These are only some of the many types of defects that can be found by building post-implementation diagrams throughout the software development life cycle. Continually creating and analyzing these diagrams helps users identify defects that may be overlooked and only found during system and acceptance testing.
The manual aspect of the integration testing technique makes it, for the most part, limited in usefulness in large projects. Automation of both the post-implementation diagram creation and the comparison of those diagrams with the pre-implementation diagrams is mandatory if the technique is to obtain widerspread acceptance.
Further enhancing the technique with the ability to generate test cases for integration testing would integrate the technique with the Revised Spiral Development Process. Test cases could be developed by providing input into either the design or the post-implementation diagrams, which would allow one to set the parameters for acceptable messages sent by one object to another.
At present the technique provides for the identification of interobject and intraobject communication. The content of messages needs to be included in order to detect message content defects. Enhancements such as this are necessary additions before one can consider the requirements for automating the entire integration testing process.
This article presents a new technique for integration testing of OO software that uses post-implementation UML diagrams to compare with pre-implementation design models. The process of producing these post-implementation diagrams and comparing them with the design models continues until the software has been compiled and is fully integrated. At that point, system and acceptance testing begins.
The technique is integrated into the Revised Spiral Software Development Life Cycle, supporting integration testing of both objects and subsystems. The Unified Software Development Process is enhanced to provide support for this technique.
The weakness inherent in the technique is the lack of automated analysis and diagrams, to be addressed in the second version of the technique. An analysis tool is being developed capable of taking C++ source code and instrumenting the code with the necessary data production statements. Phase three of the technique development will take those data and generate corresponding UML diagrams. Future efforts will be aimed at evaluating the difficulty of automating the diagram comparison process and eventual test-case generation.
Rational Software Corporation. 1998. Rose C++ tutorial. Cupertino, Calif.: Rational Software Corporation.