Public Sector IV&V©

Introduction

For about twenty years I have been involved in Information Technology (IT). I have been a business analyst, a software tester, a test manager, and an independent verification and validation (IVV) manager. Prior to that I was involved in developing evaluation strategies for human service delivery agencies. You might say that I have spent most of my career asking the same questions and receiving the same occasionally complete but mostly incomplete or not existent responses. Here are the strategic questions:

• What will this accomplish?
• How will you know it has been accomplished?
• What approach will you use?
• What obstacles have to be overcome?

The answers to these strategic questions, coupled with answers to lower level questions, such as “What is included in the design to help overcome the identified obstacles?” inevitably will determine the success or failure of the software development effort. Success is defined as having a stable and viable application within schedule and within budget

History

An early step to ensuring a successful software effort came many years ago when the Department of Defense (DoD) promulgated Standards that specified that certain engineering documents had to be prepared for all systems, including software, developed for the agency. DoD also specified the contents of these various engineering documents. Although considered burdensome by many, these documents provided a consistency of format and content among the multiple contractors creating the large and complex systems.

It did not take too many years for the success of the DoD standards to become evident. More and more civilian agencies mandated the use of the DoD standards when contracting out their own software development.

Another step came about when DoD realized that ensuring a successful software effort also required formalized processes and procedures in addition to formal documentation. Carnegie Mellon University was tasked to meet this need. The result of this tasking was the creation of the Software Engineering Institute and its subsequent identification of engineering and management improvement areas. Perhaps the most well known effort is the Capability Maturity Model (CMM). This model describes five stages of organizational maturity related to software process improvement.

Within a few years DoD saw significant software process improvement within those contractors using the SEI-CMM. It was then that DoD decreed that by the year N all contractors who wanted to do business with them had to be at a SEI-CMM Level 3 or better.

Concurrent developments were taking place in the field of software engineering, especially in the arena of software quality assurance. Software applications were being written in a variety of languages for an every increasing number of environments. Applications that started on mainframes, many years ago, were then useable on clients, and are now useable on the WEB. Some applications are even available on hand held devices such as cell phones and personal digital assistants (PDAs). Software that was once merely “debugged” by the programmer was now being tested by a separate group known as testers.

Testing, which began as an ad hoc activity, has evolved into a more formal activity with its own documentation and utilizing a software quality infrastructure. This infrastructure included change control for code and documentation, configuration management for tracking changes and their authorizations, and standards for document development as well as guidance for life cycle processes and tasks. The use of testing with this infrastructure has yielded positive results for projects based on rapid prototypeing and for those using the classic waterfall methodology.

Testers, and others, realized that a wide variety of tasks needed to be completed prior to beginning testing – if testing was to be productive and the project was to be successful. These tasks were labeled verification and validation (V&V) tasks. Verification is typically defined as those things that are done to determine that the application has been built correctly; whereas, validation determines that the correct application has been built. Prior to user acceptance testing which validates that the application is what the user expects; we engage in numerous activities that verify that the application has been correctly built.

Verification and Validation Processes and Activities

Over time these verification and validation activities and tasks became more formalized. Some verification and validation processes take place throughout the software development life cycle. Some activities begin during one phase and are completed in a subsequent phase.

Each of the processes (management, acquisition, supply, development, operation, and maintenance) includes a number of activities. Execution of specific activities depends on the integrity level of the software system, product, or service. An organization with a low level application may undertake only a minimum number of activities from each process; whereas an organization with a high level application will probably undertake all of the activities from each process described below.

The management process includes activities used to manage the V&V effort throughout the development life cycle. Activities may include the development of the Software Verification and Validation Plan, assessment of all baseline changes, review of the V&V effort, support of management and technical reviews, and interfacing with organizational and supporting processes.

The acquisition process begins with the definition of need for acquiring the system, software product, or software system. The process continues with the preparation and issuance of a request for proposal, selection of a supplier, and management of the acquisition process. Activities during this process may include assessing the V&V effort, planning interfaces with the supplier(s) and acquirer, and reviewing draft requirements contained in the request for proposal (RFP).

The supply process is initiated by the decision to prepare a proposal or by entering into a contract to provide a software system, product, or service. The process continues with a determination of procedures and resources needed to manage the project. Activities include contract verification and V&V interface with the supplier planning.

The development process contains the all the activities and tasks related to the development of the software system, product, or service. This process includes all of the activities associated with requirements analysis, design, coding, integration, testing, installation, and acceptance.

The operation process covers the operation of the software product and the concurrent operational support to users. Activities include evaluating the imnpact of proposed changes, evaluating operating procedures for compliance with the intended use and analyzing risks affecting the user and the system.

The maintenance process is activated when the software product undergoes modifications to code and associated documentation. Such modifications may be driven by an identified problem or a need for improvement or the need for adaptation.

Seldom is there a unique group assigned to V&V activities. These activities may be undertaken by managers, developers, testers, or quality assurance personnel. When the tasks are completed the application displays stability sooner, integrates smoother, and gains user acceptance easier.

Independent Verification and Validation

When these tasks are executed by persons under the control of the development manager then they are called V&V activities. However when they are completed by individuals not under the auspices of the software development manager, they are termed independent verification and validation (IVV) activities.