DESIGN REVIEWS AND FACI

1. Preliminary Design Review

The Preliminary Design Reviews for the three BUIC III CPCEIs, ADP, SEP, and UCP, are most noteworthy in that the meetings did achieve their purpose; a formal technical review of the proposed basic design approach was satisfacto- rily accomplished. For each CPCEI, the documented preliminary design pre- sented information necessary to the conduct of the formal review. The infor- mation included a brief review of the interfaces identified in the Part I Specifications, a discussion of timing, a concise description of the CPCs, storage allocation, and other information pertinent to the computer program design. This was essentially in accordance with system management guidelines contained in EST-1 and, more recently, EST-3.

The ADP PDR was significant in that it led to the subsequent cycle time analysis.

The expected cycle time given in the preliminary design exceeded the minimum specified in the Part I Specification. Since the input/output table size was predicated on the specified minimum time, a determination was made that the I/O table design should be reconsidered to make more efficient utilization of the output capabilities. The ensuing analysis revealed the cycle time problem.

Also of interest is the fact that, contrary to the understanding obtained at the ADP PDR, JOVIAL was not used as projected for the development of ADP. It was intended that new CPCs or rewritten BUIC II CPCs be written in the JOVIAL computer program language. Subsequent efforts revealed that the expansion factor prohibited the use of JOVIAL; code written in JOVIAL could not remain within the storage constraints. Consequently, a decision was made to forego JOVIAL for ADP and use machine language instead.

Thus, it is evident that the PDR served as the stage for involving the SPO/

MITRE monitors in the decision-making process with respect to significant aspects of CPCEI program design. The design agreed to at the PDRs facilitated mutual understanding by SDC and SPO/MITRE of the problems and their solutions in the computer program development that followed the PDR.

2. Critical Design Review (CDR)

As described at an earlier point (Chapter V), CDRs were scheduled and held incrementally, and for the most part in conjunction with preliminary quali- fication testing. While there were some discrepancies associated with the functional emphasis of PQTs as opposed to the computer program component (CPC) emphasis of CDRs, the reviews were generally concerned with design documen- tation for the elements undergoing test. Hence, the reviews were held at the level of completed design for those elements.

Despite their close association with PQTs, however, and in contrast with the PQT experience (see Section F below), there is a notable absence of reports of difficulty, resulting changes, or other action items related to the conduct of CDRs. In general, the CDRs occasioned very little concern or comment.

On the whole, these BUIC III CDRs did not appear to involve elements which were particularly novel as compared with prior practice. They were often attended by SPO representatives, and typically included presentations of the design, functions, timing, size, and interface characteristics of the CPC(s) being reviewed. However, beyond the point of providing visibility of technical progress, the purposes apparently did not extend to reaching significant

decisions with regard to the development cycle of the computer programs involved, This is perhaps not an unexpected circumstance, considering that the designs were already complete at the time of reviews.

In Exhibits EST-1 and EST-3, the emphasis is placed on CDRs which are held

"at the level of flow charts or computer program logical design prior to coding and testing." At that level, the stated formal objective is to identify the design documentation which will be released for coding and testing. Experience on computer program CDRs in that category is not avail- able from BUIC III or other system programs with which the authors have been associated. However, computer programmers have volunteered firm opinions to the effect that it would not have been feasible to accomplish the stated ob- jectives of a CDR at that level for a CPCEI as complex as the ADP. Comments are made that: the design which initiates coding must remain flexible during the coding process; exact interfaces among CPCs are not visible for review on an incremental basis; and the ability to conduct such reviews in a meaningful and adequate fashion would require significantly greater technical resources than the SPOs have typically had at their disposal.

While the interim design level (i.e., at flow charts) is given the primary emphasis, it is to be noted that EST-1 does provide a range of options which is broad enough to cover the BUIC III application.* Judging from the comments which have been made, and the fact that they did not contribute additional problems, it may be fortunate that the CDRs were handled as they were, in BUIC III.

3. First Article Configuration Inspection (FACI)

The first Article Configuration Inspection of the three CPCEIs was essentially accomplished in May 1968. The FACI was noteworthy in that there was very little precedent for conducting such an inspection for computer program items in accordance with recognized system management concepts.** While the FACI

* ESD Exhibit EST-1, Section H, p. 40-13.

** As noted elsewhere, one element of SEP had been FACl'd in September of the preceding year.

did not proceed without some problems, the nature of the problems did not pose severe obstacles and they were resolved to the mutual satisfaction of the part- icipants. The objectives of the FACI were achieved; the Part II specifications were audited and approved with the qualification that revisions based on SPO comments be incorporated in the basic issues of the Part IIs which were due to be published by 1 February 1969. With SPO approval, the Part IIs were recognized as the instruments defining the product configuration baseline for the re-

spective CPCEIs. For practical purposes, however, the baseline had been

established in the preceding month, April, at the onset of Formal Qualification Tests. Formal control was maintained of all changes to the draft Part II speci- fications which had been issued describing those FQT configurations.

FACI was preceded by FQT and followed by Category II testing. This relative timing is reasonable and conforms to established concepts. However, certain deviations did occur as a result of the known cycle time deficiency in ADP.

Although an approach to correct the deficiency had been agreed upon and approved, its implementation had not been completed prior to FQT and FACI. Since the

corrections entailed extensive changes and recompilation of five computer

program components, SDC and SPO/MITRE agreed to defer the FACI audit of the Part II volumes for those five CPCs. The audit of those five CPC volumes was then performed in February 1969 after implementation of the cycle time changes, thus concluding the FACI. An unusual aspect of those five volumes is that the basic issues include Specification Change Notices for ECPs and CRs that were pro- cessed against previous draft Part II volumes of the five subject CPCs.

The contrast of the approach taken to the FACI meetings for BUIC III and SEEK DAWN Interface Computer Program (SDICP) is interesting. In both cases, draft copies of the Part II specifications were delivered to the SPO in advance of the FACI meetings; BUIC III drafts-were delivered approximately 30 days before FACI; SDICP drafts were delivered approximately 90 days before FACI. Whereas the detailed review of the BUIC III Part IIs was performed at the FACI meeting, the review of SDICP Part IIs was essentially done prior to the meeting. In the latter case, SPO/MITRE comments were sent to SDC so that mutual agreement had been substantially obtained before the FACI meeting. The meeting thus served to confirm understandings that had already been reached and to resolve those few questions that had not been completely resolved earlier.

The difference in approach is of interest since the one week FACI meeting for BUIC III (disregarding the second meeting for the five deferred CPCs) permitted only a superficial examination of the Part II specifications. It is highly likely that a more comprehensive examination of the BUIC III Part IIs would have revealed many more discrepancies among the CPC descriptions, flow charts, and listings. Depending on the bulk of the Part II specifications, available SPO technical resources, and other relevant factors, the SDICP approach may be the more desirable one in conducting FACI on other CPCEIs.

configuration, was viewed in advance with some apprehension. While procedures for computer program control and accounting had been in effect for systems pre- ceding BUIC III, they were developed and used internally by the contractor, had been largely confined to control of computer program listings, and had not been carried out under the formal label of configuration management. However, with minor exceptions, anticipated difficulties did not materialize.

There were two areas of potential difficulty which deserve mention. The first related to detailed flow charts for CPCs. which proved to be expensive and

time-consuming to maintain in the draft Part II specifications during a period of some months preceding FACI. The possible continuing impact of this problem

was avoided, following FACI, by a SPO-approved change which eliminated the CPC- level flow charts from the specification.

A temporary problem arose in the course of Category II testing, relating to a proposed requirement for a new Version Description Document to cover each

daily change made in connection with the test activities. The question of what constitutes a "new version" is not unequivocally defined, and disagreements occurred on the point among personnel of the SPO, contractor, and test team.

It developed that the real issue was more a matter of test philosophy than of control procedures, and working solutions were reached following a week or two of discussion.

However, the BUIC III experience as a whole has provided clear evidence that the configuration control and accounting procedures which had been adapted for computer programs were remarkably efficient. Some data relating to rates of changes were presented in the preceding section of this chapter. Following FACI, control was extended to cover the computer programs and Part II specifi- cations, and was maintained routinely under complex circumstances. In practice, situations arose in which as many as three versions of the Air Defense Program were in existence or under development at one time, each differing signifi-

cantly from the others with respect to incorporation of approved changes and scheduled introduction into test or operational use. The process included, for example, accounting for Class II error corrections made to a given version which were either applicable to succeeding versions or not applicable because of superseding Class I changes, and also involved keeping track of change relations with other computer programs, support documentation as well as specifications, and items of system equipment. Under these conditions, con- trol was maintained at all times and with relatively minor difficulties.

Considering the frequency and volume of changes implemented, it seems clear that the BUIC III experience has also demonstrated that post-FACI control need not seriously impair the flexibility with which computer programs can be

altered to accomplish desived changes in system functions.