READ THESE TERMS AND CONDITIONS CAREFULLY BEFORE USING THIS WEBSITE. https://nrc-publications.canada.ca/eng/copyright
Vous avez des questions? Nous pouvons vous aider. Pour communiquer directement avec un auteur, consultez la
première page de la revue dans laquelle son article a été publié afin de trouver ses coordonnées. Si vous n’arrivez pas à les repérer, communiquez avec nous à [email protected].
Questions? Contact the NRC Publications Archive team at
[email protected]. If you wish to email the authors directly, please see the first page of the publication for their contact information.
NRC Publications Archive
Archives des publications du CNRC
This publication could be one of several versions: author’s original, accepted manuscript or the publisher’s version. / La version de cette publication peut être l’une des suivantes : la version prépublication de l’auteur, la version acceptée du manuscrit ou la version de l’éditeur.
Access and use of this website and the material on it are subject to the Terms and Conditions set forth at
The Merits of a formalized procedure (benchmark) for evaluating
computer-aided design drafting systems
Vanier, D. J.
https://publications-cnrc.canada.ca/fra/droits
L’accès à ce site Web et l’utilisation de son contenu sont assujettis aux conditions présentées dans le site LISEZ CES CONDITIONS ATTENTIVEMENT AVANT D’UTILISER CE SITE WEB.
NRC Publications Record / Notice d'Archives des publications de CNRC: https://nrc-publications.canada.ca/eng/view/object/?id=c5df79bc-a95e-41cb-abd0-ec7c60bbf9a5 https://publications-cnrc.canada.ca/fra/voir/objet/?id=c5df79bc-a95e-41cb-abd0-ec7c60bbf9a5
I Ser
;
THlno.
1464
National Research
Conseil national
i
I*
Coundl Canada
de recherche$ Canada
,
- -- Institute for lnstitut de
Research in recherche en Construction construction
The Merits of a Formalized Procedure
(Benchmark) for Evaluating Computer-Aided
Design Drafting Systems
by D.J. Vanier
ANALYZED
Reprinted from
Proceedings Fourth Canadian Building Congress Learning from Experience
/
Avoiding Failures Ottawa, Ontario. 6- 8 October 1985Session 6, p. 190- 196 (IRC Paper No. 1464)
Price $3.00 NRCC 27896
T h i s paper i s being d i s t r i b u t e d i n r e p r i n t form by t h e I n s t i t u t e f o r Research i n C o n s t r u c t i o n . A l i s t of b u i l d i n g p r a c t i c e and r e s e a r c h p u b l i c a t i o n s a v a i l a b l e from t h e I n s t i t u t e may be o b t a i n e d by w r i t i n g t o t h e P u b l i c a t i o n s S e c t i o n , I n s t i t u t e f o r Research i n C o n s t r u c t i o n , N a t i o n a l Research C o u n c i l of Canada, O t t a w a , O n t a r i o , KIA 0R6.
C e document e s t d i s t r i b u 6 sous forme de t irE-8-part p a r 1' I n a t i t u t de r e c h e r c h e e n c o n s t r u c t i o n . On peut o b t e n i r une l i s t e
d e s p u b l i c a t i o n s de l f I n s t i t u t p o r t a n t s u r les t e c h n i q u e s ou les recherches e n matisre
d e batiment e n 6 c r i v a n t 3 l a S e c t i o n d e s - - A- -- - - - p u b l i c ' -' - - - ? c o n s rechc KIA
THE MERITS OF A FORMALIZED PROCEDURE (BENCHMARK) FOR EVALUATING COMPUTER-AIDED DESIGN DRAFTING SYSTEMS
D.J. Vanier, Division of Building Research, National Research Council Canada, Ottawa
A myriad of Computer-Aided Design and Drafting (CADD) programs claim to address architecture, engineer- ing, and construction (AIEIC) requirements. Individual firms have great difficulty in evaluating and selecting a drafting system that will increase the productivity and effectivity of their organization. A formalized Benchmark procedure has been developed to assist purchasers in the evaluation of CADD systems. The procedure is thorough and time-consuming and should be used only at the final stage of the selection of a CADD system. The CADD Benchmark is designed to test most drafting features, as well as a number of more complex CADD operations, such as: area calculation, bills of material, and data base manipulation. It is directed at facilities management or space planning, but the software features tested cover a wide variety of drafting requirements for most building disciplines: plotting, zooming, dimensioning, scaling, rotation, filing, and symbol creation.
This paper discusses the merits of a formalized evaluation procedure, cites examples of earlier work in this field, and outlines the Benchmark procedure in depth.
In conclusion, although the evaluation method is time-consuming, the Benchmark procedure assists in the selection of CADD software for the building industry by providing expert advice to first-time puchasers, by testing a wide range of CADD operations, and by producing quantifiable results necessary for CADD system comparison.
LES AVANTAGES D'IRJE M ~ ~ ~ O D E D~FINIE D:~VALUATION DES S Y S T ~ S DE CONCEPTION KT DB DESSIN ASSISTES PAR ORDINATEUB
Une multitude de programmes de conception et de dessin assistes par ordinateur (CDAO) pretendent repondre aux besoins des arthitectes, des ingenieurs et des constructeurs (A/I/C). Certaines firmes ont beaucoup de difficultes B evaluer et 2L selectionner un systsme qui permettra d'augmenter la productivite et l'efficacitb de leur entreprise. Une methode definie d'6valuation des performances a bt.5 mise au point afin de venir en aide aux acheteurs dans le choix d'un systeme de CDAO. La procedure est minutieuse et necessite du temps; elle ne devrait Ctre utilisee qu'l la derniere &ape de la selection d'un systPme de CDAO. Elle permet de mettre 2L l'essai la plupart des options de dessin, ainsi qu'un certain nombre d'operations de CDAO plus complexes, telles que le calcul des zones, la nomenclature, et la manipulation de bases de donnles. Cette methode est concue pour la gestion d'installations ou 1,'organisation de l'espace, mais les caracteristiques logicielles mises 2L l'essai couvrent une large part des besoins en CDAO de la majorits des disciplines relevant du batiment: traGage, zoom, perspective tridimentionnelle, cadrage, affichage, classement et creation de symboles.
Cette communication discute des avantages d'une methode definie d'bvaluation, traite de certains travaux realises dans ce domaine et decrit en detail la dthode d'evaluation des performances.
En conclusion, mOme si elle necessite du temps, la methode d'evaluation des performances assiste l'industrie de la construction dans le choix de logiciels de CDAO en fournissant aux acheteurs inexperimentes des conseils d'expert, en mettant B l'essai une vaste gamme d'operations de CDAO et en produisant des resultats quantifiables necessaires 2L la comparaison des systemes de CDAO.
THE MERITS
OF A FORMALIZED PROCEDURE (BENCHMARK) FOR EVALUATING COMPUTER-AIDED DESIGN AND DRAI;TING SYSTEMSDana J. Vanier Division of Building Research National Research Council Canada
A Benchmark procedure for evaluating computer-aided design and drafting (CADD) systems for building industry applications (Vanier, 1985) was developed to provide a systematic process to objectively compare CADD systems performing standard designidrafting operations.
This paper describes the origins of this procedure for the selection of CADD systems; defines the components of the Benchmark procedure; and discusses the merits of the procedure for the architecture, engineering, and construction (AIEIC) industry.
Computer-aided design and drafting or CADD, sometimes called computer-aided drafting, is the automation of the input and the production of two-dimensional plan and elevation views of engineering and architectural graphic data. The acronym must not be confused with computer-aided &sign (CAD), which is a generic term used to cover all areas of computer applications in design, such as finite- element analysis, interference detection, stress analysis, three-dimensional representation, and geometric modelling. This paper and the associated Benchmark procedure deal only with two-dimensional automated drafting machines.
The Benchmark procedure is a systematic method for the evaluation of computer-aided design and drafting tools for the A/E/C industry. The procedure encourages a systematic approach to the evaluation and selection of CADD systems and suggests the following stages:
b .
consisting of literature study, conferences, and site visits;
Shortlist Selection, entailing definition of prerequisites, request for quotations, equipment demonstrations, and selection of shortlist vendors;
detailing invitations, Benchmark tests, and evaluation summaries;
Post-Benchmark
consisting of negotiations with vendors, evaluations of proposals, and CADD system purchase.This paper concentrates on the merits of a Benchmark procedure, however the main emphasis of the Benchmark procedure is the Benchmark test. A baseplan drawing, standard for the procedure, and a series of discrete CADD tasks are provided to all shortlist vendors. This standardizes the data received and permits an equitable and quantifiable comparison of dissimilar CADD products. The designtdrafting tasks that are tested include dimensioning, zooming, plotting, symbol creation, metric conversion, and
filing. In addition,
a
number of more sophisticated CADD features are evaluated, such as area calculation, bills of material, database manipulation, and recovery from error.Advanced technologies create many problems for building professionals. These include the added costs of these new technologies and the insecurites associated with learning a new methodology. Because computers are becoming essential components of the modem design office, professionals have to decide whether they will use these new technologies, how they will integrate the new tools, and when they will purchase their computer. One of the most difficult concerns is computer-aided design and drafting machines. The present methods for the selection of CADD tools lead to choices based more on frustration than on company requirements and equipment productivity.
There is a necessity for a thorough selection process to assist prospective CADD buyers. A Benchmark procedure is used in the computer industry to evaluate the performance of computer hardware or software. A similar evaluation procedure for CADD systems would enable design professionals to select equipment meeting the needs of the AIEIC industry. The Benchmark procedure provides a guide for the selection of the most suitable system. The procedure establishes a well-defined series of CADD operations to be tested and it permits the evaluation to be carried out by the designJdrafting professionals.
DEVELOPMENT OF THE BENCHMARK PROCEDURE Background information for a specific CADD purchase is outlined to introduce the Benchmark procedure and to show the evolution of this evaluation process for a specific application. This will permit readers to make comparisons to their requirements and will indicate how CADD systems were evaluated for a large purchase. In this way, companies planning a CAD purchase of $100 000 or more can benefit from the Federal Government's experience.
In April 1983, an Interdepartmental Task Force on Computer-Aided Facilities Management (CAFM) was formed to investigate the feasibility of Federal Government initiative in CADD for interior design, space planning, and facilities management (Interd
...,
1983). The task force recommended that software developers working closely with government designers should produce a product satisfying the requirements established by the task force. At this time the need for a Benchmark procedure was not firmly established: "...Benchmark (testing) may be required prior toCompanies an rhe Supply and Services Canada supply
list
working in computer software, architecture- design, facilities management, orCADD
development, were invited to participate. The 550 invitations that were sent solicited over100 responses.
The task force produced the Request for Roposals
IRFP)
forthe
CAFM
in late 1983 (Request, 1984). TheRFP
included guidelinesfor
the proposal evaluation, requirements ofindividual government agencies, and specifications for the "start-up" system. The "start-up" system was defined as the
CADD
package to be nsed by the succ~ssful contractor asthe base for application development Of particular interest
was the RFP requirement for a Benchmark test to evaluate
the "start-up" systems. At that time, the Benchmark test was viewed as a means of ensuring that the proposed CADD systems reflected the claims of the vendors. The request for
proposals
-
an intimidating 100 pages-
was sent to all interested companies in Januaty, 1984 and 16 proposals were received by the May deadline. The companies submitting proposaIs ranged from one-man firms to large multi-nationds. The proposals ranged from one-page letters of introduction to four-binder volumes. Because of the number of exceUent proposals, the task form realized that the Benchmark test was needed for more thana
verificationof
compliance of the "sm-up" system; it was also requiredto rank the CADD capabilities. This necessitated further investigation into evaluation processes.
Research into evduation techniques for CADD systems identified two good examples. In 198 1, the Dutch Computer Usds Group for the Construction 'Industry (CIAD) produced a document outlining the testing of two-dimensional drafting systems (The Autom.
..,
198 1). The report evaluated commercial CADD systems on their ability toperform general drafting operations for a number of engineering disciplines. The testing included a series of
mks carried out by
the software vendors. The evaluation by a team of CIAD experts took place at theCADD
vendors' sires. The procedure used by CIAD to evaluate the 12drafting systems was analysed and the ideas obtained proved to be instrumental in the formation of the Benchmark
pmedure for the CAFM task force.
In 1982, researchers at the Division of Mechanical Engineering of the National Research Council Canada required an evaluation methd for three-dimensional design
and manufacturing sys tern (CADICAM) for aerospace engineering (Aldis, 1982). In their evaIuation method, identical drawings were formally presented to five prospective companies in advance of Ihe Benchmark test to
ensure that all vendors had ample time to analyse and input the data. The CAD/CAM systems were tested on a number of tasks, including ones related to drawing information provided earlier.
The methodologies used in these two evaluation exercises
helped to develop the
CAFM
Benchmark procedare. However, the "start-up" system for the CAFM project wasdefined in prescriptive terms in the RFP, such as requirements for graphics, input, output, and "ease of use"
and as specifications for terminals, computer processing
speed, and preferred peripherals. These rqnirements were analysed, combined with the ideas obtained from other Benchmark procedures, and were translated into detailed
drafting operations for CAFM. The Benchmark procedure was developed, tested, and finalized in the summer of 1984.
During that same time, the evaluation process reduced the acceptable proposals from fourteen to a shortlist of five.
All five proposals were rested
on
separate days of the same , week in late August, 1984.The
Benchmark test quickly identified many shortcomings of a number ofCADD
systems and easily ranked the fiveCADD
systems on/
individual operations. The results proved so useful to the
-
members of the task force that they decided to place more emphasis and importance on the Benchmark results in the
CAFM
proposal evaluation than originally planned. The results of the Benchmark procedure became an integral part of the task force evaluation document (Evalua.., 1984).*
THE
BENCHMARK PROCEDUREA w e l l - d e f d pnxedure accompanies the Benchmark test; it describes investigative work up to three months before the actual test day and continues weeks after the evaluation. Brief outlines of initial studies and shortlist selection are
presented, dong with follow-up action after the Benchmark procedure. The focus for the Benchmark procedure is the action immediately before, during, and following the Benchmark tesr
This phase of the investigation includes literature search,
conference attendance, CADD consultation, and site visits.
The initial study permits interested parties to become familiar with the technology, to view CADD systems in operation, and to identify those meeting their production requirements and financial pre-requisites. This phase should reduce the field of potential systems from the wide range a d a b l e to roughly ten possible contenders.
In this phase the purchaser becomes more aware of the relationship between company requirements and existing CADD systems. A good methd is to establish company pre-requisites by formalizing budgets and company requirements. The CADD vendor's expenise can be used to produce an itemization of the entire CADD configuration. Quotations can be requested from the remaining contenders and demonstrations can be scheduled. This will lead to the
1 The "field trials" are now underway and involve $2.4
million dollars in software dtvelopmcnt funds, five
government departments, two software developers, and 15
CADD workstations.
2 CIAD has published a similar report on geometric modelling (Compu
...
1983) and an article regarding benchmarks on office automation systems is also available.I
preparation of the shortlist of contenders. This phase should
I
reduce the possible contenders from approximately ten toI five.
The letter of invitation to Benchmark informs the prospective suppliers that they are on a shortlist for the purchase of CADD equipment. This formal invititation is required to ensure equitable results from all suppliers, to
guarantee that all suppliers are well-informed of their
I responsibility, to formalize time, place, and content of the
test procedure, and to reduce the possiblity of errors or omissions. The letter identifies features of the software to be tested and how they will be evaluated. Also included with the letter of invitation is a baseplan drawing and the Benchmark Configuration Certificate. The letter states that the supplier should enter the baseplan drawing on the proposed CADD system prior to the Benchmark test date. Benchmark Bas-
The baseplan of a floor of an offie building is included with the letter of invitation for entry prior to the Benchmark test date. The baseplan will be used in the Benchmark test for placement and distribution of furniture modules and for bills of material and area calculation. As d e vendors are all provided copies of the same baseplan, these can be compared for accuracy.
Benchmark Confi~uration C e r t i f i a
This document is a formal assurance from the supplier hat the equipment used in the Benchmxk procedure is the same as that proposed in the quotation. It identifies additional features which could affect the performance of the softwarelhardware and presents specific &tails as to how the
performance will be increased.
It is important to note that there are few guarantees when it comes to the purchase of CADD equipment and software. Many items, such as cables, support services, and software modules can be considered as extras by one company and standard features by another. The document places onus on the supplier to detail every item that is included in the quotation price, leaving no costly "little surprises" after the CADD system is delivered.
A framework for this certificate is included in the Benchmark procedure document. This assists the purchasing
fm to draft a document meeting their specific needs. Legal advice can be required in the preparation of this document, as few manufacturers' guarantees or warrantees cover CADD system
wrformance.
Preparation bv Venda
The baseplan information is to be entered by the vendor during the one-week period before the Benchmark test. The data entry for the baseplan should take approximately six to
eight hours, depending on the company's software and experience. The detail of the input data will indicate the
effort made by the vendor and the ability of the system to
handle design/drafting for A/E/C applications. Benchmark Test
The Benchmark test is broken into three parts: Tenninology, General Rules, and CADD Operations. The terminology used in the Benchmark test is defined and general rules are detailed for the ven&rs to ensure equitable results from all Benchmark tests. The design and layout of a logical sequence of operations, representing typical drafting practice, are integral parts of a well-designed Benchmark
proc*.
The CADD operations form a contiguous series, permitting the evaluators to view the production of a drawing from the data capture stage through to the plotting of the final product.
The CADD operations are presented
as
a ten-page form to be completed by members of the evaluation team. There is adequate space on the form for comments and these should be encouraged by the Benchmark Director, as they will form the basis for a summary of results for each system. Common features exist for all the 18 CADD operations, such as: Operation Priority, Allotted Time, Estimated Time, and Operation Time. The operation priority is established by the evaluation team prior to the start of all Benchmark procedures and indicates to the supplier the importance the purchaser places on every operation. The latterthree
provide the time to be spent on one operation, an average time needed to complete the operation on mini-computerCADD
systems, and the actual time required for this particular operation. In some CADD operations there are additional fields for information regarding size of files and drawings and the "ease of use" or "complexity" of specific operations. These terms and others are explained in detail in the Guidelines for Evaluation included in the Benchmark Procedure publication (Vanier, 1985).
The Benchmark tests the following CADD operations: Cold Start Dimensioning
File Storing Plotting
File Recall Zoom and Scaling Module Manipulation Quick Hard Copy Bills of Materials Plot
Error Correction
Guidelines for Evaluatioq
Guidelines for Evaluation have been included to assist the purchaser in the proper evaluation of the CADD operations. All the operations have been carefully selected and they represent a cross-section of capabilities required by designers and draftspersons. The guidelines present the reasoning behind specific operations and in some cases an explanation of the computer science principles behind a CADD operation.
THE
MERITSOF THE
BENCHMARK PROCEDUREThe concepts listed below provide the framework for the Benchmark procedure and outline the logic behind the
testing process. An explanation of these key points will provide readers with a justification for considering this procedure in the selection of a CADD system to meet their
drafting requirements.
Performance specifications are becoming increasingly popular in the A/WC industry as a means of evaluating the end product. In contrast, prescriptive evaluation is predominantIy used in the selection of computer systems, such as specifying type of computer, byte word-size, version
of operating system, and speed of the computer processing unit (CPU). The Benchmark procedure evaIuates only the performance of the CADD system and is not affected by the
items on a prescriptive checkIist.
A disadvantage of prescriptive specifications is that they can
lead to sub-optimum selections: for example, a 16-bit CADD package, which might be the best overall system for a specific company, can be e h a t e d because a pre-requisite
might have stipulated 32-bit technology. As a result, the best performing 32-bit machine could be the sub-optimum winner.
The Benchmark proceduxe tests CADD operations for day-to- day drafting practice. The Benchmark does not establish criteria for three-dimensional design but does provide an exhaustive evaluation of the needs of a professional office for design/drafting.
CADD operations in this procedure have been selected for relevance to standard drafting requirements. This is a major feature of the Benchmark procedure, that the potential purchasers can see the
CADD
systems performing the time- consuming or frustrating tasks required for production designldrafting. A goal of the Benchmark procedure is to demonstrate to the evaluators system capabilities that arenormally missed in "canned demos", such as how long it
takes t file a specific drawing, how much disk space is required for a Eypical drawing, and how the system creates a symbols library. These operations are used constantly in a CADD production environment and consume the majority of operator and computer time. The simple operations, such
as the creation of symbols and the filing of drawings, consume a majority of
CADD
time and contribute to themost frustration. An exarnpIe is changing an entire drawing from i m p r i d to meaic units. The vendor may claim that
this is automatic, but in actual fact, it can require significant keyboard intervention. Toa often the purchaser is impressed by the marketing of
a
product without giving due consideration to the productivity and efficiency of the system The Benchmark test evaluates these operations uniformly across all systems and produces objective results.This procedure was developed to meet the needs of building professionals. Most professionals would develop a similar method for their evaluation, given sufficient CADD knowledge, adequate preparation time, and similar
purchasing restrictions. This procedure is now ready for use and can be easily modified to suit any additional requirements of the A/E/C industry. Professionals in faciIities management, architectural drafting, and interior design,
can
use the Benchmark procedure in its present state.However, mechanical, structural, and elecmcal drafting
1
personnel may require simple modifications to accomodatethe drafting differences in these disciplines. In addition, the principles of a procedure can be extended to different fields, such as mechanical piping layout, structural design, and even integrated office automation systems.
The day will come when we will all be able to sit behind the wheel of a CADD system and see how it performs. This is not currently possible with most systems, even in the hands of a skilled CADD user. Considerable experience is
required to master CADD systems (Lores, 1983) and a proper testing can only be carried out by well-trained CADD operators. The best substitute for "hands on" testing is to observe an expert at work.
The Benchmark procedure presents quantifiable results so that systems from various prime contenders can be compared objectively. The time required to store a drawing on one system can be compared to the time required for the identical drawing on another. In addition, subjective rating like "ease of use" can be evaluated when comparing a number of systems.
The Benchmark procedure contains considerable information on the operations of CADD systems. This information is
passed to the purchaser as various operations are tested It is not recommended to start on the procedure with no CADD knowledge, but one need not be an expert. The Benchmark procedure also eliminates the requirement for the evaluation team to be computer-literate. There is no need to know if the machine should have 32-bits, if the disk should hold 300 megabytes, whether DOS or UNIX is better, or if it must have a high resolution screen. In this procedure the evaluation staff need not have in-depth knowledge of every system being tested; that is left
to
the CADD vendor's experts.The experience with the CAFM evaluation showed that task
force members actually increased their level of CADD competance as the Benchmark tests progressed. This was beneficial in a number of ways: fmt, they were able to increase their CADD knowledge base and second, they could
re-evaluate their pre-requisites after they had viewed a number of systems. The procedure may then be viewed as a learning tool: the man-years of experience contained in the procedure are available to the evaluators during the Benchmark procedure.
Equipment demonstrations and computer shows typically feature all the best aspects of a product and salespersons will rarely discuss the demerits of their system. As a result, the information presented is naturally always biased towards their hardware and software. The majority of vendors tested to date have been open to the concept of the Benchmark procedure and have followed the procedure to the letter. Because the vendors are all rigorously following a systematic procedure, they are forced to answer specific questions when a system crashes or a system response is slow. In "canned demos", the vendor c h easily sidestep problems without the observers noticing.
Evaluates Existing Tools
The Benchmark procedure is based on the principle that immediate requirements are more important than potential applications. The procedure evaluates only what is available and not prototypical versions or vendor promises. It also tests the software on existing hardware and not hypothetical disk drives or future equipment. The purchaser should insist on having the specified hardware and software for the Benchmark procedure, as the slightest deviation from this defeats the purpose of the procedure. Experience has shown that "Murphy's Laws" have numerous corollaries in CADD.
An advantage of the Benchmark procedure is the expeditious use of personnel. The evaluation team need not include
a
principal member of the firm, since the most qualified
person to head the evaluation team may be the drafting supervisor or the design chief. It is recommended that a senior partner participate in portions of the evaluation but need not direct the operations. Of course, the evaluation team should report to the senior partners on progress and on matters concerning financial limits.
An advantage of the Benchmark procedure is the value of the exercise as perceived by the CADD vendor. This formal procedure is viewed as a potential sale rather than another "demo". Too often, browsers consume considerable time and effort of the vendor without any commitment to purchase. As a result, the quality of presentation to all buyers suffers. In this procedure, the vendor sees the evaluation team as being well aware of their requirements and will treat the process accordingly.
Benchmark procedure provides the oppommity for the
evaluation team to
see
the smngth and depth of the supportstaff.
If problems =cur in the Benchmark procedure thatcannot be resolved by their "experts", it may be difficult for
the supplier to provide adequate suppon after the purchase. The same is m e of the hardware; if the vendors are not able to obtain the purchaser's desired configuration of equipment or to make it function for the Benchmark test, they may not be able to adequately assist the purchaser after the equipment is delivered. The annual cost of hardware and software support by CADD vendors is normally 12% of the system cost. This service includes software upgrades, but does not necessarily include major software modifications, new software modules
or
custom software development. Theseextras will increase the cost of the
CADD
installation and, combined with the life cycle cost for service and support, could eventually equal the capital cwt outlay for the CADD system. These costs are related to the vendor and not the product he is selling. Many "third party" vendors presently selling CADD software on general micro-processors and mini-computers are unfadiar with their products. It is therefore important in the evaluation process to be able to observe the support staff, see their facilities, and know their sources of "expert" information.effechve for Laree CADn Purchasa
A minicomputer
CADD
system is undoubtedly the most expensive acquisition for most professiond design practices. The time spent on the selection process should represent the importance of the decision. The Benchrnark procedure is time-consuming, but represents the thoroughness required for mini-computer purchases over $50 000. The procedure would be difficult to justify for a single user, micro- computer installation. However, if the purchase involved more than three or four microcomputer installations, the procedure would be warranted.SUMMARY
The Benchmark procedure will assist the AIElC industry by providing a formalized method for the evaluation of computer-aided design and drafting tools. The procedure provides an objective comparison between a wide variety of CADD systems having little in common. It will help educate potential CADD users and make them more aware of their requirements and the capabilities of existing CADD technology.
ACKNOWLEDGEMENTS
The author would like to thank Jamie Worling for his contribution during the preparation of the original CAFM Benchmark test This paper is a contribution of the Division of Building Research, National Research Council Canada.
UD Service U DS l l p p M
Servicing of software and hardware systems is an integral, but often-overlooked, part of CADD purchases. The
REFERENCES
Aldis, C., Curley, J., Evans, F., Fuller, J., Isnor, D., Jones, D., Steele, K., Thomson, V., and Wong, K., NRC CADICAM Survey and Evaluation, LTR-AN-49, Division of Mechanical Engineering, National Research Council Canada, Ottawa, May 1982.
The Automation of Draughting Work: Final Report of the Project Group-CIAD, CIAD Association, Zoetermeer, Netherlands (English Translation
-
Construction Industry Computer Association, Cambridge, England) 198 1. Burns, D., and Venit, S., PC vs MAC: An Unfair Match,PC-the independant guide to
IBM
personal computers, Vol. 4 No. 15, July 23, 1985.Computer-Aided Designing: A Guide to the Purchasing and Introduction of CAD Equipment, CIAD Association FACE Report 5, Zoetermeer, Netherlands, October 1983. Evaluation of Proposals and Recommendations for Computer Aided Facilities Management InterdeparunentaVIndustry Field Trial Program, 1984 (Available from Technical Systems Secretariat, Public Works Canada, Confederation Heights, Riverside Drive, Ottawa, Ontario, KIA OM2).
Interdepartmental Task Force Report on Computer Aide11
Facilities Management, Public Works Canada, 198 3
(Available from Technical Systems Secretariat, Public
Works Canada, Confederation Heights, Ottawa, Ontario, KIA OM2).
Lores, ME.,Chasen, SH. , and Garner, J.
M.
,
Evaluation of 3-D Graphics Software: A Case Study, Computer Graphics and Applications, Nov 1983.Request for Proposals, 1 ET.RFP.CAFM, Supply and
Services Canada, March 1984 (Available from Supply ! and Services Canada, OASIS Directorate, 2C2 via 4C1
Place du Portage, Phase 111, 11 Laurier Street, Hull, Quebec, Mailing address: Ottawa, Ontario, KIA 0%). Vanier, Dana J., Benchmark Procedure to Evaluate
Computer-Aided Design and Drafting Systems for Building Industry Applications, Division of Building Research, Building Practice Note, National Research Council Canada, Ottawa 1985.
