Publisher’s version / Version de l'éditeur: Building Practice Note, 1985-09-01
READ THESE TERMS AND CONDITIONS CAREFULLY BEFORE USING THIS WEBSITE.
https://nrc-publications.canada.ca/eng/copyright
Vous avez des questions? Nous pouvons vous aider. Pour communiquer directement avec un auteur, consultez la
première page de la revue dans laquelle son article a été publié afin de trouver ses coordonnées. Si vous n’arrivez pas à les repérer, communiquez avec nous à [email protected].
Questions? Contact the NRC Publications Archive team at
[email protected]. If you wish to email the authors directly, please see the first page of the publication for their contact information.
NRC Publications Archive
Archives des publications du CNRC
For the publisher’s version, please access the DOI link below./ Pour consulter la version de l’éditeur, utilisez le lien DOI ci-dessous.
https://doi.org/10.4224/20375159
Access and use of this website and the material on it are subject to the Terms and Conditions set forth at
Benchmark procedure to evaluate computer-aided design and drafting
systems for building industry applications
Vanier, D. J.
https://publications-cnrc.canada.ca/fra/droits
L’accès à ce site Web et l’utilisation de son contenu sont assujettis aux conditions présentées dans le site LISEZ CES CONDITIONS ATTENTIVEMENT AVANT D’UTILISER CE SITE WEB.
NRC Publications Record / Notice d'Archives des publications de CNRC:
https://nrc-publications.canada.ca/eng/view/object/?id=73153c54-5015-4349-bd5d-9e27cb14efee https://publications-cnrc.canada.ca/fra/voir/objet/?id=73153c54-5015-4349-bd5d-9e27cb14efee
BENCHMARK PROCEDURE T O EVALUATE COMPUTER-AIDED DESIGN AND DRAFTING SYSTEMS
FOR BUILDING INDUSTRY APPLICATIONS
by
D.
J . VanierBuilding Services Section Division of Building Research
ISSN 0701-5216
Ottawa, September 1985
4 TABLE OF CONTENTS
. . .
ABSTRACT:'RESUME 1. . .
INTRODUCTIOK 2. . .
CADD BENCHM.4RK PROCEDURE 3
Initial Study
. . .
3 Shortlist Selection. . .
4. . .
Benchmark Operation 5. . .
Post Benchmark 6. . .
INVITATION T O BENCHMARK 7. . .
BENCHMARK CONFIGURATION CERTIFICATE 8
BENCHMARK FOR . COMP~JTER-AIDED DESIGN AND DRAFTING
. . .
9. . .
Terminology 9. . .
General Rules 9. . .
CADD Operations 10. . .
( I ) Introduction to System 10. . .
(2) Cold Start 11. . .
(3) Dimensioning 11. . .
(4) File St. oring 11. . .
(5) Plotting 12. . .
(6) File Recall 12. . .
(7) Zoom and Scaling 13
. . .
(8) Module Creation 13 (9) Module Modification. . .
14. . .
(10) Module Save 14. . .
(11) Quick Hard Copy I 6
. . .
(12) Module Manipulation 16 (13) Module Placement. . .
17 (14) Bills of MaterialIQuantities. . .
17. . .
(15) Plot Test 18. . .
(16) Error Correction 18. . .
(17) Area Calculation 18 (18) Additional Features .:
. . .
19 GUIDELINES F O R EVL4LUATION. . .
20. . .
ABSTRACT
A myriad of Computer-Aided Design and Drafting (CADD) programs claim to address architecture, engirleering, and constructios (A/E/C) requirements. Individual firms have great difficulty in evaluating and selecting a drafting system that will increase the pro- ductivity and effectivity of their organization. A formalized Benchmark procedure has been developed to assist purchasers in the evaluation of CADD systems. The procedure is thorough and time-consuming and should be used only at the final stage of the selection of a -CADD system. The CADD Benchmark is designed to test most drafting features, as well as a number of more complex CADD operations, such as: area calculation, bills of material, a.nd data base manipulation. It is directed at facilities management or space planning, but the software features tested cover a wide variety of drafting req~irernents for most building disciplines: plotting, zooming, dimensioning, scaling, rotation, filing, and symbol creation.
Un tr&s grand nombre de programmes de conception et de dessin assistks par ordinateur (CDAO) visent a rkpondre aux besoins des ktudes d'architecture, de gdnie, et de construc- tion (A,G,C). Les entreprises ont beaucoup de difficult6 B 6valuer et B. choisir des logiciels
de dessin qui augmenteraie~lt leur productivitd et leur efficacitk. Une mkthode d7Cvaluat,ion (Benchmark) a dtd mise au point pour aider les dventuels acheteurs de ces programmes. Cette mkthode est longue et ne doit Ctre employe'e qu'au stade final de sklection des pro- grammes. Elle vise a 6valuer la plupart des caractkristiques de dessin industriel, ainsi que d'autres opdrations plus complexes comme : le calcul des surfaces, les listes de matkriel, l'utilisation des bases de donnkes. Cette mkthode est concue principalement pour la gestion des installations ou la planification des locaux, mais les caractdristiques des logiciels mises l'essai portent sur de nombreux aspects du dessin industriel dans les divers secteurs de la construction : tracks graphiques, agrandissements, cotations, mise B l'Cchelle, rotation,
INTRODUCTION
Owing to the rapid introduction of a myriad of Computer-Aided Design and Drafting ( C ADD) programs, all claiming to address architecture, engineering, and construction (A/E/C) requirements, and the industry's unfamiliarity with these "hi-tech" tools, indi- vidual firms have great difficulty in evaluating and selecting a drafting system that will increase the productivity and effectivity of their organization. Present selection methods consist of viewing a number of vendor demonstrations, obtaining information from col- leagues and CADD consuitants, and evaluating a shortlist of firms on input of a specific building drawing. Unfortunately, this sequence does not normally produce the best results, as information from vendors, colleagues and consultants can be biased and the building drawing entry and evaluation are not carried out under controlled conditions. As a result, the selection process can be extremely subjective in nature, relying on moods of the day,
f
ashy presentations and user frustration.111 order to alleviate the deficiencies of the above method, a formalized procedure has been
developed to assist purchasers in the evaluation of CADD systems. This procedure was first used to assist the Canadian Federal Government in selecting potential contractors for a subst ant ial program for computer-assisted facilities management. The package is reproduced in full in this document.
The Interdepartmental Task Force on Computer-Aided Facilities Management required a method to evaluate potential contractors 011 their existing software and hardware products
as well as their capability in software development. It was decided that a Benchmark procedure was an efficient, thorough, and equitable method for the evaluation of computer systems. As a result, the Benchma.rk evaluation and results formed an integral part of the review process and were instrumental in the selection of the successful contractors.
A Benchmark procedure is used in the computer industry to evaluate the performance of computer hardware or software. Typically, a Benchmark defines a series of arithmetic operations run under strict conditions to evaluate the performance of specific hardware or software against a standard. An application-based Benchmark is more subjective in nature but it will also produce quantitative results that can be compared by the observer.
The Benchmark procedure is thorough and time-consuming and should be used only a t the final stage of the selection of a CADD system. It is not intended as a learning process for Computer-Aided Design Drafting, but as a rigorous evaluation method for the final selection of an integral part of a professional office. It is therefore essential that the purchasing firm be conversant with CADD terminology a.nd equipment, have defined the company CADD prerequisites, be aware of most commercial products available, and be fully committed to the concept of automated drafting.
The CADD Benchmark is designed to test most drafting features, as well as a number of more complex CADD operations, such as: area calculation, bills of material, and data base manipulation. The Benchmark is directed at facilities management or space planning, but
4
the software features tested cover a wide variety of drafting requirements for most building disciplines: plotting, zooming, dimensionifig, scaling, rotation, filing, and symbol creation. It is designed to be administered by the architectjengineer, but all of the keyboard and graphic entry is made by the hardware/softwarevendor a s per instructions in this document with guidance from the purchaser.
CADDBENCHMARKPROCEDURE
This document is divided into four sections: each an integral part of the Benchmark pro- cedure. The procedure has been used on a num'ber of occasions with good success, so it is highly recommended to foliow it. in detail. The amount of work required to plan and implement a Benchmark test is considerable, so it is therefore important that procedures are followed carefully and all potential suppliers are treated equitably. The flow chart below gives an overall view of the stages required for the proper selection of a CADD system. It is not intended as a definite sequence but a suggested schedule. The total time is dependent on the purchasing firm and a number of activities can be expedited; however, they are all necessary to ensure that the product purchased satisfies the requirements of the design firm.
Initial Study
a
Shortlist Selectiona
Benchmark Operations to Benchmark Post-Benchmark Stage Initial StudyThis phase of the CADD investigation should include the following steps: literature search, conference attendance, CADD consultation, and site visits. These will provide fundamental information for the individuals involved and will ensure that proper investigation of all potential suppliers is completed.
Literature Search This step should include investigation of professional literature for details on CADD and computer systems. Most journals have CADD surveys for their
d
specific discipline and have outlined CADD terininology and equipment. Architecture and engineering journals will have these available as past. issues and most professional libraries will have access to these articles. Addresses and suppliers' cards are provided to obtain brochure information from vendors. There are a number of textbooks on the subject and libraries will be able to supply books on the appropriate disciplines.
Conference Attendance Potential purchasers should attend a t least one national calibre corifeibence on CADD. The conferencc selected should be directed strictly a t their profession and should be of a general nature with the possibility of tutorials on a number of basic CADD topics.
CADD Consultation Every discipline tries to instill the same philosophy in its clients,
". .
.seek ,professsional advice". The same holds true with this new technology; one or twodays' consultation will save considerable research and investigation.
Site Visits Visits to professional offices employing CADD technology will assist the pur- chaser in evaluating the applicability of automation in his office. If possible the companies visited and the equipment viewed should resemble the purchaser's situation.
Shortlist Selection
In this phase the number of potential suppliers is reduced drastically. In following the process below it will be possible to identify the vendors meeting the company prerequisites.
Establish Prerequisites Any company investigating CADD must be aware of their re- quirements for automation. Essential in the establishing of prerequisites is a familiarity with existing commercial products. The prerequisites must be based on the firm's organi- zation, methods of work, personnel, and funds available. There is a Catch-22; how does one know the funds available, if one cannot evaluate the increased performance?
Request Quotations Having formulated realistic prerequisites, having identified poten- tial suppliers, and having viewed a number of CADD work sites, it is now possible to request quotations from a number of vendors. It is essential to identify the exact require- ments so as to solicit the best possible quotations. Consider this in the same light as your building quotations; a proposal with poorly defined scope of work will produce widely dispersed estimates from a number of contractors.
Equipment Demonstrations After all of the quotations have been received it is time to view the systems in a formal demonstration. The "demos" should be reduced to a manageable number and need not be attended by all members of the evaluation team. There should be time allotted in the demonstration to address the particular need of the purchasing company. It is also advisable to prevent the vendor from presenting a "canned demo" at this stage, as the purchaser should be well aware of his requirements and these presentations only deal with the advantages of CADD or a particular system and not with the disadvantages.
Select Shortlist Firms Once these four steps have been completed it is possible to identify suppliers that meet the financia! and technical requirements of the purchasing firm.
Benchmark Ogerat ion
Invitation to Benchmark The Benchmark base plan provided should be hand-delivered t c all shortlist firms one week prior to the a.ctua1 t,est date. A sample copy of an invitation has been included to ensure that the vendor is aware of the level of preparation required, the extent of testing, the degree of rigor nf the process, the time involved, and the equipment needed. Only the building base plan should be included with the invitation; the Benchmark is only to be distributed on the morning of the test. As more vendors become aware of the
CADD Benchmark, many will have already entered the base data.
Preparation by Vendor The base plan information should be entered by the vendor on the system proposed in the quotation. Entry of this building plan shou!d ~ e q u i r e six to eight hours depending on the experience of the operator, the flexibility of the CADD
package, and the complexity of the associated data base. Sufficient information is included in the invitation to Benchmark to allow the vendor to concentrate on the important details of the testing procedure.
Benchmark The location at which the Benchmark procedure is performed is left to the vendor, to ensure that proper equipment is available. The Benchmark procedure is de- signed to take approximately eight hours. It is essential that the individual directing the procedure keep on track with respect to time. A timing schedule has been included. This will assist all concerned to maintain the Benchma.rk test on cohrse and on time.
Only the Benchmark itself should be presented to the vendor on the day of the evaluation. The priorities for each Benchmark step should be entered by the Benchmark director prior to distribution of copies to the vendor and evaluators. There should be sufficient copies for all Benchmark evaluators and two additional copies for the vendor's staff.
Guidelines have been provided to explain the steps and to provide the Benchmark staff with information as to what should be observed and why a particular feature is being tested.
A Benchmark Configuration Certificate, such as the example provided, should be corn- pleted and signed by both parties on the day of the Benchmark procedure. This certificate ensures that the software and hardware, when delivered, will perform a t least as well as during the Benchmark test. It also guarantees that the equipment used during the per- formance of the Benchmark procedure is that quoted by the vendor and does not include any additional equipment or software, not included in the quotation, that would affect the results. Many items in the CADD business, such as cables, software modules, and support services, may be considered as extras by one company and standard features for
d
another. It is therefore recommended tha.t the purchasing firm seek legal advice regarding
the formation of their Configuration Certifica.te. h
i
Benchmark Summary The detailed Benchmark form should be completed by all mem- bers of the Benchma.rk staff during the test. There is sufficient space on the form for notes and comments a i d the staff should be encouraged to enter as many comments as possi- ble. These will be extremely useful because, after viewing a number of similar systems, details are forgobten. The completed forms should be collected a t the end of the day and summarized as soon as possible. This summary should form the basis for the creation of a priority list for negotiation with t,he vendors.
Post Benchmark
Re-evaluate Prerequisites Experience with the facilities management project has indi- cated that the results of the Benchmark test will affect the original prerequisites. Having tested a number of systems, the purchasing firm is now in a better position to re-establish their requirements.
Negotiate with Vendors The purchaser may wish to negotiate with the vendors to extract the best product for the firm or may wish to deal with one vendor to obtain the optimum combination of price and product. The purchaser must keep in mind that the Benchmark results are leverage in the negotiation phase. For example, if the machine's performance was slow, but faultless, perhaps the vendor could sweeten the deal with a small hardware add-on that would decrease the response time.
Evaluate Proposals It is now possible for the purchasing firm to evaluate all the proposals on their own merits.
Purchase The final stage is self-explanatory. The Benchmark Configuration Certificate should be employed to ensure that the product delivered meets the performance standards established in the Benchmark. The certificate can also be used as evidence in any legal action.
d
SAMPLE ONLY: Invitation to Benchmark
Company XYZ Ltd. Address
City
Subject: Supply and Delivery of CADD Equipment and Software for ABC Archi- tects/Engineers Ltd.
Your company is on the short list of bidders to be evaluated. The purpose of the Benchmark procedure is to eva!uate the proposed application package for typica.1 tasks carried out by ABC Architec+,s/Engineers. The procedure will be used to test the flexibility and adaptability of your proposed software for design drafting requirements. The 'Benchmark procedure will also test error handling, program documentation, and ease of use.
Your company must satisfy the require~nerlts of the Benchmark Cv~~figuration Certificate attached. The enclosed base plan is to be entered on the proposed system prior to the testing date. The vendor must provide a multi-colour plot of the base plan to the evaluation team at the start of the Benchmark procedure.
The Benchmark consists of 18 CADD operations to be followed rigidly, broken into a number of minor tasks. These will be provided to the potential contractors at the start of the Benchmark period. Only the vendor operator will be permitted to enter data on the standalone unit. However, one person may assist during the test period.
You are to carry out the Benchmark on the equipmefit proposed in your quota- tion. This will include the plotter and the hard copy device specified in your proposal.
The Benchmark procedure will be approximately eight hours in length and the estimated time to enter the base plan is six hours. There will also be a 30 minute time slot available at the end of the procedure for demonstration of additional features of the pro- posed system. The evaluation of your equipment will take place on Date- at time a t your premises at Location
.
Yours truly,
A.N. Architect, P.Eng. Senior Partner
ABC Architects/Engineers Ltd. Encl.
BENCHMARK CONFI&RATION CERTIFICATE - HARDWARE AND SOFTWARE
The certificate is to be signed by the vendor and submitted to the Benchmark director prior to the commencement of formal Benchmark testing.
The purpose of the certificate is to ensure the reliability of the Benchmark results. This certificate should be signed by all suppliers involved, identifying their area of responsibility.
This certifica.te should itemize a!! software and hardware, as well as unit cost for all components.
The Configuration Certificate should contain statements that would:
Certify that the system configuration to be tested is equal to that proposed aild that substitutions of hardware or software have been identified.
Identify any features, software or equipment installed, incorporated in, or interconnected to the system configuration proposed whose performance could cause the system used for the evaluation to exceed that of the vendor's proposed system, whether such fea- tures, software or equipment will be utilized during the Benchmark test or not. (This information should include specific details concerning how such features, software or
equipment will be excluded or appropriately degraded to more accurately emulate the proposed system's performance.)
Guarantee that a t any time after the Benchmark test, if it can be shown that the results of the Benchmark test were improved or enhanced by features, equipment or software not included in the bidder's proposal, or by any other means, the vendor shall, at no additional cost to the purchaser, provide such equipment, software or features such that each contracted system will perform t o meet or exceed the performance level achieved during the Benchmark test. It should also be noted that, in such an event, the Benchmark test shall be re-run for measurement purposes.
Clarify potential conflicts concerning differences in performance between the system (equipment, software, features) proposed/delivered and that tested. In this case, the certificate should read that a disinterested, technically competent, third party will be retained to provide a technical analysis of the situation. The certificate should mention that, if the purchaser's claim in such an instance be supported, then the vendor will be liable for the purchaser's related costs.
BENCHMARK F O COMPUTER-AIDED DESIGN AND DRAFTING ~
Date Cornpany Name
Evaluator Operat,ors
Terminology
"Vendor" refers to the prime company participating in the Benchmark procedure. "Bench- mark Staff7' refers to the individuals working on behalf of the purchasing firm evaluating the performance of the vendor for the duration of the procedure. The individuals involved should, whenever possible, remain the same for all Benchmark tests. The Benchmark staff will be identified to the vendor prior t,o the start of testing. "Benchmark Director" refers to the purchaser's representative assigned the direction of the Benchmark process. "Op- erator" refers to the individual assigned by the vendor to carry out the Benchmark test. "Assistant" refers to the individual assigned by the vendor to assist the operator. It is the responsibility of the assistant to pre-plan work, seek documentation, and investigate prob- lem areas. "CADD Operation" is a discrete step in the Benchmark procedure consisting of a number of minor "Tasks".
"Operation Priority" denotes the importance the purchaser has placed on a specific CADD operation. "Allotted Time" is the time that should be allotted to complete a CADD oper- ation in order to stay on schedule. "Estimated Time" provides an estimate of "Operation Time" based on a number of Benchmark tests from micro- and mini-CADD systems. "Op- eration Time" is the time recorded by the purchaser for completion of a specific operation or task.
General Rules
The Benchmark procedure should take approximately eight hours and all the information will be entered by the vendor's staff. Only the operator will be permitted to enter infor- mation on the system. At any time the operator or assistant may be changed on request by the vendor.
All questions to and from the vendor and the Benchmark staff will be made through the operator and the director, respectively. The director will provide additional information, if requested, and guidance if needed.
The "Operation Priority" should be given by the Benchmark director prior to the com- mencement of each task. Wherever possible "Allotted Times", the time to complete the entire CADD operation, and the "Estimated Time", the time for the computer to complete a discrete operation, are provided. The Guidelines for Evaluation provide good ranges of values for times and disk storage space.
Times required to accomplish certain CADD operations should be recorded, however, the operator is not under any pressure to carry out the Benchmark test in an allotted time.
For the sake of fairness to a n vendors, unnecessary repetition of tasks or time-consuming preparation (at the workstation) will not be permit,ted.
CADD Operations
(1) Introduction to System
Operation Priority (7-High, 0-Not required): Allot,ted Time: 15 min
The vendor is requested to indicat,e to the Benchmark staff the equipment and software employed for the Benchmark procedure. This shouid include memory size, disk space, and any other information that would influence the performance of the system during the procedure. The vendor should identify to the purchaser the components that are not listed as part or' the system in their quotation. The Benchmark Configuration Certificate should be completed by the vendor prior to the completion of the test.
A test of the error handling capabilities of the system and the software will be carried out in Step 16 of this Benchmark. However, if at any time the operator encounters a system, input, or program error, the operator will be requested to solve the problem using only available documentation. This will be carried out a t every occurence of an error message or until the Benchmark director cancels this requirement.
System as Tested
Computer Cache M e m o r y k B
Memory Size . - k B Operating System
Disk Storage
-
.
d
B
VersionD i s k e t t e k B Software _Version
Floating Point Hardware Y/N- Version
Version Version Screen Resolution
H
L
V
Colour D i s p l a y D i s p l a y a b l e V i e w a b l e Plotter Hard Copy Price as tested $-Estimated Time: N / A min. s e c . Operation Time: N / A min. sec. NOTES i ! i
i
id
(2) Cold Start
Operation Priority (7-High, 0-Not required): Allotted Time: 15 min
Start the computer (boot) as at the beginning of a session and recall the base plan from disk. The operator or assistant must itemize all operations that are being carried out
I
including program calls, disk partitioning, program loading, and file loading. The timeI
required to bring the entire drawing to the screen will be recorded. The complexity of the vendor's drawing as compared to the Benchmark drawing will also be evaluated.Estimated Time: 5 min. R e c .
Operation Time: -.set.
Corilplexity [Circle): Extreme Very Moderate Comparable Simple
, NOTES
,
1 i
(3) Dimensioning
Operation Priority (7-High, 0-Not required): Allotted Time: 15 to 30-mA
Change all of the dimensions to metric and identify to the Benchmark staff the exact operations as they take place. This will include all steps required to carry out the total operation of converting all the dimensioning information.
T h e
time taken to perform thisfunction will be recorded, but the facility of the operation is more important. Estimated Time: 30 min. L e c .
Operation Time: m i n . n e c . Number of Dimensions:
Method (Circle): Automatic Semi-automatic Manual NOTES
(4) File Storing
Operation Priority (7-High, 0-Not required) :
d
Save this information and the same information for an additional ten (10) floors on disk storage, changing the floor number in the border of each sheet. The time required to store all the information will be recorded, as well as the space requirement of the drawings. Estimated Time: 3 min. s e c . / f l o o r
Operation Time: m i n . s e c .
Space requirements for ten (10) drawings k B
i
NOTES
1
(5) Plotting
Operation Priority (7-High, 0-Not required): Allotted Time: 15 min
Plot the tenth floor of the metric drawing using different pen colours for the different components. Limit the information to be displayed to that which will be legible on an A1 drawing.
Estimated Time: 10 min. sec.
Operation Time: m i 1 1 . sec.
Complexity (Circle): Extreme Very Moderate Comparable Simple NOTES
COFFEE BREAK
(6) File Recall
Operation Priority (7-High, 0-Not required): Allotted Time: 5 min
Recall the original imperial drawing. The time required to carry out this step will be recorded.
Estimated Time: 2 min. s e c .
NOTES
(7j Z o ~ m and Scaling
Operation Priority (7-High, 0-Not required): Allotted Time: 5 min
Dispiay the base plan, column plan, column grid, and building core of an area around three colu~rlns including a portion of the central core and the entire area encompassed by the coiumns. The time required will be recorded.
Estimated Time: m i n . 5 sec. Operatkn Time: m i n . s e c .
Complexity (Circle) : Extreme Very Moderate Comparable Simple NOTES
(8) Module Creation
Operation Priority (7-High, 0-Not required): Allotted Time: 60 min
In this step the flexibility of the drafting package will be tested. The capabilities of the operator at drafting will be a major factor in this step. The operator is asked to create a module consisting of standard furniture, similiar to the module attached. These components are to be placed on a furniture layer or appropriate location to facilitate easy recall and to allow mirroring, copying, and rotating of the final module. Colours for different components would be an asset. The amount of "useable work space" must be tagged to the module. The module should incorporate roughly 10 m2. The furniture will include:
a. Table 1.0 by 1.5 m, origin a t centre.
b. Credenza 0.6 by 1.5 m, origin at bottom left.
c. Chair, semicircle 0.3 m with 100 mm extension a t front.
d. Partitions, 80 mm thick, 1.2 m modules, 80 mm diameter circular connector posts. (There must be a post at both ends of every partition and screen).
f. Circular screen, 2.0 &long along centreline of curve, 3.0 m radius, 80 mm thick, with two 80 rnm connector posts.
g. Plant octagon, rotation of branches around centre.
h. Shelf, 0.35 by 1.2 m.
Include these components in a workstation module of approximately 10 m2, as per the attached drawing. Store finished modules as "TEMPMOD".
Estimated Time:
N/A
min. s e c . Operation Time: m i n . s e c . NOTES(9) Module Modification
Operation Priority (7-High, 0-Not required): Allotted Time: 30 min
Carry out the following modifications.
a. Change connector posts to 60 mm diameter.
b. Zoom in on a connector post so post fills entire viewing screen.
c. Move plant away from screen.
d. Move the desk and credenza 50 m m from the partitions. e. Add text to the drawing.
Estimated Time: N/A min. s e c . Operation Time: d i n . L e c . NOTES
(10) Module Save
Operation Priority (7-High, 0-Not required): Allotted Time: 5 min
Save all items in this module as a unit t o be recalled by name "STETCP100". The time required to save the module, as well as the disk storage, will be recorded.
d
Estimated Time: 2 min. s e c .
Operation Time: d i n . -set.
NOTES
TABLE
CHAIR
d
(11) Quick Hard Copy
Operation Priority (7-High, 0-Not required): Allotted Time: 10 min
Provide a hard copy of the module done to scale 1:20. The time required from s t a r t t o finish wi!l be recorded. The vendor may produce the output by hardccopy unit, plotter, or both. You may carry out this operation up to a maximum of four times; the best time will be recorded.
Accuracy (Circle): Excellent VG Good Acceptable Unsatisfactory Quality (Circle) : Excellent VG Good Acceptable Unsatisfactory
Estimated Time: 3 min. sec.
Operation Time: m i n . s e c . NOTES
(12) Module Manipulation
Operation Priority (7-High, 0-Not required) :
Allotted Time: 60 min
In this task the operator will be asked to create a cluster of four workstations. This will be repeated throughout the entire base plan. Each will be centred on a column and will consist of four versions of the original workstation. This wi!! r,ct oilly be a test of the operator's understanding of the software and the software's drafting capabilities, but will test the operator's capability to carry out planning operations and object manipulations. The operator will be asked in Operation 13 to optimize the space available while provid- ing accessibility to all of the workstations. Minimum widths for circulation lanes should be 1.0 m and major arterial routes should be 2.0 m in width. Carry out the following manipulation of graphical information.
a. Place the original module a t the bottom left of the column.
b. Mirror the module around a horizontal line above the existing module.
c. Mirror the module around a vertical axis to the right of the first module. Erase the plant in the new module.
d. Rotate the original module 180 degrees around the centre of the column. Erase the trash can in the new module.
e. Store the four unit cluster under the name "B4STEMODV. Disk Space
k
B
Estimated Time:
N/A
min. sec. Operation Time: m i n . sec.NOTES
PROPOSED LUNCH BREAK
(13) Module Placement.
Operation Priorit,y (7-High, &Not required): Allotted Time: 60 min
Create a rrlatrix of the clusters "B4STEMOD" throughout the floor plan, adding additional units where possible. The operator will be asked t o optimize the space available while providing accessibility t o all of the workstations. Minimum widths for circulation lanes should be 1.0 m and major arterial routes should be 2.0 m in width. Since the plan is baaed on a 30 foot (9.144 m ) grid, this should be possible with relative ease. Do not remove extra or duplicate partitions.
Estimated Time:
N/A
min. s e c . Operation Time: m i n . s e c . NOTES(14) Bills of Material/Quantities
Operation Priority (7-High, 0-Not required): Allotted Time: 20 min
Create a report of the existing number of furniture units on the floor plan, i.e., the number of desks, credenzas, shelves, partitions, posts, plants, chairs, and trash receptacles.
a. Print this as a furniture schedule on the drawing.
b. Delete a number of plants, recalculate and update the schedule.
Estimated Time: 5 min. sec.
Operation Time: d i n . sec.
d (15) Plot Test
Operation Priority (7-High, 0-Not required): Allotted Time: 20 min
Plot an A 1 drawing of onc floor in colour using different colours for base plan, furniture, and partit,ior,s. Use a red pen for the partitions and posts.
a. R.emove the drawing from the plotter a,nd then replace the drawing to replot the pa.rtitions and furniture in green.
Estimated Time: 1 .min. sec.
Operation Time: m i n . - - s e c . NOTES
(16) Error Correction
Operation Priority (7-High, 0-Not Required) :
Allotted Time: 10 min
If no error has occurred up to this time, try to initiate an error using a number of control commands. Try to recover from the error condition without losing information already entered.
Estimated Time:
N/A
min. s e c . Operation Time: m i n . s e c . NOTES(17) Area Calculation
Operation Priority (7-High, 0-Not required): Allotted Time: 30 min
4
Calculate the following: a. Total gross area, b. Service areas,
c. Rentable area, d. Accesory area, e. Usable areas.
Estimated Time: rnin. s e c . Operation Time: m i n . L e c . NOTES
(18) Additional Features
Operation Priority (7-High, 0-Not required): Allotted Time: 30 min
You will have 30 minutes to demonstrate additional features of the system that would be considered assets to the purchaser of your product.
Estimated Time:
N/A
min. sec. Operation Time: m i n . sec. NOTESGeneral Instruct ions
Each of the 18 CADD Operations contains a number of common elements.
Task Priority
This should be completed by the Benchmark staff prior to the evaluation date. The Benchmark f o r ~ n should bc photocopied when every task has been prioritized and then distributed to t,he v-endor on the day of his Benchmark test. This priority indicates to the vendor the importance of each task.
It is not essential for all companies to carry out all aspects of the Benchmark and a "NOT POSSIBLE" entry would only exclude a contender if the purchasing firm has deemed that task to be a prerequisite. The quality of the results is a, function of the rigor of the application of the Benchmark. It is essential that the Benchmark director keep the testing on course, restricting the vendor from providing a "canned demo".
Allotted TimeIEstimated Time
The "Allotted Time" provides the Benchmark director with an indication of the time to be spent on that specific operation and remain on schedule for the day. The "Estimated Time" provides a "ballpark" time for the operation based on information from previous tests. The timings are important as they provide an absolute comparison scale between the various systems being evaluated. Care should be taken when recording timcs of less than 30 seconds. The timings should also include keyboard input, as this will be required in day-to-day operations. It is not necessary to record to the nearest second. However, experience has shown that recordings should be consistant throughout the procedure and between different vendors. The most equitable method is to start timing a t the point where the machine has received a "Carriage Return" or "Enter" and to proceed until the screen cursor appears signifying the end of the operation.
Operation Time
The Benchmark staff must enter the appropriate time. Normally one individual should be assigned to the timing operation; a stopwatch or digital watch with chronometer is suggested.
Evaluation
Some tasks require a multiple choice evaluation. The appropriate answer must be circled. A
4
is not penalized for providing more dat,a, attributes, or detail. These will be subjective evaluations used in conjunction with the opera.tion time.
Notes
Space has been left on the Benchmark to permit the evaluator to enter comments (i.e., sys- tem slow, software difficult to handle, poor user interface, etc.). After a number of visits, the mind becoines foggy; the notes refresh the memory and assist the Benchmark director to summarize the results.
CADD Operations
(1) Introduction to System
This step ensures that the purchasing agent is familiar with the equipment being evaluated. The'"error handling ca.pabilities" portion evaluates the completeness of the documentation and the familiarity of the operator with the hardware and software. This is important as
the purchasing firm will be entering a long-term contractural agreement with the supplier and the supplier must be fully aware of the purchaser's need for hardware and software support.
(2) Cold Start
The reason for the cold start is to provide the purchasing firm with the actual time for start-up. It is recognized that most mini-computers are "powered-up" for 24 hours a day, but a t times this is not possible. This task also tests reliability and stability of the system, as nothing can be tampered with before the Benchmark Eatc and the Benchmark staff can observe which equipment and program modules are being accessed. The complexity of the vendor's drawing is dependant on associated attributes, structure, or databases. Operation times can be as low as one minute or as high as ten minutes.
(3) Dimensioning
In most design projects there is a requirement to take existing plans, normally in imperial units, and translate the dimensions to metric notation. Some CADD systems carry this out automatically, others store'and display both units, some require some manual intervention, and others require re-input of the numerical text and dimensioning lines completely. (4) File Storing
This may seem redundant, but as each floor file is approximately 100 kilobytes, ten floors equal one megabyte. A large number of systems encounter difficulties when dealing with this amount of filing storage. This is also an operation that may be very time-consuming, thereby decreasing the productivity of designers or draftsmen. For each floor, filing should be between two and five minutes. There normally are different ways of saving drawings: in
rl
the CADD program, in the operating system, or in a "copy" utility. The quickest, safest method should be recorded.
(5) Plotting
The plotting allows the purchasing firm to evaluate the steps required to output, a single sheet a t a specific scale. The tenth floor of the metric drawing should be plotted using diffcrc~t colours for various layers. The plot should contain a t least as much information as the Benchmark plan. Setup time for the plotter should be minimal. The time required for the formatting of the plot file should be included in the total operation time.
(6) File Recaii
This step tests the speed of the software and hardware to recall a drawing that could already be in memory from earlier use, and to disp1a.y it. Typically in automated drafting an operator may move from one drawing to another, thereby requiring quick file recall and display. File recall can range from seconds to minutes.
(7) Zoom and Scaling
Zooms are a time-consuming, essential part of CADD drafting. Although more sophisti- cated packages and higher resolution screens allow less zooming and panning, zooming is still often used. On more advanced systems special hardware in the terminal will handle this instantaneously. This task will allow the operator to set up the proper view, remove unnecessary layers, and zoom in on one column bay. Times should vary between two sec- onds and ten seconds, depending on the amount of information displayed and which layers are "turned off". The complexity of the displayed image should be recorded.
(8) Module Creation
This task checks for two CADD features: creation of symbols and symbol manipulation. An important A/E/C operation is to draw an object that will be used throughout a drawing or series of dra.wings. Examples include standard doors, washroom fixtures, and window details. The task also tests the ease of use of the drafting package in creating rectangles, arcs, polygons, and curves. The drafting function takes up the majority of the time on a CADD system; it is therefore important to expedite these tasks.
(9) Module Modification
This operation tests the flexibility and intelligence of the stored information. Flexibility implies how the data is structured; the better the structure, the easier it is to modify and copy. If the information is nothing but a series of lines, then the object will have to be recreated each time a modification is required. Intelligence implies additional information with the graphical data. An object can resemble a chair, but if the object also contains the manufacturer's stock number, unit cost, and colour, then the information is more intelligent. The changing of the post size will alter the spacing of the partitions, the
location of the center o h h e post, and the alignment of the partitions. Observe carefully the steps followed by the operator to modify the module. The reason for zooming in on one post is to test drafting software accuracy. Normally distances and lengths on CADD systems are stored as real-world co-ordinates, that is. if an object is 1 100 mm long it is saved as that length and not as 1:100 scale. Some micro-computer systems have limited scales for recording distances. In the building iridustry a length of 100 m or 300 feet is not uncommon for a large building on a site plan. Construction tolerances do not permit us to deal with anything less than one millimetre or one-sixteenth of an inch. So the required range for an A/E/C CADD system would be 1 mm in 100 m (1116 to 300 feet, or 1/16 inch to 3600 inches, or 1 to 57 600). If a system does not allow this range it will fail as a drafting tocl. Text oil a cell will present a problem later in the procedure as the text will be mirrored when a cell is mirrored.
(10) Module Save
The. module should typically occupy between one and five kilobytes on the disk. Storage time should be to the order of ten seconds. The disk storage requirement of one module is important because over 200 similar modules on one floor plan will require 200
x
5 kilobytes or one megabyte of disk space just for the furniture.(11) Quick Hard Copy
Hard copies of screen pictures are required for sketching purposes or preliminary discus- sions. These should be quick, accurate and inexpensive. Normally a dot-matrix printer hooked up to the screen is a cheap solution, but many scaling problems occur. An E-size plotter would be overkill. Timings should be between two and five minutes; accuracy and quality should also be rated.
(12) Module Manipulation
This task is completely subjective in evaluation. Its intent is to provide the purchasing firm with the opportunity to see the draftingldesign package in use. Normally designers will be using existing furniture modules and key plans when a firm is employing CADD technology. This task will allow the observer to view the manipulation of existing objects and workstations on the building base plan. It will test the mirroring and rotating capa- bilities for sets of elements. It will also allow the creation of non-typical modules. These are created by deleting elements within modules and duplicating the new version. Objects are removed from specific copies to test the "unwrapping" or "burst" capabilities, as well as the bills of material features (see Guideline Operation 14).
The four modules should be placed so as to allow for proper circulation between bays. The resulting four-module footprint will be used around each column, so the design of the four-module unit must allow for efficient use of space. The sequence of work is important. The original module must be located at the bottom-left of the column, as per the attached drawing. The easiest way to create the duplicates is to first rotate a copy module to the upper right position. It is evident now that all partitions are correctly in place. The original
module should be broken do&, that is: burst, with the furniture reformed into a new unit and the partitions deleted. This new unit can be rotated and shifted above the horizontal and vertical axes. As a result, all furniture is in place and there are no duplications or overlapping partitions. As mentioned before, a structure is important in any CADD
system. It must be possible t80 break down, "burst", or "unwrap" any component into sub-components and subsequently into further sub-components. This should be possible in the reverse order of the assembly process. A system that automatically "bursts" an object tc? the graphic elements such as arcs and lines possesses no structure and makes it difficult to alter existing objects.
The reason for delet,ing the furniture comporlents on a number of modules will become evident, when t.he counts of material are taken in Operation 14.
(13) Module Placement
This task evaiuates the facility for placing objects on the base plan. The four-module unit should be designed to allow easy placement on a 30-foot grid system. There are many solutions to this design problem but one is obvious and allows for close packing of the four-module units. Aisle spacing should be respected and the four-module unit may be rotated 90" to pack the modules. The Benchmark director should indicate the level of detailed placement of modules. The vendor should a t least place the cluster around every column, but should not finesse the design to the last possible workstation.
The purchasing firm's major application may not be facilities management, however, it is
'important to place as many modules as possible because some computer systems may fail with this volume of data. It is also important to view the facility of rotation and placement of large symbols or objects.
(14) Bills of Material/Quantities
This task tests the data handling or bills of material software. Normally a CADD package will count entities or cells to produce a listing of components. This is relatively easy t o add to any existing package as it only counts how many times a specific object is drawn. This type of software requires each component t o be individually drawn and named. In facilities management, tables, chairs and desks would be individual components and these could be lumped together t o form workstations-or modules. The software would then count the number of tables, etc, as well as the number of each type of workstation. The resulting figures can be printed or plotted as a schedule on the drawing. If a number of workstations are deleted, added, or modified, the entire schedule would have t o be recalculated and replotted. This type of software would not be intelligent enough to add to or subtract from the total. A database program would allow this feature. If tables were deleted or added, the schedule would be updated automatically. Few commercial systems offer this flexibility.
4
(15) Plot Test
This task tests the performance of the plotting rout,ines. It will also evaluate the intelligence of the software package to handle different object colours. The vendor should be asked to carry out the plot in specific colours selected by the Benchmark director. This will also test the layering capability of the software, as furniture and partitsions should be on different layers. The second part of the task tests the ability to orient new information on the exist.ing plans SO it, meets the approval of the Benchmark staff. This feature is
used when only a snlall amount of data is added to a plan and there is no need to redraft the entire floor plan. The plan should be removed from the plotter and re-inserted and repositioned. The p a r t i t i ~ n s aud furniture should be plotted in green to demonstrate the accuracy and repeatability of the software and plotker.
(16) Error Correction
This error check operation should be carried out using published material supplied. by the vendor. In normal CADD installations there are few. if any, software application experts If the software or hardware fails and a job is due the next day, there is little recourse for the user but to go to the manual. The literature should be totally comprehensive because your contact person may not be there when you need him at 3 o'clock Monday morning, before the big presentation. Proper, accurate, and comprehensive documentation is essential for all computer installations; there is no substitute. In this task the operator must direct the Benchmark staff through the available documentation to arrive at a solution to the error. The Benchmark staff should not accept the answer, "Phone HOTLINE", for this is an indication of poor documentation and potentially poor software development.
(17) Area Calculation
Area calculation is a simple task for the computer to handle. Writing the appropriate software is the difficult part. Most CADD systems will require digitization of corner points of closed areas to identify the boundaries. More sophisticated software may trace lines or even curves to establish the boundaries. The most sophisticated will request. an interior point in the area and the software will find a closed border with assistance from the operation. This operation will indicate to the purchaser the level of sophistication of the area calculation software.
CADD software with database capabilities allows the input of area information with the creation of entities or cells as with Operation 8. A table may be calculated as two square metres or a workstation as 10 m2; these attributes could be used in calculating the "usable area". This level of sophistication is normally expected by most purchasers, however, there are very few packages that presently offer this capability in computer-aided design drafting. (18) Additiona.1 Features
Thirty minutes should be allotted for the vendor to demonstrate features of his system that would be considered as assets to the purchaser.
