Mobile Manipulator Performance Measurement Towards Manufacturing Assembly Tasks

HAL Id: hal-01476525

https://hal.archives-ouvertes.fr/hal-01476525

Submitted on 24 Feb 2017

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Distributed under a Creative Commons Attribution| 4.0 International License

Mobile Manipulator Performance Measurement Towards Manufacturing Assembly Tasks

Roger Bostelman, Sebti Foufou, Steve Legowik, Tsai Hong

To cite this version:

Roger Bostelman, Sebti Foufou, Steve Legowik, Tsai Hong. Mobile Manipulator Performance Mea- surement Towards Manufacturing Assembly Tasks. 13th IFIP International Conference on Product Lifecycle Management (PLM), Jul 2016, Columbia, SC United States. pp.411-420, �10.1007/978-3- 319-54660-5_37�. �hal-01476525�

Mobile Manipulator Performance Measurement Towards Manufacturing Assembly Tasks

Roger Bostelman*^1,2, Sebti Foufou³, Steve Legowik⁴, Tsai Hong Hong¹

*Corresponding author: roger.bostelman@nist.gov

1National Institute of Standards and Technology, Gaithersburg, MD 20899, USA

2Le2i, University of Burgundy, BP 47870, 21078 Dijon, France

3CSE Dept., College of Engineering, PO. Box 2713, Qatar University, Doha, Qatar

4Robotic Research, LLC, Gaithersburg, MD 20878, USA

Abstract. Mobile manipulator performance measurement research is relatively minimal as compared to that of robot arms. Measurement methods, such as op- tical tracking systems, are useful for measuring the performance of mobile ma- nipulators, although at a much higher relative cost as compared to artifacts. The concept of using test artifacts demonstrates to potential manufacturers and users of mobile manipulator systems that relatively low cost performance measurement methods exist. This paper discusses the concept of reconfigurable mobile ma- nipulator artifacts that were designed and built. An artifact was then used through experimentation to measure the performance of a mobile manipulator to demon- strate the feasibility of the test method. Experimental results show a promising test method to measure the performance of mobile manipulators that are to be used for manufacturing assembly tasks, where at least the mobile manipulator tested has the capability to perform assembly to 1 mm positional accuracy or greater.

Keywords: mobile manipulator, performance measurement, ASTM F45, arti- facts, ground truth

1 Introduction

Mobile robots and mobile manipulators have been popular research topics [1, 2, 3, 4, 5]. But, in general mobile manipulators have been further investigated recently and are now becoming commercial tools for industrial use [6, 7, 8]. In research, consider- ations have focused on the coordination of movements of the robot and the base since redundant degrees-of-freedom (DoF) exist by adding the moving base. An example mobile manipulator consists of a six DoF robot arm (manipulator) mounted onboard a wheeled base (e.g., automatic guided vehicle (AGV) or mobile robot) with two trans- lational and one rotational DoF in the horizontal plane for a total of nine DoF. [9]. Some mobile manipulators have more or fewer DoF and may also be equipped with vertical axis motion control of the robot arm base.

As with robot arm or AGV performance, it is important for manufacturers and users of mobile manipulators to implement performance measurements to understand their

system capabilities for appropriate application. For example, a user may wish to apply a mobile manipulator to assemble an engine having relatively high tolerances and as- sociated costs as compared to inserting relatively low tolerance and cost rivets into sheet metal covers.

Measurements of the performances of mobile manipulators performing standard tasks (poses and motions) are non-existent except for simply ensuring that the task has been more or less completed. Robot performance measurements may include path comparison and path drawing, Cartesian and polar coordinate measuring, triangulation, optical tracking, inertial measuring, as well as the difference in position and orientation, or pose, of mainly the end of arm tooling from the commanded robot pose. Ground truth measurement using motion tracking systems of various techniques provides rela- tively accurate robot joint, segment, or tool point position information such that com- parisons can be made to the commanded pose. Summarizing review of robot, mobile robot, and mobile manipulator performance measurement research shows this as being relatively new to the research community [10].

A survey of research on performance measurement of mobile manipulators [10] was published by the National Institute of Standards and Technology (NIST) as basis for research in the Robotic Systems for Smart Manufacturing (RSSM) Program [11]. The Program develops and deploys advances in measurement science that enhance U. S.

innovation and industrial competitiveness by improving robotic system performance and other aspects to achieve dynamic production for assembly-centric manufacturing.

Recently, NIST has been measuring performance of mobile manipulators using both a motion tracking system and artifacts designed at NIST. The artifacts can provide an inexpensive, yet low uncertainty method for manufacturers and users to measure the performance of mobile manipulators.

Performance standards, such as ASTM F45 [12], can also benefit from the use of low cost, high accuracy artifacts to develop generic test methods so that manufacturers and users perform similar and comparative tests. Specifically, ASTM F45.02 Docking and Navigation subcommittee is developing work item WK50379 [13] on docking un- manned ground vehicles and their onboard equipment, such as manipulators. In this work, the use of artifacts is being considered as ground truth for mobile manipulator performance measurement.

This paper discusses the design and use of the NIST artifacts, called reconfigurable mobile manipulator artifacts (RMMAs), in measuring mobile manipulator perfor- mance. The concept of using artifacts and programmed algorithms to control the ma- nipulator are discussed. An experiment demonstrating feasibility and experimental re- sults is then discussed followed by conclusions that suggest follow-on measurements.

2 Performance Measurement using Artifacts

The concept includes positioning a mobile manipulator next to an artifact as well as positioning and orienting the end of arm tool (EOAT) attached to the manipulator at specific locations above an artifact to digitally detect fiducials with known uncertainty.

The performance evaluation criteria include the:

3

 Time to register the mobile manipulator to the artifact

 Time to move from the registration points to the assembly points

 Repeatability after registration

 Number of search steps equating to the initial distance from registra- tion/assembly points

 Detection of reflectors with varying diameters

Artifacts, called reconfigurable mobile manipulator artifacts (RMMAs), were de- signed at NIST to include square, circle, triangle, straight and curved lines, and sinus- oidal geometric patterns of tapped holes drilled into machined plates with tolerance of +/- 0.025 mm. The static and index artifact, RMMA-1, is shown in Figure 1(a) beside a mobile manipulator. A dynamic RMMA, RMMA-2, is shown in Figure 1(b). Static means a stopped vehicle with the only onboard manipulator moving to detect fiducials.

Index means the mobile vehicle moves from one static location to another where the manipulator cannot physically reach all patterns from one stopped location. Dynamic means both a continuously moving vehicle and onboard manipulator to detect fiducials.

Figure 1(c) shows a 457.2 mm square pattern of four reflector fiducials located at the square corners of RMMA-1 and Figure 1(d) shows a close-up of the reflector inside the square tube reflector housing (which also supports a collimator to be explained later).

Figure 1: Reconfigurable mobile manipulator artifacts (RMMAs) showing (a) static and index version (RMMA-1) and (b) dynamic version (RMMA-2). (c) Static and index version square pattern of reflectors and (d) close-up, top view of an illuminated reflector inside the square tube. The inset in (a) shows the retroreflective laser sensor used to detect reflectors.

Both RMMAs can be in a horizontal (as tested in this research), vertical, or other orientation, at short-to-tall heights, and even configured overhead as would be typical

of assembly in manufacturing facilities for an unlimited set of performance measure- ment possibilities. The EOAT was a retroreflective laser sensor (RLS) that emits light to a reflector and is detected by the RLS. A camera, with a light source, could instead be used as the detection sensor, especially with a larger diameter reflector or other spot.

For the RLS/reflector concept, no camera software algorithm was required as the RLS connected directly into one of the manipulator digital inputs. The reflectors can have specific diameters depending upon the required uncertainty for their location. The ma- nipulator¹ used has manufacturer’s specified repeatability of 0.1 mm [14] and the AGV navigation sensor has manufacturer’s specified resolution of 1 mm [15]. No infor- mation is provided by the AGV manufacturers to specify the vehicle performance such as position accuracy. As published in [16], 6.3 mm diameter reflectors were used to test mobile manipulator uncertainty as an initial concept feasibility test.

For our tests, detector-to-reflector distance parallel to the laser axis was approxi- mately 127 mm where the minimum and maximum detection distances are 100 mm and 10 m respectively. The distance would be representative of a programmed waypoint above and in-line with the next manipulator task point aligned to grip or insert a part or perform another task. The desired uncertainty may be, for example, a part insertion alignment tolerance required for a manufacturing assembly process. Moving along this grip/insertion line, parallel to the laser, at the aligned pose to the task point, also pro- vides some knowledge of insertion performance (i.e., if the task point is continuously detected along the grip/insertion line).

Each adapter, to be screwed into the various patterns of holes, supports a background target, a circular reflector, a square tube reflector housing, and a cylinder used as a light collimator. Circular collimators are inserted into three dimensional (3D) printed (blue) square tubes that house a micro reflector, a reflector cover with a specific diameter hole, through which exposes the reflector, on top of the reflector and the collimator on top of the cover. Flat background targets, measuring 7.6 cm diameter with 6.4 mm incremen- tal rings are perpendicular to each collimator and sometimes used as a simple visual cue for the test director when the manipulator does not align with the reflector. The adapter, reflector, housing, target, and collimator can be perpendicular to the flat sur- face or rotated to pitch angles between ± 90° and yaw angles between 0° and 360°. The reflector and the collimator inside can be any diameter, dependent upon the sensor spec- ification and the desired measurement uncertainty. Experiments for this paper utilized 1 mm through 6.3 mm diameter reflector cover holes where the 1 mm diameter hole was used for registering the manipulator to the reflector center.

Without the collimator, as shown in Figure 1(c), the reflector can be detected at ap- proximately ± 20° to the vertical axis. For the collimators used, the reflector can be detected at a maximum 3.2 mm radius from the reflector center. It can be detected at

1Disclaimer: Commercial equipment and materials are identified in order to adequately specify certain pro- cedures. In no case does such identification imply recommendation or endorsement by the National In- stitute of Standards and Technology, nor does it imply that the materials or equipment identified are nec- essarily the best available for the purpose.

5

approximately ± 7° to the vertical axis when using a 12.7 mm inside-diameter collima- tor. Collimators could be made with even smaller inside-diameters to force more per- pendicular manipulator tool point axis pose to the reflector.

The RLS, shown in Figure 1(a), shows the sensor mounted in-line and perpendicular to the manipulator tool point. Initial alignment to the reflector can occur using one of several methods briefed in [16]. For our experiments, we aligned the RLS using the manipulator jog mode from the teach pendant until the laser detected the two registra- tion reflectors for both the circle and square reflector patterns. Therefore, we could read directly from the teach pendant the end-effector coordinates to return to during our ex- periment.

An optical tracking measurement system was initially used as ground truth [17] for comparison to the use of artifacts and to measure all system components simultaneously for test method development. The ground truth system has static positional accuracy of 0.02 mm, however costs approximately 20 times the cost of the artifact concept. Figure 2 shows a snapshot of the optical tracking system markers used as fiducials and posi- tioned on the square pattern of collimators. The tracking system measured the perfor- mance of the vehicle, manip- ulator, and artifact within the same system of reference. In this paper, the focus is to dis- cuss the comparison be- tween only the manipulator and artifact as a low cost, rel- atively high accuracy meas- urement method. The poten- tial for artifacts being made using three dimensional (3D) printing could lower the cost further by an order of magnitude, or 200 times, as demonstrated through machining costs of the RMMAs and the 3D printing of parts used with the artifact.

3 Experiments

The experiment consisted of moving the AGV from a home position away from the RMMA-1. The AGV control program moved the AGV to the first location where its position and orientation or pose was pre-determined by the AGV control program.

AGV orientation angles were programmed to be at 45° with respect to the RMMA-1.

Upon completion of the pattern detection for a location, the AGV moved to the second location and pose, and so forth until six locations were completed. The AGV completed the test by returning to the home position.

A modified registration method for registering the mobile manipulator to the RMMA was recently developed that uses the components shown in Figure 3, including a 3D

Figure 2: RMMA with optical tracking system mark- ers on each collimator. Markers are also shown in Fig- ure 1(a) attached to the retroreflective laser sensor held by the manipulator.

printed aperture housing, collimator, micro-reflector, and a camera iris aperture. The aperture allows the opening to the reflector to not only be any size, but to also center the opening. This ensures that the two registration fiducials are centered on the reflector even when using a much larger reflector for all other fiducials. The smallest aperture opening used was 1 mm diameter while all other fiducials were 3.2 mm or 6.3 mm diameter. The RLS did not return a ‘detect’ at a smaller diameter than 1 mm diameter aperture setting.

A circular search pattern was used in previous tests [16] to register the manipulator to the first reflector. Once the first fiducial was acquired, it was possible for a registra- tion skew to occur as only the edge of the pattern may have been detected and the opposite side of the second fiducial used for registration could cause an incorrect per- formance measurement of the mobile manipulator. The circular search began with a step increment chosen to be approximately half of the diameter of the fiducial being tested. For example, an initial step size of 3.1 mm was chosen for a 6.3 mm fiducial to be detected. However, after the initial circle of steps was completed, the step moved to the next larger 3.1 mm circle step radius (e.g., from 3.1 mm to 6.2 mm radius from the start location) and at the same step arc angle (e.g., 15°) causing much larger steps to occur as the circle pattern grew. Instead, a square pattern was tested for the research described in this paper that kept the same step size throughout the entire search. An example of the square step pattern is shown in Figure 4where the search begins away from the reflector at the chosen start point (yellow arrow dotted end) and each step moves along the small white or gray arrows until the RLS detects the reflector with the red arrow step.

Figure 3:(a) Components used for registration and (b) registration fiducial mechanism attached to the RMMA.

The Mobile Manipulator program controlled the manipulator during the tests. It in- terfaced with the AGV directly to obtain the current AGV position and orientation, and it interfaced with the AGV control program (Transport Structure) running on the Order Manager application to coordinate the motion of the arm with the motion of the AGV.

The AGV control program signaled the Mobile Manipulator program when it arrived at one of the stop or test locations. The AGV control program also sent the identifica- tion number of the test location. The Mobile Manipulator program read the current AGV pose and used it to compute the initial search location of the two registration reflectors in the target pattern (circle or square). Additional patterns could also have been used in the Mobile Manipulator program.

7

Figure 4: Example square step search pattern drawing. The pattern begins with the yellow arrow dotted end and ends when the reflector is detected with the red arrow search step.

The manipulator was first moved from a stowed location over the body of the AGV to a staging location directly in front of the AGV. The manipulator was then moved from the staging location to the first of the two registration reflectors. The staging lo- cation was chosen so that the manipulator could make a straight line motion from the staging location to a registration reflector located in front of, or to either side of, the AGV without colliding with its shoulder joint. After moving to the first registration reflector, the manipulator performed a square spiral search to determine the exact loca- tion of the reflector. When it determined the location of the first registration reflector, the program repeated the process with the second registration reflector. When the lo- cations of the two registration reflectors were determined, the program had sufficient information to compute the locations of the other fiducials in the square or circular patterns. The initial search was not counted as a performance criteria since the mobile manipulator could use various types of registration techniques, such as: physical con- tact using a touch probe [18], cameras detecting fiducials [19], or laser interferometry, theodolites, and coordinate measuring arms [20]. However, for comparison to repeat- ability, the initial registration number of iterations count was logged and included in results.

Once the locations of all reflectors in the pattern were computed, the manipulator cycled through them a set number of times – 32 times for each pattern in this experi- ment. At each reflector, the RLS checked to see if the manipulator was aligned with the reflector.

When the test was completed, the manipulator was moved to the staging location and then the stow location. When the manipulator was back in the stow location, the Mobile Manipulator program signaled to the AGV control program that it was clear to move.

The positions of the index fiducials for the targets were recorded prior to performing the repeatability tests. The AGV was first moved to a location where it could reach both of the index fiducials. The current location and orientation of the AGV was recorded.

The arm was repositioned manually until the sensor detected alignment with each of the index fiducials, and the manipulator position was recorded. This information, along with the manipulator base position relative to the vehicle’s coordinate system, allowed

the correct manipulator coordinates for the index fiducials to be calculated for an arbi- trary AGV location. This allowed the AGV to approach the target/work area from any direction and to compensate for variation in the AGV’s stopping pose.

The calibration of the manipulator base location involved recording the position of one or more fiducials from a variety of locations. Both the AGV location and the ma- nipulator coordinates of the fiducials were recorded. This data was processed using an iterative, non-linear model to find the best value of the base position and orientation.

4 Results

The mobile manipulator performance measurement results using the RMMA-1 in- cluded only the detection of reflectors for each pattern and after initial registration. By comparison, the initial number of search steps used to register the manipulator at the first reflector was recorded. Results are shown in Table 1. The repeatability perfor- mance measurement process began once the mobile manipulator was registered to the artifact after initial registration and moving through the square or circle pattern one time and with the AGV statically positioned at a pattern. Measurements of ‘detect’ or ‘1’

were logged for each RMMA-1 reflector location. If a search was required to find the fiducial after registration, the measurement at that reflector was counted as a ‘no detect’

and the number of search steps was recorded.

Table 1 shows: the consecutive position number and programmed AGV position, the AGV pose angle (heading), the circle or square pat- tern being detected, the total number of reflectors to detect for 32 pattern iterations after the registration pattern, the reflector diameters for each pattern (rounded to whole numbers), the number of reflectors detected and detection percentage, and the initial number of search steps needed to register to the first reflector after the AGV stopped. The AGV stop points programmed are shown in Figure 4. The lines leading to the stop points indicate the AGV orienta- tion.

The table shows very high repeatability results at 97% or above as shown in the “% detected” column of the table. The results demonstrate a good test procedure for determining repeatability of a mobile manipulator to register to and access assembly points within the reflector diameters chosen. Further tests are required to understand direct connections between mobile manipulator performance and system pose, for example, suggesting that AGV pose at 0° provides higher performance than at other angles. Results here do not show this since position 6 included the AGV being at 90° and yet, was repeatable to 100%.

Several additional tests are envisioned as well, such as: repeatability of the same pattern followed by the other pattern, both from different AGV poses; using the 1 mm regis- tration reflectors for all patterns followed by the same size reflectors for all, such as 3 mm; and using 1 mm diameter reflectors for all points within each pattern to provide a possible detection limit.

Figure 2: Stop points

9

Table 1: Test results of the mobile manipulator accessing the RMMA from various Stop Points (see Figure 4) and various AGV poses. The gray rectangle in the center of the Stop Points map shows the approximate RMMA square and circle pattern locations.

5 Conclusion and Future Work

As discovered in a NIST survey of mobile manipulator research [10], performance measurement of these systems is minimal as compared to robot arms. Measurement methods, such as using optical tracking systems, are useful methods for measuring mo- bile manipulator performance, although at a much higher cost. The use of known arti- facts, called reconfigurable mobile manipulator artifacts, to measure the performance of mobile manipulators is being researched at NIST to demonstrate the feasibility of the test method. The concept of using artifacts demonstrates to potential manufacturers and users of mobile manipulator systems that relatively low cost performance measure- ment methods exist. Artifacts, such as the RMMA-1 and in the future, RMMA-2, allow an unlimited number of performance measurement configurations. The measurement of mobile manipulator repeatability and accuracy for very low resolution tasks (e.g., positioning bags of product) through very high resolution tasks (e.g., assembly of parts for manufacturing) is achievable through the use of RMMAs. Static and index tests have been completed using this method and have proven feasible. Experimental results show a promising test method to measure performance of mobile manipulators that are to be used for manufacturing assembly tasks, where at least the mobile manipulator tested has the capability to perform assembly to 1 mm positional accuracy or greater.

Future test method developments should not only include dynamic mobile manipulator performance measurements, but also include the suggested tests in the results section.

Additionally, rapid registration techniques, finer retroreflective laser sensors allowing smaller diameters, and in turn, physically providing peg-in-hole measurements with variable peg and hole chamfers, are expected to provide even higher performance meas- urements towards assembly applications of mobile manipulators.

References

[1] M. Shneier and R. Bostelman, “Literature Review of Mobile Robots for Manufacturing,” NIST Internal Report #8022, 2014.

[2] D. Katz, E. Horrell, Y. Yang, B. Burns, T. Buckley, A. Grishkan, V. Zhylkovskyy, O. Brock, and E. Learned-Miller, “The UMass mobile manipulator UMan: An ex- perimental platform for autonomous mobile manipulation.” Workshop on Manip- ulation in Human Environments, at Robotics: Science and Systems, 2006.

[3] B. Hamner, S. Koterba, J. Shi, R. Simmons, and S. Singh, “An autonomous mobile manipulator for assembly tasks,” Autonomous Robot (2010) 28: 131–149. DOI 10.1007/s10514-009-9142-y

[4] S. Djebrani, A. Benali, and F. Abdessemed, “Modeling and Control of an Omnidi- rectional Mobile Manipulator,” Int. J. Appl. Math. Comput. Sci., 2012, Vol. 22, No. 3, 601–616 DOI: 10.2478/v10006-012-0046-1

[5] J. Vannoy and J. Xiao, “Real-time Adaptive Motion Planning (RAMP) of mobile manipulators in dynamic environments with unforeseen changes,” in IEEE Trans.

on Robotics, 1199—1212, 2008.

[6] “Yaskawa Motoman MH80 robot unloading trucks - from Wynright

Corporation,” http://www.youtube.com/watch?v=8wngL0BnF_4, June 18, 2013.

[7] E. Guizzo, “Meka Robotics, Announces Mobile Manipulator With Kinect and ROS,” http://spectrum.ieee.org/automaton/robotics/humanoids/meka-robotics- announces-mobile-manipulator-with-kinect-and-ros, 16 Feb 2011.

[8] T. Green, “KUKA Falls First, Buys Swisslog for $335M. Who’s Next?,”

Robotics Business Review, Sept 29, 2014.

[9] W. Miksch, D. Schroeder, “Performance-Functional Based Controller Design for a Mobile Manipulator,” . Proceedings IEEE International Conference on

Robotics and Automation, 12-14 May 1992.

[10] Roger Bostelman, Tsai Hong, Jeremy Marvel, “Survey of Research for

Performance Measurement of Mobile Manipulators”, to be published in Journal of National Institute of Standards and Technology, 2016.

[11] Robotic Systems for Smart Manufacturing Program, http://www.nist.gov/el /isd/ms/rssm.cfm, National Institute of Standards and Technology, 2016.

[12] ASTM F45 Committee on Driverless Automatic Industrial Vehicles, www.astm.org, 2016.

[13] ASTM F45.02 Docking and Navigation subcommittee, work item WK50379, www.astm.org, 2016.

[14] Universal Robots A/S-UR10 User Manual, Version 3.0 (rev. 15167), 2014.

[15] SICK 2D Laser Scanners, https://www.sick.com/media/pdf/1/41/841/

dataSheet_LMS100-10000_1041113_en.pdf.

[16] Roger Bostelman, Tsai Hong, Jeremy Marvel, “Performance Measurement of Mobile Manipulators”, SPIE 2015, Baltimore, MD, April 2015.

[17] Roger Bostelman, Joe Falco, Mili Shah, and Tsai Hong Hong, “Dynamic Metrol- ogy Performance Measurement of a Six Degree-Of-Freedom Tracking System used in Smart Manufacturing”, ASTM International “Autonomous Industrial Ve- hicles: From the Laboratory to the Factory Floor” book chapter, 2016.

[18] MasterCal, http://www.americanrobot.com/products_mastercal.html, 2005.

[19] C B. Atcheson, F. Heide, and W. Heidrich, “CALTag: High Precision Fiducial Markers for Camera Calibration,” Vision, Modeling, and Visualization, 2010.

[20] Jose Mauricio, S. T. Motta, “Robot Calibration: Modeling Measurement and Ap- plications,” Industrial Robotics: Programming, Simulation and Applications, Low Kin Huat (Ed.), ISBN: 3-86611-286-6, InTech, 2006.