Skip to main content
SpringerLink
Log in

Integrating Optical Force Sensing with Visual Servoing for Microassembly

  • Published:
Journal of Intelligent and Robotic Systems Aims and scope Submit manuscript

Abstract

For microassembly tasks uncertainty exists at many levels. Single static sensing configurations are therefore unable to provide feedback with the necessary range and resolution for accomplishing many desired tasks. In this paper we present experimental results that investigate the integration of two disparate sensing modalities, force and vision, for sensor-based microassembly. By integrating these sensing modes, we are able to provide feedback in a task-oriented frame of reference over a broad range of motion with an extremely high precision. An optical microscope is used to provide visual feedback down to micron resolutions, while an optical beam deflection technique (based on a modified atomic force microscope) is used to provide nanonewton level force feedback or nanometric level position feedback. Visually servoed motion at speeds of up to 2 mm/s with a repeatability of 0.17 μm are achieved with vision alone. The optical beam deflection sensor complements the visual feedback by providing positional feedback with a repeatability of a few nanometers. Based on the principles of optical beam deflection, this is equivalent to force measurements on the order of a nanonewton. The value of integrating these two disparate sensing modalities is demonstrated during controlled micropart impact experiments. These results demonstrate micropart approach velocities of 80 μm/s with impact forces of 9 nN and final contact forces of 2 nN. Within our microassembly system this level of performance cannot be achieved using either sensing modality alone. This research will aid in the development of complex hybrid MEMS devices in two ways; by enabling the microassembly of more complex MEMS prototypes; and in the development of automatic assembly machines for assembling and packaging future MEMS devices that require increasingly complex assembly strategies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Albrecht, T. R., Akamine, S., Carver, T. E., and Quate, C. F.: Microfabrication of cantilever styli for the atomic force microscope, J. Vac. Sci. Technol. A 8(4) (1990), 3386-3396.

    Google Scholar 

  2. Arai, F., Ando, D., and Fukuda, T.: Micro manipulation based on micro physics strategy based on attractive force reduction and stress measurement, in: Proc. of 1995 IEEE/RSJ Internat. Conf. on Intelligent Robots and Systems (IROS '95), Vol. 2, Pittsburgh, 5-9 August 1995, pp. 236-241.

    Google Scholar 

  3. Arai, F., Nonoda, Y., Fukuda, T., and Oota, T.: New force measurement and micro grasping method using laser raman spectrophotometer, in: Proc. of 1996 IEEE Internat. Conf. on Robotics and Automation, 1996, pp. 2220-2225.

  4. Benrakkad, M. S. and Benitez, M. A.: Stress measurement by microRaman spectroscopy of polycrystalline silicon structures, J. Micromech. Microengrg. 5 (1995), 132-135.

    Google Scholar 

  5. Castaño, A. and Hutchinson, S.: Visual compliance: Task-directed visual servo control, IEEE Trans. Robotics Automat. 10(3) (1994), 334-342.

    Google Scholar 

  6. Cloud, G. L.: Optical Methods of Engineering Analysis, Part VI and VII, Cambridge Univ. Press, New York, 1995.

    Google Scholar 

  7. Corke, P. I. and Paul, R. P.: Video-rate visual servoing for robots, in: V. Hayward and O. Khatib (eds), Lecture Notes in Control and Information Science, Springer, Berlin, 1989, pp. 429-451.

    Google Scholar 

  8. Diddens, D. and Reynaerts, D.: Design of a ring-shaped three-axis micro force/torque sensor, Sensors and Actuators A 46/47 (1995), 225-232.

    Google Scholar 

  9. Durrant-Whyte, H. F.: Integration, Coordination and Control of Multi-Sensor Robot Systems, Kluwer Academic, Dordrecht, 1988.

    Google Scholar 

  10. Espiau, B., Chaumette, F., and Rives, P.: A new approach to visual servoing in robotics, IEEE Trans. Robotics Automat. 8 (3) (1992), 313-326.

    Google Scholar 

  11. Fearing, R. S.: Survey of sticking effects for micro parts handling, in: Proc. of 1995 IEEE/RSJ Internat. Conf. on Intelligent Robots and Systems (IROS '95), Vol. 2, Pittsburgh, 5-9 August 1995, pp. 212-217.

    Google Scholar 

  12. Feddema, J. T. and Lee, C. S. G.: Adaptive image feature prediction and control for visual tracking with a hand-eye coordinated camera, IEEE Trans. Systems Man Cybernet. 20(5) (1990), 1172-1183.

    Google Scholar 

  13. Feinerman, A. D., Crewe, D. A., Perng, D. C., Shoaf, S. E., and Crewe, A. V.: Sub-centimeter micromachined electron microscope, J. Vac. Sci. Technol. A 10(4) (1992), 611-616.

    Google Scholar 

  14. Hager, G. D.: Task-Directed Sensor Fusion and Planning-A Computational Approach, Kluwer Academic, Dordrecht, 1990.

    Google Scholar 

  15. Hager, G. D.: Robot feedback control based on stereo vision: towards calibration-free hand-eye coordination, in: Proc. of 1994 Internat. Conf. on Robotics and Automation (ICRA '94), 1994, pp. 2850-2856.

  16. Hannaford, B., Hewitt, J., Maneewarn, T., Venema, S., Appleby, M., and Ehsresman, R.: Telerobotic remote handling of protein crystals, in: Proc. of 1997 IEEE Internat. Conf. on Robotics and Automation, 1997, pp. 1010-1015.

  17. Hashimoto, K. and Kimura, H.: LQ optimal and nonlinear approaches to visual servoing, in: K. Hashimoto (ed.), Visual Servoing: Real-Time Control of Robot Manipulators Based on Visual Sensory Feedback, World Scientific, London, 1993, pp. 165-198.

    Google Scholar 

  18. Hicks, T. R. and Atherton, P.D.: The Nano Positioning Book, Queensgate Instruments, 1997.

  19. Kato, N. and Suzuki, I.: Force-balancing force sensor with an optical lever, Rev. Sci. Instrum. 66(12) (1995), 5532-5536.

    Google Scholar 

  20. van der Werf, K.O. and Putman, C. A. J.: Compact stand-alone atomic force microscope, Rev. Sci. Instrum. 64(10), (1993), 2892-2897.

    Google Scholar 

  21. Koyano, K. and Sato, T.: Micro object handling system with concentrated visual fields and new handling skills, in: Proc. of 1996 IEEE Intrenat. Conf. on Robotics and Automation, 1996, pp. 2541-2548.

  22. Menciassi, A., Carroza, M. C., Ristori, C., Tiezzi, G., and Dario, P.: A workstation for manipulation of micron sized objects, in: Proc. of 1997 8th Internat. Conf. on Advanced Robotics, 1997, pp. 253-258.

  23. Nelson, B. J. and Khosla, P. K.: Force and vision resolvability for assimilating disparate sensory feedback, IEEE Trans. Robotics Automat. 12(5) (1996), 714-731.

    Google Scholar 

  24. Nelson, B., Papanikolopoulos, N. P., and Khosla, P. K.: Visual servoing for robotic assembly, in: K. Hashimoto (ed.), Visual Servoing-Real-Time Control of Robot Manipulators Based on Visual Sensory Feedback, World Scientific, River Edge, NJ, 1993, pp. 139-164.

    Google Scholar 

  25. Nonnenmacher, M. and Vaez-Iravani, M.: Attractive mode force microscopy using a feedbackcontrolled fiber interferometer, Rev. Sci. Instrum. 63(11) (1992), 5373-5376.

    Google Scholar 

  26. Papanikolopoulos, N. P., Khosla, P. K., and Kanade, T.: Adaptive robotic visual tracking, in: Proc. of the American Control Conf., 1991, pp. 962-967.

  27. Papanikolopoulos, N. P., Nelson, B., and Khosla, P. K.: Full 3-d tracking using the controlled active vision paradigm, in: Proc. of 1992 IEEE Internat. Symp. on Intelligent Control (ISIC-92), 1992, pp. 267-274.

  28. Peters, R. D.: Symmetric differential capacitive pressure sensor, Rev. Sci. Instrum. 64(8) (1993), 2256-2561.

    Google Scholar 

  29. Rao, B. C., Gao, R. X., and Friedrich, C. R.: Integrated force measurement for on-line cutting geometry inspection, IEEE Trans. Instrum. Meas. 44(5) (1995), 977-980.

    Google Scholar 

  30. Richardson, J. M. and Marsh, K. A.: Fusion of multisensor data, Internat. J. Robotics Res. 7(6) (1988), 78-96.

    Google Scholar 

  31. Rugar, D. and Mamin, H. J.: Force microscope using a fiber-optic displacement sensor, Rev. Sci. Instrum. 59(11) (1988), 2337-2340.

    Google Scholar 

  32. Sato, T., Kameya, T., Miyazaki, H., and Hatamura, Y.: Hand-eye system in the nano manipulation world, in: Proc. of 1995 IEEE Internat. Conf. on Robotics and Automation, 1995, pp. 59-66.

  33. Schonenberger, C. and Alvarado, C. F.: A differential interferometer for force microscopy, Rev. Sci. Instrum. 60(10) (1989), 3131-3134.

    Google Scholar 

  34. Shirai, Y. and Inoue, H.: Guiding a robot by visual feedback in assembling tasks, Pattern Recognition 5 (1973), 99-108.

    Google Scholar 

  35. Slocum, A. H.: Precision Machine Design, Prentice-Hall, Englewood Cliffs, NJ 1992

    Google Scholar 

  36. Smith, R. C. and Cheeseman, P.: On the representation and estimation of spatial uncertainty, Internat. J. Robotics Res. 5(4) (1986), 56-68.

    Google Scholar 

  37. Sulzmann, A., Breguet, J. M., and Jacot, J.: Micromotor assembly using high accurate optical vision feedback for microrobot relative 3D displacement in submicron range, in: Proc. of 1997 Internat. Conf. on Solid-State Sensors and Actuators (Transducers 97), 1997, pp. 279-282.

  38. Vikramaditya, B. and Nelson, B. J.: Visually guided microassembly using optical microscopes and active vision techniques, in: Proc. of 1997 IEEE Internat. Conf. on Robotics and Automation, Albuquerque, 21-27 April 1997, pp. 3172-3177.

  39. Weiss, L. E.: Dynamic visual servo control of robots: An adaptive image-based approach, Ph. D. Thesis, CMU-RI-TR-84-16, The robotics Institute, Carnegie Mellon University, 1984.

  40. Weiss, L. E., Sanderson, A. C., and Neuman, C. P.: Dynamic sensor-based control of robots with visual feedback, IEEE J. Robotics Automat. 3(5) (1987), 404-417.

    Google Scholar 

  41. Yamagata, Y. and Higuchi, T.: A micropositioning device for precision automatic assembly using impact force of piezoelectric elements, in: Proc. of 1995 IEEE Internat. Conf. on Robotics and Automation, 1995, pp. 666-671.

Download references

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, University of Minnesota, 111 Church St. SE, Minneapolis, MN, 55455, pU.S.A.

    Yu Zhou & Barmeshwar Vikramaditya

  2. Department of Mechanical Engineering, University of Minnesota, 111 Church St. SE, Minneapolis, MN, 55455, pU.S.A.

    Bradley J. Nelson

Authors
  1. Yu Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bradley J. Nelson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Barmeshwar Vikramaditya
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhou, Y., Nelson, B.J. & Vikramaditya, B. Integrating Optical Force Sensing with Visual Servoing for Microassembly. Journal of Intelligent and Robotic Systems 28, 259–276 (2000). https://doi.org/10.1023/A:1008136711577

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1008136711577

Search

Navigation