SEM-based vision

Contact

Sounkalo DEMBÉLÉ

Objectives

The framework of this research is instrumentation at the small scales, more particularly the characterization of small objects by means of robotics and scanning electron microscopy (SEM). Indeed, SEM provides clean environment of manipulation and high resolution images from which accurate informations can be retrieved for robot control or object geometry. But SEM exhibits some singularities, particularly at high magnification, that have to be tackled: charging of samples, drift of images, orthographic projection. The latter leads to three ambiguities: depth ambiguity that corresponds to invariance to translation along Z axis, bas-relief ambiguity that corresponds to invariance to fronto-parallel rotation by depth product, Necker ambiguity that corresponds to invariance to reflexion with respect to image plane.

Modelling of SEM imaging

Accurate and robust methods of autofocus has been developed: static and dynamic autofocus based on robust gradient descent.

Accurate autocalibration has been developed: robust optimization is used to retrieve the model of the SEM from a set of images

Motion

It is developed accurate 2D motion recovery from Fourier transform and template matching, and accurate 3D motion recovery from autofocus and autocalibration

3D structure

Linear rectification of image set and linear triangulation of equations enables 3D dense clouds.

Control

Autofocus and 2D visual servo has been combined to control a robot and to perform its calibration accurately. This results has lead to the 3D positioning of a 100 µm pollen grain and 10 µm microsphere.

Software

Pollen 3D for 3D reconstruction

People involved

Nadine PIAT, Valerian GUELPA, Rahima DJAHEL

Selection of publications

Kudryavtsev Andrey, Dembele Sounkalo and Piat Nadine, "Autofocus on Moving Object in Scanning Electron Microscope", Ultramicroscopy, vol. 182, pp. 216 - 225, jan. 2017

Marturi Naresh, Tamadazte Brahim, Dembele Sounkalo and Piat Nadine, "Image-Guided Nanopositioning Scheme for SEM.", IEEE Transactions on Automation Science and Engineering, vol. 99, pp. 1-12, jun. 2016

Dembele Sounkalo, Lehmann Olivier, Medjaher Kamal, Marturi Naresh and Piat Nadine, "Combining Gradient Ascent Search and Support Vector Machines for Effective Autofocus of a Field Emission – Scanning Electron Microscope.", Journal of Microscopy, vol. 264, 1, pp. 79 - 87, oct. 2016

Kudryavtsev Andrey, Dembele Sounkalo and Piat Nadine, "Stereo-image Rectification for Dense 3D Reconstruction in Scanning Electron Microscope", MARSS ,  Montréal - Canada, pp. 42 - 47, jul. 2017

Kudryavtsev Andrey, Dembele Sounkalo and Piat Nadine, "Full 3D Rotation Estimation in Scanning Electron Microscope", IEEE/RSJ IROS, Vancouver Canada, pp. 1 - 6, sep. 2017