Contributed Talk  - Friday, 17 September I 14:35 PM (CEST)

Javier Sotres: "Enabling Autonomous AFM Imaging of Single Molecules with Deep Learning"

Javier Sotres ¹ ²*, Hannah Boyd ¹ ², Juan F. Gonzalez-Martinez ¹ ²

¹ Department of Biomedical Science, Faculty of Health and Society, Malmö University, 20506 Malmö, Sweden

² Biofilms-Research Center for Biointerfaces, Malmö University, 20506 Malmö, Sweden

* Corresponding author. E-mail:

Scanning Probe Microscopies allow investigating surfaces at the nanoscale, in the real space and with unparalleled signal-to-noise ratio. However, these microscopies are not used as much as it would be expected considering their potential. The main limitations preventing a broader use are the need of experienced users, the difficulty in data analysis and the time-consuming nature of experiments that require continuous user supervision. In this contribution, we present a deep learning based strategy that addresses the latter. Specifically, we present an algorithm for controlling the operation of an Atomic Force Microscope (AFM) that, without the need of user intervention, allows acquiring multiple high-resolution images of different molecules. We used DNA on mica as a model sample to test our control algorithm, which makes use of two state-of-the-art deep learning techniques. One is an object detector, YOLOv3, which provides the location of molecules in the captured images. The second is a Siamese network that can identify the same molecule in different images. This allows both performing a series of images on selected molecules while incrementing the resolution, as well as keeping track of molecules already imaged at high resolution, avoiding loops where the same molecule would be imaged an unlimited number of times. Overall, this work brings SPM a step closer to full autonomous operation.


Figure 1. Consecutively acquired AFM images representative of the workflow of the autonomous imaging algorithm. The algorithm detects a suitable single DNA molecule (a), and performs successive zooms (b-c) on the selected molecule until a user-set threshold for enough resolution is achieved. Then, the algorithm zooms out, a different molecules is selected (e), continuously repeating a similar workflow.


Sotres J., Boyd H., Gonzalez-Martinez J.F. 2021. Enabling Autonomous Scanning Probe Microscopy Imaging of Single Molecules with Deep Learning. Nanoscale 13: 9193-9203.