## Abstract

T he family of scanning probe microscopy (SPM) techniques has revolutionized the study of nanostructures. The key capability of SPM is that, through a controlled combination of feedback loops, detectors, and piezoelectric actuation, it enables direct investigations of micron, nanometer, to atomic scale phenomena such as those encountered in cellular and molecular imaging. To achieve its full potential for nanobiomedical investigation, SPM will require integration with signal processing to increase both the extraction and the reliability of the data. In this article, we explore the challenges in SPM-based cellular and molecular imaging as well as signal processing solutions for these challenges. BRIEF HISTORY OF SPM IN NANOBIOLOGY The family of SPM techniques has revolutionized studies of micro and nano objects. In SPM, a sharp tip undergoes a near-field interaction with a sample surface during a piezoelectric rastermotion scan of the surface, thereby producing a map of the interaction. The inherent resolu

## I E E E P R O O F

tion of SPM is angstrom (atomic) level for scanning tunneling microscopy and nanometer (macromolecular) level for atomic force microscopy.

The potential of SPM for use in biological studies was immediately apparent, and research efforts began shortly after the invention of the scanning tunneling microscope (STM) in the mid-1980s. The original nano-Amp STM [1] was not suitable for imaging largely nonconductive biological samples, and this led directly to the invention of the atomic force microscope (AFM) for nanobiological research [2] . As the nano-Newton tip-sample interaction forces of AFM proved to be sufficient to damage soft biological tissues, pico-Newton imaging systems based on disturbances to a freely oscillating cantilever were developed (tapping mode or noncontact AFM). Liquid environment systems that greatly reduced Van der Waals and surface adhesive forces were also developed [3] , [4] . The successful operation of AFM techniques in the liquid ambient environments also provides another important capability for biological investigations: the ability to conduct direct investigations under nearly life-like conditions [5] . A pico-Amp tunneling current STM has since been developed for biological investigations [6] , as well as an SPM variant of optical microscopy, near-field scanning optical microscopy (NSOM) [7] . This article will focus on AFM imaging. The AFM record for contributions in nanobiology and medicine includes an impressive number of firsts: the first direct observations of DNA and RNA [8] , [9] , the first direct investigations of membrane proteins [10] , the first direct investigations of ligand binding by functionalized tip AFM [11] , and many more important first direct studies. A persuasive argument for the use of AFM techniques in nanobiology is that it is a direct investigative technique in a field that has largely rested on indirect or after-the-fact observations. Recognition of the importance of AFM techniques for medically relevant nanobiological investigations is growing rapidly [12] . Examples of recent studies include high-resolution combined topographical and mechanical studies for multiple living cells types [12] – [14] , direct investigations of the growth cones on living neurons [12] , and, opening up an very important nanobiological research area, the first direct single molecule investigations [12] , [15] , [16] .

Therefore, the main advantages of AFM in nanobiology are that it easily reaches macromolecular resolution, with minimal sample preparation and under nearly life-like conditions. One disadvantage is that it has been a surface technique requiring complementary investigations to provide subsurface information, but subcellular AFM is under development [12] , [17] . The disadvantage that constitutes the main focus of this article is that artifacts can make correct interpretation of images very difficult. In every tip-sample interaction there are artifacts, the reason for many of which are known but are still largely unresolved issues for scanning probe microscopists. They present even more formidable obstacles for molecular biologists, and this is holding up wide-scale utilization of AFM as an enabling direct investigative tool in nanobiology and medicine. This is a field that is ripe for the integration of signal processing techniques to take AFM for cellular and molecular imaging to a new level.

## Origins Of Artifacts In Cellular And Molecular Imaging By Afm

In AFM, a sharp tip is brought within nanometers of a sample surface by the z-motion of a piezoelectric scanner and held there by a z-feedback loop, as discussed in “Atomic Force Microscopy Basics” (Also see [12, Part I] ). The AFM tip-sample interaction is caused by the summation of all possible Coulomb forces between the tip and the sample, summed along the physical structure of the tip and over a local region of the sample surface. These forces can introduce distortions into the information that represents some given aspect of the sample surface. Any tip shape introduces significant artifacts in the form of physically missed region information. In addition to missed regions, contact at any point other than the tip will result in an image smearing or dilation artifact. The sample surface may also contribute missing region and dilation artifacts of its own, depending on its surface roughness. In the case of a yielding sample, which is the case for cellular imaging, any deformation of the sample surface also dynamically changes the interaction. While this does not necessarily introduce a missed region artifact, it does mean that dynamic modeling of the contribution of each electrostatic force component in the tip-sample interaction becomes necessary [12] – [14] , since an a priori model of such highly individual conditions cannot exist. The x-y raster motion takes place in the ambient environment within tens of nanometers of the sample surface, and surfaces are always unique littoral regions. A very common artifact is due to drag through a surface layer, which introduces torsioning of the cantilever. When a tip is raised or lowered through the surface layer while maintaining its z-feedback condition, it feels vertical adhesive forces as well. Therefore, tip shape, surface roughness, surface environment, and yielding surfaces can all result in information artifacts.

Physics-based approaches for enhancing accuracy of measurements include the work by Grimellec et al. [18] , where effort is made to reduce the tip-sample interaction for increasing the accuracy of surface height. Jiao and Schaffer [14] use additional information about the mechanical properties from spatially resolved force curves at each pixel to obtain accurate height and volume information on soft samples. Touhami et al. [13] use a similar approach for mapping of elasticity of microbial cells. Physics-based approaches are often developed when a problem with imaging artifacts has required that careful attention be paid to data interpretation. A new signal processing research field could evolve from the systematic integration of physicsbased information with powerful image processing and deconvolution techniques to increase both the extraction and the reliability of the data.

## Deconvolution In Afm

It is clear that a key component of AFM data analysis is the modeling and characterization of the tip and sample artifact sources for use in subsequent deconvolution algorithms. This

## I E E E P R O O F

is particularly germane to cell surface studies where the intrinsic roughness of the cell surface convolved with tip shape limits the resolution of many important nanobiological features [19] . In general, when modeling the AFM data as the output of a linear time-invariant system, when the sample surface involves sharp nanoscale features, the surface measured by the AFM can be interpreted as the convolution of the sample with a transfer function that depends on the properties of the tip. When the tip properties are accurately known, the true image can be reconstructed by a linear deconvolution of the measured surface. Two of the most important factors that affect AFM resolution are the apex and aspect ratio of the scanning tip. The first tips used by the inventors of the AFM were made by gluing sharply faceted diamond chips onto aluminum foil. Current tips are microfabricated from silicon or silicon nitride. Contact mode tips have an apex radius of curvature of about 25-30 nm and tapping or noncontact mode tips have an apex radius of curvature of about 5-10 nm. Standard AFM research probes are tetrahedral with a 35 • apex angle. The apex and aspect ratio may be modeled using a parabola to describe an overall tip shape.

In the following review, we will concentrate on the effects of the tip shape convolved with the sample surface features, without considering the complexity introduced by the sample plasticity or the surface environment. Two different classes of methods that are widely used in AFM image processing, namely, mathematical morphology (MM) [20] and Legendre transforms (LTs) [21] , are reviewed in the following sections. These techniques essentially serve to reconstruct, from the measured image, the sample surface if the tip geometry is known or the tip geometry if the sample surface is known.

## Morphological Processing Technique For Surface Reconstruction

The approach based on mathematical morphology for surface reconstruction, derived by Villarrubia [19] , takes into account the effects of a finite probe tip size. It is assumed here that the image distortion is due to dilation of the image features by the finite region of the probe tip. The next section discusses the inherent relationship that exists between AFM imaging and the field of mathematical morphology and describes a procedure to obtain the best possible reconstruction of the original sample surface using morphological deconvolution of the measured AFM image. We first describe AFM image formation in terms of morphological operations.

The notations in two dimensions are as follows: s(x, y) : true sample surface i(x , y ) : measured AFM image surface t(x − x , y − y ) : arbitrary position of the probe tip relative to the sample surface s(x, y).

As the probe is moved, the height of the tip above the sample surface is the measured image data ( Figure 1 ) and can be modeled as

`EQUATION (1): Not extracted; please refer to original document.`

As shown in “Mathematical Morphology,” this is equivalent to the gray-scale morphological operation of dilation

`EQUATION ( 2): Not extracted; please refer to original document.`

where ⊕ is the dilation operator and set

S = [(x, y, z)/z ≤ s(x,

## I E E E P R O O F

i(x , y ) and the probe tip t(x, y). When the probe apex is in contact with the sample, the AFM output measures the true height, i.e., i(x , y ) = s(x, y). However, when the point of contact is at a point other than the probe apex, there is a distortion in the measured surface height and i(x , y ) = s(x, y). Figure 1 shows various translates of the parabolic tip t(x − x , y − y ) relative to the sample, the corresponding points of contact, and the resultant measured surface. The dotted lines represent the simulated image obtained by the process of dilation, which results in the rounding artifact seen in measured AFM data. From the image model above, the algorithm for “deconvolving” the distortion due to the tip shape can be obtained using a gray-scale morphological erosion (inverse of dilation) operation defined as

`EQUATION ( 3): Not extracted; please refer to original document.`

where the minimum operation is carried out over those coordinates where both tip and image are defined. In terms of the corresponding sets, defined above, (3) can be written as

`EQUATION ( 4): Not extracted; please refer to original document.`

where denotes the morphological erosion operator and S r is the reconstructed surface. Figure 2 depicts the process of erosion of measured surface with a parabolic tip surface. It is important to note that the reconstruction procedure described above can also be used for estimating an unknown tip surface from the measurement of a well-characterized sample geometry as P r = I S.

c(x, y, ) =1 when S r (x, y) = S(x, y) c(x, y, ) =0 when S r (x, y) = S(x, y).

In Figure 2 , we can see that at tip position 1, t(x, y) touches the reconstructed surface at one point, (x, y), where i(x, y) +

I E E E P r o o f t(u, v) = s r (x + u, y + v).

Hence, the certainty map in this location is set to 1. In position 2, the tip contacts the sample at multiple points and hence the certainty function is defined as c(x, y, ) = 0 in this location. Figure 3 shows the performance of surface reconstruction obtained using morphological erosion on a simulated surface. The simulated AFM height image in Figure 3 (b) was generated using morphological dilation operation on the known sample surface depicted in Figure 3(a) .

## Lt Technique For Sample Surface Reconstruction

An alternate approach for deconvolving the effect of tip shape/size is developed by Keller and is based on the LT of the measured image. This technique exploits the fact that the relationship between the measured surface and true surface is nonlinear, as seen by the schematic in Figure 4 . When the slope of the sample surface is large relative to that of the tip, the probe touches the sample at a point other than the nominal tip apex position. The AFM image consequently shows a surface that is smeared and corners that are more rounded. The true image can be reconstructed from the measured image if the true contact point between probe and sample can be calculated from the image surface and tip shape. This involves the calculation of two quantities, namely, the lateral distance between true and apparent contact points ( x) and the vertical distance between true and apparent contact points ( s).

In one dimension, we define the following notation: These equations can be easily extended to a two-dimensional (2-D) image case in a straightforward manner. At the true point of contact the probe and sample surfaces have the same tangent line so that the slope of the tip surface equals slope of the true sample surface, i.e., s(x) and t( x) have the same derivative at contact

`EQUATION (5): Not extracted; please refer to original document.`

As the tip moves, x changes and so does x, i.e., x ≡

x(x ). Specifically, x = x − x or x = x + x(x ).

Taking derivative of x with respect to x , we get

`EQUATION ( 6): Not extracted; please refer to original document.`

From the definition of true surface we have

`EQUATION (7): Not extracted; please refer to original document.`

where s(x ) = t( x(x )). The derivative of (7) with respect to x is given by

`EQUATION ( 8): Not extracted; please refer to original document.`

Using (6) and (7) in (8), we get

`EQUATION ( 9): Not extracted; please refer to original document.`

Equation 9says that the slope of the measured image at apparent point of contact x is equal to the slope of the true sample surface at the point of contact x. Equation (9) can be inverted to determine x(x ) and, hence, s(x ), if t( x) is known. These values are then substituted in (7) to reconstruct the true surface s(x). A quantitative measure of the distortion due to probe surface is defined to be equal to (d x)/(dx ), which can be calculated as

`EQUATION ( 10): Not extracted; please refer to original document.`

Equation (10) simply states that when the tip has a high curvature, the distortion is less, and vice versa. In terms of the LT, the reconstruction can be expressed in a very simple manner.

where x is found according to −x = (db(m))/dm and m(x) is the solution of −x = (db/dm)(m(x)). From Figure 5 , we can see that the LTs of true surface (bold), measured surface (dotted), and probe tip surface are related by

`EQUATION ( 13): Not extracted; please refer to original document.`

where all the transforms are evaluated at m = di/dx . The true surface is therefore obtained simply by inverse Legendre transformation, which is equivalent to the operations in (5)- (9) . Figure 6 shows the surface reconstruction that results from applying the LT to simulated AFM height data of a known surface profile and a known tip geometry using linear interpolation in the inaccessible region. The simulated image in Figure 6 (b) is obtained by the morphological dilation procedure described in (2) .

## Blind-Tip Estimation

The reconstruction algorithms discussed thus far require accurate knowledge of the tip surface. However, tips can abrade or pick up debris from the sample surface during a scan. In such cases, the tip surface needs to be first estimated using a known calibration sample. In general, it is a significant challenge to produce an accurately characterized reference artifact to nanometer tolerances. This, in turn, introduces an error in the estimated tip shape, and hence we can at most estimate the upper and lower bounds of the actual tip shape. The upper bound of the tip surface allows the estimation of the lower bound of the sample surface with the measured image serving as the upper bound. Alternately, the surface can be reconstructed using iterative blind deconvolution algorithms [19] , [20] . The iterative blind reconstruction algorithm proposed by Villarrubia [19] is based on the rationale that the tip essentially serves to broaden any protrusion on the sample surface. Since the tip is assumed to be relatively narrow, independent features in the image can be treated as separate images, each of which sets an upper bound on the tip shape so that the true tip surface is sharper than the tightest of these bounds. Intersection of all the bounds results in the least blunt estimate of the tip that is consistent with the observed image. The iterative procedure for blind-tip estimation is derived in [19] and is summarized below.

1) Assume initial estimate for the probe tip surface P 0 .

2) Locate features on the image surface.

3) (i + 1)th iteration: Get a bound on the tip surface based on ith image feature.

P i (x) = P i (x − I)

where x is the location of the feature of interest. The (i + 1)th estimate is computed as

`EQUATION ( 14): Not extracted; please refer to original document.`

At convergence, we have the resulting estimate for the tip surface as P R = lim i→∞ P i . The parameters involved in this algorithm are mainly the initial tip surface and initial values for the tip height, i.e., tip shape. It can be proved that each iteration produces a result that is smaller or equal to the preceding result. At convergence, the final result gives a minimum upper bound of the true tip surface. Given an image and estimated tip geometry, the morphological erosion procedure is used for sample surface reconstruction according to (3) . The performance of blind-tip estimation depends strongly on the sharpness of the sample features.

Typical results of blind-tip estimation on synthetic data are Figure 7 (a). Using features in the synthetic image in Figure 3(a) , the blind estimation procedure results in the tip shape estimate shown in Figure 7(d) .

## Example Applications Of Deconvolution For Nanobiology

We will illustrate the application of the mathematical morphology and LT approaches, with and without blind-tip estimation, by applying them to an AFM investigation of cell growth on tissue scaffolding. Experimental AFM images of a rat kidney cell (type NRK2) attached to an electrospun carbon nanofiber tissue scaffold are shown in Figure 8 . I E E E P r o o f conspicuous projection on the cell surface that may be a real feature due the coagulating proteins of a dying cell or an artifact caused by a sharp surface protrusion imaging the tip. We begin with the problem of the evident versus the real width of an average nanofiber within the tissue scaffold network. A horizontal line section through the raw AFM data in circled region “1” is shown as the blue line in Figure 9(a) , and it indicates that a tip-shape-based rounding artifact is distorting the width measurement.

Deconvolution of the image using both the LT and MM methods was performed and the corresponding line sections through the nanofiber are shown next to the blue line in Figure 9 (a). The Legendre transform result is shown in black and the mathematical morphology deconvolution is in red. The difference in the two results is due to the fact that in the LT method, the parabolic tip is of unlimited height, which gets wider with the height of the tip, whereas in MM the probe tip is of finite height. Consequently, when the aspect ratio of the feature is comparable with that of the tip, the two methods produce similar results; yet when the aspect ratio is larger than that of the tip, the MM method produces a sharper estimate than the LT method, as in the case of the nanofiber.

The nanofiber widths were estimated as 354.25 nm for the measured image, 334.01 nm after deconvolution using LT, and 328.42 nm after deconvolution using MM method.

Next we consider the conspicuous projection on the cell surface, circled region “2” in Figure 8 Figure 8 before and after deconvolution. The lack of improvement using LT deconvolution method implies that the feature is wider than the parabolic tip curvature estimated by the blind-tip estimation procedure. Also, the difference in the LT and MM results indicate that the feature is taller than the assumed tip height (100 nm). However, the blind-tip estimation described here is rather simplistic and yields a tip shape that depends on the sharpest feature in the image. Hence, if the true tip shape is known, deconvolution would yield an impulse feature as the result, which in turn will resolve the ambiguity stated in the beginning of this section.

## I E E E P R O O F

images is a crucial problem in nanobiological investigations. SPM offers the potential for direct investigative capability at nanometer resolution necessary for imaging biological units and macromolecular protein control blocks. The distortion of the measured image due to tip-sample interaction is a major challenge for nanoscale metrology, and signal processing solutions are needed for increasing the accuracy and reliability of the data. Two candidate approaches have been described in detail in this article for modeling the tip-sample interaction from a topographical perspective, which is then used for reconstructing the sample surface from known tip geometry. When the aspect ratio of feature is comparable with that of the tip, the two methods produce similar results, but when the aspect ratio is larger than that of the tip, the MM method produces a sharper estimate than the LT method. When the tip geometry is not known, blind-tip estimations methods are needed for iterative estimations of tip and sample surfaces.

In conclusion, this is indeed a field that is ripe for the addition of signal processing techniques to take AFM for cellular and molecular imaging to a new level. By removing a serious block to the widespread use of this direct investigative tool, an entire range of new research areas in nanobiological and medical applications could see big advances.

## Mathematical Morphology

Morphological image processing is used widely for extracting components or features of objects in an image. The language of mathematical morphology is based on set theory. Sets in mathematical morphology are used to represent objects in ndimensional Eucledian space. While dealing with binary images each element of the set is a 2-tuple comprising coordinates of a black/white pixel in the image. While dealing with gray-scale images the elements of the set represents the graylevel value of each pixel in an image.

Basic concepts of set theory such as union, subset, intersection, mutual exclusivity, translation, complement, and difference are used for defining operations in mathematical morphology. This tutorial explains the basic operations of image dilation and erosion for gray-scale images using simple set theoretic operations such as union, intersection, and translation. Translation of a set S by a vector d is obtained by adding d to every element of S

S + d = {s + d |s ∈ S} .

This has been illustrated graphically in the Figure B(a) where the object represented by set S is translated by d = (d x , d y ) . The top of the translated object, T[S + d], can be represented as follows:

T[S + d](x, y) = s(x − d x , y − d y ).

Similarly, the union of two objects A, B, illustrated in Figure B The above expression is also interpreted as a union of translates as shown in Figure B(c) . Here if we take the center of the circle A to be the point a = 0, dilation is the area swept by A as it is centered over every point b in B. In Figure B(c) , the dila-tion of B by A is computed using the equation

T[A ⊕ B](x, y) = max u,v [a(x − u, y − v), b(u, v)].

The process of AFM image simulation can be explained in terms of dilation, as illustrated in Figure C [19] . Let i(x, y) be the function describing the image surface, s(x, y) the specimen surface, and t(x, y) the probe tip surface. Translates of the tip at point (x , y ) is expressed as t(x − x , y − y ). The image of the surface is obtained by lowering the tip until it touches the surface. The apex of the lowered tip would yield the height of the image at the point (x , y ) . Hence, the measured surface can be modeled as Erosion can be thought of as the inverse of dilation. While dilation causes an object to grow, erosion causes it to shrink, as illustrated in Figure D .

i

More complex morphological operations such as “Open” and “Close” are obtained by combining dilation and erosion.

## Leave a Reply