An efficient solution for correlative microscopy and co-localized observations based on multiscale multimodal machine-readable nanoGPS tags

For a long time, investigating the same regions of interest of a sample with different instruments has been recognized as a very useful approach in various scientific fields. This paper presents an original solution for spotting the same points of interest with a high degree of accuracy and simplicity using different microscopes. It is based on small patterned tags fixed to the samples or their substrates. The patterns include an image-based position-sensing technology, for which an image of a small part of the tag can be automatically converted to absolute coordinates and angular orientation. Taking a single snapshot of the tag with an imaging instrument provides a correspondence between the sample and the coordinates of the moving stage. Co-localized observations performed with scanning electron microscopes, optical microscopes, and Raman microscopes are presented. The accuracy is in the range of a few µm up to 20 µm, which is generally sufficient to remove any ambiguity between the observed objects. The different contributions to colocalization errors are investigated experimentally and it is shown that errors related to the tags are negligible and that the main source of error is related to the accuracy of the moving stages integrated into the microscopes. A straightforward estimation of the relocalization error can be performed. It is believed that this solution will save researchers time and facilitate cooperation between laboratories.


Introduction
A large variety of scientific instruments based on different techniques make it possible for scientists to investigate the properties of their samples. The structural, elemental, chemical, biological, optical, electrical, and mechanical properties of a sample can be imaged at different scales, e.g. macroscopic, microscopic, or nanoscopic, through a variety of scientific instruments.
The different imaging modalities and the different scales of observation are very often complementary. It has long been recognized that combining several modalities to observe a given sample can be very useful in science. In biology, the initial focus has been in combining electron and fluorescence microscopy [1][2][3]. The initial interest has expanded to the correlation of several other modalities [4,5], including atomic force microscopy (AFM) [6][7][8], super-resolution microscopy [7,9], and Raman microscopy [10]. The importance of correlative measurements has also been recognized in geology [11][12][13], archaeology [14], environmental sciences [15], and many other fields [16][17][18]. Some modality couplings have developed rapidly in recent years, such as the coupling of scanning electron microscopy (SEM) and Raman microscopy, the former technique providing remarkable structural and atomic information, the latter providing complementary chemical information [10,13,15,17].
To perform observations on the same regions of interest (ROI) using different techniques, scientists have developed various approaches. One approach is to rely on observational skills: navigating over the sample with different instruments, recognizing some given patterns, and recording images in the same ROI. It is then possible to use software that performs precise registration of the images obtained by the different modalities [19].
However, in some cases, the images of a given ROI are very dissimilar between two instruments, because their modalities are sensitive to very different properties of matter, and visual recognition is difficult and unreliable. This is the case, for example, for the combination of SEM and fluorescence microscopy. In all cases, visual spotting of the same ROIs is a tedious and time-consuming process. In other cases, the fields of view of the instruments can be too dissimilar to achieve reliable visual recognition of ROIs. Another difficulty comes from the fact that the different instruments may be in different locations, operated by different specialists, making visual recognition still more unreliable and impractical.
As a consequence, scientists have worked out several solutions to to allow observations to be performed on the same ROIs with different instruments. The addition of marks using pencils or scribing tools has been popular. In some cases, scientists can choose a substrate with pre-defined markers that can help with ROI identification or integrate these markers (often called fiducials) into the process of making their sample [18]. An approach proposed for relocalization in optical microscopes (OMs) was to create coded patterns on the microscopy slide, that could be read with the microscope by refocusing on the back of the slide [20]. This approach is limited to transparent samples (which is the case for most biological samples), and optical microscopy.
Instrument manufacturers also have crafted solutions to make the process of colocalization between instruments easier. One solution was to create hybrid instruments so that two modalities were applied to the same sample [7,17]. This can be very helpful when the two modalities should be used simultaneously or within a short period. However, the vision of bringing 'the best of two worlds' to the same instrument is often confronted by the reality that the instrument constraints are added. This also tends to be costly, making the hybrid instrument more expensive, and less productive, compared to two separate instruments that can be operated simultaneously by two users.
Another solution proposed by instrument manufacturers is based on 'shuttles' [14]. Shuttles are sample holders that fit on their different instruments with fiducials realizing a sample coordinate frame. Calibration is performed each time the proprietary sample holder is placed on an instrument, to obtain the coordinate transform between the sample coordinates and the instrument coordinates. This is best performed on instruments equipped with a motorized stage, and the relocalization accuracy is the result of both the fiducial calibration accuracy and the stage accuracy.
Shuttle calibration is generally performed using three fiducials on the shuttle, as the knowledge of the coordinate transformation of three points is needed in the general case to establish the coordinate transformation formula between the stage and the sample coordinates. As a variant, some solutions propose the tuning of the orientation of the sample parallel to an instrument axis on each instrument and use a single fiducial.
The shuttle approach has several shortcomings. First, it allows the correlative observation of only a limited number of samples, as only one or a small number of shuttles can be reasonably used, both for price and storage reasons. Second, most of these solutions are proprietary, and can only be used between microscopes of a given brand. Third, the calibration procedure is somewhat tedious, with the necessity to find the different fiducials.
To overcome these limitations, we developed a coordinatetransfer system based on small patterned tags that we called nanoGPS navYX 4 . This system has the following characteristics: (a) the tags are machine-readable; they simply need to be coarsely positioned in the field of view of the instrument so that an appropriate region of the tag is imaged, to obtain an accurate position and orientation in the sample reference frame; (b) the tags are multimodal and multiscale, i.e. they can be read with various instruments, including SEM, OM and AFM over a wide range of magnifications; (c) the determination of coordinate transformation between the instrument and sample coordinates requires only one image snapshot; (d) the tags are small, making them suitable for tagging numerous samples.
This paper describes the coordinate-transfer solution and several examples of how it can be used in different contexts. As an important feature of relocalization systems is relocalization accuracy, this paper provides an experimental investigation of the accuracy of the proposed system and a detailed analysis of the different factors affecting relocalization accuracy. Because different features associated with an individual instrument (such as instrument stability or stage precision) can strongly affect the accuracy, the paper also includes practical methods for obtaining relocalization accuracy estimates for an instrument. However, relocalization accuracy is only one of several aspects that make a colocalization system valuable to laboratory life. Several other features will be illustrated, such as convenience, the ability to be implemented using different instruments, the ability to support collaboration between remote laboratories, suitability to large numbers of samples, and data treatment.

Description of the nanoGPS navYX solution for correlative microscopy
The principle of the nanoGPS navYX solution is described in figure 1. A small nanoGPS tag is attached to the sample. A simple way to achieve that is to fix it permanently with glue or carbon tape. The tag realizes the sample coordinate system. An observation session on an imaging instrument is initiated by recording an image of a part of this tag ( figure 1(a)). The sample's position and orientation are determined from this single image, using nanoGPS software. The orientation of the image and the magnification are also determined. By sharing stage coordinates with the software, the relation between sample coordinates and stage coordinates is established. This coordinate transformation is valid for the whole observation session. Observations are then performed on the sample by the user (figure 1(b)) at different stage positions. The nanoGPS software transforms the stage coordinates for each observation and stores the coordinates of each observation in the coordinate system of the sample as a point of interest (POI). It also creates an image with the position, orientation and scale information in the GeoTIFF format. GeoTIFF is a public-domain metadata standard for geographical information systems, that has also been used for referencing microscopy observations [21][22][23] and that is convenient for visualization. The software also provides information on POIs recorded in previous sessions and expresses their coordinates in the stage coordinates for the current session, making it easy to revisit previously recorded POIs.

Image-based position sensing
One key technology behind nanoGPS tags is image-based position sensing, in which the precise position and orientation can be determined from a single image of a patterned scale. This technology has been implemented with a variety of types of scales and imaging systems [24][25][26][27][28][29][30][31]. Image-based position sensing has been used to build position encoders [32], measurement systems [33], and also relocalization solutions in microscopy [20,25]. It has been established that the positioning accuracy of image-based position sensing can be much better than the pixel size. Though the precision of image-based sensing depends on the details of the patterned scale, the imaging system and the performance of the position retrieval software, precisions of a few nm and a few tens of µrad have been reported. Image-based three-degreesof-freedom encoders have been made by HORIBA Scientific using the nanoGPS OxyO ® technology, with a precision better than 2 nm and 10 µrad, and an accuracy better than 150 nm over 80 mm of translation, and 63 µrad over 2π of rotation [29,31]. Figure 2(a) shows an example of a nanoGPS-patterned scale imaged using a conventional microscope, with a 10× objective lens. The size of the small squares is 10 µm × 10 µm. The in-plane coordinates and angle of the scale determined using the nanoGPS software are also displayed on this figure. Supplementary material (available online at stacks.iop.org/MST/32/045402/mmedia) includes a video featuring an experiment where this scale is translated and then rotated. A significant number of dust grains and scratches are present on the scale, but they do not affect the ability of the system to extract the position from the image.
The nanoGPS patterns are pseudoperiodic. Figure 2(a) is the representation of a pseudoperiodic cell of a nanoGPS scale or tag. An L-corner mark (labeled 'A') on the figure is used to represent the origin and the orientation of the elementary cell. The orientations of the L-corners are the same for all the cells over the whole scale, and they create a global (X, Y) axiscoordinate system that is locally readable. The 'D' zone of a cell encodes the (X, Y) position of its L-corner apex. Two 'B' zones follow a regular chessboard pattern and are used to achieve the subpixel accuracy of the positional determination. The 'C' zone encodes the tag model information, which gives access to the dimension of the unit cell in µm. has the format of a microscopy slide and is convenient for investigating the accuracy of microscopy stages [34]; it is made of float glass. The highest accuracy over large distances is obtained with quartz scales, because of the very low expansion coefficient of quartz. The scales can be accommodated to a variety of imaging systems. As an example, the scale at the bottom right corner of figure 3 has larger patterns, making it suitable to be read with a conventional camera at about 1 m distance. Other scales have been made on conductive silicon wafers, so that they can be imaged with SEM, and have been successfully used to investigate the accuracy of moving stages implemented in a commercial SEM [35].

Multimodal nanoGPS navYX tags
While the patterned scales described above are excellent solutions for high-accuracy automated position reading with an OM for a given magnification, it can easily be anticipated that imaging the same scale as that shown in figure 2 but with a much higher magnification would yield a fraction of the pseudoperiod to small to be deciphered into a position. A much lower magnification would cause the small squares to be poorly resolved, and also render position reading ineffective.
We designed multiscale multimodal tags [36] that can be read with a large range of magnifications and with different microscopes, and in particular OM and SEM. Such tags are depicted in figure 4. The tags have different zones, with the same kind of patterns that have been presented in the former paragraph in each zone. However, the pattern sizes differ between the zones. Zone information is included in every nanoGPS pseudoperiodic cell as a code in the 'C' subperiod pictured in figure 2(b). As a result, any image taken of the tag and containing at least one non-cropped pseudoperiodic cell contains all the necessary information required to determine its position and orientation in the reference frame of the tag. This task has been programmed into the nanoGPS software. The tag coordinate system has been defined so that its origin is in one corner, and its axes are parallel to the sides.
The tags are made of silicon with suitable doping so that charges can be evacuated on an SEM, with metal patterns that provide high contrast both when observed with an OM under episcopic illumination and with an SEM. The metal layer is about 140 nm thick, which also provides good contrast for AFM.
Two different models of tags are pictured in figure 4. The different zones have been designed to be suitable for automated position determination using images taken with a full set of microscopy objectives (in the case of OM) or at different magnifications (in the case of SEM). We also included patterns used for other purposes on the tags, such as visual checking, or model identification. Figures 4(e) and (f) show that the nano-GPS tags exhibit a clear contrast between the white and black squares when observed with an OM. In a later section of this paper, figure 7 will provide similar evidence for SEM observations.
We gave this technology for correlative imaging the name of nanoGPS navYX, to differentiate it from the position encoders and precision checkers that are designated as nano-GPS OxyO.

Coordinate systems and coordinate transformations
An OM with a sample has three relevant coordinate systems, as sketched in figure 5: (a) the sample coordinate system, realized by the nanoGPS tag; (b) the stage coordinate system, and (c) the image coordinate system.
In the case of an OM, the image-coordinate system is related to the camera and has different origins associated with different objective lenses. This is because imperfections of turret centering create some measurable position offsets between the observation centers of the different objectives. In the case of an SEM, the image coordinate system is determined by the origin of the beam and the scanning directions.
Even though cameras that are attached to OMs are approximately aligned along the stage axes, ignoring the angle between camera eigenaxes and stage axes may lead to large errors when moving over large distances. Manual alignment between the stage and the camera can barely achieve an error as low as 1 mrad, but this is enough to generate a 10 µm error for 10 mm of travel. Therefore, our approach includes the determination of the angle between the stage and camera coordinates by the nanoGPS navYX software. This operation needs to be done only once, at installation. It consists of visiting three non-aligned points on the nano-GPS tag, taking pictures at these points, converting them into coordinates, and recording the corresponding stage coordinates. Algebraic mathematics then leads to the determination of all the necessary coordinate transformations. It is then possible for the system to visit other points of the tag in an automated way to increase accuracy. This 'three-point calibration' only needs to be done once, at installation (or when a significant shock to the camera or the system creates a change in the orientation of the camera with respect to the stage).
Calibrating the offsets between the objective lenses is a two-point calibration: one image is taken with the reference objective lens used for the three-point calibration of a suitable zone of the tag; then, the nanoGPS navYX software instructs the stage to move to another coordinate, suitable for the magnification of the other objective lens; the user switches to the other objective lens, and an image taken by the nanoGPS navYX software allows the determination of the offset of the other lens. Another advantage of objective lens calibration is to provide a very accurate value for the magnification, expressed in µm per pixel, for each objective lens.
Initiating an observation session using a microscope requires a single-point calibration: moving the stage to observe an appropriate zone of the tag and taking a picture. The nanoGPS navYX software interprets the image and processes the stage position information into a coordinate transform between the stage coordinates and the sample coordinates, valid as long as the sample is not detached from the stage.
In summary, after installation, any observation session can be initiated using a single-point calibration using the objective lens of interest, which makes the process very simple and straightforward. After this single-point calibration, all observations performed at a known stage position are stored in the sample coordinate system, and all previously recorded POIs are expressed in the stage coordinate system, to make it easy for the user to revisit them.

Microscopes
We performed relocalization and colocalization experiments with different microscopes: (a) an OM (Olympus BX41, equipped with a Marzhauser Scan+ translation stage); (b) two Raman microscopes (LabRAM HR, from HORIBA Scientific), operated either in optical microscopy mode or in photoluminescence mapping mode, or Raman mapping mode; (c) two scanning electron microscopes (Gemini 2 and VP55, by Zeiss), and (d) a laser confocal microscope (Olympus OLS5000).

Sample holders
Observing the same sample with both an SEM and a Raman microscope can be very useful, as the two modalities provide complementary information: high-resolution structural and atomic information for the SEM and molecular information for Raman. However, the sample holders that are commonly used in the SEM community and the Raman community are quite different: stubs or metal disks are used for SEM, and microscopy slides or flat samples for Raman. Therefore, we developed a stub holder for OM that can conveniently be used to observe samples placed on an SEM stub with a Raman microscope, as shown in figure 6.

Identification of the various sources of errors in coordinate transfer
A coordinate transfer system for sample relocalization involves at least three coordinate systems, as shown in figure 5: stage, camera, and tagged sample coordinate systems. The accuracy of the coordinate transfer depends on the accuracy and stability of each coordinate system. In particular, we evidenced in a previous work [35] that the accuracy of the translation stage can be critical. The stability of the microscope frame affects the stability of the camera coordinate system, and it is critical [29] on some instruments. Switching objectives on the microscope turret and maneuvering the focusing mechanism of an OM are operations with a certain hysteresis that can affect the stability of the camera's coordinate system. It is also well known that temperature stability and low vibration levels are key ingredients for achieving precise and accurate positioning.
The following paragraph will provide a detailed assessment of the precision and accuracy of the nanoGPS tags. As a reminder of the difference between the two notions: precision refers to the repeatability of the measurements, and accuracy to their correctness.

Assessment of the nanoGPS tag's positional precision and accuracy
The accuracy of the nanoGPS tags can be affected by the manufacturing accuracy of the tags, by the capability of the nano-GPS software and by the performance of the imaging system used to take pictures of the tags. These different sources of error will be reviewed.
The registration accuracy of the patterns on the nanoGPS tags is determined by the technology that is employed to manufacture them. This technology is documented to have a registration precision (measured as root-mean-square (RMS) error) of less than 5 nm for a 5 × 4 mm 2 tag.
The positional accuracy and precision of the nanoGPS readings may be affected by local defects and imperfections in the patterns, and by imperfections of the position-reading algorithm. To investigate the precision and accuracy associated with the software, we chose to take an image and crop the initial image into a rectangle of fixed width. By moving the cropping rectangle pixel by pixel in the horizontal direction, we obtain a series of images that is equivalent to sample translation by the size of a pixel in the image plane. We applied this procedure both to perfect images and to images obtained by OM and SEM. The translation has been carried out within a range of ±100 µm for 10× magnification and ±15 µm for 100× magnification.
The procedure and the initial images obtained with the light and electronic microscopes are shown in figure 7.
The nanoGPS positional errors (in Cartesian coordinates and orientation) are presented in figure 8. Table 1 summarizes the precision (determined as RMS noise) and accuracy (determined as half maximum error over the investigated range).
Last but not least, some errors may arise from the imaging system, introducing image distortion. We investigated the effect of a small focusing imperfection. This can be relevant because it is frequently the case that the best focus of an OM is slightly different at the center compared to the sides of the sample, as shown in figure 9. Pictures of the tag were recorded without moving the sample, but with different adjustments of the focus to obtain perfect focus in the center or at the sides of the sample. It can be seen from figure 10 that this affects the reading of the position (X, Y) and the angle. For 10× magnification, the position (X, Y) is affected by up to 200 nm, while it is affected by less than 50 nm at 100× magnification. This can be understood by considering the focusing mechanism to be a translation that is not perfectly normal to the tagging plane, and, as a consequence, movement in the Z plane also introduces a shift in the XY plane. The impact of focusing on the determination of angle is as low as 80 µrad with the 10× objective lens but can be as high as a few mrad for the 50× or 100× objective lenses.
Another experiment was carried out to check the consistency of the angular reading. The stage was translated to several tag positions, including zones that were read with different objective lenses. The value of the orientation angle measured in all zones was expected to be the same, except for focusing errors and for the yaw angle of the stage, which may not behave as a perfect translation stage. Figures 10(c) and (d) show that the angular errors are within the range of values that can be attributed to focusing error.
The experiments reported in table 1 and figure 10 lead to the following conclusions: (a) the best calibration conditions for obtaining high-accuracy angular determination are with a low magnification such as 10×, which allows the obtention of an angular accuracy well below 100 µrad. Good sample flatness and precise sample focusing are required to obtain a positional accuracy better than 20 nm; (b) calibration operations with high-magnification objective lenses provide excellent accuracy in (X, Y) and can be used very reliably, for example, to determine the decentering between different objective lenses; in this case, the angular accuracy is below 1 mrad, but it is not as accurate as the case where a 10× objective is used.

Assessment of nanoGPS angular accuracy
The angular accuracy of nanoGPS systems has been investigated elsewhere, using dedicated read heads [31]. The accuracy was found to exceed 10 µrad for a small angular range, and better than 63 µrad for a 360 • excursion. However, there are differences between the nanoGPS navYX and the nanoGPS OxyO systems investigated in previous works. The imaging and illumination optics and the sizes and manufacturing methods of the patterns are different. As a consequence, we choose to investigate the angular accuracy of nanoGPS tags read with an OM. For this purpose, we used a Newport URS50BCC motorized rotating stage. The metrological characteristics of this   Although is it one of the most precise and accurate rotating stages of its class available on the market, its accuracy is not sufficient to investigate the accuracy of nanoGPS technology in a straightforward manner. However, it is possible to rely on its excellent unidirectional repeatability to determine the accuracy of the angles obtained using nanoGPS navYX technology. The measurement strategy used to determine the angular accuracy of the nanoGPS tags read with an OM, with an accuracy that is not limited by the ±350 µrad accuracy of the rotating stage, but by its ±17 µrad uni-directional repeatability, is described below and supported by figure 11.
We fixed the Newport rotation stage to an Olympus BX41 microscope, in place of the translation stage. We attached a nanoGPS tag to a microscopy slide so that it could be easily fixed to the rotating stage, with different orientations. For a slide fixed in a given position, we instructed the stage to move to several angular positions, took pictures of the nanoGPS tag and recorded the nanoGPS angle read from the tag image. We then changed the orientation of the slide on the stage and repeated the procedure, always instructing the stage to move to the same set of angular positions. The procedure was repeated a third time, with a different orientation of the tag. The three measurement sets are designated A, B, and C. Figure 11 shows measurement sets obtained by rotating the stage between 0 • and 10 • with 1 • steps and measurement sets obtained by rotating the stage between 0 • and 90 • with 10 • steps. At the top of the figure, three cropped images at the 0 • position for runs A, B, C show that the orientation of the tag is different for each run. The middle curves in figure 11 plot the difference between the orientation angle indicated by the rotating stage, and the orientation angle read from the nano-GPS tag. This difference is less than the ±350 µrad accuracy of the stage, which means that this difference cannot be interpreted as the error of the nanoGPS reading. Indeed, this difference is quite reproducible, especially in the 10 • range rotation experiment. This can be interpreted as follows: the dispersion of the measurements at a given stage position is a combination of the repositioning error of the stage and the nanoGPS angular error. This dispersion is ±24 µrad, on average, for the 10 • range experiment and ±64 µrad for the 90 • range experiment. It can be taken as an estimate of the nanoGPS angular error.
The nanoGPS angular reading is, therefore, more accurate than the rotating stage reading and the average of the angular value determined by the nanoGPS reading can be used to determine the error of the stage reading. It appears to be consistent with the 'typical accuracy of ±175 µrad' provided by the manufacturer. An estimation of the nanoGPS angular error can be performed at each position of the rotation stage, as shown by the bottom graphs of figure 11. The estimation can be condensed into a single figure of merit by taking the RMS average of the different values. It yields a typical error of 24 µrad for the 10 • range and of 54 µrad for the 90 • range.

Examples of co-localized experiments
Several colocalized observations were carried out using nano-GPS technology. Figure 12(a) shows pollen grains imaged using both an SEM and a Raman microscope in conventional light-microscopy mode. While pollen grains all look similar, investigating the very same grains using different techniques may be a good approach to highlight possible differences between individual grains. Colocalization between the SEM and OM observations makes it possible to find each pollen grain without any ambiguity, even though the sample had different orientations in the two sets of experiments.   Figure 12(b) illustrates another interesting use case for relocalization of a sample with carbon nanotubes. The nanotubes are too thin to be visualized using conventional OM, but they can be seen with the SEM (on the figure, blue arrows are used to mark the position of the nanotube filaments, to make them easier to distinguish). Colocalization using nanoGPS makes it simple to spot the nanotubes using the SEM, and then investigate their properties by performing colocalized Raman measurements at these positions. Excellent consistency is observed between the SEM and the Raman image.
When many observations of the same sample are performed with two or more instruments, visualization of observations on a sample map can be quite convenient. Dealing with maps is precisely what geographical information software (GIS) does, so it is not a surprise that microscopists have used GIS software for the visualization of correlative data [12,37], using the GeoTIFF format [21,22]. Several groups have used a popular open-source GIS called QGIS [38] as a database system with a convenient representation to store, index, and represent correlative observations [11,21,23,39]. As an example, it is in use at NASA Jet Propulsion Laboratory to store and represent observations performed on meteorites [39]. QGIS allows the view to be zoomed in and out, visualization preferences to be set for different layers, and the insertion of annotation, as a kind of 'Google Maps' for samples. The nanoGPS software allows seamless operation with QGIS, as it creates a GeoTIFF file of each recorded observation. Figure 13 provides a good illustration of what a sample map may look like, with different zoom levels. The sample consists of a Si substrate with MoS 2 flakes grown on it. It has been attached to an SEM stub using silver lacquer, and a nanoGPS tag has been attached to the same stub using carbon tape. Figure 13(a) provides a view of the whole stub, including the nanoGPS tag, along with microscopy observations performed on a LabRAM HR Raman microscope and an SEM. The nanoGPS software creates GeoTIFF files from any recorded image and organizes them into folders related to each observation session. Folders are treated as different layers by QGIS, so it is easy to select the representation option for each layer, such as color or transparency. In figure 13, we chose to represent optical micrographs using yellow-reddish tones, and SEM micrographs in grey-bluish tones. Several screenshots were taken by navigating and zooming on the sample (figures 13(b) and (c)). One flake of interest was selected, and the LabRAM microscope was set to its wavelength-resolved photoluminescence mode (figure 13(d)). Figure 14 provides an example of a sample map created using nanoGPS software from observations performed with three different microscopes: an OM, an SEM, and a confocal laser scanning microscope. The sample is a Si sample with photoresist patterns. The performance of the nanoGPS system allows non-ambiguous relocalization of the different patterns. Employing a relocalization method based on ad hoc landmarks on this sample could be misleading; several regions of the sample carry similar patterns, which would allow excellent local image correlation, but lead to erroneous localization.

Relocalization accuracy estimator
Relocalization errors determined experimentally for several relocalization experiments have been reported on the vertical axis of figure 15. They range from less than 1 µm up to 23 µm. In all cases, the error is much smaller than the field of view of the observation, and as a consequence, it is easy for the scientist to detect such small shifts, and to refine the relocalization manually or using image correlation software to obtain a perfect coincidence of all images.
It is interesting to note that during the stage calibration using nanoGPS tags, it is possible to get an estimator for localization accuracy. During instrument calibration, the coordinates of three points are determined, both in the nanoGPS tag coordinate system, and in the stage-coordinate system. The 2 × 2 matrix that transforms the coordinates of two vectors made from these three points is expected to be a rotation, possibly composed with a symmetry. Because the coordinates are determined with a finite accuracy, this matrix is not exactly a rotation. Finding the best approximates of this matrix as a rotation matrix allows the determination of the angle between the stage and the camera's coordinate system. The difference between the experimental matrix and the best-fitting rotation matrix is a matrix that carries information about coordinate transformation error. The norm, ε, of this matrix (defined as the RMS of its four elements) is a dimensionless number. We define the estimated error on localization to be the product ε × d, where d is the distance between the center of the ROI under investigation and the session calibration point. In a relocalization experiment between microscopes 1 and 2, the relocalization error can be expected to be the quadratic sum of the individual localization errors, In order to determine whether this approach is correct, for each relocalization experiment reported figure 15, we plotted the values of ε 1 × d 1 , ε 2 × d 2 and (ε 1 × d 1 ) 2 + (ε 2 × d 2 ) 2 on the horizontal axis and the experimentally determined relocalization error on the vertical axis. The results show that the estimated relocalization error is consistent with the observations. In some cases, the estimator is somewhat optimistic (i.e. the relocalization error is larger than the estimator), at most by a factor of two. In other cases, the estimator is pessimistic (i.e. the relocalization error is smaller than the estimator). Therefore, this estimator can be very useful for indicating the magnitude of possible relocalization errors. The quantity ε, defined above, can be interpreted as the ratio of the estimated localization error to the distance. Conveniently, 10 3 ε is the estimated localization error, in µm of error, per mm of distance to the session calibration point.

Identification of relocalization error contributors
Relocalization errors are related to errors in the three coordinate systems sketched in figure 5: the translation stage, the microscope frame and the sample.

Translation-stage errors.
Translation-stage errors have been investigated using nanoGPS OxyO scales. One 25 × 25 mm 2 scale based on a high conductivity silicon substrate with metallic patterns that provides high contrast for SEM investigations has been used to investigate the positioning accuracy of two SEM stages. The stage was instructed to move to 12 sets of coordinates P 0 , … P 11 , and at each set of coordinates, an image of the corresponding zone of the nano-GPS scale was recorded and converted into coordinates (Q 0 , … Q 11 ) and orientation. This procedure was repeated a second time, yielding a second set of coordinates (Q' 0 , … Q' 11 ) in the scale coordinate system.
The error of the stage's position at a position n is defined as The error in the reproducibility of the stage at position n is defined as − −− → Q n Q ′ n . These errors are plotted as a function of the distance −−→ P 0 P n in figure 16(a). The orientation variation read on the nanoGPS scale is reported in figure 16(b). This orientation has very significant variation, which suggests that the stage has some parasitic angular movement in addition to the translation movement. This rotation is commonly referred to as 'yaw.' The same procedure was applied to investigate the performance of two OM stages using a nanoGPS quartz scale and a 10× objective lens. Their performance, in terms of positioning accuracy and yaw, is shown in figures 16(c) and (d).

Frame stability errors.
Frame stability can easily be evaluated using a nanoGPS OxyO scale. As the stability depends not only on the instrument but also on its environment, including the temperature stability of the room and possible vibrations and shocks, it needs to be evaluated in the working environment, and on a timescale appropriate to the correlative observations.
It was found that for the OMs used in the investigation, the standard deviation of the X and Y coordinates was 0.03 µm. It is negligible compared to the errors reported in figure 16.
It may be useful to mention that for one SEM, we initially encountered some spurious large errors (50-100 µm). It was found that the center of the electron beam was not fixed at a given position. The problem was solved by setting the appropriate option in the SEM software.

nanoGPS tag errors.
The precision of the nanoGPS tag reading was reported in table 1. The accuracy of the position determination is described in table 1, but with OM one should also take into account some supplementary errors related to imperfect focusing, as shown in figure 10. This supplementary error can be evaluated as being 20 nm.
The accuracy of the angular determination can be defined as the standard deviation of the results reported in figure 11 for OMs. It also includes possible errors related to focusing. The accuracy of the angular determination in SEM has been estimated to be the quadratic sum of the accuracy reported in table 1, related to the angular reading of the SEM image, and the accuracy reported in figure 11, which may be accounted for by nanoGPS tag imperfections. This is believed to be a conservative approach. The resulting uncertainties are reported in table 3.

Errors due to temperature effects.
Temperature variations during an experiment can cause drifts of the instrument frame and affect the relocalization accuracy by several µm. Different experiments carried out at different temperatures are commonly the case when work takes place using several instruments situated in different laboratories. This has an impact on the sample frame of reference. The dimensional properties of the tag are certified at 21.0 • C. When used at a different temperature, the tag's coordinate system expands according to the coefficient of thermal expansion of Si, which is 2.6 × 10 −6 K −1 . In the case of observations carried out at 25 • C, the error is 10 nm per 1 mm of distance from the origin of the tag.
In the case where the sample is made of Al or fixed to an Al stub, the sample expands with the temperature according to the coefficient of thermal expansion of Al, which is about ten times larger than that of Si. For a 4 • C temperature difference between two co-localized measurements, this creates an error of 0.1 µm per 1 mm of distance from the origin of the tag. Table 3 provides a synthesis of the accuracies of the three coordinate systems used in the coordinate transfer. It shows that the largest source of errors is due to translation stages. The relocalization accuracy is not limited by the nanoGPS navYX technology, but rather by the current stage technology.

Identification of the major sources of errors.
It is also of interest to understand the influence of position and angular accuracy on the relocalization error. The positional error that arises from an angular error is the angular   error multiplied by the distance between the session calibration point and the POI. Figure 17 represents the probable localization errors that result from both positional and angular errors of the sample and stage coordinate systems. The error associated with carrying out observations on an Al stub with a 4 • C temperature difference between instruments is also represented. It can be seen that (a) the error related to the nanoGPS reading is negligible compared to the errors introduced by the stages; (b) the good positioning capability of the SEM2 stage is impaired by the mediocre angular stability of this stage; (c) the localization errors related to ROIs that are one cm to a few cm distant from the session-calibration points are consistent with the errors observed in figure 15; (d) a big advantage of the nanoGPS technology, compared to shuttle systems, is that the tags can be put close to the ROI, since any angular error in the translation stage creates a localization error that is proportional to the distance between the calibration point and the ROI; (e) with all instruments investigated here, a relocalization accuracy of about 10 µm is most often easily achievable.

Complementarity with software techniques
The raw relocalization errors are in the range from a few µm to about 10 µm, and no orientation error is perceived visually. This is largely sufficient to avoid any ambiguity in most practical cases. However, this may not be sufficient to conduct a careful interpretation of the results, and it may not be satisfactory for the presentation of the results. Performing a visual adjustment by translating the observation layers into perfect matching is quite simple, as no rotation adjustment is needed. QGIS plugins can be used for that purpose [40].
Obtaining a perfect superposition of the different observations can also be achieved using appropriate software based on pattern matching [19].
Pattern matching software is useless by itself, unless it is supplied with images with significant spatial overlap between modalities, which nanoGPS navYX provides in a very timeefficient way. As a consequence, the two types of techniques are believed to be very complementary.

nanoGPS position determination and super-resolution
The images of a nanoGPS tag observed with a conventional 10× microscope have a resolution that is limited by Abbe's law to about 1.3 µm. Their resolution is also limited because the images are made of a finite number of pixels, with a typical pixel size in the object plane of 0.3 µm to 1 µm. The nanoGPS technology makes it possible to extract positional information at the nm level from images that have a resolution at the µm level? How is it possible to obtain a precision for a position that is three orders of magnitude better than the spatial resolution of the raw images? The different theories about superresolution provide some clues [41]. Among the different ways of achieving super-resolution, one is localization microscopy, where some a priori knowledge of the imaged object is used to determine its position. It has been shown that the position of a known object can be determined from an image of this object with an accuracy that greatly exceeds the resolution of the image. This theory has been used to create an appropriate design for the nanoGPS patterns so that it has been possible to develop software that uses a priori knowledge of the design to derive the position of the pattern with an accuracy that exceeds the resolution of the pattern image by about three orders of magnitude.
The fact that the nanoGPS technology works in the superresolution regime offers huge potential for the automation of correlative observation that conventional fiducials will not be able to match. Determining the position of fiducials is essentially limited by the optical resolution of the system, while the positional reading of nanoGPS tags is not. Even though the accuracy of correlative microscopy systems is currently limited by stage accuracy, future progress in this field will make the super-localization capability of nanoGPS tags even more valuable.

Conclusion
A new approach for carrying out correlative observations has been presented. This approach relies on attaching small tags to samples or sample holders. The so-called nanoGPS navYX tags can be imaged with different microscopes, over a wide range of magnifications. They are not dependent on particular hardware; as a consequence, they can be deployed alongside a variety of imaging instruments, by individual researchers or by instrument manufacturers. A single image of a region of the tag provides both the positional and angular information required to perform coordinate transformations between the stage's coordinate system and the sample's coordinate system.
Relocalization accuracy has been experimentally investigated. Sample maps obtained with up to three instruments at various magnifications were created. Accuracy of a few µm up to 10 µm was routinely observed, and the orientational error was negligible to the human eye. This is sufficient to remove any ambiguity in colocalized observations. Manual adjustment or pattern-matching software can then be used to display images with perfect coincidence. The various sources of relocalization errors have been reviewed and experimentally investigated. It was shown that the accuracy of moving stages was the limiting factor in all experiments. This was established using nanoGPS OxyO scales that provided a precision comparable to laser interferometers but were more convenient to use for the assessment of stage performance. The position and orientation of the nanoGPS navYX tags were determined with an accuracy that was better than 50 nm and 77 µrad and was a negligible source of error compared to the stage errors and the frame drifts. Also, an estimate of the localization accuracy related to stage positioning error was provided by the nano-GPS technology.
NanoGPS navYX is expected to save time and to open new opportunities to researchers. Spotting the same areas of interest without wasting time means more time for research. Sending samples to other laboratories to get additional information on specific zones of interest is made simpler, with no risk of confusion or misunderstanding. In a scientific project consortium, collaborating through navigable sample maps may be fruitful.
NanoGPS tools may be also useful in a laboratory operating a single instrument. Young scientists may like to share their initial observations with the principal investigator, to spot the most interesting regions. NanoGPS makes it easy to return to the selected regions to perform more elaborate observations. When instruments are shared between several users or are operated as shared facilities, observations may have to be split between different sessions: easy relocalization between sessions saves scientists' time and increases the efficiency of the laboratory. In other cases, the ability to relocate between different elaboration steps or treatments is of great value to materials science and biology.