Reconstructing the past: methods and techniques for the digital restoration of fossils

During fossilization, the remains of extinct organisms are subjected to taphonomic and diagenetic processes. As a result, fossils show a variety of preservational artefacts, which can range from small breaks and cracks, disarticulation and fragmentation, to the loss and deformation of skeletal structures and other hard parts. Such artefacts can present a considerable problem, as the preserved morphology of fossils often forms the basis for palaeontological research. Phylogenetic and taxonomic studies, inferences on appearance, ecology and behaviour and functional analyses of fossil organisms strongly rely on morphological information. As a consequence, the restoration of fossil morphology is often a necessary prerequisite for further analyses. Facilitated by recent computational advances, virtual reconstruction and restoration techniques offer versatile tools to restore the original morphology of fossils. Different methodological steps and approaches, as well as software are outlined and reviewed here, and advantages and disadvantages are discussed. Although the complexity of the restorative processes can introduce a degree of interpretation, digitally restored fossils can provide useful morphological information and can be used to obtain functional estimates. Additionally, the digital nature of the restored models can open up possibilities for education and outreach and further research.

1. Introduction

By their very nature, fossils are usually incompletely preserved and deformed. Subject to millions of years of taphonomic and diagenetic processes, specimens often show the results of disarticulation, fragmentation, distortion and remineralization when they are discovered. In addition, excavation, collection and preparation can lead to further damage. This presents a considerable problem for the study of fossils, as information about phylogenetic relationships and taxonomic positions, but also about appearance, behaviour and ecology of extinct organisms, is often entirely inferred from the morphology of the preserved remains. Advances in computer-aided scanning and digitization techniques, digital visualization and computational analyses have created versatile new tools for the study of extinct (and extant) organisms [1]. Consequently, recent years have seen a phenomenal surge in the use of digital techniques in palaeontological research [2–4]. Nevertheless, the problem of preservation exists, which is an even more crucial one for functional analyses. X-ray computed tomography (CT) and digital visualization can facilitate the non-destructive extraction of fossil specimens from rocks [5–7] or the reconstruction of soft-tissue structures from fossils [8–11]. However, many computational and ecological morphospace analyses of fossils, such as geometric morphometrics (GMM) [12] and biomechanical modelling techniques, including finite-element analysis (FEA) [13,14], computational fluid dynamics (CFD) [15,16] and multi-body dynamic analysis [17–19], require accurate and complete morphological information. Validation studies have shown that results can vary when using these techniques [20,21] depending on the morphology of the studied specimens. Consequently, the results of biomechanical and functional studies require the restoration of the original morphology prior to fossilization (as close to the ‘in vivo’ state as possible), before the shape and function can be analysed rigorously.

The restoration of fossil material, especially of hominin crania, has a long-standing practice in archaeology and palaeoanthropology. Until recently, such restorations were performed physically using photographs, drawings and plaster models [22–24]; however, the widespread application of digital imaging in the last decade has brought about the development of different computational restoration methods (e.g. [25–27]). Fuelled by the intense scientific and popular interest in hominin ancestry, as well as forensic anthropology and craniofacial surgery, many restoration techniques are now routinely used in palaeoanthropology. In contrast and apart from two-dimensional interpretive drawings of skeletal reconstructions, which conventionally depict the fossil taxon in lateral view or as a shaded silhouette [28,29], anatomical restorations have been performed very rarely in vertebrate palaeontology. The handful of published three-dimensional skeletal restorations has been undertaken by using physical or digital models or a combination of both [30–34]. However, the criteria that have been used to restore these fossils are often unclear, as are the protocols and constraints that have been employed [35].

Digital reconstruction and restoration techniques offer a variety of approaches to restore the original morphology of a fossil, but they are often unknown to researchers or not described in the desired detail. Additionally, the increasing number of CT segmentation and digital visualization programs offers a confusing variety of suitable software tools. Here, different methodological steps for the digital restoration of fossils are described and evaluated. These single steps can be applied individually (if not all are required) or successively and are applicable to a wide range of fossil taxa. Different approaches and software tools (table 1) are outlined in this study and their respective advantages and disadvantages are discussed.

Given the popularity of digital visualization techniques, a variety of (often interchangeably used) terms exist. While opinions differ about the appropriateness of specific terms and definitions, the terminology in this study follows Lautenschlager [11] for consistency. In this context, the term digital restoration is used to describe the process of removing preservational and other artefacts to restore the morphology of a fossil specimen as prior to fossilization (as an alternative the term digital or virtual preparation has been suggested in the past [4]). By contrast, the term digital reconstruction is used here to describe the creation of structures, which are not directly preserved, for instance endocranial components (brain, inner ear, neurovascular structures).

2. Material and methods

Different specimens were used as examples in this study. These consist both of individual and articulated skeletal elements of vertebrate taxa and a strong focus has been put on the restoration of vertebrate fossils due to their complex nature. Consequently, only few examples for non-vertebrate fossils exist. However, the described methods are largely applicable to invertebrate fossils as well, although their preservation and relative abundance makes extensive restoration less necessary.

• (i) An articulated skull of the Upper Cretaceous therizinosaur Erlikosaurus andrewsi (IGM 100/111, Geological Institute of the Mongolian Academy of Sciences, Ulaanbaatar, Mongolia) [36,37] was CT scanned at X-Tek Systems Ltd (now Nikon Metrology), Tring, Hertfordshire, UK, using a XT-H-225ST CT scanner. Scan parameters were set at 180 kV and 145 µA for the complete skull. The resulting rotational projections (3000) were processed with custom build software provided by X-Tek Systems Ltd creating a VGI and a VOL file, containing 1998 slices with a resolution of 145 µm per slice. Visualization, segmentation and restoration steps were performed in AVIZO (v. 6 and 7; www.vsg3d.com).

• (ii) Disarticulated braincase elements of a subadult individual of Dysalotosaurus lettowvorbecki (MB.R.1370: laterosphenoid, prootic and opisthotic; MB.R.1372: parietal and supraoccipital; MB.R.1373: basioccipital and parabasisphenoid; MB.R.1377: left frontal; MB.R.1378: right frontal, Museum für Naturkunde, Berlin, Germany) were scanned at the Museum für Naturkunde, Berlin, using a Phoenix|X-ray Nanotom (GE Sensing and Inspection Technologies GmbH, Wunstorf, Germany) micro-CT scanner. Scan parameters were set at 90–100 kV and 90–110 µA (all scans: 1440 slices, resolution: 5–5.5 µm per slice) [38,39]. Additional surface scans of the left and right frontal (MB.R.1377 and MB.R.1378) were taken using a photogrammetry approach and 123DCATCH BETA (http://autodesk.com). Visualization, segmentation and restoration steps were performed in AVIZO (v. 6 and 7) and BLENDER (v. 2.65; www.blender.org).

• (iii) A museum-quality cast of the manual ungual of the Cretaceous therizinosaur Therizinosaurus cheloniformes [40] housed at the Sauriermuseum Aathal, Switzerland, was digitized using photogrammetry and AGISOFT PHOTOSCAN STANDARD (www.agisoft.ru). Visualization and restoration steps were performed in BLENDER (v. 2.65).

• (iv) A series of semi-articulated caudal vertebrae partially embedded in matrix of the Triassic dinosaur Pantydraco caducus (BMNH P64/1, Natural History Museum, London, UK) [41] was scanned at X-Tek Systems Ltd (now Nikon Metrology), Tring, Hertfordshire, UK, using an XT-H-225ST CT scanner. Scan parameters were set at 180 kV and 155 µA. The resulting rotational projections (3140) were processed with custom build software provided by X-Tek Systems Ltd creating a VGI and a VOL file, containing 1138 slices with a resolution of 105 µm per slice. Visualization, segmentation and restoration steps were performed in AVIZO (v. 6 and 7).

• (v) An articulated braincase of the Jurassic ornithischian dinosaur Stegosaurus stenops (NHMUK PV R36730, Natural History Museum, London, UK) [19] was CT scanned at the Natural History Museum, London, UK, using a Metris (now Nikon Metrology) HMX ST 225 CT scanner. Scan parameters were set at 220 kV and 160 mA. Scans were reconstructed in CT Pro (Nikon Metrology, UK) and exported from VG Studio Max (Volume Graphics, Heidelberg, Germany) as VOL files. Visualization, segmentation and restoration steps were performed in AVIZO (v. 8) and LANDMARK (www.idav.ucdavis.edu/research/EvoMorph).

3. Digitization

As presented here, the restorative steps require the digital data to have been collected already. Consequently, to restore the morphology of a fossil with the methods explained here, a digital model of the physical specimens is required first. Various technologies exist for the digitization of fossils, and their advantages and disadvantages are dependent on the respective specimen, access to hardware and software, and available funds. Different digitization methods are briefly outlined here. For a more detailed overview, the reader is referred to publications covering these methods in more depth [1,4,42].

X-ray CT scanning uses X-rays to non-destructively penetrate an object and to subsequently create a tomographic dataset. Although a range of different tomographic techniques exist, CT scanning has become the most commonly applied approach to digitize fossil specimens in the past decade [4]. Different types and variations of CT scanners exist, with different capabilities regarding resolution, scanning time, minimum and maximum specimen size and potential costs for acquisitions and operation [4]. MicroCT scanners, as found in many research institutions and commercial scanning facilities, provide high resolution (around 1–100 µm), but are mostly limited in terms of specimen size, which ranges usually between a few millimetres and approximately 50 cm. By comparison, medical CT scanners have a considerably lower resolution and scanning energy, but offer the advantage that they can accommodate large specimens and may even be used at no costs. On the other end of the spectrum, synchrotron CT produced by a particle accelerator provides ultra-high (sub-micrometre) resolution, but the effective specimen size can be restricted to a few centimetres.

As an alternative to tomographic methods, surface-based scanning techniques are available. Although the latter are not capable of capturing internal structures, they can provide good results for studies in which only the external morphology is required [4]. Laser scanning is one of the most commonly applied techniques for surface-based digitization. Here, the external surface of a specimen is actively probed and characterized by a laser beam emitted from the scanner. Depending on the system, laser scanners can achieve sub-millimetre resolution, but also be used for very large objects. A clear advantage compared to CT scanners lies in the fact that many laser scanners are mobile and have relatively low scanning times.

Photogrammetry provides a further digitization technique [43,44]. As this technique is based on the generation of a digital model by acquiring photographs of a specimen from different aspects and viewpoints, it is an easy-to-use and cost-effective alternative to the aforementioned methods. The resolution is largely dependent on the camera used to acquire photographs, but it is possible to digitize both small (i.e. few centimetres) and very large (i.e. several metres) specimens.

4. Restoration techniques

4.1. Removal of breaks and cracks

Fossil specimens are commonly riddled with small cracks, breaks and holes, resulting from the fossilization process, excavation, subsequent preparation and handling (figure 1a). Such imperfections in the fossil are often only superficial and do not notably affect or alter the original morphology of the specimen. Hence the removal of small cracks and breaks might appear purely cosmetic. However, even these subtle morphological alterations could lead to different (and most probably incorrect) results in computational studies (e.g. FEA, CFD). The removal of small breaks and cracks is therefore often a necessary first step in restoring fossil morphology.

Depending on which method was used for the digitization of specimens, different approaches for the removal of small breaks and cracks are available. For digital models derived from CT, the removal process is performed during the segmentation (automatic or manual tracing of components of interest based on greyscale thresholds) of the specimen (figure 1b) using specialized segmentation software such as AVIZO, MIMICS or SPIERS. General smoothing tools can be used to remove very small islands and cracks. However, this can also affect other, delicate parts of the labelled fossil. Many CT imaging and segmentation software packages, such as AVIZO or MIMICS, offer automatic algorithms to remove small holes or islands within the fossil (e.g. Segmentation→ Remove islands and Segmentation → Fill holes in the AVIZO segmentation editor). Additionally, a threshold can be defined to remove holes below a certain size (figure 1c). A further option is to artificially grow a segmented region to close small gaps. This process increases the outline of the thresholded region by one (or if necessary more) voxels (figure 1d; e.g. Selection → Grow → All slices in the AVIZO segmentation editor). It results in the closing of small breaks in the range of similar voxel size. After this step, a shrink operation can be performed to revert to the original outline (figure 1e; e.g. Selection → Shrink → All slices in the AVIZO segmentation editor). For larger cracks and breaks, it is, however, necessary to perform the removal manually. In most cases, such cracks are clearly visible in the individual CT slices, where they can be filled in by hand (figure 1f). This works best in an orientation perpendicular to the crack. As these small cracks and breaks do not obscure the overall morphology of the fossil or a specific fossil component, they can be removed by manually tracing the outline of the specific structure and thus by interpolating over the crack or break. Where possible, an automatic interpolation tool can be used, which calculates the outline over several slices between two (separated) segmented regions (e.g. Selection → Interpolate in the AVIZO segmentation editor; Curves → Interpolate over selected slices in SPIERSedit).

For digital models generated by a surface-based method (laser scanning, photogrammetry), the aforementioned approaches are often not applicable. However, owing to the functionality and the resolution of surface-based methods, small breaks and cracks are only surficial as no internal data are captured. In this case, a smoothing algorithm can remove small cracks (e.g. Surface Transforms → Smooth Surface in AVIZO; Objects → Smoothing in SPIERSview; Smooth Vertex in edit mode in BLENDER). However, as mentioned above, this includes the risk of altering the general morphology. Some software allows selecting individual elements and vertices, which can be used to smooth just the affected region (e.g. vertex or face selection tool in BLENDER). Alternatively, the surface can be remeshed or shrink-wrapped. Both approaches create a new polygon mesh configuration based on the object's morphology (e.g. Remesh modifier in BLENDER; figure 2). This requires the new mesh to be of sufficient resolution not to obliterate important details, but at the same time coarse enough to remove small breaks and cracks. It is therefore applicable to larger specimens with simple geometry.

Although a small amount of interpretation is introduced during this step, it is limited to small breaks and cracks, which do not alter the position of the skeletal elements and fragmentary components.

4.2. Reflection of elements

In addition to taphonomic and diagenetic artefacts, such as breaks and cracks, fossils are also often incompletely preserved. In particular, complex vertebrate skeletons, consisting of numerous distinct, articulated elements, are prone to disarticulation and fragmentation. A common method in palaeoanthropology [45–48] is replacing missing elements by exploiting the bilateral symmetry of vertebrates using reflection or mirroring of single elements (or articulated skeletal structures).

For single-component fossils or single (disarticulated) elements, the respective specimen can be reflected in most CT segmentation (e.g. using the flipcommand in the AVIZO crop editor) or three-dimensional modelling software (e.g. Mirror command in BLENDER) to create a mirror-image counterpart. This can be done by reflecting the original CT slices, the segmented labels or the resulting surface models. If articulated elements or larger regions of a fossil need to be replaced, the respective parts of the specimen can be mirrored across a midsagittal plane (figure 3). Although all these approaches lead to the same result, there can be reasons, when reflecting the original CT slices or segmented labels can become necessary; for example, when information from the CT data is required as a reference or to merge different label sets.

For the investigation of hominin crania, variations of reflection and mirror-imaging of elements have been applied using landmark-based algorithms: morphologically distinct points, the so-called landmarks, are selected either on only one side of the plane of symmetry (unpaired landmarks) or on corresponding points to each side (paired landmarks). Using one or a combination of both landmark types, different algorithms can be used (outlined in detail in [27]): (1) mirroring of whole regions across an empirical midsagittal plane through the use of unpaired landmarks selected on the complete or better preserved side. By using just one side, a midsagittal plane is calculated for the selected landmarks. This approach is recommended, if only one side of the fossil is well preserved, for instance, due to weathering and erosion of the exposed side. (2) Reflected relabelling of components by using a combination of unpaired and paired landmarks as reference points. For this approach, no plane of symmetry is calculated. Instead landmarks are mirrored (left and right side are swapped) and subsequently superimposed to create a final model. (3) Using thin-plate spline interpolation to warp complete landmarks onto an incomplete region. Here, paired and unpaired landmarks are used as control points to interpolate the surface of the less preserved region.

4.3. Superimposition

While reflection can be used to replace missing elements or regions in a fossil specimen, it has the disadvantage that it requires a complete source component on one side to create a mirror-image counterpart. However, fossils often show a different or unequal state of preservation on each side of the symmetry plane, and no single complete element might be preserved (figure 5a–c). Similarly, for fossils which do not have a bilaterally symmetrical counterpart (e.g. elements of the vertebral column), reflection cannot be used. In these cases, incomplete elements, which show different states of preservation—and thus retain different amounts of information contained within—can be superimposed onto each other. As a result, a composite, more complete element is created on the basis of the superimposed parts (figure 4).

Before two or more incomplete elements can be superimposed, it has to be ascertained that all fragments are in the same orientation and scaled to the same size. For bilaterally symmetrical elements, this can simply be done by reflecting one of them as outlined above (figure 4a,b). Scaling should be unnecessary if parts of the same specimen are used. However, for non-symmetrical elements or fragments obtained from different specimens, the individual parts have to be scaled to the same size (e.g. using the transform editor in AVIZO; transformation manipulator for scaling in BLENDER). The amount of scaling can be determined by the overlapping regions of all fragments. In a final step, the individual elements are superimposed by moving them onto the base element. Where necessary, the individual parts might have to be rotated (e.g. using the transform editor in AVIZO; transformation manipulator for translating and rotating in BLENDER). This step can be done manually or (semi-)automatically using landmarks [49,50] to align the fragments (see next step). Manual transformation (translation, rotation and scaling) is usually quicker and gives the user more control of the process, in particular, if the overlapping morphology is not fully identical. The superimposed parts can then be remeshed to create a single composite element (e.g. Remesh modifier in BLENDER).

4.4. Repositioning of elements

The term repositioning is used here in the context of reassembling broken or disarticulated parts of a fossil. Repositioning could be necessary in a variety of cases during the restoration process: (1) if a break has separated and dislocated two or more fragments of the same element and is too large to be filled in, as described above (figure 5a), (2) if two incomplete elements cannot be easily superimposed due to the lack of overlap or (3) if individual elements need to be articulated, for example as part of a cranial skeleton (figure 6). Manual repositioning involves moving an object (i.e. surface) in three-dimensional space relative to other components by manually altering the position and rotation of an object about the xyz-axes. It is therefore similar to the superimposition of elements (figure 4). However, in the aforementioned cases the lack of morphological guides (i.e. overlap of elements) makes the repositioning difficult, particularly if performed manually. Identifying the correct position requires identification of different constraints about multiple axes, such as articulation facets, size or break morphology. A landmark-based approach can further improve this step and increase its accuracy [27]. In this case, pairs of landmarks are selected on corresponding morphological points on each of the separated elements (figure 5b). Both landmark sets can then automatically be aligned via a best-fit approach in the CT segmentation software (e.g. Compute → Landmark surface warp for landmark objects in AVIZO; figure 5c). Given that elements can usually be repositioning in more than one way, this step introduces a larger degree of interpretation and hence uncertainty into the restoration.