Here we provide the automatic landmarks obtained by SRILF (Shape Regression with Incomplete Local Features) on the FRGC database. This algorithm works by calculating a set of candidate points for each landmark and performing combinatorial search, with the key assumption that some landmarks might be missed (i.e. no candidates detected); this is tackled by using partial subsets of landmarks and inferring those that are missing by maximizing their plausibility based on a statistical shape model. Such assumption is crucial for the generalizability of the model for live scanning scenarios, where pre-processing is not possible, or limited to a minimum extent, and the quality of the resulting surfaces can vary considerably.

The results are split into 3 sections, according to the following experiments:

We provide a number of videos demonstrating the search process of SRILF. An indicative example of the elements displayed in the videos is provided below, which are split into 3 sections:

·       Examples on FRGC v1 with 2-fold cross-validation

·       Examples on FRGC v2 with 2-fold cross-validation

·       Examples on FRGC v2 using a model trained on FRGC v1

Each video frame shows one step in the combinatorial search and contains:

·        The candidates for each landmark being targeted, with the symbols indicated in the legend on the left of the facial surface (the candidates are provided by the feature detectors).

·        The subset of candidates that is currently evaluated within the combinatorial search, which are highlighted by thick red circles. These candidates are assigned to landmarks, indicated at the bottom as “Detected”, and are used to infer the remaining landmarks, indicated as “Inferred”.

·        The shape resulting from the selected candidates plus the inference step is indicated by means of orange lines connecting different landmarks.

·        Note that “Detected” does not actually mean a correct localization of the specified landmarks. It only indicates that those landmarks have been selected based on candidates. For example, in the intermediate result displayed below we can see what happens with an inaccurate selection of candidates: we have an incorrect detection of the nose root combined with candidates for the mouth, nose and inner-eye corners from the wrong side of the face (i.e. these are clearly candidates from the right-side of the face assigned to landmarks from the left-side). As a result, the remaining candidates are inferred quite far away from their correct targets. This is easily discarded by SRILF, because all remaining candidates are also very far away from the inferred positions.

·        At the final stage, when the algorithm has reached its best solution, inferred landmarks are also indicated by thick orange circles and the overall error (with respect to manual annotations) is displayed (top-right corner of the figure).

Landmark abbreviations: n: nose root, prn: nose tip, pg = chin tip, en: inner-eye corners, ch = mouth corners, ac: nose corners, sn: subnasale, ls: upper lip, li: lower lip, ex: outer-eye corner. For landmarks with bilateral symmetry, Left and Right are also indicated.

An interesting by-product of the strategy followed by SRILF is that it can naturally handle cases with occlusions or missing data. To illustrate this, we provide a number of snapshots of automatic localization results on the Bosphorus dataset, focusing on scans that present occlusions and out-of-plane rotations.

·       Examples on the Bosphorus database with 2-fold cross-validation