贡献
提供了名为SFM(Surrey Face Model)的人脸模型,包含不同的分辨率层次和标注landmarks,让人脸模型的拟合更快;同时开源了一个3DMM人脸重建的代码框架,促进社区的研究。 - Surrey Face Model,包括开源非商用版本和商用版本 - eos: A lightweight header-only 3D Morphable Face Model fitting library,代码框架
数据集
使用3dMDface相机系统采集,并用Iterative Multi-resolution Dense 3D Registration (IMDR) method (Tena et al., 2006)配准及细分操作,生成不同分辨率的人脸3D模型。低分辨率网格顶点在高分辨率网格中依然存在,且序号一致。
标注
PCA结果主成分中,保留了63个网格特征向量和132个颜色特征向量,这样重建模型能保留99%的原始数据变化。特征向量和特征值都预存到数据文件中了。
论文公布了常用的landmars标注数据,代码中使用了ibug
人脸点标注。
纹理贴图
- 表示方法:isomap algorithm (Tenenbaum et al., 2000): it finds a projection from the 3D vertices to a 2D plane that preserves the geodesic distance between the mesh vertices.
- 映射算法:Rodr ́ıguez, 2007, 3D Face Modelling for 2D+3D Face Recognition. PhD thesis
具体实现
- Pose Estimation:Gold Standard Algorithm of Hartley & Zisserman (Hartley and Zisserman, 2004)
- Shape Fitting:shape-to-landmarks fitting, Aldrian, O. and Smith, W. A. P. (2013). Inverse rendering of faces with a 3D Morphable Model
解析参考
Annotations
“facial landmark detection and tracking” (Huber et al., 2016, p. 1) 🔤面部地标检测和跟踪🔤
“Surrey Face Model, a multi-resolution 3D Morphable Model” (Huber et al., 2016, p. 1) 🔤萨里面模型,一个多分辨率3D形态模型🔤
“The model contains different mesh resolution levels and landmark point annotations as well as metadata for texture remapping” (Huber et al., 2016, p. 1) 🔤该模型包含不同的网格分辨率水平和具有里程碑意义的点注释以及用于纹理重新映射的元数据🔤
“fast fitting functionality” (Huber et al., 2016, p. 1) 🔤快速拟合功能🔤
“for the community to adopt the 3D Morphable Face Model in their research” (Huber et al., 2016, p. 1) 🔤让社区在他们的研究中采用3D可变形的面部模型🔤
“spherical harmonics” (Huber et al., 2016, p. 1) 🔤球形谐波🔤
“while the BFM provides the model, they only provide fitting results for limited databases and do not provide algorithms to apply the model to novel images.” (Huber et al., 2016, p. 2) 🔤尽管BFM提供了模型,但它们仅为有限的数据库提供合适的结果,并且不提供将模型应用于新图像的算法。🔤
“First, we present the Surrey Face Model” (Huber et al., 2016, p. 2) 🔤首先,我们介绍萨里的脸模型🔤
“Second, we present a lightweight open-source Morphable Model software framework” (Huber et al., 2016, p. 2) 🔤其次,我们提出了一个轻巧的开源形式模型软件框架🔤
“Lastly, we make a low resolution shape model available together with the software” (Huber et al., 2016, p. 2) 🔤最后,我们将低分辨率形状模型与软件一起使用🔤
“where M ≤ n − 1 is the number of principal components” (Huber et al., 2016, p. 3) 🔤其中m≤n -1是主成分的数量🔤
“The Surrey Face Model is built using a number of high-resolution 3D scans that were acquired at our lab. The scans were captured using a 3dMDface2 camera system” (Huber et al., 2016, p. 3) 🔤Surrey Face模型是使用我们实验室获得的许多高分辨率3D扫描构建的。使用3DMDFACE2相机系统捕获扫描🔤
“Iterative Multi-resolution Dense 3D Registration (IMDR) method” (Huber et al., 2016, p. 3) 🔤迭代多分辨率密集3D注册(IMDR)方法🔤
“we keep 63 shape eigenvectors and 132 colour eigenvectors so that 99% of the original variation of the data is preserved” (Huber et al., 2016, p. 4) 🔤我们保留63个形状特征向量和132个颜色特征向量,以保留99%的数据变化🔤
“annotations of the most commonly used facial landmark points for all resolution levels” (Huber et al., 2016, p. 4) 🔤所有分辨率级别最常用的面部标志点的注释🔤
“manually selected landmark points on the mesh that correspond to a subset of the popular ibug facial point annotations3” (Huber et al., 2016, p. 4) 🔤在网格上手动选择的地标点,与流行的Ibug面部点注释相对应🔤
“We create such a generic representation with the isomap algorithm (Tenenbaum et al., 2000): it finds a projection from the 3D vertices to a 2D plane that preserves the geodesic distance between the mesh vertices. Our mapping is computed with the algorithm from Tena (Rodr ́ıguez, 2007).” (Huber et al., 2016, p. 5) 🔤我们使用ISOMAP算法创建了这样的通用表示(Tenenbaum等,2000):它发现了从3D顶点到2D平面的投影,该投影可保留网格顶点之间的地理距离。我们的映射使用Tena的算法(Rodŕıguez,2007年)。🔤
“We assume an affine camera model and implement the Gold Standard Algorithm of Hartley & Zisserman (Hartley and Zisserman, 2004)” (Huber et al., 2016, p. 6) 🔤我们假设一个仿射摄像头模型并实施了Hartley&Zisserman的黄金标准算法(Hartley and Zisserman,2004年)🔤
“vertex_indices are obtained with the supplied landmark annotation metadata and the mapping facilities of the library” (Huber et al., 2016, p. 7) 🔤Vertex_Indices是通过提供的具有里程碑意义的注释元数据和图书馆的映射设施获得的🔤