UNIVERSITÉ DE GENÈVE
FACULTÉ DES SCIENCES Département d’informatique Professeur José Rolim
FACULTÉ DES SCIENCES ÉCONOMIQUES ET SOCIALES Département de systèmes d’information Professeur Nadia Magnenat-Thalmann
Generating and Customizing Virtual Human Body Models
THÈSE
présentée à la Faculté des sciences de l'Université de Genève pour obtenir le grade de Docteur ès sciences, mention informatique
par
Mustafa KASAP
Malatya (Turquie) de
Thèse No 4283
GENÈVE
Atelier d’impression ReproMail 2011
A cknowledgments
First of all, I would like to thank my PhD supervisor, Professor Nadia Magnenat- Thalmann for her invaluable guidance and for having given me the opportunity to con- duct this work and learn a lot during the years at MIRALab. I am extremely grateful to her for having given to me the opportunity to work in a highly inspiring, dynamic and truly interdisciplinary working environment. In addition, I would like to thank my thesis co-director, Prof. José Rolim (University of Geneva - Switzerland), as well as the members of my jury, Prof. Noel E. O’Connor (Dublin City University - Ireland) and Prof. Franz-Erich Wolter (Leibniz University Hanover - Germany), for taking the time to evaluate my thesis work and for their very insightful comments and suggestions.
The work presented here has also benefited of many friendly collaborative discussions and work, with several MIRALab members, so thanks to all past and present staff of the lab. Taking advantage of all the software possibilities and expressing my work in terms of tremendous fashion design creations many thanks go to Nedjma, Marlene and Alessandro; to Ugo and Bart for the numerous and heterogeneous research ideas debated in our common office in Battelle. I also would like to thank Maher, Jérôme, Lazhari, Niels and Yeonha for their kindness and scientific collaboration.
I would like to sincerely thanks to my mother, father and my sisters for their support, trust and love despite the distance over the years. Finally, a huge thank you goes to my wonderful wife for her encouragement, patience and love.
A bstract
The generation of realistic or accurate 3-dimensional human body models is generally a challenging task. It involves a variety of different steps and a combination of various tools along a specific authoring pipeline. This thesis focuses on the current methodolo- gies and tools that are part of this pipeline in various contexts of application. These methodologies and tools are the result of decades of research into the generation of hu- man body models. This thesis identifies the strengths and analyzes the weaknesses of existing approaches and proposes improvements as well as new solutions.
Taking our authoring pipeline as the guiderail throughout this thesis, we analyze alter- natives, identify specific research problems and provide solutions. The pipeline ranges from capturing human bodies using 3D human body scanners, all the way to final body texturing, personalization and animation.
We start with 3d human body scanners as our point of focus. These scanners have become an important tool in authoring synthetic characters. In this pipeline they con- tribute with a point-cloud of scan data. These point clouds have to be post-processed to generate useful output. In order to create aesthetically pleasing, realistic body shapes, we present new algorithms to automatically process the scanner generated data. These algorithms are used in an automatic pipeline to generate a static mesh from a raw point cloud data. These static meshes can then be used to create body size variations. Among the contributions to the field, a new joint aware deformation technique makes it possible to change the size of a body model by keeping the integrity of the underlying layers such as the skeleton. Automatically segmenting the body into regions based on the underlying bones’ locations makes it possible to define precise deformation constraints.
With our deformable body model in place we subsequently focus the subject of anthro- pometry. Anthropometry surveys are an important resource for statistical analysis of a population’s features. In this thesis we also present a new concept by using anthro- pometry science to maintain consistent body size proportions during deformation. By creating a correspondence between the measurement landmarks in these surveys and the
With the pipeline in place to create realistic human body models, we follow with a focus on the personalization of the body models by using facial photographs of the recreated individuals. We introduce a face texture implantation method which uses a low resolution portrait photo to generate a full body texture for the model. This includes algorithms to keep the skin color consistent with that of the face by adapting the whole body color accordingly.
Besides the presentation of our contribution to state-of-the-art research we also present the practical application of the developed methodologies in applications such as a Virtual Try On, where synthetic characters are used to create a virtual fashion model with a morphology matching that of the user, or medical simulations where the characters are used to model the patient’s (internal) anatomy.
R esumé
La génération des modèles 3D réalistes et précis du corps humain est généralement une tâche difficile. Ce processus spécifique de modélisation nécessite de parcourir différentes étapes et de combiner divers outils. Cette thèse porte donc sur les méthodologies et les outils utilisés actuellement pour ce processus dans différentes applications. Ils résultent de dizaines d’années de recherches dans le domaine de la génération de modèle du corps humain. Cette thèse identifie les forces des approches existantes, en analyse les faiblesses et propose des améliorations ainsi que de nouvelles solutions.
Tout au long de cette thèse et en nous référant à notre processus, nous analysons les alter- natives possibles, nous identifions les problèmes de recherche spécifiques et proposons des solutions. Le processus débute par l’acquisition de modèles du corps humains en utilisant des scanners 3D et s’achève avec la génération d’un corps texturé, personnalisé et animé.
Nous nous intéressons tout d’abord aux scanners 3D du corps humains qui sont devenus l’un des outils majeurs dans la création de personnages de synthèse. Ils seront utilisés dans le processus pour produire des données sous forme de nuages de points qui se- ront traités par la suite pour les rendre utilisables. Afin d’obtenir des formes de corps réalistes, esthétiques et harmonieuses, nous avons développé de nouveaux algorithmes qui traitent automatiquement les données générées par le scanner. Ces algorithmes per- mettent de générer un maillage statique à partir des données brutes (nuages de points) qui peuvent être utilisés pour créer des corps de tailles variées. Parmi nos contributions dans ce domaine, une nouvelle technique de déformation des articulations permettant de changer la taille du modèle sans modifier les couches internes telles que le sque- lette. Le corps est segmenté automatiquement en régions selon les emplacements des os sous-jacents, ce qui permet de définir des contraintes de déformations précises.
Ce modèle de corps déformables nous permet par la suite de nous focaliser sur l’anthro- pométrie dont les relevés sont une ressource importante pour l’analyse statistique des caractéristiques d’une population. Un nouveau concept basé sur la science anthropomé-
relevés et les paramètres de déformations de notre système, nous parvenons à créer des modèles de corps humains de tailles variables.
Dans notre processus de création de personnages de synthèse réalistes, l’utilisation de photographies du visage du sujet est alors nécessaire pour personnaliser son modèle 3D.
Pour ce faire, nous proposons une méthode permettant de générer automatiquement une texture de visage à partir d’une photo « portrait » en basse résolution. L’étalonnage de la texture finale est géré par des algorithmes permettant d’uniformiser les teintes et les couleurs sur l’ensemble de la texture.
Outre notre contribution à l’état de l’art de ce domaine de recherche, nous présentons la mise en pratique de méthodologies développées dans des applications telles que le
« Virtual Try On ». Les personnages de synthèse y sont employés comme mannequins de mode virtuels ayant la même morphologie que celle de l’utilisateur, ou dans des simulations médicales pour visualiser certaines parties de l’anatomie du patient.
C ontents
Table of Content vii
List of Figures xi
1 Introduction 1
1.1 Problem Statement . . . 3
1.2 Motivation . . . 4
1.2.1 Rapid Production Chain . . . 5
1.2.2 Virtual Try On . . . 5
1.2.3 Populating Virtual Environments . . . 6
1.2.4 Ergonomics . . . 6
1.2.5 Medical . . . 7
1.3 Objectives . . . 7
1.4 Organization . . . 9
2 State of the Art 11 2.1 Introduction . . . 11
2.2 Geometric Modeling . . . 12
2.2.1 Basic Geometric Deformations . . . 13
2.2.2 Free-Form Deformation . . . 15
2.3 Physically Based Modeling . . . 23
2.4 Anatomic and Anthropometric Body Modeling Techniques . . . 26
2.5 Data Acquisition . . . 35
2.5.1 Range Data Acquisition . . . 36
2.5.2 Data Acquisition with Human Body Scanners . . . 38
2.6 Surface Representation . . . 40
2.7 Surface Reconstruction From Range Data . . . 41
2.7.1 Computational Geometry . . . 42
2.7.2 Surface Fitting . . . 42
2.7.3 Implicit Function Fitting . . . 43
3 Generating Template Body Models 45 3.1 3D Human Body Scanner . . . 46
3.1.1 3D Scanner System . . . 47
3.1.2 Scanner Generated Raw Data Structure . . . 50
3.1.3 Calibration . . . 50
3.1.4 Scan Prerequisites . . . 51
3.2 Post-processing Raw Data . . . 51
3.2.1 K-Nearest Neighbor Algorithm and Average Spacing . . . 52
3.2.2 Outlier Removal . . . 53
3.2.3 Smoothing - Noise filtering . . . 53
3.2.4 Simplification . . . 54
3.2.5 Triangulation . . . 56
3.3 Image Mapping . . . 57
3.3.1 Image Mapping and Cut Graph . . . 57
3.4 Animating the Model . . . 60
3.4.1 Skeleton Attachment . . . 65
3.4.1.1 Detecting Main Body Limbs . . . 65
3.4.1.2 Approximating the Joint Locations . . . 67
3.4.1.3 Joint Orientations and Bindposes . . . 69
3.4.2 Vertex Weights . . . 71
3.5 Discussions on Test Results . . . 71
4 Anthropometric Body Modeling 75 4.1 Overview of the Method . . . 76
4.1.1 Anthropometry . . . 76
4.1.2 Measurement Surveys . . . 77
4.2 Anthropometry Based Modeling . . . 78
4.2.1 Deformation Regions . . . 80
4.2.2 Constrained Deformation . . . 84
4.2.3 Normalized Local Coordinate . . . 86
4.2.4 Girth Deformation . . . 87
4.2.5 Length Deformation . . . 90
4.3 Skeleton Adaptation . . . 92
4.4 Body Measurement . . . 95
4.4.1 Height Measurement . . . 95
4.4.2 Girth Measurement . . . 95
4.4.3 From Scale Parameter to Measurement . . . 97
4.5 Discussions on the Result . . . 97
5 Skin Deformation and Rendering 99 5.1 Fast Data-Driven Skin Deformation . . . 99
5.1.1 Dynamic Skin Deformation . . . 101
5.1.2 Minimum Bounding Ellipse . . . 102
5.1.3 Generating the Muscle Map . . . 104
CONTENTS
5.1.4 Gait Electromyography . . . 106
5.1.5 Volume preserving displacement . . . 108
5.1.6 Applying the displacement map on the GPU . . . 109
5.2 Visually Realistic Skin Variation . . . 110
5.2.1 Creating Face Texture Variation . . . 111
5.2.1.1 Face Detection . . . 111
5.2.1.2 Facial Features Extraction . . . 112
5.2.1.3 Mapping Face Image . . . 112
5.2.1.4 Color Correction . . . 113
5.3 Discussions on the Result . . . 114
6 Application and Implementation 117 6.1 Stand Alone Body Customization Application . . . 117
6.1.1 System Overview . . . 119
6.2 MIRALab’s Virtual Try On . . . 121
7 Conclusion 125 7.1 Contribution . . . 125
7.2 Limitation and Future Work . . . 127
A Acronyms 129
B 3D Scanned Models 130
C Template Models 131
D List of Publications 132
Bibliography 135
L ist of F igures
1.1 Studies on human body proportions. . . 2
1.2 Evolution of the 3D models. . . 3
1.3 Virtual cloth simulation. . . 5
1.4 Simulation of an environment with a virtual crowd. . . 6
1.5 Ergonomics experiment in virtual environment. . . 7
1.6 3D Human body model production pipeline. . . 9
2.1 A historical summary of approaches to 3-D object representation. . . 13
2.2 Design of super-quadric primitive by spherical Cartesian product. . . 14
2.3 Visual results of global deformations on the leftmost object. . . 15
2.4 FFD based deformations. . . 16
2.5 A realistic animation of human leg with NFFD. . . 17
2.6 Surface refinement. . . 19
2.7 The complex surface is created by the refinement method. . . 19
2.8 Deformation in a different dimension. . . 20
2.9 Interactive deformation tool. . . 21
2.10 Deformation around the constraint and the resulting effect on a grid. . . 21
2.11 Wires deformation application for head model and inverse kinematics. . . . 22
2.12 Sweep surface generated by moving ellipsoids. . . 23
2.13 Template musculature mapped on the model. . . 23
2.14 Combination of skeleton and muscle images to construct 3D model. . . 24
2.15 Real like musculoskeletal model. . . 25
2.16 Dynamic skinning effects. . . 25
2.17 Human body modeling techniques. . . 26
2.18 Several anthropometric human body measurement landmarks. . . 27
2.19 Human factory system architecture. . . 28
2.20 Sample implementation of JLD operators in action. . . 28
2.21 Muscle abstraction through FFD. . . 29
2.22 Dynamic muscle effect based on FFD. . . 30
2.23 FFD application on body segment. . . 30
2.24 Slice collisions around the joint region. . . 31
2.25 Sample model hierarchy and coordinate frames of nodes. . . 32
2.26 Multi-layered modeling approach. . . 32
2.27 Parametric deformed cylinder for muscle modeling. . . 33
2.28 Muscle deformation. . . 34
2.29 Muscle modeling on upper arm. . . 35
2.30 Variational and Anthropometric modeling. . . 35
2.31 The pioneering work in 3D data acquisition. . . 36
2.32 Classification of range measuring techniques [JH00]. . . 37
2.33 From raw point cloud to textured solid model. . . 39
2.34 Anthropometric measurements on the scanned data. . . 40
3.1 Modeling stage of the pipeline. . . 45
3.2 A scan session with full body scanner system at MIRALab. . . 46
3.3 Arrangement of CCD cameras and laser sources. . . 47
3.4 Specific CCD camera arrangement to prevent the surface occlusion. . . 48
3.5 Analyzing the ribbon of laser light. . . 48
3.6 Laser triangulation. . . 49
3.7 30 images captured from different height and view angles. . . 49
3.8 Piecewise slices from a ribbon of laser light. . . 50
3.9 k-Nearest Neighbor Algorithm. . . 52
3.10 Foot model before and after deleting the outliers. . . 53
3.11 Box model before and after cleaning the noise. . . 54
3.12 Smoothing operator applied on a single vertex. . . 54
3.13 A regular cell grid for simplification. . . 55
3.14 Simplified versions of cleaned point cloud data. . . 55
3.15 Triangulation of a point cloud. . . 56
3.16 Model with and without image mapping. . . 57
3.17 Scan data texture map. . . 58
3.18 Generating texture map with ray casting. . . 59
3.19 Ray - triangle intersection and corresponding pixel on the texture map. . . 60
3.20 Animation stage of the pipeline. . . 61
3.21 Kinematic structure that is used to animate the models. . . 62
3.22 Joint transformed into its LCS. . . 63
3.23 Intermediate computation stages of left thigh verticesVLTanimation. . . 64
3.24 Dividing body into its main limbs. . . 67
3.25 Joint locations & skeleton fitting. . . 68
3.26 Constrained body proportions. . . 69
3.27 Joint orientation. . . 70
3.28 Skin deformation weight distribution. . . 71
3.29 Diffirent scanner outputs with artifacts. . . 72
3.30 Triangulation artifact on the arm pit. . . 73
4.1 Anthropometric human body measurement landmarks. . . 76
4.2 Statistical analyses of body measurements. . . 79
LIST OF FIGURES
4.3 Main body regions extracted from joint positions. . . 80
4.4 Segment hierarchy under the parent joint’s LCS and filtering withδmargin. 83 4.5 Defining arm regions. . . 84
4.6 Illustration of the customized Scodef deformation method. . . 85
4.7 Normalized unit local coordinate space on a bounding box. . . 87
4.8 Bézier curve with underlying 10th degree Bernstein polynomials. . . 88
4.9 Assigning deformation weights on the calf region. . . 89
4.10 Deforming the belly region along the vertex normals with two Bézier curves. 89 4.11 Scaling inside leg length and shifting non-deformed regions. . . 90
4.12 Affect of the posture to body height scaling. . . 91
4.13 Piecewise length deformation and shifting. . . 91
4.14 Redundency free mesh slice for girth measurement. . . 96
4.15 Different size models generated with our body deformation module. . . 98
5.1 Leg models with and without muscle simulation. . . 100
5.2 Averaged EMG activity from 18 subjects. . . 101
5.3 Slice to muscle map distance value projection. . . 102
5.4 The displacement map generation and registration. . . 103
5.5 Ellipse to muscle distance. . . 104
5.6 Muscle distance map. . . 106
5.7 Gait cycle and muscle contractions . . . 107
5.8 Dynamic half ellipsoid filter. . . 108
5.9 Dynamic skin surface deformation on the upper leg by taking into account the activity of muscles underneath. . . 110
5.10 Face image mapping to template model. . . 112
5.11 Customized template body model with different face texture and skin color.113 5.12 Creating visual variations of the template body model. . . 114
6.1 Main system architecture. . . 118
6.2 Internals of the scan data post-processing. . . 120
6.3 The VTO core libraries. . . 121
6.4 The VTO stand alone application in action. . . 123
C hapter 1
I ntroduction
...if Nature has planned the human body so that the members correspond in their proportions to its complete configuration...
Vitruvius, De Architectura: The Planning of Temples Researches on the human body features have a long history track, which dates back to
~15BC. Among them the De architecture of Marcus Vitruvius Pollio is known to be the first research work that contains information about the human body proportions (see fig. 1.1a). Ages after, based on Vitruvius’s work,Leonardo Da Vinciillustrated the perfect human form in geometrical terms in his work called The Vitruvian Man. Like Leonardo, numerous artists were inspired from Vitruvius to design the perfect body models in their arts. In 1883, Alphonse Bertillon defined the term anthropometry that is an identification system based on the physical measurements of the body parts. At the present time the ISO (International Organization for Standardization), which is a network of the national standards institutes of 158 countries, published an international standard to define the body measurements and landmarks (see fig. 1.1b). Later on, studies in this field have been used in a wide range of application areas such as ergonomics, health, industrial design, clothing design, computer games etc.
In Computer Graphics (CG), there has been also a keen interest in the study of human body design and modeling. With the significant hardware and software developments in CG, stick-like characters (see fig. 1.2a) give way to the realistic 3D ones (see fig. 1.2b).
This progression is mostly focused on achieving realism in virtual environments (VEs) to improve the visual perception. Depending on the functionality of each element in VEs, efficient shading, skinning, and motion algorithms are developed to simulate their phys- ical properties. Most of these techniques are focused on human body models, which are very important elements of these environments. The evolution in 3D graphics technolo-
gies are not only constrained by the software and hardware for better rendering but also by haptic devices, 3D scanners, motion capture devices, caves, etc. which are developed to meet the dramatically increasing requirements of the users for realism. With all these continuous developments, every day the gap between the virtual and the real worlds is getting smaller.
(a)Vitruvian Man, an early study on body proportions.
(b) Standard body measurement land- marks.
Figure 1.1: Studies on human body proportions.
Parallel to the recent developments in CG, the complexity of the human body modeling and animation methods has also been increasing. Many researches conducted in this field to find either an efficient or a better way to model human bodies without consider- ing any standards such as mesurements. Apart from the major interdisciplanary research that is conducted within the scope of medical field, anthropometry science should also be considered for a standardization and realism. The plausibility of the generated model not only depends on the algorithm used to generate a mesh but also the methods used to create the texture, shading, skinning, physical behavior etc. In this thesis, we analyzed the previous research on human body modeling, identified the problems, presented a new approach that integrates and improves the recent surface acquisition methods and presented new algorithms to create human body model variations.
1.1 Problem Statement
(a)Stick based character. (b)3D Mesh model with cloth.
Figure 1.2:Evolution of the 3D models.
1.1 Problem Statement
The human imagination is inspired by the objects in nature and has created their clones either physically or virtually. This inspiration has no borders and is also expanded to the creation of aesthetically pleasing ones. Having similar affects in CG, shape creation is achieved firstly by using the programs like analysis, statistics and secondly by manual design with the use of authoring tools. So in CG, shapes either have concise mathemati- cal formulations or are in free-form.
The creation of complex shapes like the virtual human body models has been of great interest in CG. In the early years of CG, they were approximated with geometric primi- tives like sticks, cylinders etc. and nowadays, 3D body scanners are developed to create a machine generated data. Precision of the resulting data is tremendously better than the previous methods but because of the cost of such systems, it is not practical to scan every individual. On the other hand, intensive operator effort is required to post-process the resulting data to make it usable in the VE. Moreover, the final form of the scanner generated model requires skinning, shading etc. similar operations to render it alive in the VE. Because the scanner generated data is in a free-form format, it is also not possible to parametrically create different size models using the existing ones.
In the past, example based modeling approaches were used to deform the free-form data in order to succeed the body size variations. These researches were based on tem- plate models where the manually pre-defined feature points, properties etc. were used
to apply the appropriate deformation. The research outlined in this thesis examines the problems in realistic body model generation for real-time applications within an automatic pipeline. In contrast to the previous approaches, we adapted the free-form deformation methods to be applied on the scan data by means of anthropometric con- straints. To do so, an automatic post-processing pipeline is implemented to take raw scan data and generate a size parametric, animatable avatars. In this way we reduce the time consumed in model post-processing and designing stages that require massive amount of skilled user interaction.
1.2 Motivation
The pursuit of realism in computer graphics revolutionizes the evolution of the graph- ical systems for a better simulation of the objects in nature. Graphic cards, which are the main components of the recent graphical systems, have dedicated processors called Graphics Processing Unit (GPU). Within a single GPU clock unit, GPU can process a massive data in a parallel manner. This feature makes it possible to generate heavy computations with a real-time efficiency that was not possible before with commodity desktop computers. Considering the modeling of an animating virtual human character, hundreds of dynamic point coordinates are computed and per-point shading, texturing operations at each frame of the simulation are applied to achieve realism. With these new developments, it is now possible to consider a broader range of application fields and simulation techniques.
There still exist hard questions about the consciousness of the human being and es- pecially the level of the sensitivity of our perception which is very high to the known objects but without a clear explanation. i.e. it is difficult to explain what is missing or what is wrong with the virtual character in contrast to the real ones. Is it just that the emotions are missing? or interaction, skin wrinkles etc.? To generate visually realistic human body models, certain criteria need to be fulfilled. Apart from the realism, the effi- ciency of the method is also important for its usability in real-time applications. Having access to all the new technological developments in MIRALab, our main interest lies in the methods to achieve the mentioned demands and the orchestration of these methods to generate an automatic post-processing pipeline. This pipeline starts with a 3D body scan session and ends with an animated, parametric virtual human body model that can be used especially in VTO like applications where the user can customize his/her own avatar and try different clothes or accessories over it. We detailed a sample VTO
1.2 Motivation
application in section 6.2 where the customized human body models are used.
1.2.1 Rapid Production Chain
Because of its complexity, modeling a human body is one of the most difficult tasks en- countered in CAD applications. Most of the CG application fields, i.e. games, medical simulation, ergonomic design etc., require models that can rapidly change posture, size, material property etc. To meet such requirements; along with the static mesh, corre- sponding animation data must also be present for further pose and motion adaptation etc. Also the additional implementations on the internal (i.e. muscle) and the external (i.e. skin) material properties of the model improve the realism. To achieve a rapid pro- duction chain, the same models can be parametrically deformed to generate variations without performing the same effort from scratch. This production pipeline which is as automatic as possible, ends with a powerful file format that is capable of holding all model properties to be shared in other authoring tools.
Figure 1.3:Virtual cloth simulation, By courtesy of MIRALab.
1.2.2 Virtual Try On
Anthropometry, the science of the body measurements is well studied both in medicine and textile and now being adapted into the CG by means of VTO applications. Espe- cially in the clothing sector, anthropometry researches focused on the methods to define body measurements that are sufficient enough to reconstruct a similar body silhouette from the measured one. Similarly in VTO, using the anthropometric measurements of a subject, it is expected to generate similar size virtual representation of the subject and try different types of clothes over the virtual one while it is animating (see fig. 1.3). VTO applications are still evolving even though a few simplified applications exist. A true
VTO [MTKV+10] should let the user specify its measurements, so that a realistic avatar can be generated. Also, the avatar should be animatable so that the worn garment can be seen in motion. Moreover, using a virtual clone of a subject that is generated by a 3D body scanner will also add further realism to the simulations.
1.2.3 Populating Virtual Environments
Let alone the complexity of a single human body model, new CG applications utilize a variety of sizes and shapes of human body models to increase the realism of the VEs (see fig. 1.4). These computationally complex and expensive methods require better hardware, especially high performance computers. With the invention of the GPU, prob- lems related to most of these performance issues are solved or new solution approaches are proposed. Because of the new processor architecture of the GPU, a single program instruction can be executed on a massive data at a time, so it provides an efficient compu- tation environment. This new hardware architecture also requires a new programming paradigm like in parallel computing. So the trend is to use more hardware resources with new architectures and increase the visual realism of the VEs.
Figure 1.4: Simulation of an environment with a virtual crowd, By courtesy of VRLab & MI- RALab.
1.2.4 Ergonomics
Ergonomics is the subject that has been tackled by many industrial fields to design a better product to fit the user. In most cases, the user of the product is a human being and the product related user scenarios, tests etc. are virtually performed by CAD programs.
i.e. the automobile sector uses this technique for testing the conformity and security of their automobiles. One of our goals is to design a parametric virtual model that can change the size and posture of a virtual human body model (VHBM) with high level parameters. It is difficult with the current methods to rapidly generate a variety of models that can change posture and size with high level industry standard parameters.
1.3 Objectives
Especially deforming the VHBMs generated by 3D body scanners require complex post- processing stages to be used in CG.
Figure 1.5:Ergonomics experiment in virtual environment.
1.2.5 Medical
Virtual surgery, transplant, aesthetics, motion simulations etc. are all based on the VHBMs that are mainly generated by 3D human body scanners. Increasing demand for a high quality medical service also increases the popularity of the use of CG in the medical field. A precise surface acquisition method is the essential requirement for this applica- tion field since precision errors in the medical applications are not tolerated. Our goal is to develop an automatic method for a precise model surface construction.
1.3 Objectives
Since the introduction of the CG systems, there has been a big demand for realistic vir- tual body models. Recent scanner technologies allow the acquisition of precise body surface structure, but starting from a scan session, the procedure required to get a para- metric animating VHBM is very complex. Considering these constraints, the objective of this thesis is to develop an efficient method to create and customize animating hu- man body models based on standardized parameters. There is a great potential in using new acquisition techniques which are evolving with low cost counterparts. But existing methods are lack of standardization, dealing with a specific part of the above mentioned goal or not considering the benefit of new acquisition techniques. The main goals of this thesis are outlined as follows:
Design a controllable, efficient and constrained deformation method:
To develop a parametric VHBM for interactive applications such as VTO, there is a need for an efficient free-form deformation (FFD) method. Constraints on the real-time effi-
ciency limits the deformation methods to be selected under the geometric deformation techniques and not the physics based ones. Additionally, we are looking for an elegant way to unify the anthropometry with deformation. FFD methods which are constrained under a surface region are extended to deform a specific part of the model with the guidance of anthropometric measurements.
Define an anthropometry based method for localized deformations:
It is highly desirable to define parameters based on standard guidelines that are accepted by many application fields. This process is accomplished by using the International Stan- dardization Organization (ISO) guidelines to define body regions on a VHBM. Our goal is to adapt this research into CG by means of VHBMs.
Design a file structure to represent a VHBM with its all properties:
Several file formats are developed for an efficient and complete data representation.
Most of them are product specific and not in a human readable form. On the other hand there is a great effort to develop a common file format that is accepted by major industry leaders. COLLADA is one of the most popular file format that is still under development by an independent consortium. The main reason for our preference for this file format is its extendibility feature. Within a single file, it is possible to store animation like features along with the mesh data or refer to a data that is stored in another file. Our goal is to extend the file format to also store the parameters that are used to deform the VHBM.
Create a method to process scanner generated data with minimal user interaction:
Data structure of a 3D scanner output is a point cloud and does not contain proximity information. The resulting massive data typically contains noise and outliers which have to be removed to get a smooth model view. This process is called post-processing and the result is generally a single mesh. Several approximation and projection methods are used to process the point cloud data to construct a mesh. Our goal is to develop an efficient method to generate a parametric VHBM from a scan data.
Design a system architecture to orchestrate the subgoals:
The clothing industry is an important sector and the automatization methods imple-
1.4 Organization
mented for this sector have been of great interest for European projects. Parallel to these research efforts, there is a big demand in VTO applications to have the customers to get their millimeter precise animated virtual clones being utilized in VTO. Develeped methods with these requirements should be integrated with possible automatization to eliminate the user interaction for model customization.
1.4 Organization
Starting with an overview of human body development techniques, in the following sections, the stages to develop a parametric human body model is explained. Orches- tration of existing techniques and our body sizing technique are represented as a single production pipeline. Overview of this pipeline is represented in figure 1.6.
3D Human Body Model Production Acquisition Animation Sizing
Calibration Scan Session Post-Processing
Skeleton Attch. Bind Poses Skinning
Segmentation Measurements Deformation
Figure 1.6:3D Human body model production pipeline.
Stages in the pipeline are categorized under the following three main headings:
Acquisition: Using 3D human body scanner, a static 3D human body model is generated.
This stage consists of three sub stages. The first substage is the calibration of the scanner to make a robust scanning of the subjects. The second substage is the scanning session where the raw point-cloud data is generated. The last substage is the post-processing of the raw data to generate a static mesh by triangulation and texturing methods.
Animation: Even with the shaders that improve the visual realism, without animation the 3D static models do not give the feeling of realism. This stage of the pipeline covers the methods to automatically attach a virtual skeleton on the model, assign deformation weights for animation and compute the bind pose matrices to be able to apply motion data.
Sizing: The last stage of the pipeline parameterize the model to generate body size ranges. The model is first segmented into vertex groups to make local deformations.
To generate a user specified body size, landmarks are defined to make tape measure- ments over the model. Finally body segment specific deformation weights are defined to achieve size ranges.
The summary of this document is as follows:
– In Chapter 2, we present the relevant previous works in CG for the problem of efficient deformation, human body modeling, scanner systems and scanner generated data processing methods.
– In Chapter 3, mainly the following questions are answered: How we generate or design VHBMs? What are the internal stages of these processes? What should be considered to have a proper data and representation structure?
– In Chapter 4, we describe the anthropometric body modeling techniques. We intro- duce a novel technique that is used in CG for parametric VHBM description and deformation.
– In Chapter 5, we focus on realistic body rendering techniques, GPU based improve- ments for efficient visualization and deformation.
– In Chapter 6, we conclude by summarizing our contributions to the field with current limitations and the future work to continue for further improvements.
C hapter 2
S tate of the A rt
2.1 Introduction
Over the last three decades there have been significant hardware and software develop- ments in computer graphics. This progression has mainly focused on achieving realism in virtual environments. Depending on the functionality of each element in virtual en- vironments, efficient shading, skinning, and motion algorithms have been developed to simulate their physical properties. Most of these techniques focused on human body models, which are very important elements of these environments. There are various stages of making virtual bodies, including designing the mesh clone and post stages like shading and skinning. Completing these stages requires a lot of design time but play- ing these realistic virtual characters in a virtual environment also requires computation power. Due to a bottleneck in the computation power of the PC, it may be necessary to sacrifice the production quality of one or more of the virtual body making stages in order to have an efficient visualization.
One of the most recent techniques for body model generation is to use 3D body scanners to acquire a virtual clone of the original model. This technique can capture data with a resolution of a few millimeters and the final output of the scanner is a single static mesh with a few million vertices. In environments such as the gaming and virtual worlds, it is not possible or efficient to use such massive data since it requires powerful hardware for processing. For the sake of efficiency, it is clearly evident that most of the applications sacrifice realism to achieve the computationally optimal representation of the virtual body by reducing the acquired polygon number. A cheaper alternative approach for the 3D data acquisition method is to use a set of 2D images taken from different camera views. Special algorithms and heuristics are used to combine these
images to generate the 3D resemblance of the model. However, the output quality of this approach is not comparable to the ones generated with 3D body scanners. This method requires professional user interaction and the results are generally not visually satisfactory.
Another way to generate body models is to design them from scratch by means of 3D design tools. When using either data acquisition or designing methods to make the bodies, the resulting production is just a static mesh if the subsequent design stages are not applied. To add further realism, these models are animated by applying walk, grasp etc. motions and then shaders are applied to simulate the skin surface reflectance behavior under different light conditions. The animation of a model is achieved by attaching model specific skinning information on the mesh. Once the skinning process is completed, then it is possible to animate the model with the existing motion capture data. When generating different sizes and types of body models there are two options;
body databases are constructed or different variations are created from a single template model.
Body modeling approaches can be classified under two main headings, structural and mathematical (fig. 2.17). In a structural approach either anthropometry or anatomy knowledge is used to design model variations. In a mathematical approach, physical or geometrical methods are used to parametrically represent the skin surface or the underlying layer by polygons, metaballs or soft objects. In most cases, combinations of these methods are used to generate the final model. In the design stage, the body is given a generic shape by using the anthropometric constraints such as the ratio of the body, the height to the limbs or the upper arm length to the lower arm length etc. When the generic model has been designed, anatomical constraints are used to give detailed surface representation like muscle bulging or fatness. As an example, while the belly, breast or muscles are modeled by geometry based methods, a finite element method (FEM) is applied to the initial shape to simulate the physical behavior in motion.
In the following, we will describe the basics of body modeling approaches using the related state of the art concepts. We conclude with the details of a recent body modeling method, using 3D body scanners which are also described.
2.2 Geometric Modeling
Modeling real objects in virtual environments is one of the main research topics in com- puter graphics. Starting with 2D in the early 1950s, we can now model very complex
2.2 Geometric Modeling
objects in 3D together with their material properties in many environmental conditions.
In the past, CAD/CAM programs used geometric primitives to model the human body.
Due to computation constraints, geometric objects like cylinders and spheres are used to represent the body limbs in simulation applications. The main application area was the ergonomic conformity test, which is a way to visualize the human body inside a vehicle under various conditions. One of the few survey articles that traces the history of the field was presented (fig. 2.1) by Requicha [RV82] in 1982.
Figure 2.1:A historical summary of approaches to 3-D object representation [RV82].
To execute performance sensitive applications with commodity computer hardware, it is necessary to balance the thresholds between all the techniques to be used in the final application. For instance, modeling animated virtual bodies together with physically accurate garments requires the integration of appropriate modeling techniques. Since the body shape details are hidden under garments but the rough body contours affect the garment’s physical behavior, using a geometric model of the body with physical garment modeling techniques gives an optimal performance.
2.2.1 Basic Geometric Deformations
One of the earliest works that may be considered as a milestone in the geometric model- ing field is represented by Barr [Bar81]. During that period, simple geometric primitives were used in model design applications to construct the desired models. Barr intro- duced super-quadric primitives and angle preserving transformations over the existing modeling techniques, making it possible to design complex models.
Given the following two dimensional two curves,
s(ω) =
⎡
⎢
⎢⎢
⎢⎣ s1(ω) s2(ω)
⎤
⎥
⎥⎥
⎥⎦
,ω0≤ω≤ω1 (2.1)
and
t(η) =
⎡⎢
⎢⎢
⎢⎣ t1(η) t2(η)
⎤⎥
⎥⎥
⎥⎦
,η0≤η≤η1 (2.2)
Barr defines a surfacex(η,ω)by a Spherical Cartesian product of these two curves,
x(η,ω) =
⎡
⎢
⎢⎢
⎢⎢
⎢
⎢⎢
⎣
t1(η) s1(ω) t1(η) s2(ω) t2(η)
⎤
⎥
⎥⎥
⎥⎥
⎥
⎥⎥
⎦
(2.3)
where ω,η are latitude and longitude parameters of the surface. Using a few sinuses, cosines and exponential functions, he modeled super-quadric primitives that were used to construct complex solids. Aside from defining these primitives, he also implemented invertible angle preserving transformations to bend and twist them in three dimensions.
These transformation methods simplify the calculation of the new tangent and normal vectors of the surface. In figure 2.2, we demonstrate a sample Cartesian product oper- ation to generate a 3D model. A 2D star shape changes its size in the z coordinate to generate the final model.
Figure 2.2:Design of super-quadric primitive by spherical Cartesian product.
After his successive work on super-quadrics, Barr developed hierarchical solid modeling operations [Bar84] that simulate twisting, bending and tapering like transformations on geometric objects. The definition of these transformation operations mainly focused on the underlying objects’ tangent and normal vector space deformations. Barr provides the general description of the deformation with the following method: If the Jacobean
2.2 Geometric Modeling
matrix of the deformation function is known, then use it to find the tangent vectors of the original model. Next, apply a tangent transformation on these vectors to find the tangent vectors of the deformed model. Finally, find the deformed model’s position vectors with the resulting tangent vectors. Later Barr’s definitions inspired researchers to animate human body models by bending the arms and legs and fattening or slimming a specific part of the body by deformation.
Figure 2.3:Visual results of global deformations on the leftmost object.
Barr’s transformation functions are used to achieve axial deformations where only one coordinate of the shape’s vertices is operated by the function. Blanc generalizes these functions to achieve global deformations [Bla94], where each vertex of the model in Euclidian space is mapped to another vertex in the same space. Implemented defor- mation functions are; pinch, taper, mould, twist, shear and bend. The results of these deformation techniques after applied on a rectangular stick like object are represented in figure 2.3.
2.2.2 Free-Form Deformation
Based on Barr’s work, Sederberg and Parry proposed the Free-Form Deformation (FFD) technique [SP86] which is commonly used and has many extensions to solve specific problems related to deformation. In this technique, a grid of arranged point set is em- bedded on a set of complex objects that will be deformed. Instead of displacing the surface vertices of the objects, a few sets of control points over the grid are displaced.
According to the deformation generated on the grid structure, underlying objects will follow similar deformation behavior. Sample deformation stages of this technique are illustrated in figure 2.4.
(a)Bending a cube and underlying objects. (b)Modeling a handset.
Figure 2.4:FFD based deformations.
In the figure 2.4a, a set of original geometric models enclosed with a cube and the defor- mation of these models along with the enclosed cube is illustrated. Sederberg and Parry demonstrated a possible application of the FFD method by modeling a handset from a stick. Mathematically the FFD method is based on tri-variate Bernstein polynomial. As it is represented in figure 2.4b, the model that will be deformed is enclosed by a grid of control points. Regarding Sederberg’s definition for this method, each pointXis defined by the parameters (s,t,u) in its local coordinate frame. In the global form, a point on the model is defined byX=X0+sS+tT+uUwhere,X0is the reference point andS,T,U are the space vectors. If the enclosing control point grid has l+1,m+1,n+1 layers and each control point is represented by Pijk in the space then the deformed point’s position will be:
Xd=
l
∑
i=0
( l i
)(1−s)l−isi(
m
∑
j=0
( m
j
)(1−t)m−jtj(
n
∑
k=0
( n k
)(1−u)n−kukPijk)) (2.4)
Later on, Coquillart extended the FFD proposed by Sederberg which is called EFFD [Coq90].
This new method uses non-parallelepipedical 3D lattice that allows arbitrary deforma- tions since standard FFD allows deformations under cubic structure. i.e. one cannot generate circular bump with standard FFD. Coquillart describes the EFFD method with the following four steps:
1. Create the EFFD lattice where control point positions not constrained to be parallel.
2. Map the EFFD lattice with the model surface or the whole model.
3. Fix the control point positions.
4. Deform the surface.
Kalra et. al. [KMTT92] presented an extension that offers one more defree of freedom to manipulate the deformations. This freedom is achieved by changing the contributions
2.2 Geometric Modeling
of control points’ weights on the deformation. To make the deformation more flexible, Lamousin [LJ94] logically extends the FFD by mapping it on a non-uniform rational B- Splines (NURBS) complex instead of a regular grid. NFFD offers much more control over the model, while this was not achieved in the prior FFD implementations. Lamousin demonstrates an interesting application of the NFFD with a human leg model as an input (fig. 2.5).
Figure 2.5:A realistic animation of human leg with NFFD.
Lamousin defines the prior FFD implementation steps with the following items:
1. Enclosing volume generation: A lattice with a set of control vertices and the corre- sponding basis functions are generated.
2. Model embedding: Enclosing the model that will be deformed with the previously generated lattice. One to one correspondence of model vertices and the control vertices are determined.
3. Deformation: Enclosing volume is deformed.
4. Update model: Control vertices positions are used to reflect the deformation on the lattice to the model. Here one to one correspondences are used to update the model vertices.
Unlike the previous FFD implementations, non-uniform lattice of control vertices are generated. Parts of the model that require detailed deformation contain more control vertices and the rest contain less. Rational B-Spline basis function knot vectors are as- signed to each non-uniform control vertices. The result is a more flexible deformation where more than one deformation lattice can be integrated. Noble [NC99] used the Lamousin’s similar NFFD method to apply on generic deformation without considering a specific local detail. Only the main difference in his method is the direct manipulation of the weights of the control points of the NURBS.
While the FFD extensions consider overcoming the limitations of the initial FFD imple-
mentation, Moccozet [MT97] developed a new approach by reconsidering the geometric foundations of the initial one. His motivation came from hand modeling which con- sisted of complex internal layer structure. His main goal was to achieve the realistic animation of the multi layered hand model. He observed that the main problem related with the initial and extended FFDs was the way of representing the object’s points in a control box. For him using rectangular local coordinates to define a point of a model was the main limitation. Instead of a local coordinate system, more generalized Sibson coordinates are used. The location of an arbitrary point inside the convex hall of a given set of points can be expressed by the linear combination of the Delaunay neighbors. It is called Sibson coordinates representation. Suppose a Dirichlet diagram D1 is associated with a given set of points and a Dirichlet diagramD2 is associated with the same points plus the arbitrary point P. The Dirichlet tile T1 of D1 overlaps with the Dirichlet tile T2 of D2 from a point Q where both tiles also overlap with P. P’s Sibson coordinate relative toQis given by the ratio ofT1 andT2. This shows that the Delaunay neighbors can be used as Bezier simplex’s support point. In this way control point set locations are represented with another approach which provides much more freedom.
Forsey [FB88] comes with a refinement approach for manipulating the surface of the models. The refinement process makes it possible to have more control on the surface deformation since it increases the number of control points of the B-Spline surface. Based on this approach, two surface deformation techniques are introduced. The first one is a direct modification of the control vertices of the surface; the other one is called "offset referencing" where the overlaying control vertices are used for refinement.
Forsey used the following terminology for the refinement process:
S(u,v) = ∑
i
∑
j
Vi,jBi,k(u)Bj,l(v) (2.5)
Here S(u,v) represents the surface to be modified,Vi,j are the control vertices, Bi,k(u), Bj,l(v) are the basis functions of order k and l. To have more control on this surface, Forsey redefined the basis functions in a piecewise manner as:
Bi,k(u) = ∑
r
αi,k(r)Nr,k(u) Bj,l(u) = ∑
s
αj,l(s)Ns,l(v) (2.6)
2.2 Geometric Modeling
And the original surface is redefined with more control points by the following formula:
S(u,v) = ∑
r
∑
s
Wr,sNr,k(u)Ns,l(v), (2.7) where
Wr,s= ∑
i
∑
s
αi,k(r)αj,l(s)Vi,j,
Here Vi,j are the control vertices that independently influence a large surface segment for deformation. After a redefinition of the surface,Wr,s become the new control points that are more numerous than the original ones.
(a) A cross section of original surface with control points
(b)Refined version of 2.6a. (c)Modified surface.
Figure 2.6:Surface refinement. By courtesy of David R. Forsey.
Figure 2.6a represents the cross section of a surface with the control points in the corre- sponding section. Figure 2.6b has more control points because of the refinement. Fig- ure 2.6c represents the deformed version of the surface which cannot be achieved with low number of control points. Also in figure 2.7, the resulting 3D model with several intermediate deformation stages is represented.
Figure 2.7: The complex surface is created by the refinement method. By courtesy of David R.
Forsey.
Another interactive deformation technique which has a complete theoretical presenta- tion with mathematical proofs is presented by Borrel [BR94]. In this approach the de-
formation is considered with two step calculations. Firstly, the product of polynomial function f in Rn into Rm is constructed. Here m>n, where production is done from a lower space to a higher one. Secondly, a different linear projection is applied to succeed the deformation back from Rm intoRn. During this projection, a transformation matrix M is used to generate appropriate deformations.
Curve in higher dimension
Projections in lower dimensions Initial curve
Deformed Projection 2 Deformed
Projection 1
Figure 2.8:Deformation in a different dimension.
Figure 2.8 presents the geometric interpretation of a deformation from 2 to 3 dimensions and backward. In the figure, the initial two dimensional curve is first transformed into the third dimension as a 3D object. Then a different transformation matrix Mis used to project the 3D object back into 2D with its deformed versions.
Based on FFD, Decaudin [Dec96] introduced a more interactive technique for modeling shapes from a simple geometric primitive like ellipse. In this technique a deformation tool composed of a convex shape is located in the space. Using this tool it is possible to produce bump or dent effects on any primitive. This tool can be visualized as a balloon which is inflating inside or outside of the model for giving a bump effect (see fig. 2.9).
The main disadvantage is the amount of user interaction required for the modeling.
Most of these deformation techniques have an unconstrained effect on the applied sur- face. A new synthesis over the existing FFD techniques is to use Borrel’s [BR94] con- strained deformation method. The controlled spatial deformation feature of this method is the main difference over the others. Defining a local constraint point which is re- lated with a B-Spline basis function, the desired deformation around the predetermined region is achieved. The constraint point is the center of this predetermined region.
In figure 2.10, C1 is the center of the constraint and the circle is the constrained region.
F1 is an arbitrary deformation function where both ends of the function evolve to zero.
The deformation function is defined in [−1, 1] domain and max(F1(Q)) = 1. Borrel
2.2 Geometric Modeling
(a)From an ellipse to a cat. (b)Deformation in and outside the model.
Figure 2.9:Interactive deformation tool. By courtesy of Philippe Decaudin.
Figure 2.10: Deformation around the constraint and the resulting effect on a grid.
defines the constrained deformation with the following formula:
d(Q) =
r
∑
i=1Mifi(Q) (2.8)
Here r is the number of constraints, Q is vertex as column order vector, d(Q) is the displacement of Q, Mi is the scale factor as column order vector, and finally fi is the normalized contribution value ofith constraint.
Using this approach, one can define small regions with more control and smoothly displace the vertices under the region. During the displacement operation, it is possible to use different types of functions like B-Splines to generate complex deformation within a region.
Another type of deformation technique where the result looks similar to the constrained deformation is called "the wires". Singh [SF98] inspired by sculptors, used wire curves to give a main deformation schema to model like armatures. Along with domain curves, wires are used to manipulate an implicit volume. The main advantage of the wire deformation comes from its independence from the complexity of the underlying object.
Once it is defined for the base object, then it is appropriate to use it multiple times with varying objects. An application domain example of this technique is represented
by modeling a human head. The inverse kinematics technique is used to simulate the highly flexible skeletons for wrinkle generation over the face.
The following tuple is used to define a wire deformation: ⟨W,R,s,r,f⟩where, W,R : Free-form parametric curves. W wire curve,Rreference curve
r : Offset surface distant from a curve s : Scale factor
f : Density function to calculate the object vertices within an offset volume
Once the wire deformation schema is defined with the above definition, it is possible to make global or local deformations on the model by changing the curve parameters.
This is best presented in figure 2.11 with local deformations on the face and global deformations on the arm.
Figure 2.11: Wires deformation application for head model and inverse kinematics. By courtesy of Karan Singh and Eugene Fiume, Autodesk Inc.
Another approach for deformation is applying the sweep based method on a model [HYKJ03]. In this approach for example a limb is approximated by sweep based ellip- soid which changes its size as it moves through the limb. Also the ellipsoid changes its orientation through the joint locations. During this sweep motion, all modified ellip- soids are interpolated to fit in the original model. The resulting approximated model is processed with a displacement map to reflect the original shape because of its smooth behavior. Stages of this process are presented in figure 2.12.
Hyun [HYKJ03] extended his sweep based approach for body deformation by adding GPU assisted collision detection for limbs during the deformation. Given a polygonal mesh approximated by control sweep surfaces, according to the joint angle changes, sweep surfaces deformed and overlapping parts are blended. Some anatomical features like elbow-protrusion, skin folding etc. are emulated in the GPU.
A recent geometric muscle deformation technique is presented by Pratscher [PCLS05].
The main idea of this technique is to use multi-shell structured ellipsoids to produce
2.3 Physically Based Modeling
Figure 2.12:Sweep surface generated by moving ellipsoids [HYKJ03].
visually realistic models as in the figure 2.13. Each shell has its own level of hardness for deforming the attached skin. Using a number of heuristics, the body mesh is partitioned into segments to determine the location of the muscles. The developed system is capable of customizing the muscle connections, size etc. and those parameters saved under a musculoskeletal template, then, can be applied on different bodies. Muscle mapping on the body with single or multiple pose is also considered.
Figure 2.13:Template musculature mapped on the model [PCLS05].
2.3 Physically Based Modeling
Increasing the precision of the simulation results in more realistic models. However simulating the details of the skin surface, the dynamic muscle bulging or the fat tissue’s behavior is computationally very complex or not realistic with geometric modeling tech- niques. Whenever such details are important in a body model generation, the inevitable choice is to use the physics based techniques. However, one must decide upon the performance requirements before switching between physical or geometric modeling techniques since physical methods require much more computation power.
The flesh layer of the model mainly requires special modeling techniques to simulate re-
alistic behavior. Animated models represent the most noticeable flesh deformation since they consist of complicated muscle and fat tissue. Under different conditions like bend- ing the arms, running with a big belly or activities that make the muscles bulge, sim- ulation becomes impossible with the geometric methods. One of the earlier researches about the simulation of physical behavior in computer graphics was implemented by Terzopoulos et. al. [TPBF87]. They modeled materials such as rubber and cloth with the use of an elasticity theory. Later, this method was used in body modeling with differ- ent approaches. Some researchers [TSB+05] implemented the muscle layer with physics based methods and updated the skin surface mesh according to the deformations gen- erated by the underlying layers. Yet, integrating both geometric and physical methods produces a simulation with optimal performance. In addition Gourret et. al. [GMTT89]
used the FEM for modeling both elastically and plastically deformations of objects. Their numerical method which is based on FEM let it possible to take into account the active forces of the fingers on the object and the reactive forces of the object on the fingers.
Figure 2.14: Combination of skeleton and muscle images to construct 3D model [NT98].
As an example, a physics based body modeling approach is to divide the model into three anthropometric layers: skeleton, muscle and skin. The main objective is to imple- ment an articulated skeleton, physical muscle deformation and a parametric surface for the skin. As an example of this approach, Nedel [NT98] used the physical deformation process to model the anatomic muscle layer. To represent the efficiency of the imple- mentation, one of the most complex muscles is modeled and deformed by a mass-spring model. Considering two attachment points of the muscles to bones, named origin and insertion, the line between these two endpoints is called the action line. This line is used to mechanically quantify the muscle forces in action. The design of the muscle model is manually generated by using a collection of artistic anatomy pictures and cor- responding action lines, as shown in figure 2.14. The designed muscle mesh requires a post processing stage to regularize the vertices in the direction of the corresponding action line. Once the muscle model is generated interactively, the skin layer is modeled.
Using the ray-casting method, the skin surface is sampled where the sampled points are
2.3 Physically Based Modeling
directly used as a control point of the spline curves. Using this method, each slice is combined to construct the skin surface.
Another approach for physics based muscle modeling is to use tetrahedrons to fill the anatomic body parts. Once this implementation succeeds, the generated model behaves as a flesh body part. Using this method, each tetrahedron can simulate different phys- ical properties to construct a heterogeneous model. As an example, Teran [TSB+05]
segmented the visible human MRI dataset to extract the real muscle shape format which will be the base envelope for the mentioned model. Once the muscle shape constructed, it is filled with tetrahedrons to apply physical behavior by means of FEM. Because the tendons and the muscle belly behave differently in their density, different parts of the model are simulated with appropriate tetrahedron material properties. The result of the simulations became much more realistic (fig.2.15) than the previously mentioned meth- ods. Without considering the modeling of skin surface etc. at all, just using this method requires powerful computation.
Figure 2.15:Real like musculoskeletal model. By courtesy of Joseph M. Teran.
The application of physical phenomena to improve the flesh parts of the body is not only constrained to the muscles. Larboulette [LCA05] proposed a fast and simple technique to enhance the standard character skinning method by adding dynamic skin effects (see fig. 2.16) through the underlying skeleton. A method called rigid skinning is applied on the existing skinning information without modifying its kinematic deformations or other post processed data. The efficiency of this technique comes from its real-time applicability of the viscoelastic properties on the body parts.
Figure 2.16: Dynamic skinning effects. By courtesy of Caroline Larboulette.
2.4 Anatomic and Anthropometric Body Modeling Techniques
Early research on human body modeling started in the fifties. In those days, it was important for many organizations that produced aircraft, automobiles etc., due to re- quirements to find the human body size ranges that fitted into their products. Later in the early eighties, many computer aided design programs were developed to accelerate this process. One of the surveys in this field was prepared by Dooley [Doo82]. Later on with the recent technological developments in the entertainment sector, computer games became one of the main fields using virtual body models. Recently, textile, medical, and sports sectors use these virtual models for their product simulation.
Physical Modeling Geometric Modeling
Anatomic Modeling Anthropometric Modeling Body Modeling
Mathematical modeling Structural modeling
Figure 2.17:Human body modeling techniques.
In the previous section, geometric and physical modeling techniques are mentioned with example application areas related to the body modeling field. As it is represented in figure 2.17, high level modeling approaches are considered over these low level mod- eling techniques. Generally, these anatomic and anthropometric modeling approaches are used together or separately to achieve realism depending on the application area of the model. In the medical research field, realistic body models are simulated with physical modeling techniques. In such applications that require accurate models, an anatomical approach with purely physical modeling produces better results. In case of computational resource constraints, a geometric modeling with anthropometric ap- proach is preferred. Because of the limitation in ordinary computer hardware, it is very important to find the best load balancing in the target application.
Anthropometry is the science of body measurement to analyze the human physical vari- ations. Specific parts of the limbs are determined to be measured in length, girth, angle
