• Aucun résultat trouvé

A million pixels, a million polygons: which is heavier? François X. Sillion

N/A
N/A
Protected

Academic year: 2022

Partager "A million pixels, a million polygons: which is heavier? François X. Sillion"

Copied!
46
0
0

Texte intégral

(1)

HAL Id: inria-00510091

https://hal.inria.fr/inria-00510091

Submitted on 17 Aug 2010

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub-

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non,

A million pixels, a million polygons: which is heavier?

François X. Sillion

To cite this version:

François X. Sillion. A million pixels, a million polygons: which is heavier?. Eurographics ’97, Sep 1997, Budapest, Hungary. Blackwell, 1997. �inria-00510091�

(2)

A million pixels, a million polygons.

A million pixels, a million polygons.

Which is heavier?

Which is heavier?

François X. Sillion François X. Sillion

iMAGIS*

iMAGIS*

Grenoble, France Grenoble, France

*A joint research project of CNRS, INRIA, INPG and UJF

*A joint research project of CNRS, INRIA, INPG and UJF

MAGIS MAGIS

(3)

Why this question?

Why this question?

Evolution of processing power and Evolution of processing power and

architectures architectures

New applications, demands and markets New applications, demands and markets

Giant databases (digital mock up)Giant databases (digital mock up) virtual reality, games...virtual reality, games...

Image-based graphics:

Image-based graphics:

current state and trendscurrent state and trends

(4)

MAGIS 1997 MAGIS 1997

A million polygons

A million polygons

(5)

Who needs a million polygons?

Who needs a million polygons?

Assemblies of CAD models Assemblies of CAD models

Integrated design/manufacturing Integrated design/manufacturing

Digital mock-ups Digital mock-ups

(6)

MAGIS 1997 MAGIS 1997

A million pixels

A million pixels

(7)

Rendering in Computer Graphics Rendering in Computer Graphics

Models for 3D geometry, light reflection Models for 3D geometry, light reflection

Global illumination simulation Global illumination simulation

Real-time rendering Real-time rendering

All of these requirements All of these requirements present difficult challenges ! present difficult challenges !

(8)

MAGIS 1997 MAGIS 1997

Subtle illumination effects

Subtle illumination effects

(9)

Real-time rendering for dynamic scenes

Real-time rendering for dynamic scenes

(10)

MAGIS 1997 MAGIS 1997

Image-based rendering (IBR) Image-based rendering (IBR)

Avoid expensive/difficult 3D model Avoid expensive/difficult 3D model

Start from a set of images Start from a set of images

Manipulate pixels to create new image Manipulate pixels to create new image

With real images, elaborate lighting With real images, elaborate lighting

effects are “free”

effects are “free”

QuicktimeVR [Chen95], [Laveau], QuicktimeVR [Chen95], [Laveau],

[McMillan95,97],...

[McMillan95,97],...

(11)

What's an image?

What's an image?

array of RGB (

array of RGB (αα) samples) samples

add depth sample add depth sample

add multiple depths, normals...

add multiple depths, normals...

(Layered Depth Image, LDI) (Layered Depth Image, LDI)

(12)

MAGIS 1997 MAGIS 1997

Tour into the picture [Horry 97]

Tour into the picture [Horry 97]

Use a single image Use a single image

Manually define simple perspective Manually define simple perspective

Manually create layers with selected Manually create layers with selected

portions of the image portions of the image

See http://www-syntim.inria.fr/~horry/images/s97slide.html

(13)

Layered depth images [Gortler97]

Layered depth images [Gortler97]

Gather multiple depth Gather multiple depth samples for each pixel samples for each pixel

See http://www.research.microsoft.com/research/graphics/cohen/SIG_97_IBR/index.htm

(14)

MAGIS 1997 MAGIS 1997

Layered depth images Layered depth images

Reproject all samples in Reproject all samples in

new image new image

no need for depth no need for depth comparisons

comparisons

splatting techniquesplatting technique

(15)

Rendering from a million polygons?

Rendering from a million polygons?

Transform 1-3M Transform 1-3M

vertices vertices

Lighting Lighting

Texturing Texturing

Memory bandwidth Memory bandwidth

Raster engine, Raster engine,

z-buffering z-buffering

20 M flop 20 M flop 10 M flop 10 M flop 15 M flop 15 M flop 100 Mb 100 Mb

??

(16)

MAGIS 1997 MAGIS 1997

Rendering from a million pixels?

Rendering from a million pixels?

Transform 1M Transform 1M

points (coherence) points (coherence)

No lighting No lighting

No z-buffering No z-buffering

Memory bandwith Memory bandwith

(coherent access) (coherent access)

6 M flop 6 M flop

8 Mb8 Mb

(17)

Rendering performance considerations Rendering performance considerations

3D rendering reaches the consumer market 3D rendering reaches the consumer market

thousands of lit,textured polygons / second.thousands of lit,textured polygons / second.

specialized boards require careful design for specialized boards require careful design for efficient integration.

efficient integration.

Image processing subsystems Image processing subsystems

video (analog/digital), video (analog/digital), texture (games),texture (games),

multimedia extensions multimedia extensions

(18)

MAGIS 1997 MAGIS 1997

Generating and obtaining IBR models Generating and obtaining IBR models

From synthetic images From synthetic images

Ray tracingRay tracing

Range images, LDIs, LumigraphsRange images, LDIs, Lumigraphs

From real images From real images

use panoramic views, vision techniquesuse panoramic views, vision techniques feature matching (difficult)feature matching (difficult)

Lumigraphs (no depth)Lumigraphs (no depth)

(19)

Link with vision Link with vision

Image based modeling (IBM...) Image based modeling (IBM...)

Use images + parameters Use images + parameters

avoid WYSIAYGavoid WYSIAYG

object class informationobject class information

interactive modeling (facade)interactive modeling (facade)

(20)

MAGIS 1997 MAGIS 1997

IBR = sampling + reconstruction IBR = sampling + reconstruction

Operate without geometry Operate without geometry

More complete representations

More complete representations (higher dimensionality)

(higher dimensionality)

Simplified representations Simplified representations

(adding simplified 3D model) (adding simplified 3D model)

(21)

Light field - Lumigraph Light field - Lumigraph

Levoy, Hanrahan 96 Levoy, Hanrahan 96 Gortler et al 96

Gortler et al 96

(22)

MAGIS 1997 MAGIS 1997

Light field - Lumigraph sampling Light field - Lumigraph sampling

Slide used with permission (M. Levoy, Pat Hanrahan)

(23)

Impostors Impostors

Create textured 3D model from images Create textured 3D model from images

simplifiedsimplified representation representation rendered as 3D geometryrendered as 3D geometry

Planar polygons

Planar polygons [Maciel95, Schaufler96, Shade96][Maciel95, Schaufler96, Shade96]

3D meshes from range images 3D meshes from range images

[Pulli 97, Darsa 97, Sillion 97]

[Pulli 97, Darsa 97, Sillion 97]

(24)

MAGIS 1997 MAGIS 1997

Textured 3D mesh from a range image Textured 3D mesh from a range image

Pulli 97 Pulli 97

Slide used with permission (K. Pulli et al.)

(25)

Blending required to combine views Blending required to combine views

without blending without blending

(z-buffer) (z-buffer)

with blending with blending

[Pulli 97]

[Pulli 97]

(26)

MAGIS 1997 MAGIS 1997

Principles of our approach: example

Principles of our approach: example

(27)

Local model (3D objects)

Local model (3D objects)

(28)

MAGIS 1997 MAGIS 1997

Distant model (3D objects)

Distant model (3D objects)

(29)

Impostor (Textured 3D mesh)

Impostor (Textured 3D mesh)

(30)

MAGIS 1997 MAGIS 1997

Combined model (local+impostor)

Combined model (local+impostor)

(31)

Combined model (local+impostor)

Combined model (local+impostor)

(32)

MAGIS 1997 MAGIS 1997

Deforming impostors Deforming impostors

Talisman

Talisman [Torborg 96][Torborg 96]

Render spritesRender sprites Layered modelLayered model Affine transformsAffine transforms

[Lengyel97]

[Lengyel97]

Impostor transition Impostor transition

Slide used with permission (J. Lengyel et al, Microsoft research.) See http://research.microsoft.com/~jedl

(33)

Applications for IBR Applications for IBR

Walkthrough / view synthesis Walkthrough / view synthesis

Stereo synthesis Stereo synthesis

Interpolation/extrapolation Interpolation/extrapolation

Latency compensationLatency compensation Frame rate equalizationFrame rate equalization Network transmissionNetwork transmission

Leverage expensive renderingLeverage expensive rendering

(34)

MAGIS 1997 MAGIS 1997

Polygons Pixels Polygons Pixels

Continuous Continuous

Modeling Modeling

Animation Animation

Level of detail Level of detail

Discrete Discrete Capture Capture

Video streams Video streams Filtering

Filtering

(35)

Pixels Pixels

Discrete, regular nature Discrete, regular nature

easy to filter: adaptation to user easy to filter: adaptation to user perceptual limitations

perceptual limitations

Work with real images Work with real images

Easy to captureEasy to capture

Let nature do the modeling/lightingLet nature do the modeling/lighting Work from existing images Work from existing images

(historical, legal, forensic applications...) (historical, legal, forensic applications...)

(36)

MAGIS 1997 MAGIS 1997

Polygons Polygons

Complete 3D model Complete 3D model

solid modelingsolid modeling

global illuminationglobal illumination

path planning, assembly checking,path planning, assembly checking, collision detection

collision detection

Common denominator for many Common denominator for many modeling systems

modeling systems

Can be simplified but it's hard to Can be simplified but it's hard to keep the model consistent

keep the model consistent

(37)

Extended notion of image-based models Extended notion of image-based models

Use Use bothboth images and 3D data images and 3D data

Combine a simplified model with images Combine a simplified model with images

model can be extracted from images or model can be extracted from images or

other information other information

(38)

MAGIS 1997 MAGIS 1997

IBR and availability of 3D models IBR and availability of 3D models

Complete 3D model Complete 3D model

IBR as graphics subsystemIBR as graphics subsystem

No 3D model No 3D model

QTVR, plenoptic renderingQTVR, plenoptic rendering The model isThe model is the image(s) the image(s)

Range data available Range data available

Scanned data is huge: need to simplifyScanned data is huge: need to simplify

(39)

Problems with current algorithms Problems with current algorithms

Holes in reconstructed images Holes in reconstructed images Image deformation (impostors) Image deformation (impostors) Volume of data

Volume of data

Sampling/filtering artifacts Sampling/filtering artifacts

(40)

MAGIS 1997 MAGIS 1997

Can we expect hardware advances?

Can we expect hardware advances?

view interpolator view interpolator

soft z-buffering and blending soft z-buffering and blending

multiple or view-dependent textures multiple or view-dependent textures

decompression decompression

memory bandwith memory bandwith

(41)

Limitations of IBR Limitations of IBR

Specularities Specularities

Lighting/geometry/reflectance changes are hard Lighting/geometry/reflectance changes are hard

Computer Vision issues: model building Computer Vision issues: model building

Images may not be available!

Images may not be available!

(42)

MAGIS 1997 MAGIS 1997

Marketability Marketability

QTVR, panoramic images QTVR, panoramic images Image-based modeling

Image-based modeling

Image-based rendering architectures Image-based rendering architectures Image caching, impostors

Image caching, impostors Network applications (QoS) Network applications (QoS) Light field

Light field

(43)

...and now?

...and now?

Simulation of global illumination Simulation of global illumination

Visibility calculations Visibility calculations

View-dependent texture mapping View-dependent texture mapping

disparity/depthdisparity/depth

specularity/shadingspecularity/shading re-lightingre-lighting

Compression of depth values Compression of depth values

(44)

MAGIS 1997 MAGIS 1997

Computer-augmented reality Computer-augmented reality

Drettakis 97 Drettakis 97

computed solution computed solution input panorama

input panorama

(45)

Conclusions Conclusions

IBR offers useful advances IBR offers useful advances

leverage cost of high-quality renderingleverage cost of high-quality rendering fast extension via specialized subsystemfast extension via specialized subsystem

Vision issues limit applicability of “pure”

Vision issues limit applicability of “pure”

IBR for real images IBR for real images

Use combined 3D models and images Use combined 3D models and images

Polygons are still useful!

Polygons are still useful!

(46)

MAGIS 1997 MAGIS 1997

Acknowledgements Acknowledgements

Yann Argotti, George Drettakis, Frédo Yann Argotti, George Drettakis, Frédo

Durand, Peter Kipfer, Céline Loscos, Durand, Peter Kipfer, Céline Loscos,

Stéphane Moreau, Cyril Soler Stéphane Moreau, Cyril Soler

Michael F. Cohen, Steven Gortler Michael F. Cohen, Steven Gortler

Marc Levoy, Pat Hanrahan, Kari Pulli, Marc Levoy, Pat Hanrahan, Kari Pulli,

Jed Lengyel, Youichi Horry, Ken-ichi Jed Lengyel, Youichi Horry, Ken-ichi

Anjyo, Kiyoshi Arai Anjyo, Kiyoshi Arai

Références

Documents relatifs

KEY WORDS: mobile mapping system, photogrammetry, multi-camera panoramic head, calibration, off-line calibration, on-line calibration, panoramic head calibration with very low

Here, we study the impact of replacing standard convolutions with equirectangular convo- lutions (EquiConvs [14]) on the task of recognizing objects. With this kind of convolutions,

Des domaines de recherche très variés paraissent s’assembler autour du renouvellement de la culture oculaire au début du XIX e : l’intensification des mobilités fait effet

We focus on this modeling step and propose an alternative solu- tion to the creation of the hair geometry: using sequences of images.. ∗ Artis is a team within the

This paper presents a novel method for computing visibility in 2.5D environments based on a novel theoretical result: the visibility from a region can be conservatively estimated

This is accomplished by synthesizing reflection profiles using a hair reflectance model, for a number of candidate directions at each pixel, and choosing the ori- entation that

In the first step, we used the k-means clustering and Horn & Schunck algorithms in order to segment the image based on color space and optical flow extraction respectively

Пример создания панорамного аэрофотоснимка на основе многополосного смешивания для набора из 32 изображений представлен на рисунке 3.. В