HAL Id: hal-01883108
https://hal.archives-ouvertes.fr/hal-01883108
Submitted on 3 Oct 2018
HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
Planar roof surface segmentation using 3D vision
Philipp Meixner, Franz Leberl, Mathieu Brédif
To cite this version:
Philipp Meixner, Franz Leberl, Mathieu Brédif. Planar roof surface segmentation using 3D vision.
19th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems,
Nov 2011, Chicago, United States. pp.9 - 15, �10.1145/2093973.2093976�. �hal-01883108�
A
In r A b in m f D a p th 3 T f b e e c r c r (A
C
G
A
K
S d
1
T m 3 p e tr th r m s in th
Pla
Phi
Institute of Graz Univ
A-801 meixne
ABSTRACT
nternet search h apid emergence Additionally, ma been started to d n a transition fr modeling of buil focus has been o Detection And aerial cameras ha point clouds from
hese point cloud 3D building mod This paper prese flow for the auto built-up areas fro extruding from, o each roof is bein can then be class
oof shapes. We compete well esearchers. Our Austria) with 18
Categories a
I.4.8 Scene Ana
General Ter
Algorithms, Mea
Keywords
Semantic segme detection, smooth
1. INTROD
The 2- dimension model of the Ear 3D GIS and its planning, archit environmental s ransactions etc.
he required lev ange from the L models with roof such as windows nclude the build he virtual city w
nar Ro
lipp Meixne
Computer Gr Vision versity of Tech
10 Graz, Aust [email protected]
T
has initially bee e of 3D buildin any commercial develop urban 3D
om the classical lding roofs is th on the use of a Ranging). How as rendered poss m high overlap ds jointly with t dels.
ents a multi-step omatic segmenta om high-resoluti or intruding into ng modeled by m sified as a specif e show that the
with LiDAR-re experimental w 86 buildings.
and Subject
alysis (Object rec
rms
asurement, Exper
ntation, roofs, r hing
DUCTION
nal GIS is rapid rth and human ha
3D virtual citi tectural design, simulations, dis
Depending on t el of detail (LO Lego-type parall f shapes as LOD s and roof detai ding interior [18 will model not o
of Surf
er
raphics &
hnology tria az.at
en a strong driv ng models of la and government D geographic inf l 2D- to the nov hus a relevant re aerial LiDAR po wever, recent pr sible the acquisit digital aerial im he image inform p processing fram ation of building ion vertical aeri
, a roof are bein means of its pla fic roof type from
results from a esults as repo work employs a
t Descriptor
construction).
rimentation, Ver
roof shapes, aer
dly morphing int abitat. Typical u ies include urba
tourist and saster prepared the input data an OD) may vary w
lelepipeds denot D-2 to building m
ls as LOD-3 and 8]. In its most so only each buildi
face Se
Fra
Institute of C Graz Unive A-801
leberl@
ving force for th arge urban area tal initiatives hav formation system vel 3D-GIS. Th esearch topic. Th oint clouds (Lig rogress in digit tion of very den magery, and to u mation to genera mework and wo g roofs in dense al images. Detai g excluded so th anar segments an m a set of standa
erial photograph orted by LiDA
test area in Gr
rs
rification.
rial images, plan
o a 3-dimension use of the resultin
an and landscap leisure activitie dness, real esta nd the applicatio widely. This ma ted as LOD-1 v models with deta d on to LOD-4 ophisticated form ing, but also eac
egmenta
anz Leberl
Computer Gra Vision ersity of Tech
0 Graz, Austr
@icg.tugraz
he as.
ve ms he he ght tal nse use ate ork ely ils hat nd ard hy AR az
ne
nal ng pe es, ate on, ay via ail to m, ch
tree, st dimens doors, suspend figure 1
Figure [C Th based o offer in Digital approac data-dr forms a forms [ driven preferre planar one ro reconst density Distanc point id Th acquisi 1000 p GSD b model wires.
comput traditio acquire
ation u
aphics &
nology ria
z.at
9
treet detail, brid sions, and many
facade elemen ded wires, stree 1). LOD-2 to LO
e 1. 3D city mod Courtesy: Aerod http://ww he state of the ar on airborne LiD nstant access to Surface Mode ches for 3D bu riven. Model-dri as presented in [1], [2]. Comple methods. Theref ed [3], [4], [5], roof elements.
oof plane. A r tructed building y of 25 pts/m
2w
ce of 20cm. It q dentification of 1 he progress in d ition of very de pts/m², based on
etween 10 cm a details of facad The DSM from ted at point int onal 2-image st ed at a density of
sing 3D
Math
MATI Institute Géo Univer 94165 Saint-M
Mathieu
dge and water y details will be nts, sidewalks, m
et signs etc., a OD-4 need a mod
del, LOD2 (deta data Internation
ww.aerodata-su rt methods of int DARs (Light Det o point clouds a el (DSM). The uilding reconstru iven methods us figure 7 and m ex roofs typical fore data-driven [6], [7], where Ideally, each d recent paper [8 g models using which correspon quotes a total st
10-15cm.
digital aerial cam ense point cloud imaging with G and 3 cm. Tho
des and roofs, m aerial photog tervals of 10 to tereo overlaps.
f 1 point per pix
D Visio
hieu Brédif
S Laboratory ographique Na
rsité Paris-Es Mandé Cedex
u.bredif@ig
body is model included such a manholes, park all as separate o
del of the roof sh
ail of Leiden, Ne nal Surveys, Be urveys.com/].
terpreting buildi tection And Ran and an easy tra two fundamen uction are mode se a predefined match the data a
lly are beyond s n roof reconstruc a roof gets seg data-plane descri 8] assesses the
LiDAR. It quo nds to a Groun
andard deviation meras has made
ds at 100 pts/m Ground Samplin se densities are street signs and graphy no long o 20 pixels asso Current point xel taking advant
n
ationale t x, France
n.fr
led in three as windows, king meters, objects (see hapes.
etherlands) elgium, ing roofs are nging). They ansition to a tal LiDAR- el-driven or
list of roof gainst those such model- ction is often
mented into
ibes exactly
quality of
otes a point
nd Sampling
n for corner
possible the
m² and up to
ng Distances
desirable to
d suspended
ger is being
ociated with
clouds get
tage of a 10-
w geometry [9], ]. Such dense ing roof lines a ysis.
presents the aut built-up areas fr ns to each roof a oof to carry sola multi-step process ts of the propos ustria) with 186 ovide us with a ofs.
ch
or the proposed r 1] and its impr building [12].
c point clouds in D point clouds as uilding’s point c steps. First is a sm ation method (TG an approach -called random Finally the roof p
moothing
ange data will b opic is a very p e there exist m ain requirement es noise but pr his reason and to using the “total has the advanta
the GPU and is onal approach fi f an energy-func e regularization ut the smoothne rces the solution er variant of thi e surfaces and t oximated by piec erm is based on rm is based on T at it reduces st
er reflects the n g the Huber nor d “Huber model
ke digital or hard oom use is granted ributed for profit notice and the full
ublish, to post on cific permission an -2011, Chicago Il, ACM 1-58113-000-
and thus to a c e DSM leads and is helpful in tomatic segmen rom high-resolut a type, and it of ar panels.
sing framework sed procedure ar 6 buildings We s success assignm
roof interpretatio rovement to de
This segments nto individual pr ssociated with ea cloud has been i moothing of the GV) [13]. Then first introduce m sampling planes are input
be noisy so that rominent one in any algorithms for such a smoo reserves sharp o accelerate the p generalized var age that it ca therefore very fa inds the solution tional that is usu n term copes w ess properties of
n to be similar is functional is therefore well su ce-wise planar su n the robust Hu
TGV. The stren tair-casing often noise model of r
rm to the data a l” is obtained.
copies of all or pa d without fee prov t or commercial a
l citation on the f n servers or to re nd/or a fee.
USA.
-0/00/0010…$10.0
concept of “supe to well-define n automating an ntation of buildin tion vertical aeri ffers a measure leading to a bas re based on a te show that vertic ment to a roof typ
on is a framewo al with extrudin
the images an roperties, buildin ach building.
isolated, roofs g DSM with a tot all roof planes g ed by [14] an
and conceptu to a classificatio
t an interpretatio n photogrammet
that discuss th othing algorithm edges and sma plane detection w iation” TGV [13 an be efficient ast.
n of the model b ually composed with the a-prio f the solution an to the input dat ideally suited f uited to buildin urfaces.
uber-L1 norm, th ngth of the Hube
n found in oth real range image and regularizatio
art of this work fo ided that copies ar advantage and tha first page. To cop
edistribute to list
00.
er- ed ny ng ial of sic est cal pe
ork ng nd ng get tal get nd ual on
on try his is all we 3].
tly by of ori nd ta.
for gs he er- her
es.
on
Ω Κ
f
1… u …
α … ε,Th piecew propert optimal the TG approxi combin norm.
min,
Ap result il of Graz within structur
Figur datas shows ra
2.2 P
Th detectio
“J-Link RANSA we wil random differen way th That m the foll
Z σ x
ior
re at py s,
min
… Image doma
… observed ran
… single observ
… sought soluti
… smoothing fa
δ … Parameter ofhe main property wise polynomial
ty to be convex l. For the appro GV regularizatio imated by pie nes TGV regular
n | |
pplying this me llustrated in Fig z, Austria. In thi the range data re of a roof.
re 2: (a )Shows R set (b) shows ra
s range image u ange data (e) sh
Plane Detec
he smoothed p on. This applies kage” method AC method. The ll present a sho m sampling to
nce to RANSAC hat neighboring p means that if a po lowing probabili
… nor
… heu
,x
j… sin
| |
ain nge images
vation ion actor (α ≥ 0) f Huber norm
y of TGV this is functions of a x, which means oximation of bui
on of the secon ecewise planar rization of secon
|ε v
ethod to a typica gure 2, taken from
is image you can are minimized
RGB image of s nge image with using TGV mod hows detail of ra
ction
oint clouds are s to façades as w introduced by e approach is de ort summary h o generate mo C is that minim points are select oint x
ihas alread ity of being draw
1
0 rmalization cons uristically chose ngle observations
| |
s that it allows to arbitrary order.
that a solution ildings it is suffi nd order since t r surfaces. Th nd order with th
| |
al roof produces m the test datase n see that outlie
by maintaining
single building hout using TGV del (d) shows de ange data using
e now the inpu well as to roofs.
[14] that res scribed in [14] i ere. The proces del hypotheses mal sets are cons ted with higher dy been selected wn:
stant
n constant s
o reconstruct It has the n is globally icient to use they can be his method he Huber-L1
|
a smoothed et in the city rs and noise g the global
of our test V model (c) etail of raw g TGV.
ut to plane We use the sembles the in detail, but
ss starts by s. The big structed in a
probability.
d, that x
jhas
th w f e th m to m tw
w th li r p
F
(
2
e d p p A p th d w c s h
RANSAC i hat is in our cas with the characte fit the point. Upo each point a pre hat belong to th meaning they are A tailored a o the same mod minimum pairwi wo preference se
where A and B a he two sets and inked together epresented as s plane detection f
Figure 3: (left) of building roo
Dimen (right) 3D Point
the two major
2.2.1 Elimina
The definiti elimination of s dormer windows There are t processing step:
planes must exc Additionally we planes are linke
herefore relevan discontinuities at within a roof ca chimneys. Figur structures have b here.
Figure 4: RGB smaller
n contrast treats se undesirable. E eristic function on the creation o eference set (set he same structu e close in the con agglomerative cl del, where at ea ise distance are ets is calculated
, |
are two sets. It m reaches from 0
if their prefer small clusters. F for one building o
Axonometric vi of (GSD 10cm) ( sions: x,y-axis [ t cloud of roof h r planes, marke planes i
ation of Smal
ion of segments smaller structure s or chimneys.
two main param First, the size ceed a threshold need to conside d and really do nt segment. And t the borders of an be found pr re 4 illustrates been eliminated o
B image of build structures (dow
s all points with Each surface poi of the set of ran of all hypotheses
t of hypothesis ure have a simil nceptual space.
lustering groups ach step the two e merged. This using the Jaccar
∪ | | ∩ |
| ∪ | measures the deg
(identical sets) t rence sets over Figure 3 shows of our test datase
iew of smoothed (points are high [in pixels], z-axi highlighted in b ed in blue and t
in green.
l Structures
of roof planes es. These could meters that are i e of the individ d for the segmen er the possibility o represent part
d second the ex f a segment. The rimarily at dorm s a building r
on the basis of th
ding and buildin wnsampled by f
h the same weig int gets associate ndom models th
one computes f it prefers). Poin lar preference se s points belongin o clusters with th distance betwee rd distance:
gree of overlap to 1. Elements a rlap. Outliers a the result of th et.
d 3D point clou hlighted in red)
is [in m]
blue; overlaid ar he two smaller
is followed by a d for example b important for th dual segments
nt to be retaine y that segments s of a larger an xistence of heig ese discontinuiti mer windows an
oof where som he rules describe
ng mask withou factor 4).
ght ed hat for nts et, ng he en
of are are he
ud
; re
an be his of ed.
of nd ght ies nd me ed
ut
2.3 R
Th classifi buildin sense to in the m footprin using a to still cloud o method It buildin may in footprin 7). Com shown more th the case illustrat thus rep be det classifi misclas enhanc emergin façades descrip footprin straight
F segme 3D f Af step we to ease a build accorda require done b charact
Figure Th are par figure connec
Roof Shapes
he roof planes ication of the ng configurations o consider the b modeling of faca nt into so-called a method introdu be complex, we of a cell gets asso d introduced in [
is necessary to ng footprints. Th nclude urban L
nt is of a buildin mplexity gets in
in figure 9: a fo han 4 faces and e for L- ,T- and ted in figures 5 presented by the termined. The ication (see figu ssifications due ce our roof in
ng masonry. W s in 3D that w ption of the com
nt. Figure 5 illus t line segments.
Figure 5: (a) bu entation result ( façade reconstru
fter the determi e have to split th a further interp ding footprint int
ance with figu ement that the nu
by defining an teristics of the bu
e 6: Decomposit he cells now det rticularly interes 7 and table 1, cted as illustrated
s
are now avai roofs. The ana s can be rather c building footprin ades in [11]. On d “cells” represe uced in [3]. Shou e review the 3D p ociated with diff 15].
differentiate be he complexity o L- and T-shapes
ng consisting of ncreased if roof otprint is thus co d if it has conne
miscellaneous o 5 and 8. The co
e complexity of big problem i ure 5b) is not
to vegetation a nterpretation we We use a meth was presented in
mplex 3D faça strates how the f
uilding visible in (c) in red modif uction (d) enha ination of the bu he footprint into pretation. A meth nto mostly quadr
ure 6. The dec umber of cells b n adequate sub
uilding.
tion of building quadrilateral c termine the roof sted the standar , and their com d in figure 9.
ilable as an in alysis is compl complex. Therefo
nts, much as ha e can decompos enting simple b uld an individual
point cloud itsel fferent height cla etween simple a of buildings and s (see figure 5) f only four faces elements are co omplex when a b ected roof eleme other buildings l omplexity of the f its footprint, an
is that the use very accurate and shadows. T e have to det od to reconstru n [11]. This re ade and a refin footprint gets rep
n vertical aerial fied building ou anced building f uilding footprin mostly quadrilat hod presented in rilateral shaped composition co be a minimum.
bset that still
footprint from cells.
f shape of the b d roof shapes il mbinations when
nput into a lex because fore it makes
s been done se a building asic shapes, l cell appear lf. The point asses using a and complex d thus roofs ). A simple s (see figure onnected, as building has ents. This is like the ones e building is nd this must ed building
because of Therefore to termine the uct building esulted in a ned building presented by
l image (b) utline using
footprint.
nts in a next teral regions n [3] divides polygons in onsiders the
This can be reflects the
m figure into
building. We
llustrated in
n roofs get
rent roof shapes ng (d) half hipp saw-tooth roof ( tomatic classific oncept introduce ted with the 3D a lower, a middle
ification of a bu classe very crucial wh mansard roofs. In both the roof egments of roo ht class. The nu ses is counted p of special interes asses is relevan
s especially e es the following er of planes pitch
ation of normal v belonging to low ency matrix (plan tant parameter
important to d anes. The result o wo roofs, it is 0 i
anes face in a si ections. The us of a roof into each roof into th
(f) (b)
s (a) gable roof ed roof (e) man (h) mansard hip cation of roofs ed in [14]. It w point cloud (see e and an upper c
uilding roof into es [15].
hen dealing with n contrast to the pitch and the b of planes are a umber of points b per plane. Points
st. The ratio betw t for the assess effective with
features of a roo
vector
wer and upper cl nes)
is the orientatio determine the or of this calculatio if two planes sta
milar direction a se of these par quadrilateral c he major roof sha (g) (c)
f (b) shed roof (c nsard roof (f) fla
pped roof.
into the standa works with “heig e figure 8): in o class.
o different heigh different kinds e method used building footprin
assigned to the belonging to eac s in the upper an ween the numbe sment of the ro hip roofs. Th of:
lass and their rat
on of the norm rientation of tw on is a function and normal to eac
and -1 if they loo rameters and th cells results in apes of table 1.
(h) (d)
c) at ard ght our
ht of in nt.
eir ch nd ers oof he
tio
mal wo of ch ok he a
Plane Roof Orien Class Ratio Adja
Plane Roof Orien Class Ratio Adja
Plane Roof Orien Class Ratio Adjac Plane Roof Orien Class Ratio Adja Plane Roof Orien Class Ratio Adja
Plane Roof Orien Class Ratio Adja
Plane Roof Orien Class Ratio Adjac
Plane Roof Orien Class Ratio Adja
Table
es f Pitch
ntation ses o acency
es f Pitch
ntation ses o acency
es f Pitch ntation ses o acency
es f Pitch ntation - ses o acency
es f Pitch ntation - ses o acency
es f Pitch ntation ses o acency
M
es f Pitch ntation ses o acency
es f Pitch
ntation ses o acency
e 1. Different ro
Gable roof
2
> 0 degree (similar) 1x is -1 (dot produc all 2 planes have low same ratio 1 neighoring
Hip roof
4
> 0 degree (each 2 a 4x is 0; 2x is -1 4 planes have lower upper class smaller 3 planes are neighor
Half Hipped r
4
> 0 degree (each 2 a 4x is 0; 2x is -1 2 planes have lower 2 planes have just up upper class smaller t 3 planes are neighbo
Shed roof
1
> 0 degree plane has lower, upp same ratio -
Flat roof
1
~ 0 degree - - -
Mansard roo
4
> 0 degree (each 2 a 2x is 1; 1x is -1 2 planes have upper 2 planes have lower -
1 neighboring
Mansard Hipped
8
> 0 degree (each 4 a 4x is 1; 1x is -1; 8x 4 planes have upper 4 planes have lower points in upper class various
Sawtooth ro
several
> 0 degree (all are s always is 1 all planes have lowe same ratio various
oof shapes
f
)
t of 2 adjacent norma wer and upper class
are similar) r and upper class
than lower class ring
roof
are similar) r and upper class
pper class than lower class oring
f
per class
of
are similar) r class r class
d roof
are similar) is 0 r class r class
s are fewer than in lo
of
similar) er and upper class
al vectors)
ower class
In addition to the principal roof types of table 1, the approach also works with the connecting roof shapes of figure 9.
Figure 9. Examples for possible connecting roof shapes One significant improvement of the proposed method over [3] is the ability to deal with non-symmetric and more general roof shapes.
3. Experiments
The test area covers 400m x 400m near the core of the city of Graz with 186 different buildings. The vertical aerial photography was taken with a GSD of 10 cm and 80% forward and 60%
sideward overlaps, using the large format digital aerial camera UltraCam-X.
Ground truth of the roof shapes was collected by hand. Table 2 summarizes the types of roofs. Since some roofs fall outside the standard types, a “miscellaneous” category with 24 entries was required. An additional classification was into “simple” and
“complex” buildings, and the latter was further classified into L- buildings (corners), T-buildings and buildings with U- and other irregular shapes. The test area has 125 simple and 61 complex buildings. Of the complex buildings 25 are L-and 17 are T- buildings, and 19 fall into the miscellaneous category.
Table 2. Ground truth: types of roof shapes for the 186- building test area
The smoothing of the DSM not only eliminates outliers, but increases the throughput of plane detections by a factor two. Plane detection requires a focus on the major planes, and thus a meaningful threshold for the acceptable minimal size for plane segments. This threshold is calculated for every building depending on the size of the building footprint and the size of the single plane segments.
The 186 buildings consist of a total of 614 major roof planes. Of these, our approach detected 567 planes. Table 3 presents the detection rates at 92%. A major limitation for the plane detection presents itself when the roofs are curved or very fragmented, thus when there are no planes. The test area has four such buildings.
Down-sampling of the DSM point clouds by a factor 4 is acceptable to first suppress minuscule plane segments and to secondly accelerate the throughput. This down-sampling eliminates small structures in advance. Nonetheless even with this down-sampling not all small structures can be eliminated especially larger dormer windows. Before the roof type can be assigned one needs to eliminate the remaining small structures caused by bigger dormers and chimneys and other extrusions or intrusions. Success in defining and removing such small structures was at 83%, identifying 612 of the 738 smaller roof structures.
Table 3. Different roof shapes roof planes smaller structures total 614 738 detected 567 612 Detection rate [%] 92 83
The assignment of a standard roof type concerns 162 of the 186 buildings, since 24 fall outside the standards. If we could detect these non allocatable buildings automatically we achieve a detection rate of 88%, thus 145 buildings out of 162 were correctly classified (see table 4). If we include the 24 non- allocatable buildings we achieve 78%.
While the flat roof would seem to be the easiest type to identify, only 88% were correct. This translates to an error in 2 of 16 buildings, and it turns out that these 2 flat roofs have gardens.
The shed roofs were identified at a rate of 86% since 2 buildings were incorrectly classified due to large extrusions in the form of large dormer windows. Gable roofs were detected at a 91%
success rate. Problems again occur with large extrusions. All hip roofs were correctly detected. However, the half-hipped roofs only were correctly classified in 43% of the cases. The problem with this category derives from the two small plane segments in the upper height class. The current approach eliminates those plane segments during plane detection
The test area did not include any mansard and mansard hipped roofs. We therefore processed 10 additional buildings from another data set (Annecy, France) for each of these two types. All 5 mansard roofs were detected. Errors occurred with the mansard hipped roofs in 2 of the 5 cases. The plane segments in such roofs are at very similar pitches so that plane detection merges plane segments when they should be kept separate.
Table 4. Evaluation of roof shape detection;
4. Conclusion
We propose a method for automatically mapping roofs and classifying them into architecturally accepted standard types, based on traditional digital large format color aerial photography.
This method relies on point clouds at 25 pts/m
2extracted from highly overlapping vertical aerial imagery, and on an image classification using color and texture to delineate building outlines and footprints. Experimental work in a Graz-test area with 186 buildings with 614 roof planes, results in correct roof planes in 92% of the cases. Small roof structures do confuse the analysis and must therefore be detected and eliminated. This is successful in 83% of all the test cases. Roof types get classified correctly at a rate of 88%. LiDAR literature quotes its success with roof type assignments at xxx%. Limitations exist with complex roof shapes that include large dormers, or with curved roofs. However, those
Flat roof Shed
roof Gable roof Hip
roof Half Hipped
Roof Mansard
roof Mansard Hipped roof
Sawtooth
roof Non-
allocatable roofs
16 14 121 2 7 0 0 2 24
Roof shape Flat
roof Shed roof Gable
roof Hip roof
Half Hipped
Roof Mansard
roof
Mansard Hipped roof
Saw- tooth
roof Total Total
number
16 14 121 2 7 0 0 2
162Detected roof
shapes
14 12 112 2 3 0 0 2
145Detection
rate (%)