SCALP: Superpixels with Contour Adherence using Linear Path

(1)

SCALP: Superpixels with Contour Adherence using Linear Path

R´ emi Giraud ^1,2

remi.giraud@labri.fr

with Vinh-Thong Ta

¹

and Nicolas Papadakis

²

1

LaBRI CNRS UMR 5800 - University of Bordeaux, FRANCE PICTURA Research Group

2

IMB CNRS UMR 5251 - University of Bordeaux, FRANCE

(2)

Superpixel Context

Simple Linear Iterative Clustering

The SCALP method

Results

Conclusion

(3)

Superpixel Context

Low-level representation of an image into homogeneous areas.

Respect of image geometry (contrary to multi-resolution approach).

Properties: • Adherence to the image contours

• Homogeneity of the color clustering

• Regularity/compactness of the decomposition

1500 su p erpixels

400 superpixels 400 superpixels

R´emi Giraud (PICTURA) SCALP: Superpixels with Contour Adherence using Linear Path 3 / 16

(4)

Simple Linear Iterative Clustering (SLIC) - (Achanta, et al. 2012)

• State-of-the-art results on standard superpixel metrics

• Produces regular superpixels in limited computational time

Regular grid initialization Constrained K-means clustering

(5)

SLIC - Distance

Distance between pixel p, and average cluster C

k

of S

k

: D(p, C k ) = d c (p, C k ) ² + d s (p, C k ) ² m

Pixel p = [(l

p

, a

p

, b

p

), X

p

]

Average feature cluster C

k

= [(l

k

, a

k

, b

k

), X

k

]

d

c

color distance d

s

spatial distance

(6)

SLIC - Limitations

Parameter m globally set for the whole image

SLIC result

Difficulty to maximize all aspects:

Adherence to contours 6= Color homogeneity 6= Decompositon regularity

(7)

Superpixels with Contour Adherence using Linear Path (SCALP)

→ Enforces respect of image contours, color homogeneity and compactness at the same time

• Definition of the linear path to superpixel barycenter

• Additional features within the clustering distance

(8)

Superpixels with Contour Adherence using Linear Path (SCALP)

→ Enforces respect of image contours, color homogeneity and compactness at the same time

• Definition of the linear path to superpixel barycenter

• Additional features within the clustering distance

SCALP result

(9)

SCALP - Linear Path Definition

Path between pixel p at position X _p and superpixel S _k of barycenter X _k Near real-time computation (Bresenham, et al. 1965)

(10)

SCALP - Features on Linear Path P ^k _p

Improvement of color distance:

d

c

(p, C

k

) = (l

p

− l

C_k

)

²

+ (a

p

− a

C_k

)

²

+ (b

p

− b

C_k

)

²

→ d

c

(p, C

_k

,P

^k_p

) = λd

c

(p, C

_k

) + (1 − λ) 1

|P

^k_p

| X

q∈P^k_p

d

c

(q, C

_k

)

(11)

SCALP - Features on Linear Path P ^k _p

Improvement of color distance:

d

c

(p, C

k

) = (l

p

− l

C_k

)

²

+ (a

p

− a

C_k

)

²

+ (b

p

− b

C_k

)

²

→ d

c

(p, C

_k

,P

^k_p

) = λd

c

(p, C

_k

) + (1 − λ) 1

|P

^k_p

| X

q∈P^k_p

d

c

(q, C

_k

)

(12)

SCALP Framework

Properties: • Regularity/compactness of the decomposition

• Homogeneity of the color clustering

• Adherence to the image contours

(13)

SCALP Framework

Properties: • Regularity/compactness of the decomposition

• Homogeneity of the color clustering

• Adherence to the image contours

(14)

Superpixel Context Simple Linear Iterative Clustering The SCALP method Results Conclusion

SCALP - Features on Linear Path P ^k _p

Use of Contour prior:

d

C

(p, C

k

, P

^k_p

) = 1 + γ max

q∈P^k_p

C(q)

Improved clustering distance:

D(p, C

k

) =

d

c

(p, C

k

, P

^k_p

) + d

s

(p, C

k

)m

d

C

(p, C

k

, P

^k_p

)

(15)

SCALP - Features on Linear Path P ^k _p

Use of Contour prior:

d

C

(p, C

k

, P

^k_p

) = 1 + γ max

q∈P^k_p

C(q)

Improved clustering distance:

D(p, C

k

) =

d

c

(p, C

k

, P

^k_p

) + d

s

(p, C

k

)m

d

C

(p, C

k

, P

^k_p

)

(16)

Example Results

GC SEEDS SLIC SCALP

(Veksler, et al. 2010) (Van den Bergh, et al. 2012) (Achanta, et al. 2012)

(17)

Example Results

GC SEEDS SLIC SCALP

(Veksler, et al. 2010) (Van den Bergh, et al. 2012) (Achanta, et al. 2012)

(18)

Quantitative Results

Validation on Berkeley segmentation dataset (BSD) (Martin, et al. 2001)

200 test images with human ground truth segmentations

• Undersegmentation Error (UE): Overlap with multiple objects

• Compactness: Regularity of the produced superpixels

• F-measure: Contour detection performances (PR) (Martin, et al. 2004)

Method UE ↓ Compactness ↑ F-measure ↑

GC

(Veksler, et al. 2010)

0.161 0.520 0.596

SEEDS

(Van den Bergh, et al. 2012)

0.134 0.201 0.581

SLIC

(Achanta, et al. 2012)

0.135 0.269 0.649

SCALP 0.130 0.278 0.673

(19)

Quantitative Results (PR)

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Recall

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Precision

[F = 0.787] Human [F = 0.596] GC [F = 0.581] SEEDS [F = 0.649] SLIC [F = 0.676] SCALP

(20)

Quantitative Results (PR) - Influence of parameters

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Recall

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Precision

[F = 0.787] Human [F = 0.649] SLIC [F = 0.660] SLIC + Color [F = 0.671] SLIC + Contour

[F = 0.676] SLIC + Color + Contour = SCALP

(21)

Conclusion

SCALP method summary

• New general method to use color and contour features along linear path

• Increase of contour adherence, compactness and respect of image objects

• Limited computational time (< 0.4s)

Work in progress

• Use of advanced color features (Li et al., 2015) → State-of-the-art results

Perspectives

• Adaptation to supervoxels (3D and video)

(22)