Tractable Lineages on Treelike Instances: Limits and Extensions

(1)

Tractable Lineages on Treelike Instances:

Limits and Extensions

Antoine Amarilli¹, Pierre Bourhis², Pierre Senellart^1,3 June 29th, 2016

1T ´el ´ecom ParisTech

2CNRS CRIStAL

3National University of Singapore

(2)

Tuple independent databases (TID)

• Probabilistic databases:modeluncertaintyabout data

• Simplest model:tuple independent databases(TID)

• Arelational databaseI

• Aprobability valuationπmapping each fact ofIto[0,1]

• Semanticsof a TID(I, π): aprobability distributiononI⁰⊆I:

• Each factF∈Iis eitherpresentorabsentwith probabilityπ(F)

• Assumeindependenceacross facts

(3)

Example: TID

S

a a 1

b v .5 b w .2

This TID(I, π)represents the followingprobability distribution: .5×.2

S a a b v b w

.5×(1−.2) S

a a

b v

(1−.5)×.2 S

a a

b w

(1−.5)×(1−.2) S

a a

(4)

Example: TID

S

a a 1

b v .5 b w .2

This TID(I, π)represents the followingprobability distribution:

.5×.2 S a a b v b w

.5×(1−.2) S

a a

b v

(1−.5)×.2 S

a a

b w

(1−.5)×(1−.2) S

a a

(5)

Example: TID

S

a a 1

b v .5 b w .2

.5×.2 S a a b v b w

.5×(1−.2) S

a a

b v

(1−.5)×.2 S

a a

b w

(1−.5)×(1−.2) S

a a

(6)

Example: TID

S

a a 1

b v .5 b w .2

.5×.2 S a a b v b w

.5×(1−.2) S

a a

b v

(1−.5)×.2 S

a a

b w

(1−.5)×(1−.2) S

a a

(7)

Example: TID

S

a a 1

b v .5 b w .2

.5×.2 S a a b v b w

.5×(1−.2) S

a a

b v

(1−.5)×.2 S

a a

b w

(1−.5)×(1−.2) S

a a

(8)

Example: TID

S

a a 1

b v .5 b w .2

.5×.2 S a a b v b w

.5×(1−.2) S

a a

b v

(1−.5)×.2 S

a a

b w

(1−.5)×(1−.2) S

a a

(9)

Probabilistic query evaluation (PQE)

Let us fix:

• Relational signatureσ

• ClassI ofrelational instancesonσ (e.g., acyclic, treelike)

• ClassQofBoolean queries(e.g., CQs, acyclic CQs)

Probabilistic query evaluation(PQE) problem forQandI:

• Fix aqueryq∈ Q

• Read aninstanceI∈ I and aprobability valuationπ

• Compute theprobabilitythat(I, π)satisfiesq

• Data complexity:measured as a function of(I, π)

(10)

Probabilistic query evaluation (PQE)

Let us fix:

• Relational signatureσ

• ClassI ofrelational instancesonσ (e.g., acyclic, treelike)

• ClassQofBoolean queries(e.g., CQs, acyclic CQs)

Probabilistic query evaluation(PQE) problem forQandI:

• Fix aqueryq∈ Q

• Read aninstanceI∈ I and aprobability valuationπ

• Compute theprobabilitythat(I, π)satisfiesq

• Data complexity:measured as a function of(I, π)

(11)

Example: PQE

Signatureσ, classQofconjunctive queries, classI ofall instances.

q:∃x y R(x)∧S(x,y)∧T(y)

R a 1 b .4 c .6

S

a a 1

b v .5 b w .2

T v .3 w .7 b 1

• The query is true iffR(b)is here and one of:

• S(b,v)andT(v)are here

• S(b,w)andT(w)are here

→ Probability:.4×(1−(1−.5×.3)×(1−.2×.7)) =.1076

(12)

Example: PQE

q:∃x y R(x)∧S(x,y)∧T(y)

R a 1 b .4 c .6

S

a a 1

b v .5 b w .2

T v .3 w .7 b 1

→ Probability:.4×(1−(1−.5×.3)×(1−.2×.7)) =.1076

(13)

Example: PQE

q:∃x y R(x)∧S(x,y)∧T(y)

R a 1 b .4 c .6

S

a a 1

b v .5 b w .2

T v .3 w .7 b 1

→ Probability:.4×(1−(1−.5×.3)×(1−.2×.7)) =.1076

(14)

Example: PQE

q:∃x y R(x)∧S(x,y)∧T(y)

R a 1 b .4 c .6

S

a a 1

b v .5 b w .2

T v .3 w .7 b 1

→ Probability:.4×(1−(1−.5×.3)×(1−.2×.7)) =.1076

(15)

Example: PQE

q:∃x y R(x)∧S(x,y)∧T(y)

R a 1 b .4 c .6

S

a a 1

b v .5 b w .2

T v .3 w .7 b 1

→ Probability:.4×(1−(1−.5×.3)×(1−.2×.7)) =.1076

(16)

Example: PQE

q:∃xyR(x)∧S(x,y)∧T(y)

R a 1 b .4 c .6

S

a a 1

b v .5 b w .2

T v .3 w .7 b 1

→ Probability:.4×(1−(1−.5×.3)×(1−.2×.7)) =.1076

(17)

Example: PQE

q:∃xyR(x)∧S(x,y)∧T(y)

R a 1 b .4 c .6

S

a a 1

b v .5 b w .2

T v .3 w .7 b 1

→ Probability:.4×(1−(1−.5×.3)×(1−.2×.7)) =.1076

(18)

Example: PQE

q:∃x y R(x)∧S(x,y)∧T(y)

R a 1 b .4 c .6

S

a a 1

b v .5 b w .2

T v .3 w .7 b 1

→ Probability:.4×(1−(1−.5×.3)×(1−.2×.7)) =.1076

(19)

Example: PQE

q:∃x y R(x)∧S(x,y)∧T(y)

R a 1 b .4 c .6

S

a a 1

b v .5 b w .2

T v .3 w .7 b 1

→ Probability:.4×(1−(1−.5×.3)×(1−.2×.7)) =.1076

(20)

Example: PQE

q:∃x y R(x)∧S(x,y)∧T(y)

R a 1 b .4 c .6

S

a a 1

b v .5 b w .2

T v .3 w .7 b 1

→ Probability:.4×(1−(1−.5×.3)×(1−.2×.7)) =.1076

(21)

Example: PQE

q:∃x y R(x)∧S(x,y)∧T(y)

R a 1 b .4 c .6

S

a a 1

b v .5 b w .2

T v .3 w .7 b 1

→ Probability:

.4×(1−(1−.5×.3)×(1−.2×.7)) =.1076

(22)

Example: PQE

q:∃x y R(x)∧S(x,y)∧T(y)

R a 1 b .4 c .6

S

a a 1

b v .5 b w .2

T v .3 w .7 b 1

→ Probability:.4×

(1−(1−.5×.3)×(1−.2×.7)) =.1076

(23)

Example: PQE

q:∃x y R(x)∧S(x,y)∧T(y)

R a 1 b .4 c .6

S

a a 1

b v .5 b w .2

T v .3 w .7 b 1

→ Probability:.4×(1−

(1−.5×.3)×(1−.2×.7)) =.1076

(24)

Example: PQE

q:∃x y R(x)∧S(x,y)∧T(y)

R a 1 b .4 c .6

S

a a 1

b v .5 b w .2

T v .3 w .7 b 1

→ Probability:.4×(1−(1−.5×.3)

×(1−.2×.7)) =.1076

(25)

Example: PQE

q:∃x y R(x)∧S(x,y)∧T(y)

R a 1 b .4 c .6

S

a a 1

b v .5 b w .2

T v .3 w .7 b 1

→ Probability:.4×(1−(1−.5×.3)×(1−.2×.7))

=.1076

(26)

Example: PQE

q:∃x y R(x)∧S(x,y)∧T(y)

R a 1 b .4 c .6

S

a a 1

b v .5 b w .2

T v .3 w .7 b 1

→ Probability:.4×(1−(1−.5×.3)×(1−.2×.7)) =.1076

(27)

Complexity of probabilistic query evaluation (PQE)

Question:what is thecomplexityof PQE

depending on the classQofqueriesand classI ofinstances?

• Existingdichotomy resultonQ[Dalvi and Suciu, 2012]

• Qare unions of CQs,Iis all instances

• There is a classS ⊆ Qofsafe queries

→ PQE isPTIMEfor anyq∈ Son all instances

→ PQE is#P-hardfor anyq∈ Q\Son all instances

• q:∃x y R(x)∧S(x,y)∧T(y)isunsafe!

(28)

(29)

(30)

(31)

(32)

Instance-based dichotomy

→ We show adichotomy resultonIinstead:

• Qare Boolean monadic second-order queries (includes all UCQs)

→ PQE is inlinear timeforQon anybounded-treewidthclassI up to arithmetic costs [Same authors, ICALP’15]

→ This talk:PQE isintractableforQ

on anyunbounded-treewidthclassI (under some assumptions)

→ Treewidthmeasures how much data is close to atree

• Treeshave treewidth1

• Cycleshave treewidth2

• k-cliquesand(k−1)-gridshave treewidthk−1

(33)

(34)

(35)

(36)

Main result: hardness of PQE

Theorem

For any arity-2 signatureσ, there is afirst-orderqueryqsuch that for any constructibleunbounded-treewidthclassI,

the PQE problem forQ={q}andIis#P-hardunder RP reductions

• Arity-2: Relations have arity≤2(and one has arity2), i.e.,graphs

• Unbounded-treewidth: for allk∈N, there isI_k ∈ I of treewidth≥k

• Constructible: givenk, we cancomputesuch an instanceIkin PTIME

• #P-hard under RP reductions: reduce in PTIME with high probability from the problem of counting accepting paths of a PTIME machine

(37)

Table of contents

Introduction

Proof sketch

Extensions Conclusion

(38)

Idea: Topological minors

• Gis atopological minorofHif:

G H

(39)

G H

(40)

3 1

2

4 3 4

2 1

maps vertices to vertices maps edges to vertex-disjoint paths

Embedding:

G H

(41)

Topological minor extraction results

Theorem [Robertson and Seymour, 1986]

For any planar graphGof degree≤3,

for any graphHofsufficiently high treewidth, Gis a topological minor ofH.

More recently:

Theorem [Chekuri and Chuzhoy, 2014] There is a certain constant c∈Nsuch that

for any planar graphGof degree≤3and graphHoftreewidth≥ |G|^c, we can embedGas atopological minorofHin PTIME with high proba

(42)

Topological minor extraction results

Theorem [Robertson and Seymour, 1986]

For any planar graphGof degree≤3,

for any graphHofsufficiently high treewidth, Gis a topological minor ofH.

More recently:

Theorem [Chekuri and Chuzhoy, 2014]

There is a certain constant c∈Nsuch that

for any planar graphGof degree≤3and graphHoftreewidth≥ |G|^c, we can embedGas atopological minorofHin PTIME with high proba

(43)

Proof sketch

• Choose aproblemfrom which to reduce:

• Must be#P-hardon planar degree-3 graphs

• Must be encodable to anFO queryq

→ We use the problem of countinggraph matchings

• Given aninput graphG, computek··=|G|^c

• Compute in PTIME aninstanceIk ofI of treewidth≥k

• Compute in randomized PTIME anembeddingofGinIk

• Construct aprobability valuationπofIk such that:

• Unneccessary edges ofIkare removed

• PQE forqgives the answerto the hard problem

→ Easy for MSO buttrickierfor FO

(44)

Proof sketch

• Compute in PTIME aninstanceIk ofI of treewidth≥k

(45)

Proof sketch

• Compute in PTIME aninstanceI_k ofI of treewidth≥k

(46)

Proof sketch

(47)

Proof sketch

• Unneccessary edges ofI_kare removed

(48)

Introduction Proof sketch Extensions

Conclusion

(49)

Non-probabilistic evaluation

Apply [Chekuri and Chuzhoy, 2014] to method of [Ganian et al., 2014]

for MSOnon-probabilisticquery evaluation (QE):

Theorem

For any arity-2 signatureσ and levelΣ^P_i of the polynomial hierarchy, there is aMSOqueryq_i such that,

for any constructible,subinstance-closed,unbounded-twclassI, the QE problem forQ={qi}andI isΣ^P_i-hardunder RP reductions

Variant:we also show a result like in [Ganian et al., 2014], with:

• weakerconstructibilityrequirement onI:

→ I isdensely unbounded poly-logarithmically

• strongercomplexity hypothesis:

→ PH does not admit2^o(n)-sized circuits

(50)

Non-probabilistic evaluation

Apply [Chekuri and Chuzhoy, 2014] to method of [Ganian et al., 2014]

for MSOnon-probabilisticquery evaluation (QE):

Theorem

For any arity-2 signatureσ and levelΣ^P_i of the polynomial hierarchy, there is aMSOqueryq_i such that,

for any constructible,subinstance-closed,unbounded-twclassI, the QE problem forQ={qi}andI isΣ^P_i-hardunder RP reductions Variant:we also show a result like in [Ganian et al., 2014], with:

• weakerconstructibilityrequirement onI:

→ I isdensely unbounded poly-logarithmically

• strongercomplexity hypothesis:

→ PH does not admit2^o(n)-sized circuits

(51)

Lower bound on OBDD representations OBDDfor a queryqon instanceI:

ordered decision diagram on the facts ofIto decide whetherqholds

q:∃x y R(x)∧S(x,y)∧T(y)

R a r1

b r2

c r3

S a a s1

b v s2

b w s3

T v t1

w t2

b t3

r2?

0

1

0 1

1

0

1 0

1

0 1

(52)

q:∃x y R(x)∧S(x,y)∧T(y)

R a r1

b r2

c r3

S a a s1

b v s2

b w s3

T v t1

w t2

b t3

r2?

0

1

0 1

1

0

1 0

1

0 1

(53)

q:∃xy R(x)∧S(x,y)∧T(y)

R a r1

b r2

c r3

S a a s1

b v s2

b w s3

T v t1

w t2

b t3

r2?

0

. . .

0

1

0 1

1

0

1 0

1

0 1

(54)

q:∃xyR(x)∧S(x,y)∧T(y)

R a r1

b r2

c r3

S a a s1

b v s2

b w s3

T v t1

w t2

b t3

r2?

0

s2?

0

1

. . . . . .

0 1

1

0

1 0

1

0 1

(55)

R a r1

b r2

c r3

S a a s1

b v s2

b w s3

T v t1

w t2

b t3

r2?

0

s2?

0

1

t1? . . .

0 1

1

0

1

0

1

0 1

(56)

R a r1

b r2

c r3

S a a s1

b v s2

b w s3

T v t1

w t2

b t3

r2?

0

s2?

0

1

t1? s3?

0 1

1

0

1

. . .

0

1

0 1

(57)

R a r1

b r2

c r3

S a a s1

b v s2

b w s3

T v t1

w t2

b t3

r2?

0

s2?

0

1

t1? s3?

0 1

1

0

1

t2?

0

1

0 1

(58)

Lower bound on OBDD representations (result)

• If we find anOBDDofqonIin PTIME, thenPQEofqonIalso is

→ Showinexistence of poly-size OBDDs(rather thanPQE hardness)

Theorem

For any arity-2 signatureσ, there is aUCQ with inequalities qs.t. for anyIof treewidthdensely unbounded poly-logarithmically, qhasno OBDDs of polynomial sizeon instances ofI

→ Meta-dichotomyon the connected UCQ⁶⁼for which this holds

(59)

→ Showinexistence of poly-size OBDDs(rather thanPQE hardness) Theorem

For any arity-2 signatureσ, there is aUCQ with inequalities qs.t.

for anyIof treewidthdensely unbounded poly-logarithmically, qhasno OBDDs of polynomial sizeon instances ofI

(60)

→ Showinexistence of poly-size OBDDs(rather thanPQE hardness) Theorem

For any arity-2 signatureσ, there is aUCQ with inequalities qs.t.

for anyIof treewidthdensely unbounded poly-logarithmically, qhasno OBDDs of polynomial sizeon instances ofI

(61)

Also in paper

• Hardness results for MSOmatch counting:

→ Count how many subsetsXare such thatIsatisfiesq(X)

• Connect thetractability resultof MSO on treelike TID to the study oftractable lineages:

→ d-DNNFs, OBDDs, formulae, etc.

• Connect the same result totractability of safe queries:

• Inversion-freequeries: subclass of safe queries, tractable OBDDs

→ We can alwaysrewritetheir input instances totreelike instances in alineage-preservingway (hence, probability-preserving)

(62)

Introduction Proof sketch Extensions Conclusion

(63)

Summary of our dichotomy

Upper. PQE forMSOontreelike instances

haslineardata complexity up to arithmetic costs

Lower. PQE forFOonanyconstructible, arity-2, unbounded-tw instance family is#P-hardunder RP reductions

→ Bounded treewidth isthe right notionfor tractability of PQE

(64)

haslineardata complexity up to arithmetic costs Lower. PQE forFOonanyconstructible, arity-2, unbounded-tw

instance family is#P-hardunder RP reductions

(65)

haslineardata complexity up to arithmetic costs Lower. PQE forFOonanyconstructible, arity-2, unbounded-tw

instance family is#P-hardunder RP reductions

(66)

Future work

• Can we show #P-hardness underusual P reductions?

→ Depends on [Chekuri and Chuzhoy, 2014]

• Can we extend the result toarbitrary aritysignatures?

• Can we extend the result to weakerquery languageslikeUCQ⁶⁼? Conjecture

PQE is hard on any constructible unbounded-tw family for: q: (E(x,y)∨E(y,x))∧(E(y,z)∨E(z,y))∧x6=z

→ This query is alerady hard in terms ofOBDDs

Thanks for your attention!

(67)

Future work

PQE is hard on any constructible unbounded-tw family for: q: (E(x,y)∨E(y,x))∧(E(y,z)∨E(z,y))∧x6=z

(68)

Future work

PQE is hard on any constructible unbounded-tw family for:

q: (E(x,y)∨E(y,x))∧(E(y,z)∨E(z,y))∧x6=z

(69)

Future work

PQE is hard on any constructible unbounded-tw family for:

q: (E(x,y)∨E(y,x))∧(E(y,z)∨E(z,y))∧x6=z

(70)

References I

Amarilli, A., Bourhis, P., and Senellart, P. (2015).

Provenance circuits for trees and treelike instances.

InProc. ICALP, LNCS, pages 56–68.

Chaudhuri, S. and Vardi, M. Y. (1992).

On the equivalence of recursive and nonrecursive Datalog programs.

InPODS.

Chekuri, C. and Chuzhoy, J. (2014).

Polynomial bounds for the grid-minor theorem.

InSTOC.

(71)

References II

Dalvi, N. and Suciu, D. (2012).

The dichotomy of probabilistic inference for unions of conjunctive queries.

JACM, 59(6):30.

Ganian, R., Hlinˇen`y, P., Langer, A., Obdrˇz´alek, J., Rossmanith, P., and Sikdar, S. (2014).

Lower bounds on the complexity of MSO1 model-checking.

JCSS, 1(80).

Robertson, N. and Seymour, P. D. (1986).

Graph minors. V. Excluding a planar graph.

J. Comb. Theory, Ser. B, 41(1).

(72)

Encoding treelike instances [Chaudhuri and Vardi, 1992]

Instance:

N a b b c c d d e e f

S a c b e

Gaifman graph: a

b

c d

e f

Tree decomp.:

a b c b c e c d e e f

Tree encoding:

N(a1,a2) N(a2,a3) S(a1,a3) S(a2,a4) N(a3,a1) N(a1,a4)

N(a4,a1)

(73)

Instance:

S a c b e

Gaifman graph:

a

b

c d

e f

Tree decomp.:

a b c b c e c d e e f

Tree encoding:

N(a4,a1)

(74)

Instance:

S a c b e

Gaifman graph:

a

b

c d

e f

Tree decomp.:

a b c b c e c d e e f

Tree encoding:

N(a4,a1)

(75)

Instance:

S a c b e

Gaifman graph:

a

b

c d

e f

Tree decomp.:

a b c b c e c d e e f

Tree encoding:

N(a4,a1)

(76)

Defining the query

G Ik

1 2 ⇒ ₁ ₂

• In the embedding, edges ofGcan becomelong pathsinI_k

• qmust answer the hard problem onGdespite subdivisions

→ Easy in MSO but tricky inFO!

→ Ourqrestricts to asubset of the worldsof known weight and gives the right answerup to renormalization

(77)

Defining the query

G Ik

1 2 ⇒ ₁ ₂

• In the embedding, edges ofGcan becomelong pathsinI_k

• qmust answer the hard problem onGdespite subdivisions

→ Easy in MSO but tricky inFO!

→ Ourqrestricts to asubset of the worldsof known weight and gives the right answerup to renormalization