July10th,2015 AntoineAmarilli ,PierreBourhis ,PierreSenellart ProvenanceCircuitsforTreesandTreelikeInstances

(1)

Introduction Bool[X]-provenance N[X]-provenance Conclusion

Provenance Circuits for Trees and Treelike Instances

Antoine Amarilli¹, Pierre Bourhis², Pierre Senellart^1,3

1Télécom ParisTech

2CNRS-LIFL

3National University of Singapore

July 10th, 2015

(2)

General idea

We consider a query and arelational instance

Often it is notsufficient to merelyevaluate the query:

→ We needquantitative information

→ We need the link from theoutputto theinput data

→ Computequery provenance!

(3)

General idea

We consider a query and arelational instance

Often it is notsufficient to merelyevaluate the query:

→ We needquantitative information

→ We need the link from theoutputto theinput data

→ Computequery provenance!

(4)

Example 1: security for a conjunctive query

Consider theconjunctive query: ∃xyz R(x,y)∧R(y,z) Consider therelational instance below:

R a b b c d e e d f f

Result: true

Add security annotations: Public, Confidential, Secret, Top secret, Never available

What is the minimal security clearancerequired?

→ Result: Confidential

(5)

Example 1: security for a conjunctive query

R a b b c d e e d f f

Result: true

(6)

Example 1: security for a conjunctive query

R a b Public b c Secret d e Confidential e d Confidential f f Top secret

Result: true

(7)

Example 1: security for a conjunctive query

Result: true

(8)

Example 1: security for a conjunctive query

Result: true

(9)

Example 1: security for a conjunctive query

Result: true

(10)

Example 1: security for a conjunctive query

R a b Public b c Secret d e Confidential e d Confidential f f Top secret Result: true

(11)

Example 1: security for a conjunctive query

Result: true

(12)

Example 2: bag queries

Consider again: ∃xyz R(x,y)∧R(y,z).

R

a b

b c

d e

e d

f f

Result: true

Add multiplicity annotations How many query matches?

→ Result: 1 + 1 + 1 + 1 = 4

(13)

Example 2: bag queries

R

a b

b c

d e

e d

f f

Result: true

Add multiplicity annotations How many query matches?

→ Result: 1 + 1 + 1 + 1 = 4

(14)

Example 2: bag queries

R

a b 1

b c 1

d e 1

e d 1

f f 1

Result: true

Add multiplicity annotations

How many query matches?

→ Result: 1 + 1 + 1 + 1 = 4

(15)

Example 2: bag queries

R

a b 1

b c 1

d e 1

e d 1

f f 1

Result: true

Add multiplicity annotations How manyquery matches?

→ Result: 1 + 1 + 1 + 1 = 4

(16)

Example 2: bag queries

R

a b 1

b c 1

d e 1

e d 1

f f 1

Result: true

→ Result: 1

+ 1 + 1 + 1 = 4

(17)

Example 2: bag queries

R

a b 1

b c 1

d e 1

e d 1

f f 1

Result: true

→ Result: 1 +1

+ 1 + 1 = 4

(18)

Example 2: bag queries

R

a b 1

b c 1

d e 1

e d 1

f f 1

Result: true

→ Result: 1 + 1 +1

+ 1 = 4

(19)

Example 2: bag queries

R

a b 1

b c 1

d e 1

e d 1

f f 1

Result: true

→ Result: 1 + 1 + 1 +1

= 4

(20)

Example 2: bag queries

R

a b 1

b c 1

d e 1

e d 1

f f 1

Result: true

→ Result: 1 + 1 + 1 + 1 = 4

(21)

Example 3: uncertain facts

R

a b

b c

d e

e d

f f

Result: true

Assume facts areuncertain, give thematomic annotations For which subinstancesdoes the query hold?

→ Result: (f₁∧f₂)∨(f₃∧f₄)∨f₅

(22)

Example 3: uncertain facts

R

a b

b c

d e

e d

f f

Result: true

→ Result: (f₁∧f₂)∨(f₃∧f₄)∨f₅

(23)

Example 3: uncertain facts

R a b f₁ b c f₂ d e f₃ e d f₄ f f f₅

Result: true

Assume facts areuncertain, give thematomic annotations

For which subinstancesdoes the query hold?

→ Result: (f₁∧f₂)∨(f₃∧f₄)∨f₅

(24)

Example 3: uncertain facts

Result: true

→ Result: (f₁∧f₂)∨(f₃∧f₄)∨f₅

(25)

Example 3: uncertain facts

R

a b f₁

b c f₂

d e f₃ e d f₄ f f f₅

Result: true

→ Result: (f₁∧f₂)

∨(f₃∧f₄)∨f₅

(26)

Example 3: uncertain facts

R a b f₁ b c f₂

d e f₃

e d f₄

f f f₅

Result: true

→ Result: (f₁∧f₂)∨(f₃∧f₄)

∨f₅

(27)

Example 3: uncertain facts

R a b f₁ b c f₂

d e f₃

e d f₄

f f f₅

Result: true

→ Result: (f₁∧f₂)∨(f₃∧f₄)

∨f₅

(28)

Example 3: uncertain facts

R a b f₁ b c f₂ d e f₃ e d f₄

f f f₅

Result: true

→ Result: (f₁∧f₂)∨(f₃∧f₄)∨f₅

(29)

Example 3: uncertain facts

Result: true

→ Result: (f₁∧f₂)∨(f₃∧f₄)∨f₅

(30)

Example 4: the universal semiring N [X]

Annotate input factswith atomic annotationsX=f₁, . . . ,f_n Most general semiring: N[X]of polynomials onX

→ Result: (f₁⊗f₂)⊕(f₃⊗f₄)⊕(f₄⊗f₃)⊕(f₅⊗f₅)

(31)

Example 4: the universal semiring N [X]

→ Result:

(f₁⊗f₂)⊕(f₃⊗f₄)⊕(f₄⊗f₃)⊕(f₅⊗f₅)

(32)

Example 4: the universal semiring N [X]

Consider again: ∃xyzR(x,y)∧R(y,z).

R

a b f₁

b c f₂ d e f₃ e d f₄ f f f₅

→ Result:

(f₁⊗f₂)⊕(f₃⊗f₄)⊕(f₄⊗f₃)⊕(f₅⊗f₅)

(33)

Example 4: the universal semiring N [X]

R

a b f₁

b c f₂ d e f₃ e d f₄ f f f₅

→ Result: (f₁⊗f₂)

⊕(f₃⊗f₄)⊕(f₄⊗f₃)⊕(f₅⊗f₅)

(34)

Example 4: the universal semiring N [X]

R a b f₁ b c f₂

d e f₃

e d f₄ f f f₅

→ Result: (f₁⊗f₂)

⊕(f₃⊗f₄)⊕(f₄⊗f₃)⊕(f₅⊗f₅)

(35)

Example 4: the universal semiring N [X]

R a b f₁ b c f₂

d e f₃

e d f₄ f f f₅

→ Result: (f₁⊗f₂)⊕(f₃⊗f₄)

⊕(f₄⊗f₃)⊕(f₅⊗f₅)

(36)

Example 4: the universal semiring N [X]

R a b f₁ b c f₂ d e f₃

e d f₄

f f f₅

→ Result: (f₁⊗f₂)⊕(f₃⊗f₄)

⊕(f₄⊗f₃)⊕(f₅⊗f₅)

(37)

Example 4: the universal semiring N [X]

R a b f₁ b c f₂ d e f₃

e d f₄

f f f₅

→ Result: (f₁⊗f₂)⊕(f₃⊗f₄)⊕(f₄⊗f₃)

⊕(f₅⊗f₅)

(38)

Example 4: the universal semiring N [X]

f f f₅

→ Result: (f₁⊗f₂)⊕(f₃⊗f₄)⊕(f₄⊗f₃)

⊕(f₅⊗f₅)

(39)

Example 4: the universal semiring N [X]

f f f₅

→ Result: (f₁⊗f₂)⊕(f₃⊗f₄)⊕(f₄⊗f₃)⊕(f₅⊗f₅)

(40)

Example 4: the universal semiring N [X]

→ Result: (f₁⊗f₂)⊕(f₃⊗f₄)⊕(f₄⊗f₃)⊕(f₅⊗f₅)

(41)

Specialization and homomorphisms

These examples are captured by commutative semirings:

securitysemiring(K,min,max,Public,Never available) bagsemiring(N,+,×,0,1)

Booleansemiring(PosBool[X],∨,∧,f,t) universalsemiring(N[X],+,×,0,1)

N[X]is theuniversal semiring:

The provenance forN[X] can bespecializedto anyK[X] Bycommutation with homomorphisms, atomic annotations inXcan be replaced by their value in K

→ Computing N[X]provenancesubsumes all tasks

→ It can be done inPTIME data complexity for CQs

(42)

Specialization and homomorphisms

Booleansemiring(PosBool[X],∨,∧,f,t) universalsemiring(N[X],+,×,0,1) N[X]is theuniversal semiring:

The provenance forN[X] can bespecializedto anyK[X]

Bycommutation with homomorphisms, atomic annotations inXcan be replaced by their value in K

(43)

Specialization and homomorphisms

Booleansemiring(PosBool[X],∨,∧,f,t) universalsemiring(N[X],+,×,0,1) N[X]is theuniversal semiring:

The provenance forN[X] can bespecializedto anyK[X]

Bycommutation with homomorphisms, atomic annotations inXcan be replaced by their value in K

(44)

Provenance and probability

Probabilistic query evaluation:

FixedCQq, and inputTID instance:

R a b 0.6 b c 0.9

→ Computing theprobabilityof thePosBool[X]-provenance

→ #P-hard in data complexity

(45)

Provenance and probability

R a b 0.6 b c 0.9

(46)

Provenance and probability

R a b 0.6 b c 0.9

(47)

Trees and treelike instances

Idea: restrict the instances to trees andtreelike instances Tree decompositionof an instance: cover all facts

Treewidth: minimal width (bag size) of a decomposition Treeshave treewidth1

Cycleshave treewidth2

k-cliquesandk-gridshave treewidthk−1 Treelike: thetreewidthis bounded by aconstant

(48)

Trees and treelike instances

Idea: restrict the instances to trees andtreelike instances Tree decompositionof an instance: cover all facts Treewidth: minimal width (bag size) of a decomposition

Treeshave treewidth1 Cycleshave treewidth2

k-cliquesandk-gridshave treewidthk−1 Treelike: thetreewidthis bounded by aconstant

(49)

Trees and treelike instances

k-cliquesandk-gridshave treewidthk−1

Treelike: thetreewidthis bounded by aconstant

(50)

Trees and treelike instances

k-cliquesandk-gridshave treewidthk−1 Treelike: thetreewidth is bounded by aconstant

(51)

Problem statement

Many tasks have tractable data complexity ontreelike instances:

MSO query evaluationislinear[Courcelle, 1990]

MSO result countingislinear [Arnborg et al., 1991]

Probability evaluationislinearfortrees[Cohen et al., 2009]

(MSOcovers relational algebra, UCQs, monadic Datalog...)

→ Can we define provenance in this setting?

→ Can we computeit efficiently?

→ Can we generalizethe above results?

(52)

Problem statement

(53)

Problem statement

(54)

Problem statement

(55)

General idea

Bool[X]-provenance on treesand treelike instances The world of trees:

Query: MSO on trees The world of treelike instances:

Query: MSO on the instance

→ Reduces to trees[Courcelle, 1990]

→ Start with Bool[X]-provenance for queries ontrees

(57)

General idea

Bool[X]-provenance on treesand treelike instances The world of trees:

Query: MSO on trees The world of treelike instances:

Query: MSO on the instance

→ Reduces to trees[Courcelle, 1990]

→ Start with Bool[X]-provenance for queries ontrees

(58)

Uncertain trees

1

5

7 6

2

4 3

A valuationof a tree decides whether to keepor discard node labels.

Example query:

“Is there both a red and a green node?”

Valuation: {1,2,3,4,5,6,7} The query is true

(59)

Uncertain trees

1

5

7 6

2

4 3

Example query:

Valuation: {1,2,5,6} The query is false

(60)

Uncertain trees

1

5

7 6

2

4 3

Example query:

Valuation: {2,7} The query is true

(61)

Provenance formulae and circuits

1

5

7 6

2

4 3

Whichvaluations satisfy the query?

→ Provenance formula of a query q on an uncertain treeT:

Boolean formulaϕ onvariablesx₁. . .x₇

→ ν(T)satisfiesqiffν(ϕ)istrue Provenance circuit ofq onT [Deutch et al., 2014]

Boolean circuitC withinput gatesg1. . .g7

→ ν(T)satisfiesqiffν(C)is true

(62)

Provenance formulae and circuits

1

5

7 6

2

4 3

Boolean formulaϕ onvariablesx1. . .x7

→ ν(T)satisfiesqiffν(ϕ)istrue

Provenance circuit ofq onT [Deutch et al., 2014]

(63)

Provenance formulae and circuits

1

5

7 6

2

4 3

Boolean formulaϕ onvariablesx1. . .x7

→ ν(T)satisfiesqiffν(ϕ)istrue Provenance circuit ofq onT [Deutch et al., 2014]

(64)

Example

1

5

7 6

2

4 3

Is there both a red and agreennode?

Provenance formula: (x₂∨x₃)∧x₇ Provenance circuit:

∧

∨ g₇

g₂ g₃

(65)

Example

1

5

7 6

2

4 3

Provenance formula: (x₂∨x₃)∧x₇

Provenance circuit:

∧

∨ g₇

g₂ g₃

(66)

Example

1

5

7 6

2

4 3

Provenance formula: (x₂∨x₃)∧x₇ Provenance circuit:

∧

∨ g₇

g₂ g₃

(67)

Our main result on trees

Theorem

For any fixedMSO query q (first order + quantify on sets), for any inputtree T,

we can build aBool[X]provenance circuitof q on T inlinear time in T.

→ Key ideas:

Compile q to a tree automaton[Thatcher and Wright, 1968] Write thepossible transitions of the automaton onT

Corollary

If tree nodes have a probabilityof being independently kept, we can compute thequery probability in linear time.

(68)

Our main result on trees

Theorem

→ Key ideas:

Compile q to a tree automaton[Thatcher and Wright, 1968]

Write thepossible transitions of the automaton onT

Corollary

If tree nodes have a probabilityof being independently kept, we can compute thequery probability in linear time.

(69)

Our main result on trees

Theorem

→ Key ideas:

Compile q to a tree automaton[Thatcher and Wright, 1968]

Write thepossible transitions of the automaton onT Corollary

If tree nodes have aprobability of being independently kept, we can compute thequery probability in linear time.

(70)

Treelike instances

Tree encodings: represent treelike instances as trees

MSO queryon an instance→MSO query on the tree encoding

Uncertain instance: each fact can be present or absent

→ Possible subinstancesare possible valuationsof the encoding R

a b b c b d

R(a₁,a₂)

R(a₂,a₃) R(a₂,a₃)

(71)

Treelike instances

MSO queryon an instance→MSO query on the tree encoding Uncertain instance: each fact can be present or absent

a b b c b d

R(a₁,a₂)

R(a₂,a₃) R(a₂,a₃)

(72)

Treelike instances

a b b c b d

R(a₁,a₂)

R(a₂,a₃) R(a₂,a₃)

(73)

Treelike instances

a b b c b d

R(a₁,a₂)

R(a₂,a₃) R(a₂,a₃)

(74)

Treelike instances

a b b c b d

R(a₁,a₂)

R(a₂,a₃) R(a₂,a₃)

(75)

Our result and consequences

Theorem

For any fixedMSO query q and k∈N, for any inputinstance I of treewidth≤k,

we can build inlinear time aBool[X]provenance circuit of q on I.

Corollary

MSO queries havelineardata complexity on treelike TID instances.

Corollary

MSO counting haslinear timecomplexity (already known).

(76)

Our result and consequences

Theorem

Corollary

(77)

Our result and consequences

Theorem

Corollary

(78)

First problem: non-monotone queries

We want to movefrom Bool[X]toN[X]

Semirings and negationdon’t mix [Amsterdamer et al., 2011]

Our previous construction builds circuits with NOT-gates

→ q monotoneif I|=q impliesI^′ |=q for all I^′ ⊇I

→ Provenance circuits for monotone queries can bemonotone

(80)

First problem: non-monotone queries

(81)

First problem: non-monotone queries

(82)

Second problem: intrinsic definition

Boolean provenance has an intrinsic definition:

“Characterize which subinstances satisfy the query”

→ Independent from how the query iswritten

→ Independent from itsencodingon trees N[X]-provenance was definedoperationally

→ Depends on how the query iswritten

→ We restrict to (Boolean) UCQs from now on

(83)

Second problem: intrinsic definition

Boolean provenance has an intrinsic definition:

“Characterize which subinstances satisfy the query”

→ Independent from how the query iswritten

→ Independent from itsencodingon trees N[X]-provenance was definedoperationally

→ Depends on how the query iswritten

→ We restrict to (Boolean) UCQs from now on

(84)

Provenance of a Boolean CQ

R a a x₁ b c x₂ c b x₃

Query: q:∃xy R(x,y)∧R(y,x)

Provenance:

(x₁⊗x₁)⊕(x₂⊗x₃)⊕(x₃⊗x₂) akax²₁+ 2x₂x₃

Definition:

Sumover query matches Multiplyover matched facts How isN[X]more expressive thanPosBool[X]?

→ Coefficients: counting multiple derivations

→ Exponents: using facts multiple times

(85)

Provenance of a Boolean CQ

Query: q:∃xyR(x,y)∧R(y,x) Provenance:

(x₁⊗x₁)⊕(x₂⊗x₃)⊕(x₃⊗x₂) akax²₁+ 2x₂x₃

Definition:

(86)

Provenance of a Boolean CQ

(x₁⊗x₁)

⊕(x₂⊗x₃)⊕(x₃⊗x₂) akax²₁+ 2x₂x₃

Definition:

(87)

Provenance of a Boolean CQ

(x₁⊗x₁)

⊕(x₂⊗x₃)⊕(x₃⊗x₂) akax²₁+ 2x₂x₃

Definition:

(88)

Provenance of a Boolean CQ

(x₁⊗x₁)⊕(x₂⊗x₃)

⊕(x₃⊗x₂) akax²₁+ 2x₂x₃

Definition:

(89)

Provenance of a Boolean CQ

(x₁⊗x₁)⊕(x₂⊗x₃)

⊕(x₃⊗x₂) akax²₁+ 2x₂x₃

Definition:

July10th,2015 AntoineAmarilli ,PierreBourhis ,PierreSenellart ProvenanceCircuitsforTreesandTreelikeInstances

Provenance Circuits for Trees and Treelike Instances

General idea

General idea

Example 1: security for a conjunctive query

Example 1: security for a conjunctive query

Example 1: security for a conjunctive query

Example 1: security for a conjunctive query

Example 1: security for a conjunctive query

Example 1: security for a conjunctive query

Example 1: security for a conjunctive query

Example 1: security for a conjunctive query

Example 2: bag queries

Example 2: bag queries

Example 2: bag queries

Example 2: bag queries

Example 2: bag queries

Example 2: bag queries

Example 2: bag queries

Example 2: bag queries

Example 2: bag queries

Example 3: uncertain facts

Example 3: uncertain facts

Example 3: uncertain facts

Example 3: uncertain facts

Example 3: uncertain facts

Example 3: uncertain facts

Example 3: uncertain facts

Example 3: uncertain facts

Example 3: uncertain facts

Example 4: the universal semiring N [X]

Example 4: the universal semiring N [X]

Example 4: the universal semiring N [X]

Example 4: the universal semiring N [X]

Example 4: the universal semiring N [X]

Example 4: the universal semiring N [X]

Example 4: the universal semiring N [X]

Example 4: the universal semiring N [X]

Example 4: the universal semiring N [X]

Example 4: the universal semiring N [X]

Example 4: the universal semiring N [X]

Specialization and homomorphisms

Specialization and homomorphisms

Specialization and homomorphisms

Provenance and probability

Provenance and probability

Provenance and probability

Trees and treelike instances

Trees and treelike instances

Trees and treelike instances

Trees and treelike instances

Problem statement

Problem statement

Problem statement

Problem statement

Table of contents

General idea

General idea

Uncertain trees

Uncertain trees

Uncertain trees

Provenance formulae and circuits

Provenance formulae and circuits

Provenance formulae and circuits

Example

Example

Example

Our main result on trees

Our main result on trees

Our main result on trees

Treelike instances

Treelike instances

Treelike instances

Treelike instances

Treelike instances

Our result and consequences

Our result and consequences

Our result and consequences

Table of contents

First problem: non-monotone queries