Enumeration for MSO Queries on Trees via Circuits

(1)

Enumeration for MSO Queries on Trees via Circuits

Antoine Amarilli¹ Pierre Bourhis², Louis Jachiet², Stefan Mengel³, Matthias Niewerth⁴

July 14, 2019

1Télécom Paris, Institut Polytechnique de Paris

2CNRS CRIStAL

(2)

Enumeration for MSO Queries on Trees via Circuits

July141414141414141414141414,1414141414 2019

2CNRS CRIStAL

3CNRS CRIL

4Universität Bayreuth 1/35

(3)

Circuits pour l’énumération de MSO sur des arbres

July1414141414141414141414141414141414,2019

2

(4)

Query evaluation

The main database problem isquery evaluation DataD: what we know

?

QueryQ: question asked about the data

i

^Result: all results of the queryQon the dataD

Measure of efficiency:computational complexity:

•

Combined complexity:D andQareinputs

•

Data complexity:Qisfixed,Dis theinput

2/35

(5)

The main database problem isquery evaluation

DataD: what we know

?

i

•

(6)

DataD: what we know

?

i

•

2/35

(7)

DataD: what we know

?

i

•

Combined complexity:DandQareinputs

(8)

Structured data: trees

•

Several kinds of data are structured as atree (HTML pages, XML, folder hierarchies...)

• Important special case:text

•

On this kind of data, we can usemore efficient query evaluation techniques

•

Natural query language:monadic second-order logic(MSO)

• Very expressive

• Corresponds totree automata

• Data complexity is inlinear time

3/35

(9)

•

• Very expressive

(10)

•

• Very expressive

3/35

(11)

Query evaluation on trees

Data: a treeTwhere nodes have a color from an alphabet

?

^Query^{Q: a}^Booleanformula in monadic second-order logic (MSO)

•P (x)means “xis blue”

•x→ymeans “xis the parent ofy”

“Is there both a pink and a blue node?”

∃x y P (x)∧P (y)

i

Result: YES/NO indicating if the treeT satisfies the queryQ

•

Data complexity: linear inT

•

Combined complexity:cannot be elementary

(12)

?

∃x y P (x)∧P (y)

i

•

4/35

(13)

?

∃x y P (x)∧P (y)

i

•

(14)

?

∃x y P (x)∧P (y)

i

•

4/35

(15)

Beyond Boolean queries

•

^Booleanqueries only givesYES/NOanswers

→ YES, there is both a pink and a blue node

•

Better idea:non-Boolean queries

→ e.g.,Q(x,y): “xis a pink node andyis a blue node”

•

^Challenge:how to define complexity for non-Boolean queries?

• Justwriting the outputis slow in the worst case

• Easy solution: thenaive algorithmthat tests all pairs

→ We need anew definitionof complexity

(16)

•

5/35

(17)

•

(18)

•

5/35

(19)

•

(20)

•

5/35

(21)

Idea: Enumeration algorithms

Idea:In real life, we do not want to computeall the matches we just need to be able toenumeratematches quickly

→ Formalization:enumeration algorithms

(22)

6/35

(23)

(24)

6/35

(25)

(26)

6/35

(27)

Formalizing an enumeration algorithm

Input

Step 1: Indexing

in O(input) Indexed input

Step 2: Enumeration

in O(result)

A B C

a b c a’ b c a b’ c a’ b’ c

Results

State

(28)

Input

Step 1:

Indexing in O(input)

Indexed input

Step 2: Enumeration

in O(result)

A B C

Results

State

7/35

(29)

Input

Step 1:

Indexing

Step 2: Enumeration

in O(result)

A B C

Results

State

(30)

Input

Step 1:

Indexing

Step 2:

Enumeration in O(result)

A B C

Results

State

7/35

(31)

Input

Step 1:

Indexing

Step 2:

A B C a b c

a’ b c a b’ c a’ b’ c

Results

State

(32)

Input

Step 1:

Indexing

Step 2:

A B C a b c

Results

State 0011

7/35

(33)

Input

Step 1:

Indexing

Step 2:

A B C a b c

Results

State 0011

(34)

Input

Step 1:

Indexing

Step 2:

A B C

a b c

a’ b c

a b’ c a’ b’ c

Results

State 010001

7/35

(35)

Input

Step 1:

Indexing

Step 2:

A B C

a b c a’ b c

a b’ c

a’ b’ c

Results

State 01100111

(36)

Input

Step 1:

Indexing

Step 2:

A B C

a b c a’ b c a b’ c

a’ b’ c Results

State

⊥

7/35

(37)

Main results

•

We are given:the treeT, a non-Boolean queryQin MSO

→ Example:Q(x,y)asks for pairs of a blue nodexand a pink nodey

•

^{We want:}enumerate the results of the query

→ Here, the pairs of a pink node and blue node Theorem

[Bagan, 2006, Kazana and Segoufin, 2013]

For anyfixedMSO queryQ, given a treeT,

we can preprocessT inlinear time in Tand then enumerate each result inlinear time in the result

This was already known, so what’s new?

•

Modular proofbased on a notion ofset circuits

•

Tractable in combined complexityforQgiven as an automaton

•

Efficientlyupdatethe preprocessing when the treechanges

(38)

Main results

•

→ Here, the pairs of a pink node and blue node

Theorem

•

8/35

(39)

Main results

•

→ Here, the pairs of a pink node and blue node Theorem

•

(40)

Main results

•

Theorem [Bagan, 2006, Kazana and Segoufin, 2013]

we can preprocessT inlinear time in Tand then enumerate each result inlinear time in the result This was already known, so what’s new?

•

8/35

(41)

Main results

•

Theorem [Bagan, 2006, Kazana and Segoufin, 2013]

we can preprocessT inlinear time in Tand then enumerate each result inlinear time in the result This was already known, so what’s new?

•

(42)

Structure of the talk

•

Building a set circuit:given a treeT and automatonA, we can build aset circuitCthat represents the results

•

Enumeration on set circuits:given a set circuitC,

we can enumerate efficiently the results that it captures (under some assumptions onC)

•

^{New stuff:}

• Tractability in theautomatonand application to text

• Efficientupdatesof the index

9/35

(43)

•

^{New stuff:}

(44)

•

^{New stuff:}

9/35

(45)

Building a set circuit

(46)

Monadic second-order logic (MSO)

•

^P ^(x)^{means “x}is blue”; alsoP (x),P (x)

•

^x^→^y^{means “}^xis the parent ofy”

•

Propositional logic:formulas withAND∧,OR∨,NOT¬

• P (x)∧P (y)means“Nodexis pink and nodeyis blue”

•

First-order logic:addsexistential quantifier∃and universal quantifier∀

• ∃x y P (x)∧P (y)means“There is both a pink and a blue node”

•

Monadic second-order logic (MSO):addsquantifiers over sets

• ∃S∀x S(x)means“there is a setScontaining every elementx”

• Can expresstransitive closurex→^∗y, i.e.,“xis an ancestor ofy”

• ∀x P (x)⇒ ∃y P (y)∧x→^∗y

means“There is a blue node below every pink node”

10/35

(47)

•

• ∀x P (x)⇒ ∃y P (y)∧x→^∗y

(48)

•

• ∀x P (x)⇒ ∃y P (y)∧x→^∗y

10/35

(49)

•

(50)

Tree automata

Tree alphabet:

•

Bottom-up deterministictree automaton

•

“Is there both a pink and a blue node?”

•

^States:^{⊥,^B,^P,^>}

•

Final states: {>}

•

Initial function: ⊥ P B

•

Transitions(examples): P

P ⊥

> B P

⊥

11/35

(51)

Tree automata

Tree alphabet:

•

^States:^{⊥,^B,^P,^>}

•

Final states: {>}

•

P ⊥

> B P

⊥

(52)

Tree automata

Tree alphabet:

•

^States:^{⊥,^B,^P,^>}

•

Final states: {>}

•

P ⊥

> B P

⊥

11/35

(53)

Tree automata

Tree alphabet:

•

^States:^{⊥,^B,^P,^>}

•

Final states: {>}

•

P ⊥

> B P

⊥

(54)

Tree automata

Tree alphabet:

•

^States:^{⊥,^B,^P,^>}

•

Final states: {>}

•

P ⊥

> B P

⊥

11/35

(55)

Tree automata

Tree alphabet:

B

⊥ P ⊥

•

^States:^{⊥,^B,^P,^>}

•

Final states: {>}

•

P ⊥

> B P

⊥

(56)

Tree automata

Tree alphabet:

B

⊥ P ⊥

•

^States:^{⊥,^B,^P,^>}

•

Final states: {>}

•

Transitions(examples):

P P ⊥

>

B P

⊥

11/35

(57)

Tree automata

Tree alphabet:

B

⊥ P

P ⊥

•

^States:^{⊥,^B,^P,^>}

•

Final states: {>}

•

P P ⊥

>

B P

⊥

(58)

Tree automata

Tree alphabet:

>

B

⊥ P

P ⊥

•

^States:^{⊥,^B,^P,^>}

•

Final states: {>}

•

P P ⊥

>

B P

⊥

11/35

(59)

Boolean MSO query evaluation via automata

Theorem [Thatcher and Wright, 1968]

MSOandtree automatahave the sameexpressive poweron trees

→ Given a Boolean MSO query, we can compute a tree automaton that accepts precisely the trees on which the query holds

→ Complexity (in the query) is generally nonelementary Corollary

Evaluating a Boolean MSO query on a tree is inlinear timein the tree

(60)

→ Complexity (in the query) is generally nonelementary

Corollary

12/35

(61)

→ Complexity (in the query) is generally nonelementary Corollary

(62)

A small hack for non-Boolean queries

Remember our problem on non-Boolean queries:

Data: AtreeT

AtreeTwithmore colorsto select nodes

?

^Query:^Anon-BooleanMSOQ(x)formula

ABooleanMSO formulaQ⁰

“What are the pairs of a pink and blue node?”

“Are the two selected nodes pink and blue?”

i

^Result^{: All the}^a^{such that}^Q(a)^holds

All thewaysν to colorT such thatQ⁰ holds onν(T)

Fortechnical reasonsit will be simpler to:

•

Write the choice for variablesas colors on the tree

•

Replace the non-Boolean queryQby a Boolean queryQ⁰

•

Enumerate theways to color the tree

13/35

(63)

Data: AtreeT

?

i

•

(64)

Data: AtreeT

?

i

All thewaysν to colorT such thatQ⁰ holds onν(T) Fortechnical reasonsit will be simpler to:

•

13/35

(65)

Data: AtreeT

?

i

•

(66)

Data: AtreeT

AtreeT withmore colorsto select nodes

?

i

•

13/35

(67)

Data: AtreeT

?

i

•

(68)

Data: AtreeT

?

^Query:^Anon-BooleanMSOQ(x)formula ABooleanMSO formulaQ⁰

i

•

13/35

(69)

Data: AtreeT

?

i

•

(70)

Data: AtreeT

?

i

^Result_{All the}^{: All the}^a^{such that}^Q(a)^holds

waysν to colorT such thatQ⁰ holds onν(T) Fortechnical reasonsit will be simpler to:

•

Enumerate theways to color the tree _13/35

(71)

Uncertain trees

Now: Boolean query on a tree where the color of nodes isuncertain 1

5

6 7 2

4 3

Avaluationof a tree decides whether to keep(1) ordiscard(0) node labels Valuation: {2,3,77→1, ∗ 7→0}

A:“Is there both a pink and a blue node?”

→ Theresultsthat we want to enumerate are allvaluationsofT that makeAaccept

(72)

Uncertain trees

5

6 7 2

4 3

Avaluationof a tree decides whether to keep(1) ordiscard(0) node labels

Valuation: {2,3,77→1, ∗ 7→0}

14/35

(73)

Uncertain trees

5

6 7 2

4 3

(74)

Uncertain trees

5

6 7 2

4 3

Avaluationof a tree decides whether to keep(1) ordiscard(0) node labels Valuation: {27→1, ∗ 7→0}

14/35

(75)

Uncertain trees

5

6 7 2

4 3

Avaluationof a tree decides whether to keep(1) ordiscard(0) node labels Valuation: {2,77→1, ∗ 7→0}

(76)

Uncertain trees

5

6 7 2

4 3

14/35

(77)

Uncertain trees

5

6 7 2

4 3

The tree automatonAaccepts

(78)

Uncertain trees

5

6 7 2

4 3

Avaluationof a tree decides whether to keep(1) ordiscard(0) node labels Valuation: {27→1, ∗ 7→0}

The tree automatonArejects

14/35

(79)

Uncertain trees

5

6 7 2

4 3

The tree automatonAaccepts

(80)

Set circuits

We want to represent these results as aset circuit:

∪

×

x

>

×

y

•

Directed acyclic graph ofgates

•

^Output^gate:

•

^Variable^gates: ^x

•

^Constant^gates: ^> ^⊥

•

^Internal^gates: ^× ^∪

15/35

(81)

Set circuits

∪

×

x

>

×

y

•

^Output^gate:

•

^Variable^gates: ^x

•

(82)

Set circuits

∪

×

x

>

×

y

•

^Output^gate:

•

^Variable^gates: ^x

•

15/35

(83)

Set circuits

∪

×

x

>

×

y

•

^Output^gate:

•

^Variable^gates: ^x

•

(84)

Set circuits

∪

×

x

>

×

y

•

^Output^gate:

•

^Variable^gates: ^x

•

15/35

(85)

Semantics of set circuits

∪

×

x

>

×

y

{{x}} {{y}} {{}}

{{x}} {{x,y}} {{x},{x,y}}

Every gategcapturesa setS(g)

•

^Variablegate with labelx:S(g) :={{x}}

•

^>^-gates: ^{S(g) =}^{{}}

•

^⊥^-gates: ^{S(g) =}^∅

•

^×-gate with childreng1,g2:

S(g) :={s₁∪s₂|s₁∈S(g₁),s₂∈S(g₂)}

•

^∪-gate with childreng₁,g₂: S(g) :=S(g₁)∪S(g₂)

(86)

∪

×

x

>

×

y {{x}} {{y}}

{{}}

{{x}} {{x,y}} {{x},{x,y}}

•

^>^-gates: ^{S(g) =}^{{}}

•

^⊥^-gates: ^{S(g) =}^∅

•

S(g) :={s₁∪s₂|s₁∈S(g₁),s₂∈S(g₂)}

•

16/35

(87)

∪

×

x

>

×

y {{x}} {{y}}

{{}}

{{x}} {{x,y}} {{x},{x,y}}

•

^>^-gates: ^{S(g) =}^{{}}

•

^⊥^-gates: ^{S(g) =}^∅

•

S(g) :={s₁∪s₂|s₁∈S(g₁),s₂∈S(g₂)}

•

(88)

∪

×

x

>

×

y {{x}} {{y}}

{{}}

{{x}} {{x,y}} {{x},{x,y}}

•

^>^-gates: ^{S(g) =}^{{}}

•

^⊥^-gates: ^{S(g) =}^∅

•

S(g) :={s₁∪s₂|s₁∈S(g₁),s₂∈S(g₂)}

•

16/35

(89)

∪

×

x

>

×

y {{x}} {{y}}

{{}}

{{x}} {{x,y}}

{{x},{x,y}}

•

^>^-gates: ^{S(g) =}^{{}}

•

^⊥^-gates: ^{S(g) =}^∅

•

S(g) :={s₁∪s₂|s₁∈S(g₁),s₂∈S(g₂)}

•

(90)

∪

×

x

>

×

y {{x}} {{y}}

{{}}

{{x}} {{x,y}}

{{x},{x,y}}

•

^>^-gates: ^{S(g) =}^{{}}

•

^⊥^-gates: ^{S(g) =}^∅

•

S(g) :={s₁∪s₂|s₁∈S(g₁),s₂∈S(g₂)}

•

16/35

(91)

Set circuit for a query on a tree 1

5 6 7 2

4 3

Set circuitfor automatonAon uncertain treeT:

•

Variable gates: nodes ofT

•

^Condition:^Let^νbe a valuation ofT, thenA acceptsν(T)iff the setS(g0)of the output gateg₀ contains{n∈T |ν(n) =1}.

Query:Is there both a pink and a blue node?

×

∪ 7

2 3

×

× ×

∪ ∪ ∪ ∪

> 1 > 4 > 5 > 6

(92)

5 6 7 2

4 3

•

×

∪ 7

2 3

×

× ×

∪ ∪ ∪ ∪

> 1 > 4 > 5 > 6

17/35

(93)

5 6 7 2

4 3

•

×

∪ 7

2 3

×

× ×

∪ ∪ ∪ ∪

> 1 > 4 > 5 > 6

(94)

5 6 7 2

4 3

•

×

∪ 7

2 3

×

× ×

∪ ∪ ∪ ∪

> 1 > 4 > 5 > 6

17/35

(95)

5 6 7 2

4 3

•

×

∪ 7

2 3

×

× ×

∪ ∪ ∪ ∪

(96)

Building set circuits

Theorem

For any bottom-uptree automatonAand inputtreeT, we can build a set circuitofAonTinO(|A| × |T|)

•

^Alphabet:

•

^Automaton:“Is there both a pink and a blue node?”

•

^States:

{⊥,B,P,>}

•

^Final: ^{>}

•

Transitions:

> P ⊥

P P ⊥

n

∪ ∪ ∪ ∪

⊥ B P >

∪ ∪ ∪ ∪

⊥ B P >

∪ ∪ ∪ ∪

⊥ B P >

×

18/35

(97)

Theorem

•

^Alphabet:

•

^States:

{⊥,B,P,>}

•

^Final: ^{>}

•

Transitions:

>

P ⊥

P P ⊥

n

∪ ∪ ∪ ∪

⊥ B P >

∪ ∪ ∪ ∪

⊥ B P >

∪ ∪ ∪ ∪

⊥ B P >

×

(98)

Theorem

•

^Alphabet:

•

^States:

{⊥,B,P,>}

•

^Final: ^{>}

•

Transitions:

>

P ⊥

P P ⊥

n

∪ ∪ ∪ ∪

⊥ B P >

∪ ∪ ∪ ∪

⊥ B P >

∪ ∪ ∪ ∪

⊥ B P >

×

18/35

(99)

Theorem

•

^Alphabet:

•

^States:

{⊥,B,P,>}

•

^Final: ^{>}

•

Transitions:

>

P ⊥

P P ⊥

n

∪ ∪ ∪ ∪

⊥ B P >

∪ ∪ ∪ ∪

⊥ B P >

∪ ∪ ∪ ∪

⊥ B P >

×

(100)

Theorem

•

^Alphabet:

•

^States:

{⊥,B,P,>}

•

^Final: ^{>}

•

Transitions:

>

P ⊥

P P ⊥

n

∪ ∪ ∪ ∪

⊥ B P >

∪ ∪ ∪ ∪

⊥ B P >

∪ ∪ ∪ ∪

⊥ B P >

×

18/35

(101)

Theorem

•

^Alphabet:

•

^States:

{⊥,B,P,>}

•

^Final: ^{>}

•

Transitions:

>

P ⊥

P P ⊥

n

∪ ∪ ∪ ∪

⊥ B P >

∪ ∪ ∪ ∪

⊥ B P >

∪ ∪ ∪ ∪

⊥ B P >

×