Query Lineages and Knowledge Compilation

(1)

Query Lineages and Knowledge Compilation

Antoine Amarilli¹ November 13, 2019

(2)

Query Lineage Definitions

•

Relational database instanceI: set of facts

•

Boolean queryQ: take an instance and answer yes/no

•

^Lineageôf^QônÎ^{: a}Boolean circuitover the facts ofI accepting exactly thesubsetsofIwhereQis true

Example: queryQ:

∃xyz R(x,y)∧S(y,z) R

a b a⁰ b

S b c

R(a,b) R(a⁰,b)

∨ S(b,c)

∧

(3)

•

Example: queryQ:

∃xyz R(x,y)∧S(y,z)

R a b a⁰ b

S b c

R(a,b) R(a⁰,b)

∨ S(b,c)

∧

(4)

•

Example: queryQ:

R S

R(a,b) R(a⁰,b)

∨ S(b,c)

∧

(5)

•

Example: queryQ:

R S

R(a,b) R(a⁰,b)

∨ S(b,c)

∧

(6)

•

Example: queryQ:

R S

∨ S(b,c)

∧

(7)

Why bother? Applications of query lineages

•

Evaluation:the lineage gives you the query answer

•

^Counting:

• Compute theprobabilitythat the query is true

• Counthow many subinstancessatisfy the query

•

Enumeration:efficientlyenumeratethe subinstances

•

Explanation:

• Representation ofwhythe query is true

• What-if:is the query still true without these facts?

(8)

•

^Counting:

•

Explanation:

(9)

•

^Counting:

•

Explanation:

(10)

•

^Counting:

•

Explanation:

(11)

Another application: Query answers

Lineages can also represent theanswerstonon-Boolean queries!

•

Study answers ofnon-Boolean query Q(x,y)on instanceI

Q(x,y) :∃z R(x,y)∧S(y,z) I:R(a,b),R(a⁰,b),S(b,c)

•

^Addassignment facts X(v),Y(v)toI for each elementv(linear)

X(a),X(a⁰),X(b),X(c) Y(a),Y(a⁰),Y(b),Y(c)

•

Consider theBoolean query Q⁰ :X(x)∧Y(y)∧Q(x,y)

X(x)∧Y(y)∧(∃z R(x,y)∧S(y,z))

•

^{Compute a}^lineage^C⁰ ^of^Q⁰

onIplus assignment facts

(X(a)∧R(a,b)∨X(a⁰)∧R(a⁰,b))

∧Y(b)∧S(b,c)

•

^Define^Cby replacing all variables by1 except assignment facts

(X(a)∨X(a⁰))∧Y(b)

→ The circuitCrepresents thequery answers (a,b)and(a⁰,b) We cancountthe answers,enumeratethem, etc.

(12)

•

X(x)∧Y(y)∧(∃z R(x,y)∧S(y,z))

•

(X(a)∧R(a,b)∨X(a⁰)∧R(a⁰,b))

∧Y(b)∧S(b,c)

•

(X(a)∨X(a⁰))∧Y(b)

(13)

•

X(x)∧Y(y)∧(∃z R(x,y)∧S(y,z))

•

(X(a)∧R(a,b)∨X(a⁰)∧R(a⁰,b))

∧Y(b)∧S(b,c)

•

(X(a)∨X(a⁰))∧Y(b)

(14)

•

^Addassignment factsX(v),Y(v)toI for each elementv(linear)

•

X(x)∧Y(y)∧(∃z R(x,y)∧S(y,z))

•

(X(a)∧R(a,b)∨X(a⁰)∧R(a⁰,b))

∧Y(b)∧S(b,c)

•

(X(a)∨X(a⁰))∧Y(b)

(15)

•

X(x)∧Y(y)∧(∃z R(x,y)∧S(y,z))

•

(X(a)∧R(a,b)∨X(a⁰)∧R(a⁰,b))

∧Y(b)∧S(b,c)

•

(X(a)∨X(a⁰))∧Y(b)

(16)

•

X(x)∧Y(y)∧(∃z R(x,y)∧S(y,z))

•

(X(a)∧R(a,b)∨X(a⁰)∧R(a⁰,b))

∧Y(b)∧S(b,c)

•

(X(a)∨X(a⁰))∧Y(b)

(17)

•

X(x)∧Y(y)∧(∃z R(x,y)∧S(y,z))

•

(X(a)∧R(a,b)∨X(a⁰)∧R(a⁰,b))

∧Y(b)∧S(b,c)

•

(X(a)∨X(a⁰))∧Y(b)

(18)

•

X(x)∧Y(y)∧(∃z R(x,y)∧S(y,z))

•

(X(a)∧R(a,b)∨X(a⁰)∧R(a⁰,b))

∧Y(b)∧S(b,c)

•

(X(a)∨X(a⁰))∧Y(b)

(19)

•

X(x)∧Y(y)∧(∃z R(x,y)∧S(y,z))

•

(X(a)∧R(a,b)∨X(a⁰)∧R(a⁰,b))

∧Y(b)∧S(b,c)

•

(X(a)∨X(a⁰))∧Y(b)

(20)

•

X(x)∧Y(y)∧(∃z R(x,y)∧S(y,z))

•

(X(a)∧R(a,b)∨X(a⁰)∧R(a⁰,b))

∧Y(b)∧S(b,c)

•

(X(a)∨X(a⁰))∧Y(b)

(21)

•

X(x)∧Y(y)∧(∃z R(x,y)∧S(y,z))

•

(X(a)∧R(a,b)∨X(a⁰)∧R(a⁰,b))

∧Y(b)∧S(b,c)

•

(22)

•

X(x)∧Y(y)∧(∃z R(x,y)∧S(y,z))

•

(X(a)∧R(a,b)∨X(a⁰)∧R(a⁰,b))

∧Y(b)∧S(b,c)

•

(a,b)and(a⁰,b) We cancountthe answers,enumeratethem, etc.

(23)

•

X(x)∧Y(y)∧(∃z R(x,y)∧S(y,z))

•

(X(a)∧R(a,b)∨X(a⁰)∧R(a⁰,b))

∧Y(b)∧S(b,c)

•

We cancountthe answers,enumeratethem, etc.

(24)

•

X(x)∧Y(y)∧(∃z R(x,y)∧S(y,z))

•

(X(a)∧R(a,b)∨X(a⁰)∧R(a⁰,b))

∧Y(b)∧S(b,c)

•

^Define^Cby replacing all variables by1 (X(a)∨X(a⁰))∧Y(b)

(25)

Related work: Semiring provenance

Semiring provenance([Green et al., 2007], PODS ToT award): annotate results of arelational algebraquery with asemiring expression

What is the difference?

•

^Lineage= provenance in thesemiring of Boolean functions

• Nomultiplicityof facts or derivations

→ Essentially only make sense forrelational algebra

•

Circuit representation:more concise

(26)

• Nomultiplicityof facts or derivations

→ Essentially only make sense forrelational algebra

•

(27)

•

(28)

•

(29)

Computing lineages: theory

•

Unions of Conjunctive Queries(UCQ) Theorem

For anyUCQ, given an instance, we can construct its lineage inpolynomial time.

(disjunction of all matches)

•

Acyclic Conjunctive Queries(ACQ) Theorem

For anyACQ, given an instance, we can construct its lineage

inlinear time. (following a join tree)

•

Monadic Second Order queries(MSO) Theorem

For anyMSO query, given a tree (or word), we can construct

its lineage inlinear time. (automaton product)

•

^Datalog:see [Deutch et al., 2014],PTIME

(30)

•

For anyUCQ, given an instance, we can construct its lineage

inpolynomial time. (disjunction of all matches)

•

(31)

•

For anyACQ, given an instance, we can construct its lineage inlinear time.

(following a join tree)

•

(32)

•

(33)

•

(automaton product)

•

(34)

•

(35)

•

(36)

Computing lineages: practice

•

^ProvSQL:PostgreSQL extension to compute query lineages

•

Keeps track of thelineageof query results as acircuit

You can run it!https://github.com/PierreSenellart/provsql

(37)

Computing lineages: practice

•

^ProvSQL:PostgreSQL extension to compute query lineages

•

Keeps track of thelineageof query results as acircuit

(38)

Knowledge compilation and tractable circuit classes Knowledge compilation:

•

Translate your problem to acircuit

•

^Designgeneral-purpose algorithmson the circuits

→ Satisfiability, counting, probability computation, enumeration...

Without knowledge compilation: O(n²)algorithms

Setting A Task 1 Task 2

Setting B Task 1 Task 2

With knowledge compilation: O(n)algorithms

Setting A Circuit Setting B Circuit Circuit Task 1 Circuit Task 2

→ Tractability: use tractablecircuit classes

(39)

•

Setting B Task 1

With knowledge compilation: O(n)algorithms

Setting A Circuit Setting B Circuit Circuit Task 1 Circuit Task 2

(40)

•

Setting B Task 1

With knowledge compilation:

O(n)algorithms

Setting A Circuit Setting B Circuit

Circuit Task 1 Circuit Task 2

(41)

•

Setting B Task 1

O(n)algorithms

Setting A Circuit Setting B Circuit Circuit Task 1

(42)

•

Setting B Task 1

O(n)algorithms

Setting A Circuit Setting B Circuit Circuit Task 1

(43)

Tractable circuit classes: Theory vs practice There is a wholezooof tractable circuit classes...

But inpracticethere are solvers for arbitrary circuits:

•

Satisfiability (SAT):MapleSAT, Cadical, Glucose, etc.

•

^Counting:c2d, d4, dsharp, etc.

(44)

Tractable circuit classes: Theory vs practice There is a wholezooof tractable circuit classes...

But inpracticethere are solvers for arbitrary circuits:

(45)

A tractable circuit class: d-SDNNF

∨

∧

x ∧

y ¬

z

∧

¬

x ∧

y z

•

• y z x

•

Negation Normal Form: negations only applied to the leaves

•

Decomposable: inputs of∧-gates are independent(no variable has a path to two different inputs of the same

∧-gate)

→ efficientsatisfiability

•

Deterministic: inputs of∨-gates are mutually exclusive

→ efficientcounting

•

^Structured: there is avtreethat structures the∧-gates

→ efficientenumeration

(46)

∨

∧

x ∧

y ¬

z

∧

¬

x ∧

y z

•

• y z x

•

∧-gate)

•

(47)

∨

∧

x ∧

y ¬

z

∧

¬

x ∧

y z

•

• y z x

•

∧-gate)

•

(48)

∨

∧

x ∧

y ¬

z

∧

¬

x ∧

y z

•

• y z x

•

∧-gate)

•

(49)

∨

∧

x ∧

y ¬

z

∧

¬

x ∧

y z

•

• y z x

•

∧-gate)

•

(50)

∨

∧

x ∧

y ¬

z

∧

¬

x ∧

y z

•

• y z x

•

∧-gate)

•

(51)

∨

∧

x ∧

y ¬

z

∧

¬

x ∧

y z

•

• y z x

•

∧-gate)

•

(52)

∨

∧

x ∧

y ¬

z

∧

¬

x ∧

y z

•

• y z x

•

∧-gate)

•

(53)

Computing lineages in tractable classes

•

Self-join-Free Conjunctive Queries(SFCQs) Theorem [Jha and Suciu, 2013]

For anysafe SFCQ, given an instance, we can construct anOBDD representationof the lineage inpolynomial time.

•

Monadic Second Order queries (MSO) Theorem [Amarilli et al., 2015]

For anyMSO query, given a tree (or word), we can construct ad-SDNNF representation(or OBDD) of the lineage inlinear time.

•

Unions of Conjunctive Queries(UCQ) Conjecture (see [Monet, 2020])

For anysafe UCQ, given an instance, we can construct ad-Drepresentation of the lineage inpolynomial time.

Thanks for your attention!

(54)

•

Unions of Conjunctive Queries(UCQ) Conjecture (see [Monet, 2020])

For anysafe UCQ, given an instance, we can construct ad-Drepresentation of the lineage inpolynomial time.

(55)

•

Unions of Conjunctive Queries(UCQ)

(56)

•

Unions of Conjunctive Queries(UCQ)

(57)

References i

Amarilli, A., Bourhis, P., and Senellart, P. (2015).

Provenance Circuits for Trees and Treelike Instances.

InICALP.

Deutch, D., Milo, T., Roy, S., and Tannen, V. (2014).

Circuits for Datalog Provenance.

InICDT.

Green, T. J., Karvounarakis, G., and Tannen, V. (2007).

Provenance semirings.

(58)

References ii

Jha, A. and Suciu, D. (2013).

Knowledge compilation meets database theory: compiling queries to decision diagrams.

Theory of Computing Systems, 52(3).

Monet, M. (2020).

Solving a Special Case of the Intensional vs Extensional Conjecture in Probabilistic Databases.

InPODS. To appear.