Uncertain Data Management Open-World Query Answering

(1)

Uncertain Data Management Open-World Query Answering

Antoine Amarilli¹, Silviu Maniu² November 28th, 2016

1Télécom ParisTech

2LRI

1/37

(2)

Table of contents

Basics

Contexts Languages Chase

Advanced topics

(3)

Incompleteness

• We have aninstanceI

• Thetruestate of the world isW

• Wemay haveI6=W

• Imay becorrect:I⊆W

• Imay becomplete:W⊆I

→ Today,Iiscorrectbut notcomplete

(4)

Incompleteness

• Wemay haveI6=W

(5)

Incompleteness

• Wemay haveI6=W

(6)

Incompleteness and query evaluation

• Weknow:evaluate a queryQonI

• Wewant:evaluateQonW

• Wedon’t haveW

→ What can we do?!

(7)

Incompleteness and query evaluation

• Weknow:evaluate a queryQonI

• Wewant:evaluateQonW

• Wedon’t haveW

→ What can we do?!

(8)

Constraints to the rescue!

• We know thatI⊆W (correct)

• We know thatWsatisfies somelogical constraintsΘ

• QisentailedifW|=QforallW ⊇Isuch thatW |= Θ Definition (Open-World Query Answering – OWQA)

Given an instanceI, Boolean CQQ, and constraintsΘ, decide whether allW⊇Ithat satisfyΘalso satisfyQ.

Combined complexity. Input isI,Θ,Q Data complexity. Input isI

(9)

• QisentailedifW|=QforallW ⊇Isuch thatW |= Θ

Definition (Open-World Query Answering – OWQA)

(10)

(11)

(12)

Example

RelationClassinI

date teacher resp name num

2016-11-28 Antoine Fabian Uncert. Data Mgmt 2 2016-12-05 Antoine Fabian Uncert. Data Mgmt 3

BookinI

date room prof

2016-12-05 E242 John

Θ:∀date,prof,r,n,i, Class(date,prof,r,n,i)⇒

∃room, Book(date,room,prof)

“Every class has a booking.” Q:∃t rBook(“2016-11-28”,t,r)

“Is there a room booked on Nov 28th?”

(13)

Example

RelationClassinI

BookinI

2016-12-05 E242 John

“Every class has a booking.”

Q:∃t rBook(“2016-11-28”,t,r)

(14)

Example

RelationClassinI

BookinI

2016-12-05 E242 John

“Every class has a booking.”

Q:∃t rBook(“2016-11-28”,t,r)

(15)

Basics

Contexts

Languages Chase

Advanced topics

(16)

Logical satisfiability

OWQA is equivalent to:

isI∧Θ∧ ¬Qsatisfiable?

Is it just logical satisfiability then?!

• Iis where we want toscale

• ΘandQare usuallydifferentlanguages...

• ... if we express both in thesamelanguage, it will be hard to achieve goodcomplexities! (or evendecidability...)

(17)

(18)

(19)

(20)

• ... if we express both in thesamelanguage, it will be hard to achieve goodcomplexities!

(or evendecidability...)

(21)

Repairing the database

Class

2016-11-28 Antoine Fabian Uncert. Data Mgmt 2 2016-12-05 Antoine Fabian Uncert. Data Mgmt 3 2016-12-12 Antoine Fabian Uncert. Data Mgmt 4 2017-01-09 Silviu Fabian Uncert. Data Mgmt 5

? Fabian ? ? ?

? ? ? ? 1

Constraints:

• Theresponsiblefor a class must teachsomeclass

• Every class must have afirstsession

→ What can wededuce?

→ Qis true iff it is true onallcompletions

(22)

Class

? Fabian ? ? ?

? ? ? ? 1

Constraints:

(23)

Class

? Fabian ? ? ?

? ? ? ? 1

Constraints:

(24)

Class

? Fabian ? ? ?

? ? ? ? 1

Constraints:

(25)

Class

? Fabian ? ? ?

? ? ? ? 1

Constraints:

(26)

But why deal with broken databases?

• The data may have come from adifferent source

• The constraints may have been imposedafter the fact

• User input may beincorrect

• You want aresilientsystem...

(27)

But why deal with broken databases?

• The data may have come from adifferent source

• The constraints may have been imposedafter the fact

• User input may beincorrect

• You want aresilientsystem...

(28)

Reasoning (AI)

• Artificialreasoning: drawconsequencesfrom what you know

• Icontains thefacts

• Θare thereasoning rules

• Qis what we want tofigure out

→ Can wededuceQfromIusingΘ?

→ IsQcertainto hold?

(29)

Reasoning (AI)

• Artificialreasoning: drawconsequencesfrom what you know

• Icontains thefacts

• Θare thereasoning rules

• Qis what we want tofigure out

→ Can wededuceQfromIusingΘ?

→ IsQcertainto hold?

(30)

Data integration

• IcontainsI1, . . . ,In, the course databases of all D&K schools

• We want to create a virtualglobaldatabaseIof classes

• Fix arelationClass for the global database

• Θ: whenever someI_i contains a class,createit inR Class1

date name

2016-11-28 UDM 2016-12-05 UDM 2016-12-12 UDM 2017-01-14 UDM

Class

2016-11-28 ? ? UDM ?

2016-12-05 ? ? UDM ?

2016-12-12 ? ? UDM ?

2017-01-14 ? ? UDM ?

(31)

date name

2016-11-28 UDM 2016-12-05 UDM 2016-12-12 UDM 2017-01-14 UDM

Class

2016-11-28 ? ? UDM ?

2016-12-05 ? ? UDM ?

2016-12-12 ? ? UDM ?

2017-01-14 ? ? UDM ?

(32)

date name

2016-11-28 UDM 2016-12-05 UDM 2016-12-12 UDM 2017-01-14 UDM

Class

2016-11-28 ? ? UDM ?

2016-12-05 ? ? UDM ?

2016-12-12 ? ? UDM ?

2017-01-14 ? ? UDM ?

(33)

date name

2016-11-28 UDM 2016-12-05 UDM 2016-12-12 UDM 2017-01-14 UDM

Class

2016-11-28 ? ? UDM ?

2016-12-05 ? ? UDM ?

2016-12-12 ? ? UDM ?

2017-01-14 ? ? UDM ?

(34)

Ontology-based data access

• In general: use acommonschema for reasoning

• Icontains heterogeneousdata sources

• Θdescribesmappingsfrom sources to common schema andreasoning rulesandconstraintson the common schema

• Qis thequery posed the common schema

��

(35)

Basics Contexts

Languages

Chase

Advanced topics

(36)

First-order logic

AllconstraintsforΘare infirst-order logic(FO):

• containsatomsR(x,y,z)

• closed under Boolean AND, OR, NOT

• existential quantification∃xφ(x)

• universal quantification∀xφ(x)

→ Why not just use FO for constraints then?!

(37)

First-order logic

AllconstraintsforΘare infirst-order logic(FO):

• containsatomsR(x,y,z)

• closed under Boolean AND, OR, NOT

• existential quantification∃xφ(x)

• universal quantification∀xφ(x)

→ Why not just use FO for constraints then?!

(38)

FO is undecidable!

Given an input FO formulaΘ, is itsatisfiable? (i.e., OWQA withI=∅andQ:False).

→ This problem isundecidable!

• Proof: Encode a tiling system, or

encode transition rules for a Turing machine

→ We considerweaker languages

. . . . . . . . . ... ... ... ... ... ...

(39)

. . . . . . . . . ... ... ... ... ... ...

(40)

. . . . . . . . . ... ... ... ... ... ...

(41)

. . . . . . . . . ... ... ... ... ... ...

(42)

Tuple-generating dependencies

Tuple-generating dependencies(TGDs), classical database rules:

∀xQ⁰(x)⇒ ∃yQ⁰⁰(x,y)

whereQ⁰ andQ⁰⁰areCQs.

∀date,prof,r,n,i, Class(date,prof,r,n,i)⇒

∃room, Book(date,room,prof) Intuition:facts causemore factsto be created

Useful for:

• Integrity constraints:see above

• Schema mappings:copy facts fromI1 toI

• Reasoning:∀xHuman(x)⇒Mortal(x)

(43)

∀xQ⁰(x)⇒ ∃yQ⁰⁰(x,y)

Useful for:

(44)

∀xQ⁰(x)⇒ ∃yQ⁰⁰(x,y)

Useful for:

(45)

OWQA for TGDs is undecidable!

• Satisfiabilityof TGDsΘistrivial...

→ takeW=∅

• Satisfiabilityof TGDsΘand instanceIiseasy...

→ always possible –infiniterepair of violations (thechase– see later)

• OWQAforI,ΘandQisundecidable!

from [Chandra et al., 1981, Beeri and Vardi, 1981]

→ We needless expressivelanguages

(46)

→ takeW=∅

(47)

→ takeW=∅

(48)

→ takeW=∅

(49)

→ takeW=∅

(50)

→ takeW=∅

(51)

Inclusion dependencies

Inclusion dependencies(IDs), classical database rules:

∀xA⁰(x)⇒ ∃yA⁰⁰(x,y)

whereAandA⁰areatomsrather than CQs.

The TGD example was in factalso an ID:

(52)

Inclusion dependencies

Inclusion dependencies(IDs), classical database rules:

∀xA⁰(x)⇒ ∃yA⁰⁰(x,y)

whereAandA⁰areatomsrather than CQs.

The TGD example was in factalso an ID:

(53)

OWQA for IDs is decidable

• Satisfiabilityfor IDs isstill trivial

• OWQA:determine whetherQis implied byIand IDsΘ?

• Decidable!

• PSPACE-completecombined complexity [Johnson and Klug, 1984]

• Data complexityis PTIME, even AC⁰

→ Intuition: we canrewritethe queryQ

→ We will study otherdecidable classesof TGDs

(54)

• Decidable!

(55)

• Decidable!

(56)

• Decidable!

(57)

Basics Contexts Languages Chase

Advanced topics

(58)

Chase example

Class

∃room, Book(date,room,prof) Book

2016-12-05

?1

Antoine 2016-12-12

?2

Antoine

(59)

Chase example

Class

Book

2016-12-05

?1

Antoine 2016-12-12

?2

Antoine

(60)

Chase example

Class

2016-12-05

?1

Antoine 2016-12-12

?2

Antoine

(61)

Chase example

Class

2016-12-05 ?1 Antoine 2016-12-12 ?2 Antoine

(62)

Chase

• OWQA:test if instanceIand TGDsΘentailqueryQ

• Thechase: a most genericrepair ofIbyΘ

• Iterative process:start withI

• At each stage, findviolationsof each TGD inΘ

• TGD∀xQ⁰(x)⇒ ∃yQ⁰⁰(x,y)

• findasuch thatQ⁰(a)but notQ⁰⁰(a,b)for anyb

• Createnew elementsb

• Createnew facts to makeQ⁰(a,b)true

→ Take theinfinite resultof this process

(63)

Chase

(64)

Chase

(65)

Chase

(66)

Chase

(67)

Infinite chase example

∀t uMentor(t,u)⇒ ∃sMentor(s,t) Mentor

master padawan

Antoine Arthur Dent Silviu Arthur Dent

?1 Antoine

?2 Silviu

?3 ?1

?4 ?2

?5 ?3

?6 ?4

... ...

(68)

master padawan Antoine Arthur Dent Silviu Arthur Dent

?1 Antoine

?2 Silviu

?3 ?1

?4 ?2

?5 ?3

?6 ?4

... ...

(69)

?1 Antoine

?2 Silviu

?3 ?1

?4 ?2

?5 ?3

?6 ?4

... ...

(70)

?1 Antoine

?2 Silviu

?3 ?1

?4 ?2

?5 ?3

?6 ?4

... ...

(71)

?1 Antoine

?2 Silviu

?3 ?1

?4 ?2

?5 ?3

?6 ?4

... ...

(72)

?1 Antoine

?2 Silviu

?3 ?1

?4 ?2

?5 ?3

?6 ?4

... ...

(73)

Chase universality

• The chase is themost genericcompletion

• Can be shown usinghomomorphisms

→ A query is true in the chaseiffit is entailed Theorem

For any instanceI, TGDsΘ, Boolean CQQ, the following areequivalent:

• IandΘentailQ

• the chase ofIbyΘsatisfiesQ

→ How toreasonabout this infinite chase?

(74)

→ A query is true in the chaseiffit is entailed

Theorem

(75)

(76)

(77)

Chase termination

• Sometimes, the chase ofIbyΘisfinite

• We can thendecidewhether a queryQis entailed:

• Constructthe chase

• EvaluateQon the chase

→ When is the chasefinite?

(78)

(79)

(80)

Full dependencies

• If no TGD has an∃, then the chase isfinite

→ Nonew elementsare created

• Good:∀d r pBook(d,r,p)⇒Room(r)

• Bad:∀x Mentor(x)⇒ ∃yMentor(x,y)

(81)

Full dependencies

• If no TGD has an∃, then the chase isfinite

→ Nonew elementsare created

• Good:∀d r pBook(d,r,p)⇒Room(r)

• Bad:∀x Mentor(x)⇒ ∃yMentor(x,y)

(82)

Acyclicity

• Simplesufficient conditionfor finite chase:

If arelation nameoccurs at theleftof a TGD then it does not occur at theright

• Good:∀xClass(x)⇒ ∃y Book(x)

• Bad: ∀xMentor(x)⇒ ∃yMentor(x,y)

• More generalacyclicity conditions

(83)

Acyclicity

• Good:∀x Class(x)⇒ ∃y Book(x)

(84)

Acyclicity

• Good:∀x Class(x)⇒ ∃y Book(x)

(85)

Infinite chase

• What can we do if the chase isinfinite?

• Bounded derivation depth:we cantruncatethe chase:

• we fixΘand look at thesizeofQ

• bound the maximaldepthin the chase whereQcan be made true

• Bounded treewidth:the chase is like atree:

• we can reason aboutinfiniteandregulartrees

• usetree automata, following Courcelle’s theorem

• some rules preserve this, e.g., theguarded fragments

(86)

Infinite chase

• What can we do if the chase isinfinite?

• Bounded derivation depth:we cantruncatethe chase:

• we fixΘand look at thesizeofQ

• bound the maximaldepthin the chase whereQcan be made true

• Bounded treewidth:the chase is like atree:

• we can reason aboutinfiniteandregulartrees

• usetree automata, following Courcelle’s theorem

• some rules preserve this, e.g., theguarded fragments

(87)

Basics Contexts Languages Chase

Advanced topics

(88)

Query rewriting

• Thechase:reason aboutconsequencesofIunderΘ

• Other option: reason abouthow to proveQ

Q:∃t rBook(“2016-11-28”,t,r)

Q2:∃prof,r,n,i, Class(“2016-11-28”,prof,r,n,i)

(89)

Query rewriting

Q:∃t rBook(“2016-11-28”,t,r)

(90)

Query rewriting

Q:∃t rBook(“2016-11-28”,t,r)

(91)

Query rewriting

Q:∃t rBook(“2016-11-28”,t,r)

(92)

Query rewriting and inclusion dependencies

• To show that OWQA forinclusion dependenciesis decidable...

∀xA⁰(x)⇒ ∃yA⁰⁰(x,y)

• Rewrite all atoms in the query inall possible ways

→ Each atom rewritten byonly one atom

→ The query size does notincrease

• ReplaceQby aunionof conjunctive queries

→ OWQA for IDs isdecidable

→ OWQA for IDs hastractabledata complexity

(93)

∀xA⁰(x)⇒ ∃yA⁰⁰(x,y)

(94)

∀xA⁰(x)⇒ ∃yA⁰⁰(x,y)

(95)

Description logics

• TGDscannot expresseverything

∀xQ⁰(x)⇒ ∃yQ⁰⁰(x,y)

• Disjunction:ifAthen BorC

• Negation:youcannothave bothAandB

→ Description logics:expressive rules

• signature must havearityat most2

(96)

∀xQ⁰(x)⇒ ∃yQ⁰⁰(x,y)

• Disjunction:ifAthenBorC

(97)

∀xQ⁰(x)⇒ ∃yQ⁰⁰(x,y)

• Disjunction:ifAthenBorC

(98)

Description logics (2)

• Description logics have a specificsyntax

Teacher v Proft(∃Advisor⁻.Prof)

• Description logics exist in manyvariants

• Idea:precisecomplexityof OWQA depending on variant

Complexity Languages UNA Combined complexity Data complexity

core NLogSpace inAC⁰ inAC⁰

DL-Lite[F|N |(HF)|(HN)]

horn yes P ≤[Th.5.8, 5.13] inAC⁰ inAC⁰≤[Th.7.1]

krom NLogSpace≤[Th.5.7,5.13] inAC⁰ coNP DL-Lite[F|N |(HF)|(HN)]

bool NP ≤[Th.5.6, 5.13] inAC⁰≤[Cor.6.2] coNP

DL-Lite^[F|(HF)]_core/horn P≤[Cor.8.8] ≥[Th.8.7] P≥[Th.8.7] P

DL-Lite^[F|(HF)]_krom P≤[Cor.8.8] P coNP

DL-Lite^[F|(HF)]bool no NP P≤[Cor.8.8] coNP

DL-Lite^{[N |(HN)]}_core/horn NP≥[Th.8.4] coNP≥[Th.8.4] coNP DL-Lite^{[N |(HN)]}_krom/bool NP≤[Th.8.5] coNP coNP DL-Lite^HFcore/horn ExpTime≥[Th.5.10] P≥[Th.6.7] P≤[D] DL-Lite^HF_krom/bool

yes/no ExpTime coNP≥[Th.6.5] coNP

DL-Lite^HNcore/horn ExpTime coNP≥[Th.6.6] coNP

DL-Lite^HNkrom/bool ExpTime≤[F] coNP coNP≤[E]

(99)

DL-Lite^{[N |(HN)]}_core/horn NP≥[Th.8.4] coNP≥[Th.8.4] coNP DL-Lite^{[N |(HN)]}_krom/bool NP≤[Th.8.5] coNP coNP DL-Lite^HFcore/horn ExpTime≥[Th.5.10] P≥[Th.6.7] P≤[D] DL-Lite^HF_krom/bool

DL-Lite^HNkrom/bool ExpTime≤[F] coNP coNP≤[E]

(100)

Satisﬁability Instance checking Query answering DL-Lite^[core^|H] NLogSpace≥[A] inAC⁰ inAC⁰ DL-Lite^[horn^|H] yes/no P≤[Th.8.2] ≥[A] inAC⁰ inAC⁰≤[C]

DL-Lite^[|H]_krom NLogSpace≤[Th.8.2] inAC⁰ coNP≥[B]

DL-Lite^[bool^|H] NP≤[Th.8.2] ≥[A] inAC⁰≤[Th.8.3] coNP DL-Lite[F|N |(HF)|(HN)]

DL-Lite^{[N |(HN)]}_core/horn NP≥[Th.8.4] coNP≥[Th.8.4] coNP DL-Lite^{[N |(HN)]}_krom/bool NP≤[Th.8.5] coNP coNP DL-Lite^HFcore/horn ExpTime≥[Th.5.10] P≥[Th.6.7] P≤[D]

DL-Lite^HF_krom/bool

(101)

Equality-generating dependencies

• Another important constraint forΘ: functional dependencies

• There can’t betwo bookingsfor one room at the same time

• There can’t betwo roomsfor one session

• Functional dependencies can beaddedtoΘfor OWQA

• Decidablefor description logics

• Undecidablewith inclusion dependencies

(102)

Equality-generating dependencies

• Another important constraint forΘ: functional dependencies

• There can’t betwo bookingsfor one room at the same time

• There can’t betwo roomsfor one session

• Functional dependencies can beaddedtoΘfor OWQA

• Decidablefor description logics

• Undecidablewith inclusion dependencies

(103)

Finite models

Given an instanceI, Boolean CQQ, and constraintsΘ, decide whether all

finite

W ⊇Ithat satisfyΘsatisfyQ.

• The worldW is actuallyfinite

• Shouldn’t wereflectthis?

• If thechaseis infinite, it no longer works

• Imposing finiteness may make adifference

→ Veryhardto reason about FOWQA!

(104)

Finite models

Given an instanceI, Boolean CQQ, and constraintsΘ, decide whether all

finite

W ⊇Ithat satisfyΘsatisfyQ.

• The worldWis actuallyfinite

(105)

Finite models

Definition (Finite Open-World Query Answering – FOWQA)

Given an instanceI, Boolean CQQ, and constraintsΘ, decide whether allfiniteW ⊇Ithat satisfyΘsatisfyQ.

(106)

Finite models

Definition (Finite Open-World Query Answering – FOWQA)

Given an instanceI, Boolean CQQ, and constraintsΘ, decide whether allfiniteW ⊇Ithat satisfyΘsatisfyQ.

(107)

Partial completeness

• We have assumed thatIwasincomplete

• Sometimes, we knowwhich relationsare complete

• e.g., the list ofroomsmay becomplete

• the list ofclassesmay beincomplete

→ Partially complete databases[Razniewski et al., 2015]

• We may knowother things:

• If I know the lecturer of a class, then I knowall lecturers

• If I knowone sessionof a class, I knowall sessions

→ Partial completeness assumption[Galárraga et al., 2013]

(108)

(109)

(110)

(111)

Slide credits

• Slide 34:

http://www.slideshare.net/MartnRezk/slides-swat4-ls, slide 17, licence CC-BY-SA 3.0¹

• Slides 16 and 36: Jaques Rouxel,Les Shadoks(reproduit en vertu du droit de citation)

• Slide 34: [Artale et al., 2009], p 18

(112)

References I

Abiteboul, S., Hull, R., and Vianu, V. (1995).

Foundations of Databases.

Addison-Wesley.

http://webdam.inria.fr/Alice/pdfs/all.pdf.

Artale, A., Calvanese, D., Kontchakov, R., and Zakharyaschev, M.

(2009).

The DL-Lite family and relations.

Journal of artificial intelligence research.

https://www.jair.org/media/2820/live-2820-4662-jair.pdf.