On the Complexity of Deriving Schema Mappings from Database Instances

On the Complexity of Deriving Schema Mappings from Database Instances

Pierre Senellart^1;2 Georg Gottlob³

1

2

3

30 November 2007, Gemo Seminar

Different sources organize the same data differently

Jeffrey D. Ullman

List of publications from the DBLP Bibliography Server FAQ

Coauthor Index Ask others: ACM DL/Guide CiteSeer CSB Google MSN Yahoo

Home Page

2007

240 EE Foto N. Afrati, Chen Li, Jeffrey D. Ullman: Using views to generate efficient evaluation plans for queries. J. Comput. Syst. Sci. 73(5): 703724 (2007)

2005

239 EE Jeffrey D. Ullman: Gradiance OnLine Accelerated Learning. ACSC 2005: 36

238 EE Serge Abiteboul, Rakesh Agrawal, Philip A. Bernstein, Michael J. Carey, Stefano Ceri, W. Bruce Croft, David J. DeWitt, Michael J. Franklin, Hector GarciaMolina, Dieter Gawlick, Jim Gray, Laura M. Haas, Alon Y. Halevy, Joseph M. Hellerstein, Yannis E. Ioannidis, Martin L. Kersten, Michael J. Pazzani, Michael Lesk, David Maier, Jeffrey F. Naughton, HansJörg Schek, Timos K. Sellis, Avi Silberschatz, Michael Stonebraker, Richard T. Snodgrass, Jeffrey D. Ullman, Gerhard Weikum, Jennifer Widom, Stanley B.

Zdonik: The Lowell database research selfassessment. Commun. ACM 48(5): 111118 (2005) 237 EE Serge Abiteboul, Richard Hull, Victor Vianu, Sheila A. Greibach, Michael A. Harrison, Ellis Horowitz,

Daniel J. Rosenkrantz, Jeffrey D. Ullman, Moshe Y. Vardi: In memory of Seymour Ginsburg 1928 2004.

SIGMOD Record 34(1): 512 (2005)

2003

236 EE Jeffrey D. Ullman: A Survey of New Directions in Database System. DASFAA 2003: 3

235 EE Jeffrey D. Ullman: Improving the Efficiency of DatabaseSystem Teaching. SIGMOD Conference 2003:

13

234 EE Jim Gray, HansJörg Schek, Michael Stonebraker, Jeffrey D. Ullman: The Lowell Report. SIGMOD Conference 2003: 680

233 EE Serge Abiteboul, Rakesh Agrawal, Philip A. Bernstein, Michael J. Carey, Stefano Ceri, W. Bruce Croft, David J. DeWitt, Michael J. Franklin, Hector GarciaMolina, Dieter Gawlick, Jim Gray, Laura M. Haas, Alon Y. Halevy, Joseph M. Hellerstein, Yannis E. Ioannidis, Martin L. Kersten, Michael J. Pazzani, Michael Lesk, David Maier, Jeffrey F. Naughton, HansJörg Schek, Timos K. Sellis, Avi Silberschatz, Michael Stonebraker, Richard T. Snodgrass, Jeffrey D. Ullman, Gerhard Weikum, Jennifer Widom, Stanley B.

Zdonik: The Lowell Database Research Self Assessment CoRR cs.DB/0310006: (2003)

232 EE Anand Rajaraman, Jeffrey D. Ullman: Querying websites using compact skeletons. J. Comput. Syst. Sci.

66(4): 809851 (2003)

2001

231 EE Chen Li, Mayank Bawa, Jeffrey D. Ullman: Minimizing View Sets without Losing QueryAnswering Power.

ICDT 2001: 99113

230 EE Anand Rajaraman, Jeffrey D. Ullman: Querying Websites Using Compact Skeletons. PODS 2001 229 EE Foto N. Afrati, Chen Li, Jeffrey D. Ullman: Generating Efficient Plans for Queries Using Views. SIGMOD

Conference 2001: 319330

228 EE Edith Cohen, Mayur Datar, Shinji Fujiwara, Aristides Gionis, Piotr Indyk, Rajeev Motwani, Jeffrey D.

Ullman, Cheng Yang: Finding Interesting Associations without Support Pruning. IEEE Trans. Knowl.

Data Eng. 13(1): 6478 (2001)

2000

227 Hector GarciaMolina, Jeffrey D. Ullman, Jennifer Widom: Database System Implementation PrenticeHall 2000

226 EE Jeffrey D. Ullman: A Survey of AssociationRule Mining. Discovery Science 2000: 114

225 EE Edith Cohen, Mayur Datar, Shinji Fujiwara, Aristides Gionis, Piotr Indyk, Rajeev Motwani, Jeffrey D.

Ullman, Cheng Yang: Finding Interesting Associations without Support Pruning. ICDE 2000: 489499 224 EE Shinji Fujiwara, Jeffrey D. Ullman, Rajeev Motwani: Dynamic MissCounting Algorithms: Finding

Different sources organize the same data differently

Advanced Scholar Search

Scholar Preferences Scholar Help

Scholar All articles Recent articles Results 1 10 of about 12 for author:"jd ullman". (0.07 seconds)

jd ullman J Ullman J Hopcroft A Rajaraman B Konikow ska A Aho

Querying websites using compact skeletons all 11 versions »

A Rajaraman, JD Ullman Journal of Computer and System Sciences, 2003 Elsevier Several commercial applications, such as online comparison shopping and process automation, require integrating information that is scattered across multiple w ebsites or XML documents. Much research has been devoted to this problem, ...

Cited by 13 Related Articles Web Search

[BOOK] Wprowadzenie do teorii automatów, jezyków i obliczen JE Hopcroft, JD Ullman, B Konikow ska 2003 Wydaw . Naukow e PWN Cited by 15 Related Articles Web Search

Improving the efficiency of databasesystem teaching all 3 versions »

JD Ullman Proceedings of the 2003 ACM SIGMOD international conference …, 2003 portal.acm.org ABSTRACT The education industry has a very poor record of produc tivity gains.

In this brief article, I outline some of the w ays the teaching of a college course in database systems could be made more ecient, and sta time used ...

Cited by 4 Related Articles Web Search

A survey of new directions in database systems all 5 versions »

JD Ullman Database Systems for Advanced Applications, 2003.(DASFAA …, 2003 ieeexplore.ieee.org A survey of new directions in database systems. Ullman, JD Stanford University;

This paper appears in: Database Systems for Advanced Applications, 2003.

(DASFAA 2003). Proceedings. Eighth International ...

Cited by 3 Related Articles Web Search [CITATION] ????

AV Aho, R Sethi, JD Ullman 2003 ??: ???????

Cited by 6 Related Articles Web Search [BOOK] Automi, linguaggi e calcolabilità

… Hopcroft, R Motw ani, JD Ullman, L Bernardinello, L … 2003 Pearson Education Italia Cited by 5 Related Articles Web Search

[CITATION] ???????

H GarciaMolina, JD Ullman, J Widom 2003 ??: ???????

Cited by 4 Related Articles Web Search [BOOK] Implementacja systemów baz danych

H GarciaMolina, J Widom, M Jurkiew icz, JD Ullman 2003 Wydaw nictw a Naukow oTechniczne Cited by 3 Related Articles Web Search

[BOOK] Projektowanie i analiza algorytmów: klasyczna praca z teorii algorytmów komputerowych AV Aho, JE Hopcroft, JD Ullman, W Derechow ski 2003 Helion

Cited by 2 Related Articles Web Search [CITATION] ???????

AV AHO, JE HOPCROFT, JD ULLMAN 2003 ??: ???????

Cited by 1 Related Articles Web Search

Result Page: 1 2 Next

Google Home About Google About Google Scholar

©2007 Google

Motivation

Context

Multiple data sources containing information about similar entities, with someredundancy (e.g., sources of the deep Web).

Several different ways to present this information, i.e., several different schemata.

No a prioriinformation about (some of) these schemata.

How to know the relationshipsbetween these schemata, by just looking at the instances?

Other way to see this problem: Match operator on schema mappings, in the setting ofdata exchange.

Motivation

Context

Multiple data sources containing information about similar entities, with someredundancy (e.g., sources of the deep Web).

Several different ways to present this information, i.e., several different schemata.

No a prioriinformation about (some of) these schemata.

How to know the relationshipsbetween these schemata, by just looking at the instances?

Other way to see this problem: Match operator on schema mappings, in the setting ofdata exchange.

Motivation

Context

Multiple data sources containing information about similar entities, with someredundancy (e.g., sources of the deep Web).

Several different ways to present this information, i.e., several different schemata.

No a prioriinformation about (some of) these schemata.

How to know the relationshipsbetween these schemata, by just looking at the instances?

Other way to see this problem: Match operator on schema mappings, in the setting ofdata exchange.

Problem definition

Problem

Given two (relational) database instances I and J with different schemata, what is the optimal description of J knowing I (with a finite set of formulas in some logical language)?

What does optimal implies:

Conciseness of description.

Validity of facts predicted byI and . All facts ofJ explainedby I and .

(Note the asymmetry between I and J; context ofdata exchange where J is computed fromI and ).

Problem definition

Problem

Given two (relational) database instances I and J with different schemata, what is the optimal description of J knowing I (with a finite set of formulas in some logical language)?

What does optimal implies:

Conciseness of description.

Validity of facts predicted byI and . All facts ofJ explainedby I and .

(Note the asymmetry between I and J; context ofdata exchange where J is computed fromI and ).

Outline

Source-to-target tuple-generating dependencies

Definition (Source-to-target tgd) First-order formula of the form:

8x'(x) ! 9y (x;y) with:

'conjunctionof source relation atoms;

conjunctionof target relation atoms;

all variables ofx bound in'.

Example

8x18x2R1(x1;x2) ^R2(x2) ! 9y R⁰(x1;y)

Particular tgds

We consider two ways of having simplertgds:

Disallow existential quantifiers on the right hand-side: full tgds.

Disallow cycles on both left- and right-hand sides: acyclic tgds.

(Classical notion of acyclicity on hypergraphs extending the basic notion of acyclicity on graphs.)

Examples

8x₁8x₂8x₃R₁(x₁;x₂) ^R₂(x₂;x₃) ^R₃(x₃;x₁) !R⁰(x₁)iscyclic(and full).

8x18x28x3R1(x1;x2) ^R2(x2;x3) !R⁰(x1)isacyclic(and full).

We then consider the languages:

Ltgd: arbitrary source-to-target tgds;

Lfull: full tgds;

Lacyc: acyclic tgds;

Lfacyc: full and acyclic tgds.

Particular tgds

We consider two ways of having simplertgds:

Disallow existential quantifiers on the right hand-side: full tgds.

Disallow cycles on both left- and right-hand sides: acyclic tgds.

(Classical notion of acyclicity on hypergraphs extending the basic notion of acyclicity on graphs.)

Examples

8x₁8x₂8x₃R₁(x₁;x₂) ^R₂(x₂;x₃) ^R₃(x₃;x₁) !R⁰(x₁)iscyclic(and full).

8x18x28x3R1(x1;x2) ^R2(x2;x3) !R⁰(x1)isacyclic(and full).

We then consider the languages:

Ltgd: arbitrary source-to-target tgds;

Lfull: full tgds;

Lacyc: acyclic tgds;

Lfacyc: full and acyclic tgds.

Particular tgds

We consider two ways of having simplertgds:

Disallow existential quantifiers on the right hand-side: full tgds.

Disallow cycles on both left- and right-hand sides: acyclic tgds.

(Classical notion of acyclicity on hypergraphs extending the basic notion of acyclicity on graphs.)

Examples

8x₁8x₂8x₃R₁(x₁;x₂) ^R₂(x₂;x₃) ^R₃(x₃;x₁) !R⁰(x₁)iscyclic(and full).

8x18x28x3R1(x1;x2) ^R2(x2;x3) !R⁰(x1)isacyclic(and full).

We then consider the languages:

Ltgd: arbitrary source-to-target tgds;

Lfull: full tgds;

Lacyc: acyclic tgds;

Lfacyc: full and acyclic tgds.

How to define the pertinence of a set of tgds?

Example

R R⁰

a b c d

a a b b c a d d g h 0 = ?

1 = f8x R(x) !R⁰(x;x)g 2 = f8x R(x) ! 9y R⁰(x;y)g 3 = f8x8y R(x) ^R(y) !R⁰(x;y)g 4 = f9x9y R⁰(x;y)g

How to define the pertinence of a set of tgds?

Example

R R⁰

a b c d

a a b b c a d d g h 0 = ?

1 = f8x R(x) !R⁰(x;x)g 2 = f8x R(x) ! 9y R⁰(x;y)g 3 = f8x8y R(x) ^R(y) !R⁰(x;y)g 4 = f9x9y R⁰(x;y)g

Idea

Size of a formula: number of occurrences of variables and constants.

Costof a schema mapping : Size of theminimum repairof that is valid and explains all facts of J.

Types of repairs considered:

“fix” a universal quantifier by adding conditions (x =a or x 6=a);

“fix” an existential quantifier by giving corresponding constants ((x) !y=a with a conjunction of conditions on universally quantified variables);

addground factsto the target instance.

The problem is then to find a schema mapping ofminimal cost.

Example of cost computation

Example

R R⁰

a b c d

a a b b c a d d g h 8x R(x) !R⁰(x;x)

Cost: 17

PredictedR⁰ a a b b c c d d

Example of cost computation

Example

R R⁰

a b c d

a a b b c a d d g h 8x R(x)^x 6=c !R⁰(x;x)

Cost: 17

PredictedR⁰ a a b b d d

Example of cost computation

Example

R R⁰

a b c d

a a b b c a d d g h 8x R(x)^x 6=c !R⁰(x;x)

R(c;a)

R(g;h) Cost: 17

PredictedR⁰ a a b b c a d d

Example of cost computation

Example

R R⁰

a b c d

a a b b c a d d g h 8x R(x)^x 6=c !R⁰(x;x)

R(c;a) R(g;h)

Cost: 17

PredictedR⁰ a a b b c c d d g h

Example of cost computation

Example

R R⁰

a b c d

a a b b c a d d g h 8x R(x)^x 6=c !R⁰(x;x)

9x9y R(x;y) ^x =c^y =a 9x9y R(x;y) ^x =g^y=h

Cost: 17

PredictedR⁰ a a b b c c d d g h

Example of cost computation

Example

R R⁰

a b c d

a a b b c a d d g h 8x R(x) ^x 6=c !R⁰(x;x)

9x9y R(x;y) ^x =c^y =a 9x9y R(x;y) ^x =g^y=h

Cost: 17

PredictedR⁰ a a b b c c d d g h

Problems considered

Decision problemsof interest:

Cost : Is the cost of a given schema mapping less thanK? Optimality : Is a given schema mapping optimal?

Complexity? Algorithms?

Problems considered

Decision problemsof interest:

Cost : Is the cost of a given schema mapping less thanK? Optimality : Is a given schema mapping optimal?

Complexity? Algorithms?

Outline

Behavior for simple operators

Consider the elementary operatorsof the relational algebra:

Projection Intersection

Selection (conjunction of atomic conditions) Cross Product

Join (on a given attribute) Theorem

For any elementary operator , the tgd naturally associated with is optimal with respect to (I; (I)) (or ((J);J)), under some basic assumptions.

Behavior for simple operators

Consider the elementary operatorsof the relational algebra:

Projection Intersection

Selection (conjunction of atomic conditions) Cross Product

Join (on a given attribute) Theorem

For any elementary operator , the tgd naturally associated with is optimal with respect to (I; (I)) (or ((J);J)), under some basic assumptions.

Examples of naturally associated tgds

Examples

Condition I andJ Optimal tgd

Projection

I 6= ? J= 1(I) R(x;y) !R⁰(x) 1(J) \ 2(J) = ?,

j1(J)j >2 I = 1(J) R(x) ! 9y R⁰(x;y) Selection j'(I)j >^size(')+₃ ² J= '(I) R(x) !R⁰(x)

'(J) 6= ? I = '(J) R(x) !R⁰(x) Product R1^I6= ?,R^I26= ? J=R^I1R^I2 R1(x) ^R2(y) !R⁰(x;y)

R⁰1

J6= ?,R⁰2

J 6= ? I =R1⁰ JR⁰2

J R(x;y) !R⁰1(x) ^R2⁰(y)

(Combined) Complexity Results

Ltgd Lfull

Cost ^P₃,^P₂-hard ^P₂, (co)NP-hard Optimality ^P4, (co)NP-hard ^P3, (co)NP-hard

Lacyc Lfacyc

Cost ^P2, (co)NP-hard NP-complete Optimality ^P₃, (co)NP-hard ^P₂, (co)NP-hard

(Combined) Complexity Results

Ltgd Lfull

Cost ^P₃,^P₂-hard ^P₂, (co)NP-hard Optimality ^P4, (co)NP-hard ^P3, (co)NP-hard

Lacyc Lfacyc

Cost ^P2, (co)NP-hard NP-complete Optimality ^P₃, (co)NP-hard ^P₂, (co)NP-hard

(Combined) Complexity Results

Ltgd Lfull

Cost ^P₃,^P₂-hard ^P₂, (co)NP-hard Optimality ^P4, (co)NP-hard ^P3, (co)NP-hard

Lacyc Lfacyc

Cost ^P2, (co)NP-hard NP-complete Optimality ^P₃, (co)NP-hard ^P₂, (co)NP-hard

(Combined) Complexity Results

Ltgd Lfull

Cost ^P₃,^P₂-hard ^P₂, (co)NP-hard Optimality ^P4, (co)NP-hard ^P3, (co)NP-hard

Lacyc Lfacyc

Cost ^P2, (co)NP-hard NP-complete Optimality ^P₃, (co)NP-hard ^P₂, (co)NP-hard

(Combined) Complexity Results

Ltgd Lfull

Cost ^P₃,^P₂-hard ^P₂, (co)NP-hard Optimality ^P4, (co)NP-hard ^P3, (co)NP-hard

Lacyc Lfacyc

Cost ^P2, (co)NP-hard NP-complete Optimality ^P₃, (co)NP-hard ^P₂, (co)NP-hard

Vertex-Cover in r -partite r -uniform hypergraph

Vertex-Cover: find a set of vertices of minimal size that cover all (hyper)edges in a (hyper)graph.

NP-complete for general (hyper)graphs.

PTIME for bipartite graphs (Kőnig’s theorem).

Lemma

Vertex-Cover is NP-complete for r -partite r -uniform hypergraphs for r >3.

r-partite: partition of the set of vertices intor sets, with no hyperedge spanning vertices of two different sets.

r-uniform: every hyperedge spansr vertices.

Encoding of 3-SAT

x1

x₁

x2

x2

x3

x3

y₁

y1

y₂

y2

y₃

y3

z₁

z₁ z₂

z2

z₃

z3

:z _x _y

Cost is NP-hard for L

Reduction from Vertex-Cover in 3-partite 3-uniform hypergraphs.

Withoutx =a repairs on the left-hand side of a tgd:

R(x₁;x₂;x₃) !R⁰(x₁) Source instance: hypergraph Target instance: empty

Cost: size of the tgd plus twice the minimum size of a vertex cover.

With x =a repairs: a little more difficult, but feasible!

Cost is NP-hard for L

Reduction from Vertex-Cover in 3-partite 3-uniform hypergraphs.

Withoutx =a repairs on the left-hand side of a tgd:

R(x₁;x₂;x₃) !R⁰(x₁) Source instance: hypergraph Target instance: empty

Cost: size of the tgd plus twice the minimum size of a vertex cover.

With x =a repairs: a little more difficult, but feasible!

Cost is NP-hard for L

Reduction from Vertex-Cover in 3-partite 3-uniform hypergraphs.

Withoutx =a repairs on the left-hand side of a tgd:

R(x₁;x₂;x₃) !R⁰(x₁) Source instance: hypergraph Target instance: empty

Cost: size of the tgd plus twice the minimum size of a vertex cover.

With x =a repairs: a little more difficult, but feasible!

Cost is NP-hard for L

Reduction from Vertex-Cover in 3-partite 3-uniform hypergraphs.

Withoutx =a repairs on the left-hand side of a tgd:

R(x₁;x₂;x₃) !R⁰(x₁) Source instance: hypergraph Target instance: empty

Cost: size of the tgd plus twice the minimum size of a vertex cover.

With x =a repairs: a little more difficult, but feasible!

Outline

Extension to Relational Calculus

Definition of repairs can be extended to relational calculus.

Same definition of cost, optimality.

Costis not recursive (but co-r.e.).

Computability ofOptimality: open(!).

Other Cost Functions

Why not counting the number of tuples to add or remove in J? . . . because it can beexponential in the size of the schema mapping!

Why not counting the number of tuples to add or remove in I orJ? . . . because selections are not captured!

Other Cost Functions

Why not counting the number of tuples to add or remove in J? . . . because it can beexponential in the size of the schema mapping!

Why not counting the number of tuples to add or remove in I orJ? . . . because selections are not captured!

Other Cost Functions

Why not counting the number of tuples to add or remove in J? . . . because it can beexponential in the size of the schema mapping!

Why not counting the number of tuples to add or remove in I orJ? . . . because selections are not captured!

Other Cost Functions

Why not counting the number of tuples to add or remove in J? . . . because it can beexponential in the size of the schema mapping!

Why not counting the number of tuples to add or remove in I orJ? . . . because selections are not captured!

Outline

In summary...

Formal framework for the discovery ofsymbolic relations between two data sources.

High complexity(up to fourth level of PH).

Link with Inductive Logic Programming?

Heuristics?

Generalization of acyclicity?

In summary...

Formal framework for the discovery ofsymbolic relations between two data sources.

High complexity(up to fourth level of PH).

Link with Inductive Logic Programming?

Heuristics?

Generalization of acyclicity?

Merci.