Uncertain Data Management NULLs

(1)

SQL Semantics V-tables c-tables

Uncertain Data Management NULLs

Antoine Amarilli¹, Silviu Maniu²

1Télécom ParisTech

2LRI

November 30th, 2015

(2)

NULLs

Representmissing informationin a relation.

Book

date teacher class room 2015-11-23 Antoine UDM C47 2015-11-30 Antoine UDM C017 2015-12-07 Antoine UDM NULL 2015-12-14 NULL UDM NULL Other name: Codd tables.

(3)

NULLs

Book

date teacher class room 2015-11-23 Antoine UDM C47 2015-11-30 Antoine UDM C017 2015-12-07 Antoine UDM NULL 2015-12-14 NULL UDM NULL

Other name: Codd tables.

(4)

NULLs

Book

date teacher class room 2015-11-23 Antoine UDM C47 2015-11-30 Antoine UDM C017 2015-12-07 Antoine UDM NULL 2015-12-14 NULL UDM NULL Other name: Codd tables.

(5)

Semantics

EachNULL can be replacedindependently byanydomain value

Book

date teacher class room 2015-11-23 Antoine UDM C47 2015-11-30 Antoine UDM C017 2015-12-07 Antoine UDM

2015-12-14 UDM

(6)

Semantics

EachNULL can be replacedindependently byanydomain value Book

date teacher class room 2015-11-23 Antoine UDM C47 2015-11-30 Antoine UDM C017 2015-12-07 Antoine UDM NULL 2015-12-14 NULL UDM NULL

(7)

Semantics

date teacher class room 2015-11-23 Antoine UDM C47 2015-11-30 Antoine UDM C017 2015-12-07 Antoine UDM C42 2015-12-14 Silviu UDM C42

(8)

Semantics

date teacher class room 2015-11-23 Antoine UDM C47 2015-11-30 Antoine UDM C017 2015-12-07 Antoine UDM xbecz 2015-12-14 gruiiik UDM buuuk

(9)

Tricky semantics

How can we evaluatequeries on Codd tables?

Book

date teacher class room 2015-11-23 Antoine UDM C47 2015-11-30 Antoine UDM C017 2015-12-07 Antoine UDM NULL 2015-12-14 NULL UDM NULL SELECT * FROM Book WHERE room='C47';

Book

date teacher class room 2015-11-23 Antoine UDM C47

→ Tricky semantics, oftencriticized!

(10)

Tricky semantics

Book

(11)

Tricky semantics

Book

(12)

Tricky semantics

Book

(13)

Three-valued logic

Usually, we evaluate operations as Boolean:

WHERE a='42' OR (b=c AND NOT (c=d))

→ WHERE False OR (True AND NOT (True))

→ False

In SQL, values can be True,False, or Unknown(NULL) Essentially anythingthat involvesNULL is NULL

(14)

Three-valued logic

→ False

In SQL, values can be True,False, or Unknown(NULL)

Essentially anythingthat involvesNULL is NULL

(15)

Three-valued logic

→ False

In SQL, values can be True,False, or Unknown(NULL) Essentially anythingthat involvesNULL is NULL

(16)

Three-valued logic (example)

WHERE 42=43 OR (42=NULL OR 42=43)

→ False OR (Unknown OR False)

→ False OR Unknown

→ Unknown

WHERE 42=45 OR (42=NULL OR 42=42)

→ False OR (Unknown OR True)

→ False OR True

→ True

(17)

Three-valued logic (example)

WHERE 42=43 OR (42=NULL OR 42=43)

→ Unknown

WHERE 42=45 OR (42=NULL OR 42=42)

→ False OR True

→ True

(18)

Three-valued logic (example)

WHERE 42=43 OR (42=NULL OR 42=43)

→ Unknown

WHERE 42=45 OR (42=NULL OR 42=42)

→ False OR True

→ True

(19)

Three-valued logic (example)

WHERE 42=43 OR (42=NULL OR 42=43)

→ Unknown

WHERE 42=45 OR (42=NULL OR 42=42)

→ False OR True

→ True

(20)

Three-valued logic (example)

WHERE 42=43 OR (42=NULL OR 42=43)

→ Unknown

WHERE 42=45 OR (42=NULL OR 42=42)

→ False OR True

→ True

(21)

Three-valued logic (example)

WHERE 42=43 OR (42=NULL OR 42=43)

→ Unknown

WHERE 42=45 OR (42=NULL OR 42=42)

→ False OR True

→ True

(22)

Three-valued logic (example)

WHERE 42=43 OR (42=NULL OR 42=43)

→ Unknown

WHERE 42=45 OR (42=NULL OR 42=42)

→ False OR True

→ True

(23)

Three-valued logic (example)

WHERE 42=43 OR (42=NULL OR 42=43)

→ Unknown

WHERE 42=45 OR (42=NULL OR 42=42)

→ False OR True

→ True

(24)

Three-valued logic (AND table)

AND True False

NULL

True True False

NULL

False False False

False NULL NULL False NULL

(25)

Three-valued logic (AND table)

AND True False NULL True True False

NULL

False False False

False

NULL

NULL False NULL

(26)

Three-valued logic (AND table)

AND True False NULL True True False

NULL

False False False False NULL

NULL

False

NULL

(27)

Three-valued logic (AND table)

AND True False NULL True True False NULL False False False False NULL NULL False NULL

(28)

Three-valued logic (OR table)

OR True False

NULL

True True True

True

False True False

NULL NULL True NULL NULL

(29)

Three-valued logic (OR table)

OR True False NULL True True True

True

False True False

NULL

True NULL NULL

(30)

Three-valued logic (OR table)

OR True False NULL True True True True False True False

NULL

NULL True

NULL NULL

(31)

Three-valued logic (OR table)

OR True False NULL True True True True False True False NULL NULL True NULL NULL

(32)

Three-valued logic (traps)

What is NULL * 42?

→ NULL

What is NULL / 0?

→ Implementation-dependent: NULLor error What is NULL =NULL?

→ NULL

What does the following do?

SELECT * FROM Book WHERE room=NULL

→ Returns anempty result What does the following do?

SELECT * FROM Book WHERE room='C42' OR room<>'C42'

→ Return everything whereroomisnotNULL

(33)

Three-valued logic (traps)

What is NULL * 42?

→ NULL

What is NULL / 0?

→ NULL

(34)

Three-valued logic (traps)

What is NULL * 42?

→ NULL

What is NULL / 0?

→ NULL

(35)

Three-valued logic (traps)

What is NULL * 42?

→ NULL

What is NULL / 0?

→ Implementation-dependent: NULLor error

What is NULL =NULL?

→ NULL

(36)

Three-valued logic (traps)

What is NULL * 42?

→ NULL

What is NULL / 0?

→ NULL

(37)

Three-valued logic (traps)

What is NULL * 42?

→ NULL

What is NULL / 0?

→ NULL

(38)

Three-valued logic (traps)

What is NULL * 42?

→ NULL

What is NULL / 0?

→ NULL

(39)

Three-valued logic (traps)

What is NULL * 42?

→ NULL

What is NULL / 0?

→ NULL

→ Returns anempty result

(40)

Three-valued logic (traps)

What is NULL * 42?

→ NULL

What is NULL / 0?

→ NULL

(41)

Three-valued logic (traps)

What is NULL * 42?

→ NULL

What is NULL / 0?

→ NULL

→ Return everything where isnot

(42)

Three-valued logic (fixes)

IS NULL

→ test if an expression isNULL Law of excluded fourth:

[COND] IS TRUE OR [COND] IS FALSE OR [COND] IS NULL

(43)

Three-valued logic (more complaints)

This issillyin terms of semantics!

Book

date teacher class room 2015-12-07 Antoine UDM NULL

Possible worlds:

Either theNULL is'C42' ... or theNULL is something else

→ ... so the tuple should match in either case!

(44)

Three-valued logic (more complaints)

Book

SELECT * FROM Book WHERE room='C42' OR room<>'C42' Possible worlds:

(45)

Three-valued logic (more complaints)

Book

SELECT * FROM Book WHERE room='C42' OR room<>'C42' Possible worlds:

(46)

Three-valued logic (more traps!)

Book

date teacher class room 2015-11-23 Antoine UDM C47 2015-11-30 Antoine UDM C017 2015-12-03 Antoine UDM NULL

Repairs room cause C42 lavatory leak NULL leopard

SELECT * FROM Book WHERE room NOT IN (SELECT room FROM Repairs)

→ Empty result!

SELECT * FROM Book WHERE room NOT IN

(SELECT room FROM Repairs WHERE room IS NOT NULL)

→ Does notcontain theNULL for 2015-12-03 SELECT * FROM Book WHERE

(room IN (SELECT room FROM Repairs) IS NOT TRUE)

(47)

Three-valued logic (more traps!)

Book

→ Empty result!

(48)

Three-valued logic (more traps!)

Book

→ Empty result!

(49)

Three-valued logic (more traps!)

Book

→ Empty result!

(50)

Three-valued logic (more traps!)

Book

→ Empty result!

(51)

Three-valued logic (more traps!)

Book

→ Empty result!

→ Does notcontain theNULL for 2015-12-03

SELECT * FROM Book WHERE

(52)

Three-valued logic (more traps!)

Book

→ Empty result!

→ Does notcontain theNULL for 2015-12-03

(53)

Even more traps with NULLs

SELECT * FROM R NATURAL JOIN S

→ NULLs willneverjoin

SELECT a FROM R UNION SELECT a FROM S

→ multiple NULLs willnotbe kept SELECT DISTINCT a FROM R

→ multiple NULLs willnotbe kept

(54)

Even more traps with NULLs

(55)

Even more traps with NULLs

(56)

Even more traps with NULLs

SELECT DISTINCT a FROM R

(57)

Even more traps with NULLs

(58)

Even more traps with NULLs

(59)

Even, even more traps about NULLs

SELECT COUNT(*) FROM R

→ NULLswill be counted SELECT COUNT(a) FROM R

→ NULLswill beignored! SELECT SUM(a) FROM R

→ NULLs will beignored

SELECT AVG(a),SUM(a)/COUNT(*) FROM R

→ values maydiffer

(60)

Even, even more traps about NULLs

(61)

Even, even more traps about NULLs

→ NULLswill be counted

SELECT COUNT(a) FROM R

(62)

Even, even more traps about NULLs

(63)

Even, even more traps about NULLs

→ NULLswill beignored!

SELECT SUM(a) FROM R

(64)

Even, even more traps about NULLs

(65)

Even, even more traps about NULLs

(66)

Even, even more traps about NULLs

(67)

Even, even more traps about NULLs

(68)

Semantics

We fix a signatureσ:

→ relationnames

→ associatedarity

We define uncertain interpretationsfor each relation

(70)

Uncertain relation

An uncertain relation: set ofpossible worlds

Booking date tch room

23 A. a

23 A. b

23 A. c

· · ·

(71)

Uncertain relation

An uncertain relation: set ofpossible worlds

23 A. a

23 A. b

23 A. c

· · ·

(72)

Relational algebra

Extend relational algebra operators to uncertain relations The possible worldsof the result should be...

take allpossible worldsof the inputs

apply the operation and get apossible output

Booking 23 A. a

Booking 23 A. b

Booking 23 A. c

...

∪

Booking 30 a C017

Booking 30 b C017

Booking 30 c C017

...

=

Booking 23 A. a 30 b C017

Booking 23 A. b 30 a C017

...

(73)

Relational algebra

Booking 23 A. a

Booking 23 A. b

Booking 23 A. c

∪

Booking 30 a C017

Booking 30 b C017

Booking 30 c C017

...

=

...

(74)

Relational algebra

Booking 23 A. a

Booking 23 A. b

Booking 23 A. c

∪

Booking 30 a C017

Booking 30 b C017

Booking 30 c C017

...

=

...

(75)

Relational algebra

Booking 23 A. a

Booking 23 A. b

Booking 23 A. c

∪

Booking 30 a C017

Booking 30 b C017

Booking 30 c C017

=

...

(76)

Relational algebra

Booking 23 A. a

Booking 23 A. b

Booking 23 A. c

∪

Booking 30 a C017

Booking 30 b C017

Booking 30 c C017

=

...

(77)

Relational algebra

Booking 23 A. a

Booking 23 A. b

Booking 23 A. c

∪

Booking 30 a C017

Booking 30 b C017

Booking 30 c C017

=

..

(78)

Representation system

Tables withNULL are arepresentation of uncertain tables

Booking

23 A. NULL stands for

Booking 23 A. a

Booking 23 A. b

Booking 23 A. c

...

(79)

Representation system

Booking 23 A. NULL

stands for

Booking 23 A. a

Booking 23 A. b

Booking 23 A. c

...

(80)

Representation system

Booking

Booking 23 A. a

Booking 23 A. b

Booking 23 A. c

...

(81)

Representation system

Booking

Booking 23 A. a

Booking 23 A. b

Booking 23 A. c

...

(82)

Back to the silly example

Book

date teacher class room 2015-12-07 Antoine UDM NULL 2015-12-14 Silviu UDM C47 SELECT * FROM Book WHERE teacher='Antoine' AND

(room='C42' OR room<>'C42')

→ How to representthe result?

(83)

Back to the silly example

Book

date teacher class room 2015-12-07 Antoine UDM NULL 2015-12-14 Silviu UDM C47 SELECT * FROM Book WHERE teacher='Antoine' AND

→ How to representthe result?

(84)

Representing the output

07 A. UDM NULL 14 S. UDM C47

represents 07 A. UDM a

14 S. UDM C47

07 A. UDM b 14 S. UDM C47

07 A. UDM C42 14 S. UDM C47 · · · SELECT * FROM Book WHERE teacher='A.' AND

07 A. UDM a 07 A. UDM b 07 A. UDM C42 · · · represented as

07 A. UDM NULL

(85)

Representing the output

represents

07 A. UDM a 14 S. UDM C47

07 A. UDM NULL

(86)

Representing the output

14 S. UDM C47

07 A. UDM C42 14 S. UDM C47 · · ·

SELECT * FROM Book WHERE teacher='A.' AND (room='C42' OR room<>'C42')

07 A. UDM NULL

(87)

Representing the output

14 S. UDM C47

07 A. UDM NULL

(88)

Representing the output

14 S. UDM C47

07 A. UDM a 07 A. UDM b 07 A. UDM C42 · · ·

represented as 07 A. UDM NULL

(89)

Representing the output

14 S. UDM C47

07 A. UDM NULL

(90)

Representing the output

14 S. UDM C47

(91)

Representation system definition

Uncertain instance: set of possible worlds

Uncertainty framework: concise way to represent uncertain instances

Here, Codd tables Query language: here, relational algebra

Definition (Strong representation system) For any query in the language,

on uncertain instances represented in the framework, the uncertain instance obtained by evaluating the query can also be represented in the framework.

→ Are Codd tables astrong representation system?

(92)

Representation system definition

Uncertain instance: set of possible worlds Uncertainty framework: concise way to represent

uncertain instances Here, Codd tables

Query language: here, relational algebra

(93)

Representation system definition

uncertain instances Here, Codd tables Query language: here, relational algebra

(94)

Representation system definition

(95)

Representation system definition

on uncertain instances represented in the framework,

the uncertain instance obtained by evaluating the query can also be represented in the framework.

(96)

Representation system definition

on uncertain instances represented in the framework, the uncertain instance obtained by evaluating the query

can also be represented in the framework.

(97)

Representation system definition

(98)

Representation system definition

(99)

Are Codd tables a representation system?

Member id class

1 UDM

2 UDM

3 IE

Booking

Can we representMember ▷◁Booking? Member▷◁Booking

id date teacher class room 1 2015-12-07 Antoine UDM NULL 2 2015-12-07 Antoine UDM NULL

→ Can you spot the problem?

(100)

Are Codd tables a representation system?

Member id class

1 UDM

2 UDM

3 IE

Booking

Can we representMember ▷◁Booking?

Member▷◁Booking

(101)

Are Codd tables a representation system?

Member id class

1 UDM

2 UDM

3 IE

Booking

Member▷◁Booking

(102)

Are Codd tables a representation system?

Member id class

1 UDM

2 UDM

3 IE

Booking

Member▷◁Booking

(103)

Multiple values

When querying Codd tables, we may duplicateNULLs

→ We cannot represent that two NULLs are the same This may causeproblems!

(104)

Multiple values example

Booking

date teacher class room 2015-12-07 Antoine UDM NULL 2015-12-14 Silviu UDM C47

Πroom(Booking)−Πroom(σteacher=“Antoine”(Booking))

According toSQL According tosemantics C47

But if we try to representintermediate expressions?

Π_room(Booking) Π_room(σteacher=“Antoine”(Booking)) NULL

C47

NULL

(105)

Multiple values example

Booking

Πroom(Booking)−Πroom(σteacher=“Antoine”(Booking)) According to SQL According tosemantics

C47

NULL

(106)

Multiple values example

Booking

C47

NULL

(107)

Multiple values example

Booking

C47

Πroom(Booking) Πroom(σteacher=“Antoine”(Booking))

NULL NULL

(108)

v-tables

Idea: give each NULL itsown name

→ namedNULLs

Initially, all NULLs are distinct Propagate their identities

Member id class

1 UDM

2 UDM

3 IE

Booking

date teacher class room 2015-12-07 NULL₁ UDM NULL₂

Member▷◁Booking

id date teacher class room 1 2015-12-07 NULL₁ UDM NULL₂ 2 2015-12-07 NULL₁ UDM NULL₂

(110)

v-tables

→ namedNULLs

Member id class

1 UDM

2 UDM

3 IE

Booking

Member▷◁Booking

id date teacher class room 1 2015-12-07 NULL₁ UDM NULL₂ 2 2015-12-07 NULL₁ UDM NULL₂

(111)

v-tables

→ namedNULLs

Member id class

1 UDM

2 UDM

3 IE

Booking

Member▷◁Booking

id date teacher class room 1 2015-12-07 NULL₁ UDM NULL₂

(112)

v-table semantics

1 2015-12-07 NULL₁ UDM NULL₂ 2 2015-12-07 NULL₁ UDM NULL₂

1 2015-12-07 aa UDM bb 2 2015-12-07 aa UDM bb

1 2015-12-07 ccc UDM ddd 2 2015-12-07 ccc UDM ddd

1 2015-12-07 e UDM e

2 2015-12-07 e UDM e

(113)

v-table semantics

1 2015-12-07 aa UDM bb 2 2015-12-07 aa UDM bb

1 2015-12-07 e UDM e

2 2015-12-07 e UDM e

(114)

v-table semantics

1 2015-12-07 aa UDM bb 2 2015-12-07 aa UDM bb

1 2015-12-07 e UDM e

2 2015-12-07 e UDM e

(115)

v-table semantics

1 2015-12-07 aa UDM bb 2 2015-12-07 aa UDM bb

1 2015-12-07 e UDM e

2 2015-12-07 e UDM e

(116)

Are v-tables a representation system?

Member id class

1 UDM

2 UDM

3 NULL₀

Booking

Member▷◁Booking id date teacher class room 1 2015-12-07 NULL₁ UDM NULL₂ 2 2015-12-07 NULL₁ UDM NULL₂

3 2015-12-07 NULL₁ UDM NULL₂ ifNULL₀ is “UDM”

(117)

Are v-tables a representation system?

Member id class

1 UDM

2 UDM

3 NULL₀

Booking

Member ▷◁Booking id date teacher class room 1 2015-12-07 NULL₁ UDM NULL₂ 2 2015-12-07 NULL₁ UDM NULL₂

3 2015-12-07 NULL₁ UDM NULL₂ ifNULL₀ is “UDM”

(118)

Problem

v-tables cannot represent optional rows

→ the number of rows iscertain

Whenselection,joinapplies to a NULL: we do not knowhow to evaluate

we areuncertainabout whether the tuple matches

→ Add conditions to rows!

(119)

Problem

Whenselection,joinapplies to a NULL:

we do not knowhow to evaluate

(120)

Problem

Whenselection,joinapplies to a NULL:

we do not knowhow to evaluate

Uncertain Data Management NULLs