XML typing MPRI 2.26.2: Web Data Management

(1)

XML typing

MPRI 2.26.2: Web Data Management

Antoine Amarilli^a Friday, December 14th

aBased on slides from the Webdam book by Serge Abiteboul, Ioana Manolescu, Philippe Rigaux, Marie-Christine Rousset, Pierre Senellart https://webdam.inria.fr/Jorge/files/TypingXML.ppt

1/51

(2)

Introduction

(3)

XML typing

•

XML documents must bewell-formed

•

^However,^typingis not required

•

Why typing?

• Formal speciﬁcationof a format

• Ensureinteroperabilitybetween programs

• Automatically detectbugswhen generating

• Make it easier toauthora document

• Make it easier toqueryit

• Avoid introducing errors whenupdating

• Make querying moreefﬁcient

•

Many languages!

https://en.wikipedia.org/wiki/List_of_types_of_XML_schemas

2/51

(4)

Improving storage

(5)

Veriﬁcation

•

^Easy: when data is produced/received, check the schema at runtime

•

^Hard: verify statically that a program willalwaysproduce valid output

•

Explored in theCducelanguagehttp://www.cduce.org/

•

Applicable toprograms, toqueriesin XQuery, toXSLT, etc.

4/51

(6)

Veriﬁcation example

for $p in doc("parts.xml")//part[color="red"]

return <part>

</part>

•

The type of the result will be:

(part (name (string) desc (any) )^∗

•

If the type ofparts.xml//part/descisstring then the type will be:

(part (name (string) desc (string) )^∗

(7)

Generating complicated types

for $x in $d/*, $y in $d/*

return

•

When we run this on an input document of the form(<a/>)ⁿ it willproduce()ⁿ²

•

No hope to describe this as areasonable type

6/51

(8)

DTDs

(9)

Type declaration

•

The simplest (and oldest) typing mechanism is based on Document Type Deﬁnitions(DTD).

•

A DTD may be speciﬁed in the prologue with the keyword DOCTYPEusing an ad hoc syntax.

•

Note: the DTD is not itself anXML document

7/51

(10)

Example of an XML document with its DTD

<?xml version="1.0" standalone="yes" ?>

<!DOCTYPE email [

<!ELEMENT email ( header, body )>

<!ELEMENT header ( from, to, cc? )>

<!ELEMENT to (#PCDATA)>

<!ELEMENT from (#PCDATA)>

<!ELEMENT cc (#PCDATA)>

<!ELEMENT body (paragraph*) >

<!ELEMENT paragraph (#PCDATA)> ]>

<email>

<from> [email protected] </from>

<to> [email protected] </to>

</header>

</email>

(11)

Document Type Deﬁnition

A DTD may also be speciﬁedexternallyusing an URI:

<!DOCTYPE docname SYSTEM "DTD-URI" [local-declarations]>

Some DTDs have establishedpublic identiﬁers:

<?xml version="1.0" encoding="utf-8"?>

<!DOCTYPE html PUBLIC

"-//W3C//DTD XHTML 1.0 Transitional//EN"

"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

9/51

(12)

Simple declarations in a DTD

Declaring anempty element:

<!ELEMENT br EMPTY>

Declaring an element containingtext:

Declaring an element containinganything:

<!ELEMENT from ANY>

Declaring an element containingtext or some elementsamong a list:

<!ELEMENT from (#PCDATA | a1 | a2 | a3)*>

Note: there can beonly onedeclaration per element name!

(13)

More complex declarations in a DTD

Declaring an element with complex content:

<!ELEMENT complex (E)>

whereEis adeterministicregular expression built with:

•

,: concatenation

•

^{|: or}

•

?: 0 or 1 occurrence

•

*: arbitrary number of occurrences

•

+: non-zero number of occurrences Whitespacethat shouldn’t exist is ignored

11/51

(14)

Deterministic regexp

•

To make validation easier, the regexp must bedeterministic:

while matching any word of element names, there must never be any choice in how to continue

•

More formally: for any position in the regexp and element name, there must be at most one position where we can go

•

Non-deterministic examples:(e1 e2 | e1 e3)ore1? e1

•

Some regular languages haveno deterministic regular

expressions, e.g.,(a|b)* a (a|b), see Bruggemann-Klein and Wood,One-Unambiguous Regular Languages, I&C 1998.

https://core.ac.uk/download/pdf/82738828.pdf

•

We can test inlinear timeif a regular expression is deterministic: see Groz, Maneth, and Staworko,Deterministic Regular

Expressions in Linear Time, PODS 2012.

https://hal.inria.fr/inria-00618451/document

(15)

Deterministic regexp

•

To make validation easier, the regexp must bedeterministic:

while matching any word of element names, there must never be any choice in how to continue

•

More formally: for any position in the regexp and element name, there must be at most one position where we can go

•

Non-deterministic examples:(e1 e2 | e1 e3)ore1? e1

•

Some regular languages haveno deterministic regular

expressions, e.g.,(a|b)* a (a|b), see Bruggemann-Klein and Wood,One-Unambiguous Regular Languages, I&C 1998.

https://core.ac.uk/download/pdf/82738828.pdf

•

We can test inlinear timeif a regular expression is deterministic:

see Groz, Maneth, and Staworko,Deterministic Regular Expressions in Linear Time, PODS 2012.

https://hal.inria.fr/inria-00618451/document

12/51

(16)

Attributes

Declaring an attributeatton elementelt:

<!ATTLIST element attribute type default>

Thetypecan be one of the following (non-exhaustive):

•

^CDATA:^anything

•

v1 | v2 | ... | vn: one of thesevalues(XML tokens)

•

ID: a (unique)XML ID(at most one such attribute per element)

•

^{IDREF: a}^referenceto some ID value Thedefaultcan be:

•

"value"Somedefault value(so the attribute is optional)

•

^#IMPLIEDsaying that the attribute isoptional(no default)

•

^#REQUIREDsaying that the attribute isrequired

•

#FIXED "value"

(17)

Entities

DTDs allow us to deﬁneentities(like macros):

<!ENTITY author "John Doe">

and then to refer to them in the XML:

&author;

There are also mechanisms forexternal entities.

14/51

(18)

External DTDs

The use of external DTDs poses someproblems:

•

^{We may be}^ofﬂine

•

They may bedown

•

Fetching them causesloss of privacy

If we do not want to access the external DTD, we:

•

^Cannot^validatethe document (duh)

•

^Cannot^replaceentities deﬁned in the DTD

•

^Cannot^knowabout default attribute values

(19)

XML Schema

(20)

Key points of XML Schema (XSD for XML Schema Deﬁnition))

•

^More^recent^{and more}^expressivethan the DTD standard

•

^More^complex

•

Support fornamespaces

•

Constraints ondata types, e.g., dates, numbers, etc. (19 primitive types)

•

May have multipledeﬁnitionsfor the same element in different contexts

•

More powerfuluniquenessconstraints (can be scoped to part of the document)

•

An XML schema is anXML documentso there is anXML Schema schema:

https://www.w3.org/2009/XMLSchema/XMLSchema.xsd

•

In XSD 1.1,assertionswith XPath expressions

(21)

Example of XML Schema

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">

<xsd:element name="book">

<xsd:complexType>

<xsd:sequence>

<xsd:element name="title" type="xsd:string"/>

<xsd:element name="author" type="xsd:string"/>

<xsd:element name="character" minOccurs="0" maxOccurs="unbounded">

<xsd:complexType>

<xsd:sequence>

<xsd:element name="name" type="xsd:string"/>

<xsd:element name="friend-of" type="xsd:string"

minOccurs="0" maxOccurs="unbounded"/>

<xsd:element name="since" type="xsd:date"/>

<xsd:element name="qualification" type="xsd:string"/>

</xsd:sequence>

</xsd:complexType>

</xsd:element>

</xsd:sequence>

<xsd:attribute name="isbn" type="xsd:string"/>

</xsd:complexType>

</xsd:element>

</xsd:schema> ^17/51

(22)

XML Schema simple types

•

^Built-in^types: ^string,^integer,^date,^{time, ...}

•

^Modifyingbasic types, e.g.:

</restriction>

</simpleType>

</element>

Also: enumeration of values, patterns, length, etc.

→ Thesesimple typesare used forattributesand for elements that containonly text

(23)

More XML Schema features

•

all: we must have all elements but in an arbitrary order

<all>

</all>

</complexType>

</element>

•

^Giving^namesto types and reusing them afterwards:

...

</complexType>

•

^Assertionsusing XPath 19/51

(24)

More XML Schema features (2)

•

^Groupsof elements and attributes

•

^any^andanyAttributeto allow anything

•

Substitution groupsto indicate that an element can be substituted for another

•

Including anexisting schema(possibly in a different namespace)

<import namespace="http://..."

schemaLocation="http://..." />

•

^Redeﬁningan existing schema:

...

</redefine>

(25)

XML Schema vs DTD

<!ELEMENT note

(to, from, heading, body)>

<!ELEMENT to (#PCDATA)>

<!ELEMENT heading (#PCDATA)>

<!ELEMENT body (#PCDATA)>

<?xml version="1.0"?>

<xs:schemaxmlns:xs="http://www.w3.org/2001/XMLSchema">

<xs:element name="note">

<xs:complexType>

<xs:sequence>

<xs:element name="to"type="xs:string"/>

<xs:element name="from"type="xs:string"/>

<xs:element name="heading" type="xs:string"/>

<xs:element name="body"type="xs:string"/>

</xs:sequence>

</xs:complexType>

</xs:element>

</xs:schema>

21/51

(26)

Criticism of XML Schema

•

^Verycomplex: 461 pages

•

^Extremely^verbose

•

Some unexpectedshortcomings: no simple way to say what the root element must be

•

Not sufﬁcientlyexpressive, in particular notself-describing

•

Still requiresdeterministicregular expressions

(27)

Beyond XML Schema

•

^{Relax NG:}

• Simplerthan XML Schema

• Moreminimalisticdefault types

• No way to specify a document’s schemawithin the document

• No way to adddefault values

• Nodeterminismrequirement

•

Schematron:

• Different approach: list ofrulesthat the document must pass

• Can be used inconjunctionwith DTD/XSD/RelaxNG

•

Many possiblechoices!

https://en.wikipedia.org/wiki/XML_schema#Languages

23/51

(28)

RELAX NG example

<start>

</oneOrMore>

</element>

</start>

<text/>

</element>

</define>

</grammar>

Also ashort syntax:

start = element book { page+ } page = element page { text }

(29)

Schematron example

<?xml version="1.0" encoding="UTF-8"?>

<sch:schema xmlns:sch="http://www.ascc.net/xml/schematron">

<sch:pattern name="Check structure">

<sch:rule context="Person">

<sch:assert test="@Title">Person must have Title attribute</sch:assert>

<sch:assert test="count(*)=2 and count(Name)=1 and count(Gender)=1">

The element Person should have the child elements Name and Gender.

</sch:assert>

<sch:assert test="*[1] = Name">Name must be first.</sch:assert>

</sch:rule>

</sch:pattern>

<sch:pattern name="Check co-occurrence constraints">

<sch:rule context="Person">

<sch:assert test="(@Title='Mr' and Gender='Male') or @Title!='Mr'">

If the Title is "Mr" then the gender of the person must be "Male".

</sch:assert>

</sch:rule>

</sch:pattern>

</sch:schema>

25/51

(30)

Tree automata

(31)

General presentation

•

Tree automatageneralizeword automata

•

Remember that a deterministicword automatonAspeciﬁes:

• AnalphabetΣof labels

• A setQofstates

• Aninitial stateq0∈Q

• A setF⊆Qofﬁnal states

• Atransition functionδ:Q×Σ→Q

26/51

(32)

Automaton reminders

•

The automaton can be:

• deterministic: the transition functionδis a function

• non-deterministic: δis a relation⊆Q×Σ×Q;

we may also haveϵ-transitions

•

^{We can}determinizeautomata but at the cost of an exponential blowup

•

Some languages, e.g.,{aⁿbⁿ |n∈N}, cannot be recognized by a ﬁnite word automaton

(33)

Bottom-up tree automata AlphabetΣ:

•

Bottom-up deterministictree automaton

•

^States^Q: ^{⊥^,^B,^P,^⊤}

•

Final statesF⊆Q: {⊤}

•

Initial functionι: ⊥ P B

•

Transitionsδ(examples): P

⊥ P

⊤ B P

⊥

The automaton is...

•

deterministicifδis afunction: δ :Q²×Σ→Q

•

non-deterministicifδis arelation: δ ⊆Q²×Σ×Q

→ Determinizationfor bottom-up tree automata (exp blowup).

28/51

(34)

Bottom-up tree automata

AlphabetΣ:

•

^States^Q: ^{⊥^,^B,^P,^⊤}

•

⊥ P

⊤ B P

⊥

The automaton is...

•

(35)

AlphabetΣ:

•

^States^Q: ^{⊥^,^B,^P,^⊤}

•

⊥ P

⊤ B P

⊥

The automaton is...

•

28/51

(36)

AlphabetΣ:

•

^States^Q: ^{⊥^,^B,^P,^⊤}

•

⊥ P

⊤ B P

⊥

The automaton is...

•

(37)

AlphabetΣ:

•

^States^Q: ^{⊥^,^B,^P,^⊤}

•

⊥ P

⊤ B P

⊥

The automaton is...

•

28/51

(38)

⊥ B

⊥ P

•

^States^Q: ^{⊥^,^B,^P,^⊤}

•

⊥ P

⊤ B P

⊥

The automaton is...

•

(39)

⊥ B

⊥ P

•

^States^Q: ^{⊥^,^B,^P,^⊤}

•

Transitionsδ(examples):

P

⊥ P

⊤ B P

⊥

The automaton is...

•

28/51

(40)

B

⊥ B P

⊥ P

•

^States^Q: ^{⊥^,^B,^P,^⊤}

•

P

⊥ P

⊤ B P

⊥

The automaton is...

•

(41)

⊤

B

⊥ B P

⊥ P

•

^States^Q: ^{⊥^,^B,^P,^⊤}

•

P

⊥ P

⊤ B P

⊥

The automaton is...

•

28/51

(42)

⊤

B

⊥ B P

⊥ P

•

^States^Q: ^{⊥^,^B,^P,^⊤}

•

P

⊥ P

⊤ B P

⊥

The automaton is...

•

→ Determinizationfor bottom-up tree automata (exp blowup). ^28/51

(43)

Example: Boolean (tree) circuit evaluation

•

^Alphabet: ^{Σ =}^{0,^1,^∧,^∨}

•

^States: ^Q⁼^{q⁰^,^q¹^}

•

Final states: F={q1}

•

Initial function:

• ι(0) :=q0

• ι(1) :=q1

•

Transition function:

• δ(q_i,q_j,∧) :=q_i∧j

• δ(q_i,q_j,∨) :=q_i∨j

29/51

(44)

Top-down deterministic tree automata

Likebottom-up deterministic automatabut differences:

•

^{We add two}dummy leaveswith some special label⊥to all leaves

•

^Thetransition functionis redeﬁned:

• Nowδ:Q×Σ→Q×Q

• Semantics: when we are at stateqon a node with labela

the state of the automaton on the left and right children isδ(q,a)

•

There is now aninitial stateq0 for the root

•

The semantics forﬁnal statesis thatall leavesshould be in a ﬁnal state

(45)

Top-down deterministic tree automata are weaker

•

Consider theﬁnite tree languagecontaining the two trees:

•

It can be accepted by abottom-up tree automaton

•

However, with a deterministic top-down tree automaton...

• Let(q,q^′) :=δ (q0,

)

• Let(q1,q2) :=δ

(q, )

and(q^′₁,q^′₂) :=δ (q^′,

)

• The ﬁrst tree is accepted soq1,q2are ﬁnal

• The second tree is accepted soq^′1,q^′2are ﬁnal

• Hence the automaton accepts:

31/51

(46)

Top-down nondeterministic tree automata

•

We still adddummy leaveswith label⊥

•

^Alphabet: ^Σ

•

^States: ^Q

•

^Multipleinitial statesI⊆Q

•

Final states: F⊆Q

•

^Transition^relation^δ^⊆^Q^×^Σ^×^Q²

Can berewrittento a bottom-up automatonA^′:

•

^Iare the ﬁnal states ofA^′

•

^Fdeﬁnes the initial states ofA^′

•

^Reversing^δ gives the transition relation ofA^′

(47)

Connections and properties

•

Regular tree language: accepted by a bottom-up tree automaton

•

Equivalently: deterministic bottom-up tree automaton

•

Regular tree languages are closed under union, intersection, complement

•

Tree automata are equivalent tomonadic second-order(MSO) logic:

• First-order logic: variables, Boolean connectives∧,∨,¬, quantiﬁers∀,∃

• Quantiﬁcation over sets (unary predicates): ∀S,∃S

• Example: everya-node has ab-descendant:

∀x(

a(x)→ ∀X(

(X(x)∧β_E)→(∃y(X(y)∧b(y))))) where we have the following (Edenotes the edge relation):

β_E:=∀yz(E(y,z)∧X(y)→X(z))

33/51

(48)

Unranked tree automata

•

XML documents areunranked: each node can have multiple children

•

We need speciﬁcautomata formalismsto work on unranked trees

•

How can we specifyinﬁnitely manytransition rules?

• δ(q0,∧) =q0

• δ(q0,q0,∧) =q0

• δ(q0,q0,q0,∧) =q0

• δ(q0,q0,q0,q0,∧) =q0

• ...

•

^Intuition: deﬁne transitions likeδ((q0+q1)^∗q1(q0+q1)^∗,∨) =q1

(49)

Deﬁning transitions for unranked tree automata

•

^Idea: ^{use an}automaton on wordsto deﬁne the new state

•

We still have thealphabetΣ, thestatesQ, theﬁnal statesF⊆Q, theinitial functionι: Σ→Q

•

^{For each}^a^∈^Σ^{we have a}word automatonΞaon the alphabetQ, whose states are also identiﬁed withQ

Ξ_∧ Ξ_∨

q1

start q0

q0

q1 q0,q1

q0

start q1

q1

q0 q0,q1

•

Deterministic: the word automata are deterministic

35/51

(50)

Building on ranked trees

a

b

a b

a

b

a b

Ranked tree: FirstChild-NextSibling F: encoding into a ranked tree

• F is a bijection F^-1: decoding

(51)

Master Informatique Web Data Management Typing semistructured data 10/9/2007 53

Building on

bottom-up ranked trees (2)

• For each Unranked TA A, there is a Ranked TA accepting F(L(A))

• For each Ranked TA A, there is an unranked TA accepting F

^-1

(L(A))

• Both are easy to construct

Consequence: Unranked TA are closed under

union, intersection, complement

(52)

Top-down unranked tree automata

•

^We^adaptthe deﬁnition of top-down unranked tree automata

•

We have oneinitial stateat the root

•

^{For each}^a^∈^Σ^{and each}^q^∈^Q^{we have a}word transducerΞ_a,q

•

^Semantics:knowing the label and state of theparent, we read the labels of thechildrenand we give them astate

•

^Toaccept, the transducers runs must beaccepting, and all leaves must be labeled with aﬁnal state

Ξ_∧_,q₀ Ξ_∨_,q₁

start ∨,∧,0 :q0

•:q0,q1 •:q0,q1

start ∨,∧,1 :q1

•:q0,q1 •:q0,q1

(53)

DTD validation

DTDs can be checked with adeterministic pushdown automaton:

•

^The^stackis used to remember thenestingof elements

•

^Whileînsidean element we can check that the word of its children satisfies thedeterministic regular expressionused to define it

•

Can be implemented with aSAX parser

39/51

(54)

Very efficient validation (2)

<!ELEMENT a ( b c ) >

<!ELEMENT b ( d+ ) >

a

b c

d d

s t u

b c

A_a

s’ t’

d A_b d

<a><d/><d/><c/></a>

(55)

Very efficient validation (2)

<!ELEMENT b ( d+ ) >

a

b c

d d

s t u

b c

A_a

s’ t’

d A_b d

<a><d/><d/><c/></a>

(56)

Very efficient validation (2)

<!ELEMENT b ( d+ ) >

a

b c

d d

s t u

b c

A_a

s’ t’

d A_b d

<a><d/><d/><c/></a>

s

(57)

Very efficient validation (2)

<!ELEMENT b ( d+ ) >

a

b c

d d

s t u

b c

A_a

s’ t’

d A_b d

<a><d/><d/><c/></a>

s’

s t

(58)

Very efficient validation (2)

<!ELEMENT b ( d+ ) >

a

b c

d d

s t u

b c

A_a

s’ t’

d A_b d

<a><d/><d/><c/></a>

s’

s t t’

(59)

Very efficient validation (2)

<!ELEMENT b ( d+ ) >

a

b c

d d

s t u

b c

A_a

s’ t’

d A_b d

<a><d/><d/><c/></a>

s’

s t t’

(60)

Very efficient validation (2)

<!ELEMENT b ( d+ ) >

a

b c

d d

s t u

b c

A_a

s’ t’

d A_b d

<a><d/><d/><c/></a>

s t

(61)

Very efficient validation (2)

<!ELEMENT b ( d+ ) >

a

b c

d d

s t u

b c

A_a

s’ t’

d A_b d

<a><d/><d/><c/></a>

s t u

(62)

Very efficient validation (2)

<!ELEMENT b ( d+ ) >

a

b c

d d

s t u

b c

A_a

s’ t’

d A_b d

<a><d/><d/><c/></a>

Accept

(63)

Warning

• The previous example can be checked with a simple automata on words

• But not the following one

**<!ELEMENT part ( part* ) >**

• The stack is needed for accepting

<a>…<a></a>…</a>

n <a> n </a>

(64)

Some bad news for DTD

• Not closed under union

DTD1 …

<!ELEMENT used( ad*) >

<!ELEMENT ad ( year, brand )>

DTD2 …

<!ELEMENT new( ad*) >

<!ELEMENT ad ( brand )>

• L(DTD1)  L(DTD2) cannot be described by a DTD but can be described easily by a tree automata

– Problem with the type of ad that depends of its parent

• Also not closed under complement

• Limited expressive power

(65)

XML Schema validation for version 1.0

•

The strange requirements ofXSD version 1.0make it a subset of the languages that can be checked withdeterministic top-down tree automata

•

With version 1.1, the use ofassertionsmeans that the tree language is no longerregular:

count(//a) = count(//b) and count(//b) = count(//c)

51/51

(66)

Sources

•

Slide 11–14: https://www.irif.fr/~carton/

Enseignement/XML/Cours/DTD/

•

^{Slide 17:} ^http:

//xmlfr.org/documentations/tutoriels/001218-0001

•

^{Slide 24:}https://en.wikipedia.org/wiki/RELAX_NG

•

^{Slide 25:}^https:

//www.xml.com/pub/a/2003/11/12/schematron.html