How to Split Recursive Automata

HAL Id: inria-00341770

https://hal.inria.fr/inria-00341770

Submitted on 25 Nov 2008

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

How to Split Recursive Automata

Isabelle Tellier

To cite this version:

Isabelle Tellier. How to Split Recursive Automata. 9th International Colloquium ICGI, Alexander

Clark, François Coste, Laurent Miclet, 2008, St Malo, France. pp.200-212. �inria-00341770�

IsabelleTellier

LIFL-InriaLilleNordEurope

universityofLille

isabelle.tellieruniv-lille3. fr

⋆⋆

In this paper, we interpret in terms of operations applying on extended nite

stateautomatasomealgorithmsthathavebeenspeiedonategorialgrammars

to learnsublassesof ontext-freelanguages.Thealgorithms onsideredimple-

mentspeializationstrategies.Thisnewperspetivealsohelpstounderstandhow

itispossibletoontroltheombinatorialexplosionthatspeializationtehniques

havetofae,thankstoatypingapproah.

1 Introdution

Thereareoftenseveralwaystorepresentalanguage:itiswellknownthatevery

regular language an be speied either by a regular grammar or by a deter-

ministi nitestateautomaton.Context-freelanguagesanalsobespeiedby

dierentkindsofdevies.Inreentpreviouspapers[17,18℄,wehaveshownthat

somelassesofategorialgrammars(CGsinthefollowing),generatingontext-

free languages, ould easily be represented by a family of extended automata

alled reursiveautomata (RA).Thistranslationallowedtoexhibit onnexions

betweentwopreviouslydistintapproahesofgrammatialinferenefromposi-

tiveexamples:theoneusedin[3,13,14℄tolearnCGs,andtheoneusedtolearn

regulargrammarsrepresentedbynitestateautomata[1,10℄.Thiswaspossible

beausebothemployageneralization strategy.Inpartiular,thegeneralization

operatorsusedin bothontextswereshowntobesimilar.

Now,wewanttoapplythesameproessforspeializationstrategiesfrompos-

itiveexamples.Insuhstrategies,theinitialhypothesisistoogeneralagrammar

(or set ofgrammars) andeahexampleis onsidered asaonstraint whihre-

stritsthesearhspae,untilitisreduedtothetargetgrammar.Weshowhere

thatthetranslationofCGsintoRA,whihhashelpedtobetterunderstandthe

familyofgeneralizationstrategies,analsohelptobetterunderstandthefamily

of speialization strategies.As a matterof fat, although barely used, speial-

ization approaheshave been proposed independentlyin bothbakgrounds: to

learnsublassesofCGsintheonehand[16℄,andtolearnregulargrammarsrep-

resented by nite stateautomata in the otherhand [11℄.A rst movetowards

that diretion has been briey proposed in [19℄, but limited to unidiretional

CGs.Inthispaper,wegeneralizetheapproahtoitsfullgenerality.

⋆⋆

into aRA preservingthestrutures produed, inbothunidiretional andbidi-

retionalases. Insetion 3,werst brieypresentthe speialization strategy

desribedbyMoreauin[16℄,allowingtolearnrigidCGsfrompositiveexamples.

We thenexplain how it relatesto thespeializationstrategy proposedby Fre-

douilleand Miletin[11℄,whihtargetsregularlanguagesrepresentedbynite

stateautomata. Weshow that Moreau'salgorithm anbe interpretedassome

kindofwellfoundedstatesplittingstrategyapplyingonRA.Finaly,thewhole

pitureisompletedinsetion4,byanewinterpretationofyetanotheralready

knownalgorithmallowingtolearnCGsfromsentenesenrihedbylexialtypes

[8,7℄.Itappearstobeaneientlyontroledspeializationapproah.

This paperthus proposes neither any newalgorithm orresult, noranyex-

periment,but itsuggestsanewstimulatinglookonalreadyknownstrategies.

2 From ategorial grammars to reursive automata

2.1 Basi denitionsof ategorialgrammars

Denition1 (Categories, CategorialGrammarsand theirLanguage).

Let

B

^{be aset (atmostountable)of basiategories ontaining a}distinguished ategory

S ∈ B

^{,alledtheaxiom.}

Cat (B)

^{isthesmallestsetsuhthat}

B ⊂ Cat (B)

andfor any

A, B ∈ Cat(B)

^:

A/B ∈ Cat(B)

^and

B\A ∈ Cat (B)

^.Unidiretional variantsallow onlyoneoftheseoperators(either

/

^or

\

^{)butnotboth.Forevery}

nitevoabulary

Σ

^{andforeveryset}

B

^ontaining

S

^{,aategorialgrammar(or}

CG) isaniterelation

G

^over

Σ × Cat (B)

^.Wenote

hv, Ci ∈ G

^{the assignment}

oftheategory

C ∈ Cat (B)

^{totheelementofthevoabulary}

v ∈ Σ

^.Thesyntati

rules ofaCG take theformof tworewriting shemes:

∀A, B ∈ Cat (B)

FA

^(ForwardAppliation):

A/B B → A

BA

^(Bakward Appliation) :

B B\A → A

Unidiretional CGs make use of only one of these rules (either

FA

^or

BA

^{) but}

notof both.The languagegenerated(orreognized)byaCG

G

^is:

L(G)

⁼

{w = v 1 . . . v n ∈ Σ ⁺ | ∀i ∈ {1, . . . , n}

^,

∃C i ∈ Cat (B)

^suhthat

hv i , C i i ∈ G

^and

C 1 . . . C n → ^∗ S}

^,

where

→ ^∗

^{is the reexive and transitive losure of the relation}

→

^{, dened by}

FA

^and

BA

^{shemes. For every}

w ∈ L(G)

^{, a syntati analysis struture an}

be produed, taking the form of a binary-branhing tree whose leaf nodes are

assignmentsof

G

^{andwhose internalnodesarelabelledeitherby}

FA

^or

BA

^and

by aategory (seeFigure1).

Example 1 (asimpleCG).CGshavemainlybeenusedtorepresentnaturallan-

guagesyntax,asillustratedbythisexample.Let

B = {S, T, CN }

^where

T

^stands

fortermand

CN

^{forommonnoun,}

Σ = {John, runs, a, man, f ast}

^and

G = {hJohn, T i, hruns, T \Si, hman, CN i, ha, (S/(T \S))/CN i, hf ast, ((T \S)\(T \S))i}

^.

Thisover-simpleCGreognizessenteneslikeJohnruns oraman runsfast

withthesyntatianalysisstruturesofFigure1.

S BA

T

John

T \S

runs

S FA

S/(T \S) FA

(S/(T \S))/CN

a

CN

man

T \S BA

T \S

runs

(T \S)\(T \S)

fast

Fig.1.SyntatianalysisstruturesproduedbyaCG

2.2 Reursive Automataand theirLanguage

Denition2 (Reursive Automaton). Areursive automaton

R

^isa

5

^-

tuple

R = hQ, Σ, γ, q 0 , F i

^with

Q

^{athenitesetof states,}

Σ

^{anite voabulary,}

q 0 ∈ Q

^{a (unique) initial state and}

F ∈ Q

^{a (unique) nal state.}

γ

^{is the}

transitionfuntion of

R

^,denedfrom

Q × (Σ ∪ Q)

^to

2 ^Q

^.

Werestritourselveshereto reursiveautomata(RAin thefollowing)with

uniqueinitialandnalstates,butitisnotaruialhoie.Theonlyimportant

dierenebetweenthisdenitionandtheusualdenitionofnitestateautomata

isthat,inaRA,itispossibletolabelatransitioneither byanelement of

Σ

^or

byanelementof

Q

^{.Touseatransitionlabelledbyastate,youhavetoprodue}

astringbelonging tothe languageofthis state.RA anthus be onsidered as

speialasesofreursivetransitionnetworks orRTRs[20℄.But,dependingon

the notionof statelanguage used, there exist in fat twodistint notions of

RA whih will be alled, for reasons that will beome lear soon,

RA F A

^and

RA BA

^.Ina

RA F A

^,thelanguage

L F A (q)

^{assoiatedwiththestate}

q ∈ Q

^isthe

setofstringsstartingfrom

q

^{andreahingthenalstate}

F

^,whereasina

RA BA

^,

L BA (q)

^{istheset ofstringsstartingfromtheinitialstate}

q 0

^andreahing

q

^.

Denition3 (LanguageReognizedbya RA).Let

R = hQ, Σ, γ, q 0 , F i

^be

a

RA F A

^(resp.a

RA BA

^).Forevery

q ∈ Q

^{wedenethelanguage}

L F A (q)

^(resp.

L BA (q)

^{) assoiatedwith}

q

^{asthe smallestset} satisfying:

ǫ ∈ L F A (F)

^(resp.

ǫ ∈ L BA (q 0 )

^);

if there exists a transition labelled by

a ∈ Σ

^between

q

^and

q ^′ ∈ Q

^{, i.e.}

q ^′ ∈ γ(q, a)

^then:

a.L F A (q ^′ ) ⊆ L F A (q)

^(resp.

L BA (q).a ⊆ L BA (q ^′ )

^);

ifthere existsatransition labelledby

r ∈ Q

^between

q

^and

q ^′ ∈ Q

^{, i.e.}

q ^′ ∈ γ(q, r)

^then:

L F A (r).L F A (q ^′ ) ⊆ L F A (q)

^(resp.

L BA (q).L BA (r) ⊆ L BA (q ^′ )

^).

The language

L F A (R)

^{of the}

RA F A

^{(resp. the language}

L BA (R)

^{of the}

RA BA

⁾

isdenedby:

L F A (R) = L F A (q 0 )

^(resp.

L BA (R) = L BA (F )

^).

For astate

q ∈ Q

^{suh that}

q 6= F

^(resp.

q 6= q 0

^{), thedenition of}

L F A (q)

(resp.of

L BA (q)

^{)maybereursive:when itexists, itis asmallestx-point. A}

realreursionourswhen,ina

RA F A

^{,thereexistsapathstartingfromastate}

q

^{,usingatransitionlabelledby}

q

^andreahing

F

^(resp.,ina

RA BA

^,whenthere

existsapathstartingfrom

q 0

^{,usingatransitionlabelledby}

q

^{andreahingthe}

state

q

^{).Unlikenitestateautomata,RAarenotlimitedtoproduingattrees,}

beausereursivetransitionsallowarealbranhing.Wehaveshownin[19℄that

RA F A

^and

RA BA

^are respetively linked with the two possible unidiretional CGs.Thisproperty,whih justiestheirname,isdetailedin thefollowing.

2.3 FromUnidiretional CGs to RA

Eah one of the two families of unidiretional CGs an produe any

ǫ

^-free

ontext-freelanguage[2℄.Here,weshowthatevery

F A

-unidiretional(resp.

BA

^-

unidiretional)CG anbeeasily transformedinto astronglyequivalent

RA F A

(resp.

RA BA

^{),i.e.generatingthesamestrutural}desriptions[19℄.Theproess, foragiven

F A

-unidiretional(resp.

BA

-unidiretional)CG

G

^{,isthefollowing:}

thevoabulary

Σ

^{oftheRAisthesameastheoneof}

G

^.

let

N

^{be theset of everysubategoryof aategoryassigned to a member}

of the voabulary in

G

^{(a ategory is a subategory of itself). The set of}

statesforthe

RA F A

^(resp.

RA BA

^)tobebuiltis

N ∪ {F }

^with

F / ∈ N

^(resp.

N ∪ {I}

^with

I / ∈ N

^{).Theinitialstateis}

S

^(resp.

I

^{),thenaloneis}

F

^(resp.

S

^).

forevery

C ∈ N

^{,deneatransitionlabelledby}

C

^{betweenthestates}

C

^and

F

^{(resp.between}

I

^ans

C

^),i.e.

F ∈ γ(C, C)

^(resp.

C ∈ γ(I, C)

^).

forevery

A/B ∈ N

^(resp.

A\B ∈ N

^{),dene a transition labelled by}

A/B

(resp.

A\B

^{) between the states}

A

^and

B

^{, that is:}

B ∈ γ(A, A/B)

^(resp.

B ∈ γ(A, A\B)

^).

forevery

hv, ,iC ∈ G

^{,addatransitionlabelledby}

v

^{betweenthestate}

C

^and

F

^{, i.e.}

F ∈ γ(C, v)

^{(resp.add atransition between}

I

^and

C

^labelledby

v

^,

i.e.

C ∈ γ(I, v)

^).

2.4 Mutually Reursive Automata

Both families of unidiretionalCGs havethe expressivityof

ǫ

^-freeontext-free languagesatthestringlevel,but bidiretionalCGsareusefulforlinguistipur-

poses,beauseofthestrutures theyprodue,andpartiularlythelabels

FA

^or

BA

^{assignedtoeahinternalnode.Itisthusnaturaltotrytoextendournotion}

of RA to the generalase of bidiretionalCGs,where both

F A

^and

BA

^rules

are used. As we haveseen, itis possibleto represent theuse of

F A

^{rules in a}

RA F A

^andtheuseof

BA

^rulesina

RA BA

^{.So,weproposetorepresenta(bidi-}

retional)CGbyapair ofmutuallyreursiveautomata(MRAinthefollowing):

one element of the pairis a

RA F A

^{, theother one is a}

RA BA

^{. Fora syntati}

analysisthatusesboth

F A

^and

BA

^{rules,mutualallsbetweenthetwoRAwill}

beneessary.After an introduing example, we provide ageneral denition of

Example 2 (Example of aMRA).Letus translatetheCG

G

^{givenin Example}

1 into a MRA (f. Figure 2). The states of eah of these RA orrespond to

every possible subategory of a ategory assigned by

G

^{to a element of the}

voabulary,plusanalstate

F

^inthe

RA F A

^{(above),andaninitialstate}

I

^inthe

RA BA

^(under).Thetransitionshavebeendesignedexatlyasexplainedbefore.

Then,eahRAhasbeensimpliedforreadability(someun-neessarystatesand

transitions are deleted), but not asmuh aspossible: here,wehave hosen to

preservetherepresentationofallthenal voabulary

Σ

^{in bothautomata.}

T

S T\S F (T\S)\(T \S)

S/(T \S) CN

S/(T \S)

T

John

T \S

runs

(T\S)\(T \S)

fast

a

(S/(T \S))/CN

man

CN

CN (T \S)\(T\S)

I T S

(S/(T\S))/CN T \S

man CN

T

John

T\S

runs

runs

T \S

a

(S/(T\S)/CN

fast

(T\S)\(T \S)

(T\S)\(T \S)

Fig.2.Apairofmutuallyreursiveautomata:the

RA _{F A}

^andthe

RA _BA

Denition4 (MRA and their Language). A pair of mutually reur-

sive automata (or MRA) is a pair

M = (R F A , R BA )

^where

R F A = hQ ∪ {F}, Σ, γ F A , S F A , F i

^{is a}

RA F A

^and

R BA = hQ ∪ {I}, Σ, γ BA , I, S BA i

^{is a}

RA BA

^{sharingthe samevoabulary}

Σ

^{andthesamesetofstatenames}

Q

^exept

for the nal state of the

RA F A

⁽

F / ∈ Q

^{) and for the initial stateof the}

RA BA

(

I / ∈ Q

^{). Weonsider}

ǫ ∈ L F A (I)

^and

ǫ ∈ L BA (F )

^{and for every state}

q ∈ Q

^,

the language

L M (q)

^{of thestate}

q

ⁱⁿ

M

^{isthe smallestsetsuhthat:}

L F A (q) ∪ L BA (q) ⊆ L M (q)

if there existsa transition labelledby

r ∈ Q

^between

q

^and

q ^′ ∈ Q

ⁱⁿ

R F A

(resp.in

R BA

^),i.e.

q ^′ ∈ γ F A (q, r)

^(resp.

q ^′ ∈ γ BA (q, r)

^)then:

L M (r).L F A (q ^′ ) ⊆ L F A (q)

^(resp.

L BA (q).L M (r) ⊆ L BA (q ^′ )

^).

Wedene thelanguage ofthe MRA as:

L(M ) = L M (S F A ) ∪ L M (S BA ).

ForeveryCG

G

^{, thereexists aMRA}

M = (R F A , R BA )

^{stronglyequivalent}

with

G

^{,i.e.generatingthesamestrutures.}

3 Learning by speialization

3.1 Learning rigid CGfrom positiveexamples

A rigid CG is a CG in whih every

v ∈ Σ

^{is assigned at most one ategory.}

Kanazawa has proved [13,14℄ that the set of every (bidiretional) rigid CG

is learnable in the limit (i.e. in the sense of [12℄) from positive examples, i.e.

fromsentenes.Twodistintlearningalgorithmsarenowavailable forthis pur-

pose. The best known is Kanazawa's, derived from BP (proposed earlier by

Buszkowski and Penn [3℄) and is alassial generalization strategy. The other

one, alled RGPL (Rigid GrammarPartial Learning) is desribed by Moreau

in [16℄. It is this seond algorithm that we will onentrateonhere. Although

itsauthordidnotpresentitthisway,weshowthatitisin fataspeialization

strategy.

Letusrstillustratehowitworksonasimpleexample.Wesupposethatthe

availablesetofpositiveexamplesis

{

^{Johnruns,amanrunsfast}

}

^.Atitsrst

step,thealgorithmassignstoeahmemberofthevoabularyusedatleastone

in theexamplesadistintvariable. Thisinitialassignmentisthushere:

A = {hJohn, x 1 i, hruns, x 2 i, ha, x 3 i, hman, x 4 i, hf ast, x 5 i}

^.

Evenifawordisusedseveraltimesin theexamples,onlyonevariableisintro-

duedbeausethetargetgrammarisrigid.Infat,

A

^{impliitely speiesaset}

of grammars: theset ofrigidCGsbuiltontheusedvoabulary.Asamatterof

fat,everysuhrigidCG

G

^{anbeobtainedbyapplying a}substitution

σ

^from

thesetofvariablestoasetofategoriesto

A

^suhthat:

G = σ(A) = {hv, σ(C)i|hv, Ci ∈ A}

Thesubstitution

σ

^{hasonlytheeetofrenamingthevariablesintoategories.}

Ofourse,

A

^{an alsobe}representedby aMRA

M = (R F A , R BA )

^{.In this}

MRA,

R F A

^(resp.

R BA

^)has

{x 1 , x 2 , x 3 , x 4 , x 5 , F }

^(resp.

{I, x 1 , x 2 , x 3 , x 4 , x 5 }

^)as

set of states, and eah state

x i

^for

1 ≤ i ≤ 5

^{is onneted to}

F

^(resp.

I

^is

onneted to

x i

^{) by atransition labelled by the} orresponding word (another transition labelled by

x i

^{should be addedbut it is uselessat thispoint).As}

S

appearsnowherein this MRA, thelanguageit reognizesis empty. Butit isa

ompatwaytorepresentthe wholelass ofrigid CGsbuilt on

Σ

^.

Then, eah sentene is syntatially parsedwith theassigmentsin

A

^{, bya}

CYK-likealgorithm.TheonlytwopossiblewaystoparseJohnruns are:

eithertoreplae

x 1

^by

S/x 2

^:thena

F A

^{ruleanbeapplied}

eithertoreplae

x 2

^by

x 1 \S

^:thena

BA

^{ruleanbeapplied}

These kindsof substitutions express aonstraint that thevariables (

x 1

^or

x 2

⁾

must satisfy:

A

^{must thus be updatedto a disjuntion of sets of} assignments, eahsubsetorrespondingtoasublassofrigidCGs.Asimplerwaytostorethe

urrentsubsetsof possiblesolutionsisto storetheset ofpossiblesubstitutions

thatanbeappliedto

A

^{.Inourase,thissetismadeof}

{σ 1 , σ 2 }

^,with

σ 1 (x 1 ) = S/x 2

^{and is equalto the idential funtion elsewhere, and}

σ 2 (x 2 ) = x 1 \S

^and

is equal to the idential funtion elsewhere.

σ 1 (A)

^{, as well as}

σ 2 (A)

^{, an be}

represented by a MRA derived from the previous one. This time, both MRA

reognizeexatlythesenteneJohnruns.

Toparseamanrunsfast,manymoresolutionsarepossible.Themaximum

theoretialnumberis

5 ∗ 2 ³ = 40

^{beausethere are}

5

^{possiblebinarybranhing}

trees with

4

^{leaves(this anbe omputed in the general aseby the Catalan}

number),andeahofthem has

3

^{internal nodeswhih anreeiveeithera}

F A

or a

BA

^{label. This makes}

40 ∗ 2 = 80

^{theoretial possible} substitutions by ombining theonstraintsobtained from both sentenes(the ombinaisonis a

lassialomposition of funtions). But, among them, some areontraditory:

asthetargetgrammarisrigid,theuniqueategoryassignedtothewordruns

annotbeoftheform

x i /x j

^and

x k \x l

^{atthesametime.Wethusseewherethe}

initiallassplaysarolein thelearningstrategy.

Itiseasytoseethatthemainproblemwiththisalgorithmistheombinato-

rialexplosionithastofae,espeiallywhenexamplesdonotshareanyommon

word.Thisisnotsurprising,sinetheproblem oflearningrigid CGsfrom sen-

tenes isknownto beNP-hard [4℄.Tolimitthis explosion,Moreauproposes to

exploitasmuhinitialknowledgeaspossible,intheformofaninitialgrammar,

that is,initiallyknownategoriesforsomeusualwords(forexamplethelexial

ones)whihannotberenamed,asitistheaseforthevariableategories.

Furthermore,thereisnoguaranteeatallthatthisstrategyalwaysonverges

toauniquesolution.Intheory,tofullltherequirementsoflearnabiblityinthe

limit, when several possibleompatible grammars are available,inlusion tests

should beperformed toseletthe onegenerating thesmallest language.This

problemalsoourredwithKanazawa'salgorithm,whenappliedto sentenes.

3.2 State mergesand state splits

The previousstrategy annowbe interpretedin termsof operations applying

on MRA. As we have seen, at every step of this algorithm, the searh spae

is a disjuntion of sets of assignmentsof the form

σ(A)

^{for some} substitution

σ

^{, and eah of them anbe} representedby a MRA. The MRA orresponding

with

A

^{reognizesno sentene. But, assoonas at leastone examplehas been}

treated(and,thus,theategory

S

^beenintrodued),

σ(A)

^{speiesasetofCGs}

reognizingatleastthisexample.WhatistheeetofaonstraintonaMRA?

Theonstraintsalwaystaketheform:

x k = x l

^,where

x k

^and

x l

^arealready

introduedvariablesorequalto

S

^,or

x k = X m /X n

^or

x k = X m \X n

^,with

X m

and

X n

^{anyategorybuiltontheset ofeveryvariableunionS.}

theeet ofaonstraintoftheform

x k = x l

^{onaMRAisastatemerge in}

both the

RA F A

^{and the}

RA BA

^{of the MRA. As, in MRA,}

x k

^{an also be}

usedastransitionlabels,orrespondingtransition merges analsoour.

theeet of aonstraintof theform

x k = X m /X n

^(resp.

X m \X n

^{)an be}

deomposedintofour steps:

1.

X m /X n

^(resp.

X m \X n

^)replaes

x k

^{everywhereintheMRA;}

2. every subategory of

X m

^and

X n

^(inluding themselves) not already identied (i.e.notalready asub-ategoryof thepreviousset of assign-

ments)beomesanewstateinbothRA:inthe

RA F A

^{,itislinkedtothe}

state

F

^(resp.inthe

RA BA

^fromthestate

I

^{)byatransitionlabelledby}

itsname;

3. inthe

RA F A

^{(resp.in the}

RA BA

^{),anewtransitionlabelledby}

X m /X n

(resp.

X m \X n

^{) links the states}

X m

^and

X n

^{, and the sameours for}

everynewlyidentiedsubategory;

4. thestatesandtransitionsofthesamenamearemergedineahRA.

This operation an now be ompared to the state splitting strategy pro-

posedbyFredouilleandMiletin[11℄tolearnregularlanguagesrepresentedby

nite stateautomata byspeialization.Forexample,theonstraint

x 1 = S/x 2

hastheeet of splittingthe state

x 1

^{into twonewstates:}

S

^and

x 2

^{. Then, as}

astatenamed

x 2

^{already exists,the newoneis mergedwiththe previousone.}

But our speialization operation is more generalthan Fredouilleand Milet's,

beause of the reursivenature of the automata on whih it applies. Further-

more,their algorithm wasaspeialization strategyat the language level: their

initialhypothesiswasthemostgeneralregularlanguage

Σ ^∗

^{andonstraintswere}

used to speialize the language. Moreau's algorithmis aspeialization strategy

at the set of grammars level: its initial hypothesis is theset of possible gram-

mars, and the examplesare used to introdue onstraints that redue this set

tosubsets.TheorrespondingMRArepresentsaset ofgrammars andnotonly

a spei language.Forthese reasons, ourapproah annot be easily adapted

to usualnite stateautomata.Butwebelievethatourstatesplitting operator

isbetterfoundedthanthepreviousone,beauseitistheformalounterpartof

well-denedsubstitutions.

4 Learning from typed examples revisited

We now show that the algorithm proposed in [8,7℄ to learn CGs from typed

examplesan be onsidered asaspeialization strategywhere state splits and

statemergesareontrolledbyatypingapproah.

4.1 Learning from semantiallytyped examples

Theideaof learningCGs from typedexampleswasrstintroduedin [8℄.The

typesonsideredinthisworkareborrowedfromMontague'stheory[6℄:theyare

fromtypedexamplesisognitivelyrelevantbeausetypesanbeinterpretedas

semantiinformationavailableintheenvironmentorpreviouslylearned.Inthis

setion,webrieyreallthisnotioninageneralfashionandgivetheonditions

underwhihtheyanhelp learning.

ThenotionoftypesusefulforlearningCGs isbasedon:

a nite set

τ

^{of basi types among whih is a} distinguished type

t ∈ τ

standingfortruthvalues:usually,this setis

τ = {e, t}

^where

e ∈ τ

^isthe

typeofentities.Montaguealsousedatype

s

^{forintensions thatwillnot}

beusedin thefollowing;

the set

Types(τ)

^{of every typeis the smallestset suh that}

τ ⊂ Types(τ)

andforeverytype

α, β ∈ Types(τ)

^,

hα, βi ∈ Types(τ)

^.

hα, βi

^{is thetypeof}

funtionsthatrequireanargumentoftype

α

^{andprovidearesultoftype}

β

^.

Typesin

Types(τ)

^{are usefulfor learning aCG only ifthey are onneted}

with itssyntatiategories in

Cat (B)

^{.Morepreisely, theneessaryondition}

tobefullledisthat thereexistsahomomorphism

h

^suhthat:

for everybasi ategory

C ∈ B

^,

h(C)

^{is dened and belongs to}

Types(τ)

^.

Thedistinguishedategory

S ∈ B

^{isassoiatedwiththe}distinguishedtype

t ∈ τ

^:

h(S) = t

^.

for every other ategory in

Cat (B)

^{of the form}

A/B

^or

A\B

^{, we have:}

h(A/B) = h(B\A) = hh(B), h(A)i

^.

Example 3 (lassial semantitypes for natural languages). Let

τ = {e, t}

^.The

wordsofthegrammardenedinExample1reeivethefollowingsemantitypes:

Johnanbeonsideredasanentityoftype

e

^;

runsandman areone-plaeprediates;theirtypeis:

he, ti

^;

fastisaone-plae-prediatemodier,i.e.ittransformsaprediateofarity

oneintoanotheroneofthesamearity:itthushasthetype

hhe, ti, he, tii

^;

naly,ifwefollowMontague'sintuitionaboutthepropertreatmentofquan-

tiation[6℄,thedeterminerahasthemostomplextype:

hhe, ti, hhe, ti, tii

^.

Theorrespondinghomomorphism

h

^isdenedby:

h(S) = t

^,

h(T ) = e

^,

h(CN) = he, ti

^{.Asrequired,if}

hv, Ci ∈ G

^{,thesemantitypeof}

v

^is

h(C)

^.

4.2 How types helpto ontrol state splitsand state merges

Learning from typed examples means learningfrom sentenes where eah ele-

ment of the voabulary

v ∈ Σ

^{, whih should be assigned}

C ∈ Cat (B)

^{by the}

targetgrammar

G

^{toanalysethis sentene,is provided withthe}orresponding type

h(C) ∈ Types(τ )

^{. As we will see, the learningstrategy proposed in [8,7℄}

an alsobeinterpretedin termsofoperationsapplyingonMRA.Weillustrate

thisalgorithmonourexample.Theinputdataarenowoftheform:

John runs

e he, ti

e x 1 he, ti

hhe, ti, hhe, ti, tii he, ti he, ti hhe, ti, he, tii x 2 hx 3 he, ti, x 4 hx 5 he, ti, tii x 6 he, ti x 1 he, ti x 7 hx 8 he, ti, x 9 he, tii

Inthesetypedexamples,thethirdlineistheresultofasimplepre-treatment

whih onsistsin introduing variables in frontof everyfuntional type.The

variablesarealldistint,exeptwhenthesameoupleword,typeours(asit

istheaseherefortheoupleruns,

he, ti

^{).Thesevariableswilleventuallytake}

thevalue

/

^or

\

^{duringthelearningproess.Theinitialsetof}assignements is,thistime:

A = {hJohn, ei, hruns, x 1 he, tii, ha, x 2 hx 3 he, ti, x 4 hx 5 he, ti, tiii, hman, x 6 he, tii, hf ast, x 7 hx 8 he, ti, x 9 he, tiii}

^.

Aspreviously,

A

^{impliitelyspeiesasetof grammars.Thissetismuhlarger}

than the one of rigid CGs:it is the set of CGs whih an assign an arbitrary

number of distint ategories to eah word (so, it intersets everylass of

k

^-

valuedCGs),butforwhihthereexistsahomomorphismsuhthateverydistint

ategoryassignedtothesamewordgivesrisetoadistinttype.Informalterms,

itissuhthatthereexists ahomomorphism

h

satisfying:

∀hv, C 1 i, hv, C 2 i ∈ G

^,

h(C 1 ) = h(C 2 ) = ⇒ C 1 = C 2

^.

We haveshown [9℄ that forevery

ǫ

^-freeontext-free language,it is possible to deneaCGgeneratingthislanguage,asetoftypesandahomomorphismsuh

thatthispropertyissatised.Thisnewtargetlassislearnableinthelimitfrom

typedexamples[8,7℄.

As previously,

A

^{an also be} represented by aMRA. But the information arried by thetypesis muh riherthan theone arried by thebasivariables

Moreauused: typesan be interpretedassomekindof maximalbound onthe

possibleategoriestheyreplae;theydisplayalltheirpotentialrenaming.

Thelearningalgorithmappliesasinsetion3.1:itonsistsintryingtoparse

eah sentene with the rules

FA

^and

BA

^{adapted to typesso as to reah the}

type

t

^{at theroot, bydening onstraintson thevariables(see [7℄for details).}

The onlypossibletype-ompatible way to parsethe rsttyped exampleJohn

runs isto have:

x 1 = \

^{, meaningthat onlya}

BA

^{rule is ompatible with the}

typeassignments.runs shouldthusnalyreeivetheategory

e\t

^{.This time,}

there is only one type-ompatible way to parse a man runs fast: this parse

(isomorphitotheoneinFigure1) isshownonFigure3.Both typedexamples

lead to the following (unique) set of onstraints:

x 2 = /

^,

x 3 = x 6

^,

x 7 = \

^,

x 8 = x 1 = \

^,

x 4 = /

^,

x 5 = x 9

^.

Theset ofassignmentsisthusupdatedto:

A = {hJohn, ei, hruns, \he, tii, ha, /hx 3 he, ti, /hx 5 he, ti, tiii, hman, x 3 he, tii, hf ast, \h\he, ti, x 5 he, tiii}

^.

Ifweapplytheproess ofsetion2.4 tothis set(after re-orderingthetypesto

makethemsimilartosyntatiategoriesand

t

^{playingtheroleof}

S

^),weobtain

theMRAofFigure4.Inthisexample,withonlytwotypedexamples,weobtain

auniqueMRAwhihisnearlyisomorphitothetargetone.

Inthis ontext,theonstraintstaketheform

x i = x j

^,

x i = /

^or

x i = \

^and

t FA : x 4 = /

x ⁵ = x ⁹

x 4 hx 5 he, ti, ti FA : x ² = /

x 3 = x 6

x 2 hx 3 he, ti, x 4 hx 5 he, ti, tii

a

x 6 he, ti

man

x 9 he, ti BA : x ⁷ = \

x 8 = x 1

x 1 he, ti

runs

x 7 hx 8 he, ti, x 9 he, tii

fast

Fig.3.parsetreeforatypedexample

e\t e

t x 5 he, ti F (e\t)\x 5 he, ti

t/x 5 he, ti x 3 he, ti

t/(x 5 he, ti)

e

John

x 5 he, ti e\t

runs

(e\t)\(x 5 he, ti)

fast

a

(t/x 5 he, ti)/x 3 he, ti

man

x 3 he, ti

x 3 he, ti (e\t)\x 5 he, ti

I e t

(t/(x 5 he, ti))/x 3 he, ti e\t x 5 he, ti

man

x 3 he, ti

e

John

e\t

runs

runs

e\t

a

(t/x 5 he, ti)/x 3 he, ti

fast

(e\t)\(x 5 he, ti)

(e\t)\x 5 he, ti

fast

Fig.4.MRAfortypeassignments

statesplit,butthesituationisabitmoreomplex.Inourexample,toreahthe

target,onlyoneonstraintismissing:

x 5 = \

^{.Thetypedexample}orresponding tothesenteneJohnrunsfast wouldprovidethisonstraint.Itsrsteeton

the MRAwould be to renamethe state

x 5 he, ti

^{both in the}

RA F A

^{and in the}

RA BA

^by

\he, ti

^{, that is}

e\t

^{. But, doingso, thisstate beomes idential to an}

alreadyexistingoneand thenmustbemergedto it.

The table of Figure 5 explains why types help the algorithm to avoid a

ombinatorialexplosionandtoonvergequiker.Wehaveseenthattherealways

exists ahomomorphism

σ

^{betweenolumn 2andolumn 3,whih isthetarget}

of the learning proess. Hypotheses about types ensure that there also exists

ahomomorphism

h

^{betweenolumn 3 and olumn 4.This situation is similar}

to the onedesribed in [15℄,and analyzedin [5℄. Thetwo learningalgorithms

presentedhere are bothspeialization strategies at the set of grammars level,

buttheirinitialhypothesisiseitheralowerboundoranupperboundoftheset

of ategories of the target grammar.Types are eient beausethey allow to

ontrolthepossiblerenamings.

voabulary Moreau'sinitialtargetategory pre-treated

assigment initialassignment types

John

x 1 T e

a

x 2 (S/(T \S)/CN x 2 hx 3 he, ti, x 4 hx 5 he, ti, tii

man

x 3 CN x 6 he, ti

runs

x 4 T \S x 1 he, ti

fast

x 5 (T \S)\(T \S) x 7 hx 8 he, ti, x 9 he, tii

Fig.5.tabularshowingthestartingpointsandtargetofthetwoalgorithms

5 Conlusion

In grammatial inferene from positive examples, two soures of information

are usually available: the target lass and theset of examples. Generalization

tehniquesusetheexamplestogenerate amostspei grammar ompatible

withthem(theprextreeautomatonintheaseofregularlanguages),andthen

use thetarget lass to generalizeit. Speialization tehniques dothe ontrary:

thelassisthestartingpointand theexampleshelptospeializeit.

Inthis paper, we propose anew perspetive onthese tehniques.First, we

see that disjuntions of MRA are able to represent the searh spae of suh

learning algorithms. Seond, we show that the algorithm to learn CGs from

typedexamplesproposed in[8,7℄introduestypeontrolinto theproess.The

initial semanti types assoiated with the elements of the voabulary speify

some kindof maximal bound on the possible renamings, allowingto limit the

1. D.Angluin. InfereneofReversibleLanguages JournaloftheACM3:p.741765,

1982.

2. Y.BarHillelandC.GaifmanandE.Shamir.OnCategorialandPhraseStruture

Grammars. BulletinoftheResearh CounilofIsrael,9F,1960.

3. W.BuszkowkiandG.Penn.Categorialgrammarsdeterminedfromlinguistidata

byuniation,newblokStudiaLogia,p.431454,1990.

4. C. Costa-Florenio. Consistent Identiation in the Limit of Rigid Grammars

fromStrings IsNP-hard. proeedings oftheIGGI: Algorithms andAppliations,

p.4962,LNAI2484,2002.

5. F. Coste, D. Fredouille, C. Kermovant,C. de laHiguera. Introduingdomain

and typingbiasinautomatainferene, inproeedings of the7thICGI,Vol.3264

ofLNAI,p.115126,2004.

6. D. Dowty and R.E. Wall and S. Peters. Introdution to Montague Semantis,

LinguistisandPhilosophy,Reidel,1981.

7. D. Dudau-Sofronie. Apprentissage de grammaires atégorielles pour simuler

l'aquisition du langage naturel à l'aide d'informations sémantiques, thèse

d'informatique,universitéLille3,2004.

8. D. Dudau-Sofronie, I. Tellier, and M. Tommasi. Learning ategorial grammars

fromsemantitypes, inproeedingsofthe13thAmsterdamColloquium,p.7984,

2001.

9. D.Dudau-Sofronie,I.Tellier,andM.Tommasi. ALearnableClassofCCGsfrom

Typed Examples, inproeedings of the 8thonferene onFormal Grammars,p.

7788,2003.

10. P.Dupont and L. Milet andE. Vidal. What is the searhspae of the regular

inferene. proeedingsofICGI,p.2537,LNCS862,1994.

11. D.Fredouille,andL.Milet.Experienessurl'inferenedelangageparspeialisa-

tion, inproeedings ofCAP'2000,p.117130, 2000.

12. E.M. Gold. Language identiation inthe limit. Information and Control, 10:

p.447474, 1967.

13. M.Kanazawa.Identiationinthelimitofategorialgrammars,JournalofLogi,

Language andInformation5(2),p.115155,1996.

14. M. Kanazawa. Learnable Classes of Categorial Grammars, CSLI Publiations.

1998.

15. C.Kermovant,C.delaHiguera Learninglanguagewithhelp proeedingsofICGI,

p.161173, LNAI,2002.

16. E.Moreau.Apprentissagepartieldegrammaireslexialisées, TAL45(3),p.71102,

2004.

17. I. Tellier. When ategorial grammars meet regular grammatial inferene, in

proeedings ofthe5thLACL,Vol.4492 ofLNAI,p.317332, 2005.

18. I.Tellier.Learningreursiveautomatafrompositiveexamples,Revued'Intelligene

ArtiielleNewMethodsinMahineLearning(20/2006),p.775804, 2006

19. I.Tellier. Grammatialinferenebyspeializationasastatesplittingstrategy, in

proeedings ofthe16thAmsterdam Colloquium,p.223-228,2007.

20. W.A.WoodsTransitionNetworkGrammarsofNaturalLanguageAnalysis Com-

muniationof theACMvol.13,p.591606, 1970.