Composing extended top-down tree transducers

Composing extended top-down tree transducers

Aur´elie Lagoutte

Ecole normale sup´erieure de Cachan, D´epartement Informatique´ [email protected]

Fabienne Braune and Daniel Quernheim and Andreas Maletti University of Stuttgart, Institute for Natural Language Processing {braunefe,daniel,maletti}@ims.uni-stuttgart.de

Abstract

A composition procedure for linear and nondeleting extended top-down tree trans- ducers is presented. It is demonstrated that the new procedure is more widely applica- ble than the existing methods. In general, the result of the composition is an extended top-down tree transducer that is no longer linear or nondeleting, but in a number of cases these properties can easily be recov- ered by a post-processing step.

1 Introduction

Tree-based translation models such as syn- chronous tree substitution grammars (Eisner, 2003; Shieber, 2004) or multi bottom-up tree transducers (Lilin, 1978; Engelfriet et al., 2009;

Maletti, 2010; Maletti, 2011) are used for sev- eral aspects of syntax-based machine transla- tion (Knight and Graehl, 2005). Here we consider the extended top-down tree transducer (XTOP), which was studied in (Arnold and Dauchet, 1982; Knight, 2007; Graehl et al., 2008; Graehl et al., 2009) and implemented in the toolkit TIBURON (May and Knight, 2006; May, 2010).

Specifically, we investigate compositions of linear and nondeleting XTOPs (ln-XTOP). Arnold and Dauchet (1982) showed that ln-XTOPs compute a class of transformations that is not closed under composition, so we cannot compose two arbitrary ln-XTOPs into a single ln-XTOP. However, we will show that ln-XTOPs can be composed into a (not necessarily linear or nondeleting) XTOP. To illustrate the use of ln-XTOPs in machine transla- tion, we consider the following English sentence together with a German reference translation:

∗ All authors were financially supported by the EMMY

NOETHERproject MA / 4959 / 1-1 of the German Research Foundation (DFG).

RC PREL

that C NP VP

7→ C

NP VP

C NP VP VAUX VPART NP

7→

C NP VP

VAUX NP VPART

Figure 1: Word drop [top] and reordering [bottom].

The newswire reported yesterdaythat the Serbs have completed the negotiations.

Gestern [Yesterday] berichtete [reported] die [the]

Nachrichtenagentur [newswire] die [the] Serben [Serbs]h¨atten[would have]die[the]Verhandlungen [negotiations]beendet[completed].

The relation between them can be described (Yamada and Knight, 2001) by three operations:

drop of the relative pronoun, movement of the participle to end of the clause, and word-to-word translation. Figure 1 shows the first two oper- ations, and Figure 2 shows ln-XTOP rules per- forming them. Let us now informally describe the execution of an ln-XTOP on the top rule ρ of Figure 2. In general, ln-XTOPs process an in- put tree from the root towards the leaves using a set of rules and states. The state pin the left- hand side ofρcontrols the particular operation of Figure 1 [top]. Once the operation has been per- formed, control is passed to statesp_NP andp_VP, which use their own rules to process the remain- ing input subtree governed by the variable below them (see Figure 2). In the same fashion, an ln- XTOP containing the bottom rule of Figure 2 re- orders the English verbal complex.

In this way we model the word drop by an ln- XTOPM and reordering by an ln-XTOPN. The syntactic properties of linearity and nondeletion yield nice algorithmic properties, and the mod-

p RC PREL

that C y1 y2

→

C pNP

y1

pVP

y2

q C z1 VP

z2 z3 z4

→

C qNP

z1

VP qVA

z2

qVP

z4

qNP

z3

Figure 2: XTOP rules for the operations of Figure 1.

ular approach is desirable for better design and parametrization of the translation model (May et al., 2010). Composition allows us to recombine those parts into one device modeling the whole translation. In particular, it gives all parts the chance to vote at the same time. This is especially important if pruning is used because it might oth- erwise exclude candidates that score low in one part but well in others (May et al., 2010).

Because ln-XTOP is not closed under compo- sition, the composition ofMandN might be out- side ln-XTOP. These cases have been identified by Arnold and Dauchet (1982) as infinitely “over- lapping cuts”, which occur when the right-hand sides ofM and the left-hand sides ofN are un- boundedly overlapping. This can be purely syn- tactic (for a given ln-XTOP) or semantic (inher- ent in all ln-XTOPs for a given transformation).

Despite the general impossibility, several strate- gies have been developed: (i) Extension of the model (Maletti, 2010; Maletti, 2011), (ii) online composition (May et al., 2010), and (iii) restric- tion of the model, which we follow. Composi- tions of subclasses in which the XTOPN has at most one input symbol in its left-hand sides have already been studied in (Engelfriet, 1975; Baker, 1979; Maletti and Vogler, 2010). Such compo- sitions are implemented in the toolkit TIBURON. However, there are translation tasks in which the used XTOPs do not fulfill this requirement. Sup- pose that we simply want to compose the rules of Figure 2, The bottom rule does not satisfy the re- quirement that there is at most one input symbol in the left-hand side.

We will demonstrate how to compose two lin- ear and nondeleting XTOPs into a single XTOP, which might however no longer be linear or non- deleting. However, when the syntactic form of

δ^(ε) q⁽¹⁾

x⁽¹¹⁾₁

σ⁽²⁾ α⁽²¹⁾ q⁽²²⁾

x⁽²²¹⁾₂ γ⁽³⁾ γ⁽³¹⁾ p⁽³¹¹⁾ x⁽³¹¹¹⁾₃

δ q x1

α γ γ p x3

Figure 3: Linear normalized treet∈TΣ(Q(X))[left]

andt[α]2[right] withvar(t) ={x1, x2, x3}. The posi- tions are indicated intas superscripts. The subtreet|2

isσ(α, q(x2)).

the composed XTOP has only bounded overlap- ping cuts, post-processing will get rid of them and restore an ln-XTOP. In the remaining cases, in which unbounded overlapping is necessary or occurs in the syntactic form but would not be nec- essary, we will compute an XTOP. This is still an improvement on the existing methods that just fail. Since general XTOPs are implemented in TIBURONand the new composition covers (essen- tially) all cases currently possible, our new com- position procedure could replace the existing one in TIBURON. Our approach to composition is the same as in (Engelfriet, 1975; Baker, 1979; Maletti and Vogler, 2010): We simply parse the right- hand sides of the XTOP M with the left-hand sides of the XTOPN. However, to facilitate this approach we have to adjust the XTOPsM andN in two pre-processing steps. In a first step we cut left-hand sides of rules ofN into smaller pieces, which might introduce non-linearity and deletion into N. In certain cases, this can also intro- duce finite look-ahead (Engelfriet, 1977; Graehl et al., 2009). To compensate, we expand the rules of M slightly. Section 4 explains those prepa- rations. Next, we compose the prepared XTOPs as usual and obtain a single XTOP computing the composition of the transformations computed by M andN (see Section 5). Finally, we apply a post-processing step to expand rules to reobtain linearity and nondeletion. Clearly, this cannot be successful in all cases, but often removes the non- linearity introduced in the pre-processing step.

2 Preliminaries

Our trees have labels taken from an alphabet Σ of symbols, and in addition, leaves might be labeled by elements of the countably infinite

σ x1 γ

δ β β x2

7→θ

σ α γ

δ β β x2

←θ[ σ α x3

Figure 4: Substitution whereθ(x1) =α,θ(x2) =x2, andθ(x₃) =γ(δ(β, β, x₂)).

set X = {x₁, x₂, . . .}of formal variables. For- mally, for every V ⊆ X the set T_Σ(V) of Σ-trees withV-leaves is the smallest set such that V ⊆ T_Σ(V) andσ(t₁, . . . , t_k) ∈ T_Σ(V) for all k∈N,σ ∈Σ, andt₁, . . . , t_k∈T_Σ(V). To avoid excessive universal quantifications, we drop them if they are obvious from the context.

For each treet∈T_Σ(X)we identify nodes by positions. The root ofthas positionεand the po- sition iw withi ∈ Nandw ∈ N^∗ addresses the position w in the i-th direct subtree at the root.

The set of all positions in tis pos(t). We write t(w)for the label (taken fromΣ∪X) oftat po- sitionw∈pos(t). Similarly, we use

• t|_w to address the subtree oftthat is rooted in positionw, and

• t[u]_w to represent the tree that is ob- tained from replacing the subtree t|_w at w byu∈TΣ(X).

For a given setL⊆Σ∪Xof labels, we let pos_L(t) ={w∈pos(t)|t(w)∈L}

be the set of all positions whose label belongs toL. We also writepos_l(t) instead ofpos_{l}(t).

The treet∈T_Σ(V)islinearif|pos_x(t)| ≤1for everyx∈X. Moreover,

var(t) ={x∈X |pos_x(t)6=∅}

collects all variables that occur in t. If the vari- ables occur in the orderx1, x2, . . . in a pre-order traversal of the treet, thentisnormalized. Given a finite setQ, we writeQ(T) with T ⊆ TΣ(X) for the set{q(t) |q ∈ Q, t ∈ T}. We will treat elements of Q(T) as special trees of TΣ∪Q(X).

The previous notions are illustrated in Figure 3.

A substitutionθis a mappingθ:X→TΣ(X).

When applied to a treet ∈ T_Σ(X), it will return the treetθ, which is obtained fromtby replacing all occurrences of x ∈ X (in parallel) by θ(x).

This can be defined recursively byxθ =θ(x)for allx ∈ X andσ(t₁, . . . , t_k)θ = σ(t₁θ, . . . , t_kθ)

q_S S x₁ VP

x2 x3

→

S’

q_V x2

q_NP x1

q_NP x1

t q_S

S

t₁ VP

t2 t3

⇒

t S’

qV

t2

qNP

t1

qNP

t1

Figure 5: Rule and its use in a derivation step.

for allσ ∈Σandt₁, . . . , t_k∈T_Σ(X). The effect of a substitution is displayed in Figure 4. Two substitutions θ, θ⁰: X → TΣ(X) can be com- posed to form a substitution θθ⁰: X → T_Σ(X) such thatθθ⁰(x) =θ(x)θ⁰for everyx∈X.

Next, we define two notions of compatibility for trees. Lett, t⁰ ∈TΣ(X)be two trees. If there exists a substitutionθsuch thatt⁰ =tθ, thent⁰ is aninstanceoft. Note that this relation is not sym- metric. Aunifierθfortandt⁰ is a substitutionθ such thattθ = t⁰θ. The unifierθis amost gen- eral unifier(short: mgu) fortandt⁰ if for every unifierθ⁰⁰fortandt⁰there exists a substitutionθ⁰ such thatθθ⁰=θ⁰⁰. The setmgu(t, t⁰)is the set of all mgus fortandt⁰. Most general unifiers can be computed efficiently (Robinson, 1965; Martelli and Montanari, 1982) and all mgus for t andt⁰ are equal up to a variable renaming.

Example 1. Let t = σ(x₁, γ(δ(β, β, x₂))) and t⁰ = σ(α, x₃). Then mgu(t, t⁰) containsθ such thatθ(x1) =αandθ(x3) =γ(δ(β, β, x2)). Fig- ure 4 illustrates the unification.

3 The model

The discussed model in this contribution is an extension of the classical top-down tree trans- ducer, which was introduced by Rounds (1970) and Thatcher (1970). The extended top-down tree transducer with finite look-ahead or just XTOP^Fand its variations were studied in (Arnold and Dauchet, 1982; Knight and Graehl, 2005;

qS

S x1 VP

x2 x3

S’

q_V x2

q_NP x1

q_NP x3

→

q_S S’

x2 x1 x3

S qNP

x1

VP q_V x2

q_NP x3

→

Figure 6: Rule [left] and reversed rule [right].

Knight, 2007; Graehl et al., 2008; Graehl et al., 2009). Formally, an extended top-down tree transducer with finite look-ahead (XTOP^F) is a systemM = (Q,Σ,∆, I, R, c)where

• Qis a finite set ofstates,

• Σand ∆ are alphabets of input andoutput symbols, respectively,

• I ⊆Qis a set ofinitial states,

• Ris a finite set of (rewrite)rulesof the form

` → r where ` ∈ Q(TΣ(X))is linear and r∈T_∆(Q(var(`))), and

• c:R×X → T_Σ(X) assigns alook-ahead restrictionto each rule and variable such that c(ρ, x)is linear for eachρ∈Randx∈X.

The XTOP^F M islinear (respectively,nondelet- ing) ifr is linear (respectively,var(r) = var(`)) for every rule` → r ∈ R. It hasno look-ahead (or it is an XTOP) if c(ρ, x) ∈ X for all rules ρ∈Randx∈X. In this case, we drop the look- ahead componentc from the description. A rule

` → r ∈ R isconsuming(respectively, produc- ing) ifpos<sub>Σ</sub>(`)6= ∅(respectively,pos_∆(r) 6=∅).

We letLhs(M) ={l| ∃q, r:q(l)→r ∈R}.

LetM = (Q,Σ,∆, I, R, c)be an XTOP^F. In order to facilitate composition, we define senten- tial forms more generally than immediately nec- essary. Let Σ⁰ and ∆⁰ be such that Σ ⊆ Σ⁰ and ∆ ⊆ ∆⁰. To keep the presentation sim- ple, we assume that Q ∩ (Σ⁰ ∪ ∆⁰) = ∅. A sentential form of M (using Σ⁰ and ∆⁰) is a tree of SF(M) = T∆⁰(Q(TΣ⁰)). For every ξ, ζ ∈ SF(M), we writeξ ⇒_M ζ if there exist a positionw∈pos_Q(ξ), a ruleρ=`→r∈R, and a substitutionθ:X →TΣ<sup>0</sup> such thatθ(x)is an in- stance ofc(ρ, x)for everyx∈Xandξ=ξ[`θ]w

and ζ = ξ[rθ]_w. If the applicable rules are re- stricted to a certain subsetR⁰ ⊆ R, then we also writeξ ⇒_R⁰ ζ. Figure 5 illustrates a derivation step. Thetree transformation computed byM is τ_M ={(t, u)∈T_Σ×T_∆| ∃q∈I:q(t)⇒^∗_M u}

where ⇒^∗_M is the reflexive, transitive closure of⇒_M. It can easily be verified that the definition

p C y1 y2

→

RC PREL

that

C pNP

y₁

pVP

y₂ Figure 7: Top rule of Figure 2 reversed.

ofτ_M is independent of the choice ofΣ⁰and∆⁰. Moreover, it is known (Graehl et al., 2009) that each XTOP^F can be transformed into an equiva- lent XTOP preserving both linearity and nondele- tion. However, the notion of XTOP^Fwill be con- venient in our composition construction. A de- tailed exposition to XTOPs is presented by Arnold and Dauchet (1982) and Graehl et al. (2009).

A linear and nondeleting XTOP M with rules R can easily be reversed to obtain a linear and nondeleting XTOP M⁻¹ with rulesR⁻¹, which computes the inverse transfor- mation τ_M⁻¹ = τ_M⁻¹, by reversing all its rules.

A (suitable) rule is reversed by exchanging the locations of the states. More precisely, given a rule q(l) → r ∈ R, we obtain the rule q(r⁰) → l⁰ ofR⁻¹, where l⁰ = lθ andr⁰ is the unique tree such that there exists a substitution θ:X → Q(X) with θ(x) ∈ Q({x}) for every x ∈ X and r = r⁰θ. Figure 6 displays a rule and its corresponding reversed rule. The reversed form of the XTOP rule modeling the insertion op- eration in Figure 2 is displayed in Figure 7.

Finally, let us formally define composition.

The XTOPM computes the tree transformation τM ⊆ TΣ ×T∆. Given another XTOP N that computes a tree transformationτ_N ⊆ T_∆×T_Γ, we might be interested in the tree transforma- tion computed by the composition of M and N (i.e., runningM first and thenN). Formally, the compositionτ_M ;τ_N of the tree transformations τM andτN is defined by

τM;τN ={(s, u)| ∃t: (s, t)∈τM,(t, u)∈τN} and we often also use the notion ‘composition’ for XTOP with the expectation that the composition ofM andN computes exactlyτ_M ;τ_N.

4 Pre-processing

We want to compose two linear and nondelet- ing XTOPs M = (P,Σ,∆, I_M, R_M) and

LHS(M⁻¹) LHS(N)

C y₁ y₂

C z₁ VP

z2 z3 z4

Figure 8: Incompatible left-hand sides of Example 3.

N = (Q,∆,Γ, I_N, R_N). Before we actually per- form the composition, we will prepareM andN in two pre-processing steps. After these two steps, the composition is very simple. To avoid com- plications, we assume that (i) all rules of M are producing and (ii) all rules ofN are consuming.

For convenience, we also assume that the XTOPs M and N only use variables of the disjoint sets Y ⊆XandZ⊆X, respectively.

4.1 Compatibility

In the existing composition results for subclasses of XTOPs (Engelfriet, 1975; Baker, 1979; Maletti and Vogler, 2010) the XTOPN has at most one input symbol in its left-hand sides. This restric- tion allows us to match rule applications ofN to positions in the right-hand sides of M. Namely, for each output symbol in a right-hand side ofM, we can select a rule of N that can consume that output symbol. To achieve a similar decompo- sition strategy in our more general setup, we in- troduce a compatibility requirement on right-hand sides of M and left-hand sides of N. Roughly speaking, we require that the left-hand sides ofN are small enough to completely process right- hand sides of M. However, a comparison of left- and right-hand sides is complicated by the fact that their shape is different (left-hand sides have a state at the root, whereas right-hand sides have states in front of the variables). We avoid these complications by considering reversed rules ofM. Thus, an original right-hand side ofM is now a left-hand side in the reversed rules and thus has the right format for a comparison. Recall that Lhs(N) contains all left-hand sides of the rules ofN, in which the state at the root was removed.

Definition 2. The XTOP N iscompatibletoM if θ(Y) ⊆ X for all unifiers θ ∈ mgu(l₁|_w, l₂) between a subtree at a ∆-labeled position w∈pos_∆(l₁)in a left-hand sidel₁∈Lhs(M⁻¹) and a left-hand sidel₂∈Lhs(N).

Rule ofM⁻¹ Rule ofN

δ p₁ y1

p₂ y2

α ← p σ y1 y2

q σ β σ

z1 z2

→ σ q1

z1

q2

z2

Figure 9: Rules used in Example 5.

Intuitively, for every∆-labeled positionwin a right-hand sider₁ ofM and any left-hand sidel₂ ofN, we require (ignoring the states) that either (i)r1|_w andl2 are not unifiable or (ii) r1|_w is an instance ofl₂.

Example 3. The XTOPs for the English-to- German translation task in the Introduction are not compatible. This can be observed on the left-hand side l₁ ∈ Lhs(M⁻¹) of Figure 7 and the left-hand side l₂ ∈ Lhs(N) of Fig- ure 2[bottom]. These two left-hand sides are il- lustrated in Figure 8. Between them there is an mgu such thatθ(Y) 6⊆ X (e.g., θ(y₁) = z₁ and θ(y2) = VP(z2, z3, z4)is such an mgu).

Theorem 4. There exists an XTOP^F N⁰ that is equivalent toN and compatible withM.

Proof. We achieve compatibility by cutting of- fending rules of the XTOPN into smaller pieces.

Unfortunately, both linearity and nondeletion of N might be lost in the process. We first let N⁰ = (Q,∆,Γ, I_N, R_N, c_N)be the XTOP^Fsuch thatcN(ρ, x) =xfor everyρ∈RN andx∈X.

IfN⁰ is compatible withM, then we are done.

Otherwise, letl₁∈Lhs(M⁻¹)be a left-hand side, q(l2) → r2 ∈ RN be a rule, andw ∈ pos_∆(l1) be a position such that θ(y) ∈/ X for some θ∈mgu(l₁|_w, l₂)andy∈Y. Letv∈pos_y(l₁|_w) be the unique position ofyinl1|_w.

Now we have to distinguish two cases: (i) Ei- ther var(l₂|_v) = ∅ and there is no leaf in r₂ la- beled by a symbol fromΓ. In this case, we have to introduce deletion and look-ahead intoN⁰. We replace the old ruleρ = q(l₂) → r₂ by the new ruleρ⁰ =q(l2[z]v) →r2, wherez ∈X\var(l2) is a variable that does not appear inl2. In addition, we letc_N(ρ⁰, z) = l₂|_v andc_N(ρ⁰, x) =c_N(ρ, x) for allx∈X\ {z}.

(ii) Otherwise, letV ⊆var(l2|_v)be a maximal set such that there exists a minimal (with respect to the prefix order) positionw⁰ ∈ pos(r₂) with

Another rule ofN q

σ z₁ σ

z2 z3

→ δ q₁ z1

q₂ z2

q₃ z3

Figure 10: Additional rule used in Example 5.

var(r2|_w⁰)⊆var(l2|_v)andvar(r2[β]w⁰)∩V =∅, whereβ ∈Γis arbitrary. Letz∈X\var(l2)be a fresh variable, q⁰ be a new state of N, and V⁰ = var(l2|_v) \ V. We replace the rule ρ=q(l2)→r2ofRN by

ρ₁ =q(l₂[z]_v)→trans(r₂)[q⁰(z)]_w⁰ ρ2 = q⁰(l2|_v)→r2|_w⁰ .

The look-ahead for z is trivial and other- wise we simply copy the old look-ahead, so cN(ρ1, z) = zandcN(ρ1, x) = cN(ρ, x)for all x ∈ X\ {z}. Moreover, c_N(ρ₂, x) = c_N(ρ, x) for allx ∈X. The mapping ‘trans’ is given for t=γ(t1, . . . , tk)andq⁰⁰(z⁰⁰)∈Q(Z)by

trans(t) =γ(trans(t1), . . . ,trans(t_k)) trans(q⁰⁰(z⁰⁰)) =

(hl₂|_v, q⁰⁰, v⁰i(z) ifz⁰⁰∈V⁰ q⁰⁰(z⁰⁰) otherwise, wherev⁰= pos_z⁰⁰(l₂|_v).

Finally, we collect all newly generated states of the form hl, q, vi in Ql and for every such state with l = δ(l₁, . . . , l_k) and v = iw, let l⁰ =δ(z₁, . . . , z_k)and

hl, q, vi(l⁰)→

(q(z_i) ifw=ε hl_i, q, wi(z_i) otherwise be a new rule ofN without look-ahead.

Overall, we run the procedure untilN⁰is com- patible with M. The procedure eventually ter- minates since the left-hand sides of the newly added rules are always smaller than the replaced rules. Moreover, each step preserves the seman- tics ofN⁰, which completes the proof.

We note that the look-ahead ofN⁰after the con- struction used in the proof of Theorem 4 is either trivial (i.e., a variable) or a ground tree (i.e., a tree without variables). Let us illustrate the construc- tion used in the proof of Theorem 4.

µ1:

q C z₁ z

→

C qNP

z1

q⁰ z

µ₂:

q⁰ VP z₂ z₃ z₄

→

VP q_VA

z₂ q_VP

z₄ q_NP

z₃ Figure 11: Rules replacing the rule in Figure 7.

Example 5. Let us consider the rules illustrated in Figure 9. We might first note that y1 has to be unified withβ. Sinceβ does not contain any variables and the right-hand side of the rule ofN does not contain any non-variable leaves, we are in case (i) in the proof of Theorem 4. Conse- quently, the displayed rule ofN is replaced by a variant, in whichβis replaced by a new variablez with look-aheadβ.

Secondly, with this new rule there is an mgu, in whichy₂ is mapped toσ(z₁, z₂). Clearly, we are now in case (ii). Furthermore, we can select the set V ={z₁, z₂}and position w⁰ = . Cor- respondingly, the following two new rules forN replace the old rule:

q(σ(z, z⁰))→q⁰(z⁰)

q⁰(σ(z1, z2))→σ(q1(z1), q2(z2)) , where the look-ahead forzremainsβ.

Figure 10 displays another rule ofN. There is an mgu, in whichy2is mapped toσ(z2, z3). Thus, we end up in case (ii) again and we can select the set V = {z₂} and position w⁰ = 2. Thus, we replace the rule of Figure 10 by the new rules

q(σ(z₁, z))→δ(q₁(z₁), q⁰(z), q₃(z)) (?) q⁰(σ(z₂, z₃))→q₂(z₂)

q3(σ(z1, z2))→q3(z2) , whereq₃ =hσ(z₂, z₃), q₃,2i.

Let us use the construction in the proof of The- orem 4 to resolve the incompatibility (see Exam- ple 3) between the XTOPs presented in the Intro- duction. Fortunately, the incompatibility can be resolved easily by cutting the rule ofN (see Fig- ure 7) into the rules of Figure 11. In this example, linearity and nondeletion are preserved.

4.2 Local determinism

After the first pre-processing step, we have the original linear and nondeleting XTOP M and an XTOP^F N⁰ = (Q⁰,∆,Γ, I_N, R⁰_N, c_N) that is equivalent to N and compatible with M. How- ever, in the first pre-processing step we might have introduced some non-linear (copying) rules inN⁰(see rule (?) in Example 5), and it is known that “nondeterminism [in M] followed by copy- ing [inN⁰]” is a feature that prevents composition to work (Engelfriet, 1975; Baker, 1979). How- ever, our copying is very local and the copies are only used to project to different subtrees.

Nevertheless, during those projection steps, we need to make sure that the processing inM pro- ceeds deterministically. We immediately note that all but one copy are processed by states of the form hl, q, vi ∈ Ql. These states basically pro- cess (part of) the treeland project (with stateq) to the subtree at positionv. It is guaranteed that each such subtree (indicated byv) is reached only once. Thus, the copying is “resolved” once the states of the form hl, q, vi are left. To keep the presentation simple, we just add expanded rules toMsuch that any rule that can produce a part of a treel immediately produces the whole tree. A similar strategy is used to handle the look-ahead ofN⁰. Any right-hand side of a rule ofM that produces part of a left-hand side of a rule of N⁰ with look-ahead is expanded to produce the re- quired look-ahead immediately.

LetL⊆T_∆(Z)be the set of treeslsuch that

• hl, q, viappears as a state ofQl, or

• l = l₂θfor some ρ₂ = q(l₂) → r₂ ∈ R⁰_N of N⁰ with non-trivial look-ahead (i.e., cN(ρ2, z) ∈/ X for some z ∈ X), where θ(x) =c_N(ρ₂, x)for everyx∈X.

To keep the presentation uniform, we assume that for every l ∈ L, there exists a state of the form hl, q, vi ∈ Q⁰. If this is not already the case, then we can simply add useless states with- out rules for them. In other words, we assume that the first case applies to eachl∈L.

Next, we add two sets of rules to RM, which will not change the semantics but prove to be use- ful in the composition construction. First, for every tree t ∈ L, let R_t contain all the rules p(l) → r, where p = p(l) → r is a new state withp∈P, minimal normalized treel∈T_Σ(X), and an instance r ∈ T_∆(P(X)) of t such that

q p σ y1 y2

δ i p_s y1

q ρ y2

q⁰ ρ y2

→

i p_s s⁰ y₁

→ s i p_s y₁

i p_s

→

q ρ_s

σ y₁ y₂

i ps

y₁

→

q⁰ ρ_s σ y₁ y₂

q p y₂

→

q ρ_s,s⁰/ρ⁰_s,s0

δ y₁ y₂ y₃

i ps

y₁

→

q⁰ ρ⁰_s,s0

δ y₁ y₂ y₃

σ i p_s⁰ y₂

i pα

y₃

→

q⁰ ρ_s,s0

δ y₁ y₂ y₃

δ i p_s⁰ y₂

q ρ y₃

q⁰ ρ y₃

→

Figure 12: Useful rules for the compositionM⁰;N⁰of Example 8, wheres, s⁰ ∈ {α, β}andρ∈P_σ(z2,z3).

p(l) ⇒^∗_M0 ξ ⇒_M⁰ r for some ξ that is not an instance oft. In other words, we construct each rule of Rt by applying existing rules of RM in sequence to generate a (minimal) right-hand side that is an instance oft. We thus potentially make the right-hand sides ofM bigger by joining sev- eral existing rules into a single rule. Note that this affects neither compatibility nor the seman- tics. In the second step, we add pure ε-rules that allow us to change the state to one that we constructed in the previous step. For every new state p¯ = p(l) → r, let base(¯p) = p. Then R⁰_M = R_M ∪R_L∪R_E andP⁰ = P ∪S

t∈LPt

where R_L= [

t∈L

R_t and P_t={`(ε)|`→r ∈R_t} RE ={base(¯p)(x1)→p(x¯ 1)|p¯∈ [

t∈L

Pt} . Clearly, this does not change the semantics be- cause each rule of R⁰_M can be simulated by a chain of rules ofRM. Let us now do a full ex- ample for the pre-processing step. We consider a nondeterministic variant of the classical example by Arnold and Dauchet (1982).

Example 6. Let M = (P,Σ,Σ,{p}, R_M) be the linear and nondeleting XTOP such that P = {p, p_α, p_β}, Σ = {δ, σ, α, β, }, and RM contains the following rules

p(σ(y₁, y₂))→σ(p_s(y₁), p(y₂)) (†)

p(δ(y₁, y₂, y₃))→σ(p_s(y₁), σ(p_s⁰(y₂), p(y₃))) p(δ(y1, y2, y3))→σ(ps(y1), σ(p_s⁰(y2), pα(y3)))

ps(s⁰(y1))→s(ps(y1)) ps()→

for every s, s⁰ ∈ {α, β}. Similarly, we let N = (Q,Σ,Σ,{q}, R_N) be the linear and non- deleting XTOP such thatQ={q, i}andR_N con- tains the following rules

q(σ(z₁, z₂))→σ(i(z₁), i(z₂))

q(σ(z1, σ(z2, z3)))→δ(i(z1), i(z2), q(z3)) (‡) i(s(z1))→s(i(z1))

i()→

for alls ∈ {α, β}. It can easily be verified that M andN meet our requirements. However,N is not yet compatible with M because an mgu be- tween rules (†) ofM and (‡) ofN might mapy2

toσ(z2, z3). Thus, we decompose (‡) into q(σ(z₁, z))→δ(i(z₁), q(z), q⁰(z)) q⁰(σ(z₂, z₃))→q(z₃)

q(σ(z1, z2))→i(z1)

whereq =hσ(z₂, z3), i,1i. This newly obtained XTOPN⁰is compatible withM. In addition, we only have one special treeσ(z2, z3)that occurs in states of the formhl, q, vi. Thus, we need to com- pute all minimal derivations whose output trees are instances of σ(z₂, z₃). This is again simple since the first three rule schemes ρ_s, ρ_s,s⁰, and ρ⁰_s,s0 of M create such instances, so we simply create copies of them:

ρs(σ(y1, y2))→σ(ps(y1), p(y2))

ρs,s⁰(δ(y1, y2, y3))→σ(ps(y1), σ(ps⁰(y2), p(y3))) ρ⁰_s,s0(δ(y₁, y₂, y₃))→σ(p_s(y₁), σ(p_s⁰(y₂), p_α(y₃)))

for all s, s⁰ ∈ {α, β}. These are all the rules ofR_σ(z2,z3). In addition, we create the following rules ofRE:

p(x1)→ρs(x1) p(x1)→ρ_s,s⁰(x1) p(x1)→ρ⁰_s,s⁰(x1) for alls, s⁰ ∈ {α, β}.

Especially after reading the example it might seem useless to create the rule copies inR_l[in Ex- ample 6 forl = σ(z₂, z₃)]. However, each such rule has a distinct state at the root of the left-hand side, which can be used to trigger only this rule.

In this way, the state selects the next rule to apply, which yields the desired local determinism.

hq, pi

RC PREL

that C x1 x2

→

C hq_NP, p_NPi

x1

hq⁰, pVPi x2

Figure 13: Composed rule created from the rule of Fig- ure 7 and the rules ofN⁰displayed in Figure 11.

5 Composition

Now we are ready for the actual composition. For space efficiency reasons we reuse the notations used in Section 4. Moreover, we identify trees of T_Γ(Q⁰(P⁰(X)))with trees ofT_Γ((Q⁰×P⁰)(X)).

In other words, when meeting a subtreeq(p(x)) withq ∈ Q⁰, p ∈ P⁰, andx ∈ X, then we also view this equivalently as the treehq, pi(x), which could be part of a rule of our composed XTOP.

However, not all combinations of states will be allowed in our composed XTOP, so some combi- nations will never yield valid rules.

Generally, we construct a rule ofM⁰;N⁰by ap- plying a single rule ofM⁰ followed by any num- ber of pureε-rules of RE, which can turn states base(p) into p. Then we apply any number of rules ofN⁰and try to obtain a sentential form that has the required shape of a rule ofM⁰;N⁰. Definition 7. LetM⁰ = (P⁰,Σ,∆, I_M, R⁰_M)and N⁰ = (Q⁰,∆,Γ, IN, R⁰_N) be the XTOPs con- structed in Section 4, whereS

l∈LP_l ⊆ P⁰ and S

l∈LQ_l⊆Q⁰. LetQ⁰⁰=Q⁰\S

l∈LQ_l. We con- struct the XTOPM⁰;N⁰ = (S,Σ,Γ, IN×IM, R) where

S= [

l∈L

(Ql×Pl)∪(Q⁰⁰×P⁰)

andRcontains all normalized rules`→r(of the required shape) such that

`⇒_M⁰ ξ ⇒^∗_R

E ζ ⇒^∗_N0 r for someξ, ζ ∈TΓ(Q⁰(T∆(P⁰(X)))).

The required rule shape is given by the defi- nition of an XTOP. Most importantly, we must have that ` ∈ S(T_Σ(X)), which we identify with a certain subset of Q⁰(P⁰(T_Σ(X))), and r ∈ TΓ(S(X)), which similarly corresponds to a subset ofT_Γ(Q⁰(P⁰(X))). The states are sim- ply combinations of the states ofM⁰ andN⁰, of

q p σ y1 σ

y2 y3

→

σ i p_s y1

i p_s y2

q p y3

Figure 14: Successfully expanded rule from Exam- ple 9.

which however the combinations of a stateq ∈Q_l with a statep /∈P_lare forbidden. This reflects the intuition of the previous section. If we entered a special state of the formhl, q, vi, then we should use a corresponding state p ∈ P_l ofM, which only has rules producing instances ofl. We note that look-ahead ofN⁰ is checked normally in the derivation process.

Example 8. Now let us illustrate the composition on Example 6. Let us start with rule (†) ofM.

q(p(σ(x₁, x₂)))

⇒_M⁰ q(σ(ps(x1), p(x2)))

⇒_RE q(σ(ps(x1), ρs⁰,s⁰⁰(x2)))

⇒_N⁰ δ(i(p_s(x₁)), q(ρ_s⁰_,s⁰⁰(x₂)), q⁰(ρ_s⁰_,s⁰⁰(x₂))) is a rule ofM⁰ ;N⁰ for everys, s⁰, s⁰⁰ ∈ {α, β}.

Note if we had not applied theR_E-step, then we would not have obtained a rule of M ;N (be- cause we would have obtained the state combina- tionhq, piinstead ofhq, ρ_s⁰_,s⁰⁰i, andhq, piis not a state ofM⁰ ;N⁰). Let us also construct a rule for the state combinationhq, ρ_s⁰_,s⁰⁰i.

q(ρs⁰,s⁰⁰(δ(x1, x2, x3)))

⇒_M⁰ q(σ(p_s⁰(x₁), σ(p_s⁰⁰(x₂), p(x₃))))

⇒_N⁰ q⁰(p_s⁰(x₁))

Finally, let us construct a rule for the state combi- nationhq⁰⁰, ρ_s⁰_,s⁰⁰i.

q⁰⁰(ρ_s⁰_,s⁰⁰(δ(x1, x2, x3)))

⇒_M⁰ q(σ(p_s⁰(x1), σ(p_s⁰⁰(x2), p(x3))))

⇒_RE q(σ(ps⁰(x1), σ(ps⁰⁰(x2), ρs(x3))))

⇒_N⁰ q(σ(p_s⁰⁰(x₂), ρ_s(x₃)))

⇒_N⁰ δ(q⁰(p_s⁰⁰(x₁)), q(ρ_s(x₂)), q⁰⁰(ρ_s(x₂))) for everys∈ {α, β}.

After having pre-processed the XTOPs in our introductory example, the devicesM andN⁰can be composed intoM ;N⁰. One rule of the com- posed XTOP is illustrated in Figure 13.

q p σ y1 δ y₂ y₃ y₄

→

σ i

p_s y1

i p_s⁰

y2

δ i p_s⁰⁰

y3

q ρ⁰ y4

q⁰ ρ⁰ y4

Figure 15: Expanded rule that remains copying (see Example 9).

6 Post-processing

Finally, we will compose rules again in an ef- fort to restore linearity (and nondeletion). Since the composition of two linear and nondeleting XTOPs cannot always be computed by a single XTOP (Arnold and Dauchet, 1982), this method can fail to return such an XTOP. The presented method is not a characterization, which means it might even fail to return a linear and nondelet- ing XTOP although an equivalent linear and non- deleting XTOP exists. However, in a significant number of examples, the recombination succeeds to rebuild a linear (and nondeleting) XTOP.

LetM⁰;N⁰ = (S,Σ,Γ, I, R)be the composed XTOP constructed in Section 5. We simply in- spect each non-linear rule (i.e., each rule with a non-linear right-hand side) and expand it by all rule options at the copied variables. Since the method is pretty standard and variants have al- ready been used in the pre-processing steps, we only illustrate it on the rules of Figure 12.

Example 9. The first (top row, left-most) rule of Figure 12 is non-linear in the variabley2. Thus, we expand the callshq, ρi(y₂)andhq⁰, ρi(y₂). If ρ = ρ_s for somes∈ {α, β}, then the next rules are uniquely determined and we obtain the rule displayed in Figure 14. Here the expansion was successful and we could delete the original rule for ρ = ρs and replace it by the displayed ex- panded rule. However, ifρ =ρ⁰_s0,s⁰⁰, then we can also expand the rule to obtain the rule displayed in Figure 15. It is still copying and we could repeat the process of expansion here, but we cannot get rid of all copying rules using this approach (as ex- pected since there is no linear XTOP computing the same tree transformation).

References

Andr´e Arnold and Max Dauchet. 1982. Morphismes et bimorphismes d’arbres. Theoretical Computer Science, 20(1):33–93.

Brenda S. Baker. 1979. Composition of top-down and bottom-up tree transductions. Information and Control, 41(2):186–213.

Jason Eisner. 2003. Learning non-isomorphic tree mappings for machine translation. In Proc. ACL, pages 205–208. Association for Computational Lin- guistics.

Joost Engelfriet, Eric Lilin, and Andreas Maletti.

2009. Composition and decomposition of extended multi bottom-up tree transducers. Acta Informatica, 46(8):561–590.

Joost Engelfriet. 1975. Bottom-up and top-down tree transformations—A comparison. Mathemati- cal Systems Theory, 9(3):198–231.

Joost Engelfriet. 1977. Top-down tree transducers with regular look-ahead. Mathematical Systems Theory, 10(1):289–303.

Jonathan Graehl, Kevin Knight, and Jonathan May.

2008. Training tree transducers. Computational Linguistics, 34(3):391–427.

Jonathan Graehl, Mark Hopkins, Kevin Knight, and Andreas Maletti. 2009. The power of extended top- down tree transducers. SIAM Journal on Comput- ing, 39(2):410–430.

Kevin Knight and Jonathan Graehl. 2005. An over- view of probabilistic tree transducers for natural language processing. In Proc. CICLing, volume 3406 of LNCS, pages 1–24. Springer.

Kevin Knight. 2007. Capturing practical natural language transformations. Machine Translation, 21(2):121–133.

Eric Lilin. 1978. Une g´en´eralisation des transduc- teurs d’´etats finis d’arbres: les S-transducteurs.

Th`ese 3`eme cycle, Universit´e de Lille.

Andreas Maletti and Heiko Vogler. 2010. Composi- tions of top-down tree transducers withε-rules. In Proc. FSMNLP, volume 6062 of LNAI, pages 69–

80. Springer.

Andreas Maletti. 2010. Why synchronous tree sub- stitution grammars? InProc. HLT-NAACL, pages 876–884. Association for Computational Linguis- tics.

Andreas Maletti. 2011. An alternative to synchronous tree substitution grammars. Natural Language En- gineering, 17(2):221–242.

Alberto Martelli and Ugo Montanari. 1982. An effi- cient unification algorithm. ACM Transactions on Programming Languages and Systems, 4(2):258–

282.

Jonathan May and Kevin Knight. 2006. Tiburon: A weighted tree automata toolkit. InProc. CIAA, vol- ume 4094 of LNCS, pages 102–113. Springer.

Jonathan May, Kevin Knight, and Heiko Vogler. 2010.

Efficient inference through cascades of weighted tree transducers. InProc. ACL, pages 1058–1066.

Association for Computational Linguistics.

Jonathan May. 2010. Weighted Tree Automata and Transducers for Syntactic Natural Language Pro- cessing. Ph.D. thesis, University of Southern Cali- fornia, Los Angeles.

John Alan Robinson. 1965. A machine-oriented logic based on the resolution principle. Journal of the ACM, 12(1):23–41.

William C. Rounds. 1970. Mappings and grammars on trees. Mathematical Systems Theory, 4(3):257–

287.

Stuart M. Shieber. 2004. Synchronous grammars as tree transducers. InProc. TAG+7, pages 88–95.

James W. Thatcher. 1970. Generalized² sequential machine maps. Journal of Computer and System Sciences, 4(4):339–367.

Kenji Yamada and Kevin Knight. 2001. A syntax- based statistical translation model. InProc. ACL, pages 523–530. Association for Computational Lin- guistics.