Recursive Construction of Periodoc Steady State for Neural Networks

(1)

HAL Id: hal-02101956

https://hal-lara.archives-ouvertes.fr/hal-02101956

Submitted on 17 Apr 2019

HAL is a multi-disciplinary open access

archive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Recursive Construction of Periodoc Steady State for Neural Networks

Martin Matamala

To cite this version:

Martin Matamala. Recursive Construction of Periodoc Steady State for Neural Networks. [Research Report] LIP RR-1993-23, Laboratoire de l’informatique du parallélisme. 1993, 2+20p. �hal-02101956�

(2)

LIP

Laboratoire de l’Informatique du ParallélismeEcole Normale Supérieure de Lyon Unité de recherche associée au CNRS n°1398

Martn Matamala

July 1993

Research Report No 93-23

Ecole Normale Supérieure de Lyon

46, Allée d’Italie, 69364 Lyon Cedex 07, France, Téléphone : + 33 72 72 80 00; Télécopieur : + 33 72 72 80 80;

Adresses électroniques :

(3)

State for Neural Networks

Martn Matamala

July 1993

Abstract

We present a strategy in order to build neural networks with long steady state periodic behavior. This strategy allows us to obtain 2nnon equivalent neural networks of size

n, when the equivalence

relation is the dynamical systems one. As a particular case, we build a neural network withnneurons admitting a cycle of period

2n.

Keywords:

neural networks, dynamical systems

Resume

Nous presentons une strategie pour construir des reseaux neu-ronaux qui ont cycles de grand taille. Cette strategie nous per-mets d'obtenir 2n reseaux neuronaux de taille

n, qui sont no

equivalents pour la relation de equivalence habituelle dans les sys-t emes dynamiques. Comme un cas particulier, nous construisons un reseaux neuronal de taille n, qui a un seul cycle de longueur

2n.

(4)

State for Neural Networks

Martn Matamala

y

Laboratoire de l'Informatique du Parallelisme Ecole Normale Superieure de Lyon 46, Allee d'Italie, 69364 Lyon Cedex 07, France.

July 21, 1993

Abstract

We present a strategy in order to build neural networks with long steady state periodic behavior. This strategy allows us to obtain 2n

non equivalent neural networks of sizen, when the equivalence relation

is the dynamical systems one. As a particular case, we build a neural network with nneurons admitting a cycle of period 2

n. 1 Introduction

A neural network of sizen, is a discrete dynamical system acting onf;1 1g n,

whose transition function,F

A, is given in term of an nxnreal matrixA= (a ij) as follows: F A( x) =sgn(Ax) (Ax) i = n X j=1 a ij x j i= 1 ::: n sgn:IR n !f;1 1g n sgn(y) i= sgn(y i) i= 1 ::: n (1)

Partially supported by FONDECYT 0047/90

yAdress: From December 93: Departamento de Ingeniera Matematica, U. de Chile,

Casilla 170-correo 3 Santiago, Chile

(5)

sgn(u) = 1

u0 ;1 u<0

We are interested in the reverberation neural networks, i.e. neural net-works where each state of the system, after a nite number of steps comes back to himself ( hypercube permutations). We study the question: how many reverberation neural networks have really di erent dynamical behavior ?. A related question is asked in 2] where an equivalence relation is dened whose equivalence classes are characterized and the number of elements in any class is proved to be 2n

n! wheren is the size of the neural network, but

nothing is said concerning the number of dierent classes.

Our approach is slightly dierent and it consists in considering the neural networks , in particular reverberation neural networks, as dynamical systems. In order to give a partial answer to our question, we dene also an equiva-lence relation and we prove, by building recursively 2n non equivalent neural

networks, that there exists at least 2n dierent classes when we consider this

relation in the set of the reverberation neural networks of sizen.

This work is divided in two parts: the recursive construction of neural networks and the enumeration of dierent dynamical behaviors.

In the former, we give a process which permits us to build recursively neural networks satisfying two properties: strictness and variability which are weak enough so that one can nd a large number of neural networks satisfying them. This process is supported by lemma 1 and 2 lemma 1 gives a way to build from a threshold function f : f;1 1g

n

! f;1 1g

another threshold function g :f;1 1g n+1

!f;1 1g such that over a vector

(x u) belongs to f;1 1g nx f;1 1gnfy 1 y 2 y 3 y 4 g, g(x u) = f(x) and g(y i)

for i = 1 2 3 4 is xed by the construction. So, one can describe easily the

dynamical evolution of g in term of those of f. Lemma 2, gives a way in

order to build threshold functions which have an a priori desired behavior. From these two lemmas we give in theorem 1 a recursive way for the construction of matrices , i.e., given a matrixAof sizensatisfying hypotheses

(a0) and (b0), dened in the next section, we build two matrices

B and C of

size n+ 1, satisfying also (a

0) and (b0) . This process will permit us to nd

a large number of neural network which are dened by the matrices given in theorem 1.

In the second part we dene an equivalence relation on P

n, the set of

bijective functions from f;1 1g n into

f;1 1g

n and we build a function

associating to each element in P

n a vector of size 2

n. We prove that this

(6)

function characterizes the equivalence relation, i.e., two functions F and G

are equivalent i (F) = (G). Hence, we prove that the extensions B and C given in theorem 1 dene non equivalent neural networks by proving that (F

B) and (F

C) are dierents, where F

B(resp. F

C) is the transition function

associated a B (resp. C). Latter, we prove that given two non equivalent

neural networksA and A

0 their extensions are also non equivalent. This fact

implies that increasing the size of the neural networks by one unit, one can double the non equivalent neural network number. That explains why we nd 2n non equivalent neural networks.

As a corollary, we build a neural network A of size n which has only one

cycle of period 2n.

2 Recursive construction of neural network

The following properties are important in our construction and represent the possibility of modication for a vector.

Denition 1

Consider a 2IR n. We say that (a) a is strictif 8x2f;1 1g n ax= n P j=1 a j x j 6 = 0 (b) a is variable if 9I 2f;1 1g n aI <0 such that 8x2f;1 1g n x6=I ax<0)ax<aI

Observe that for a vector a satisfying (a) and (b) we have the scheme given

in gure 1 which we adopt in order to give a more clear vision of the results.

a I.

a −a I._a

x / a x < 0. _x _/ _{a x > 0}.

Figure 1: Scheme of the values of ax where x 2f;1 1g n and

a is a strict

variable vector.

(7)

For a vectorasatisfying (a) and (b) one can have onlyxsuch thatax<0

or ax > 0 and then between aI a and

;aI

a, in gure 1, there no exist

any value ax.

Denition 2

Previous denitions apply to a realnxnmatrixAby imposing

that each row of Asatises them. More precisely, given a matrix A we say :

(a0)

A isstrict if each row of A, a i, for

i= 1 ::: n, is strict.

(b0)

A is variable if there exists a vector I

A such that a i satises (b) with I =I A, for every i= 1 ::: n.

When there exists a vectorI

A (resp. I a ) satisfying (b 0)(resp. b ) we say that A(resp. a ) is I A( resp. I a )-variable.

In the sequel we will work with vectors and matrices verifying properties (a) and (b). So, we dene

M n(

I

R) =fA:A is a strict variable nxn real matrixg I

R n =

fa2IR

n is a strict variable vector g

Denition 3

The transition functionF

a associated to vector a2IR nis given by: F a: f;1 1g n !f;1 1g x! F a( x) =sgn(ax) Observe that F

A given in equation 1 can be written as follows: F A( x) i = F a i(x) =sgn(a i x) i= 1 ::: n where a i is the ith row of A.

The following lemmas give the vector basic extensions. In this lemmas several technical details are concentrated and in the sequel only its conclu-sions will be used.

In lemma 1 we build from a vector a2IR n

another vector b2IR

n+1 such

that the function F

b is an extension of the function F a from IR n nfI a ;I a g to IR n+1 nf(I a

u)= u 2 f;1 1gg and such that F b over

f(I a

u)= u 2 f;1 1gg takes values depending only in the n+ 1th coordinate. In order to

(8)

get a better understanding of lemma 1 we show the meaning of the concepts used on, by giving an example.

Consider the vector a2IR

2 given by a t = (1 ; 1 2) (2)

Compute the values ax for x2f;1 1g

2. Since ax=;a(;x) we get: a(1 1) = 1; 1 2 = 1 2 a(;1 ;1) = ;1 2 a(;1 1) =;1; 1 2 = ; 3 2 a(1 ;1) = 32

Clearly a is strict. LetI t a= ( ;1 ;1). Then since ; 3 2 <; 1 2 <0< 1 2 < 3 2 a is I a-variable. Let D ; n( a),D + n( a) be given by D ; n( a) =fx2f;1 1g n =ax<0 x6=I a g (3) D + n( a) =fx2f;1 1g n =ax >0 x6=;I a g (4) In this case: n = 2, I t a = ( ;1 ;1) and a t = (1 ; 1 2). Then D ; 2 ( a) = f(;1 1)g D + 2( a) = f(1 ;1)g. Let h

a be the maximum value in D ; n( a) given by: h a= max fax: x2D ; n( a)g (5) Then h a = ; 3 2. Let > 0 be such that 2aI a = ;1 < ; < ; 1 2 = aI a and ha+aIa 2 = ;1<; <; 1 2 = aI a. Taking = 3 4 and v= (I a)2 = ;1 we dene b2IR 3 by b 1 = a 1 = 1 b 2 = a 2+ v 2 = ; 1 2 ; 3 8 = ; 7 8 b n+1 = ; 2 = ; 3 8 Then bis as follows: b t = (1 ; 7 8 ; 3 8) (6) and b(x u) for (x u)2f;1 1g 2x

f;1 1g is given by (see gure 2)

(9)

b(1 1 1) =;b(;1 ;1 ;1) =; 1 4 b(;1 1 1) =;b(1 ;1 ;1) = ; 9 4 b(1 ;1 1) =;b(;1 1 ;1) = 3 2 b(1 1 ;1) =;b(;1 ;1 1) = 1 2 For a vector I 2 IR n and an element v 2 IR we denote by (I v) t the extension of I fromIR n into I R n+1 whose n+ 1th coordinate is v.

So, bis strictand takingI t b = (

;I

a 1) = (1 1 1) one obtains that bI b = ; 1 4 and h b = ; 1 2 and then bis I b-variable. Moreover, F b and F a are related by: F b( ;1 1 1) =F b( ;1 1 ;1) =F a( ;1 1) =;1 F b(1 ;1 1) =F b(1 ;1 ;1) =F a(1 ;1) = 1 and F b( I a 1) = ;1 =F b( ;I a 1), F b( I a ;1) = 1 =F b( ;I a ;1). So, F b( x u) =F a( x) x6=I a x6=;I a (7) and F b( I a u) =;u u2f;1 1g (8) Taking a t = ( ; 1 2 1) (9) we deduce that I t a = ( ;1 ;1), h a = ; 3 2 and aI a = ; 1 2. For a we dene vector bby: b t= ( ; 1 2 1 + (;1)38 ; 3 8) = (; 1 2 58 ; 3 8) (10)

It is easy to see that bis strict, I t b = (

;I a

v) = (1 1 1)-variable and that F

b satises equations 7 and 8. The generalization of this result is given in

lemma 1. 0 − a I a . h a a I a . _{h a} 1 3 − 4 −−1 4 − 1 − 2 3 − 2 1 − 3 − 2 1 − 2 − − h b . bI_b

Figure 2: Scheme of the values of (1 ;1 2 )

x where x2f;1 1g 2.

(10)

Lemma 1.

Leta 2IR n

. Then, there exists b2IR n+1 satisfying: (a) I t b = ( ;I a 1), (b) 8(x u)2f;1 1g nx f;1 1g x 6=;I a x6=I a F b( x u) =F a( x) (c)8 u2f;1 1g F b( I a u) =;u.

Proof

Let D ; n( a),D + n(

a) be given as in equations 3 and 4.

Since a is a strictvector we have the following equivalence x2f;1 1g n i x2f;I a I a g _x2D ; n( a) _x2D + n( a) (11) Let h

a be the maximum value in D

; n(

a) given in equation 5. Since a is I

a-variable we have that h

a

<aI a

<0 and then ( see gure 3 ):

2aI a <aI a and h a+ aI a 2 <aI a (12)

so, there exists >0 such that

2aI a <;<aI a and h a+ aI a 2 <;< (13)

which is equivalent to:

;(aI a+ )<0 h a+ <;(aI a+ ) and aI a <;(aI a+ ) (14)

Observe that in gure 3 we suppose that ha+aIa 2

> 2aI a

> h

a which

is not the general case. From denition of h

a and equation 14 we have for x2D ; n( a) that: ax+h a+ <;(aI a+ )<0 (15) hence, <jaxj. Since x2D ; n( a) i ;x2D + n( a) we obtain 8x2D ; n( a)D + n( a) jaxj> (16) Dene b2IR n+1 by 7

(11)

F ( D ( a ) )_a n − 0 a I a . h a a I a . h a + 2 a I a . 2 a I a . ₊ ( ) − −

Figure 3: Scheme of the values of dierents parameters dened in lemma 1.

b i = a i i= 1 ::: n;1 b n = a n+ v 2 b n+1 = ; 2 where (I a)n=

v. It is clear that for (x u)2f;1 1g nx f;1 1gwe have b(x u) = ax+ (vx n ;u) 2 (17) and then jb(x u);axj (18)

We shall prove thatbisvariable. For that we want to nd (x u) satisfying b(x u)< 0. Let x 2 D ; n( a)D + n(

a) then from equations 16 and 18 we

have: ax>0 )b(x u)ax; >0 (19) and ax<0 )b(x u)ax+<0 (20) Let x=I a with =;1 1. Since (I a)n=

v, from equation 17 we have: b(I a 1) = aI a+ ( vv;1) 2 =aI a+ ( ;1) 2 (21) so, from equations 14 and 21 we obtain:

= 1)b(I a 1) = aI a <;(aI a+ )<0 (22) =;1)b(;I a 1) = ;aI a+ ( ;1;1) 2 = ;(aI a+ )<0 (23) 8

(12)

i.e.,b(I a 1)

<0 for=;1 1. Let I b = (

;I

a 1). Then from equation 23

we get bI b

< 0 and applying equations 19, 20, 22 and 23 we obtain the

following equivalence: b(x u)<0 and (x u)6=I b i ax<0^x6=I a _(x u) = (I a 1) (24) Let (x u)2f;1 1g n+1 such that b(x u)< 0^(x u)6= I b. From equation

24 there are only two possibilities for (x u). For the rst one i.e, ax < 0

and x6=I

a we know, from equations 15, 20 and 23 that b(x u)ax+<;(aI

a+

) =bI b

For the second one, from equation 22 we get that

b(I a 1) = aI a <bI b

which proves that bis I

b-variable. Observe that the inequalities in equations

19, 20, 22 and 23 are strict, so,

8(x u)2f;1 1g n+1

b(x u)<0_b(x u)>0

which says that bis strict.

Finally, from equations 19, 20, 22 and 23 we get:

8(x u)2f;1 1g n+1 x 2D ; n( a)D + n( a) F b( x u) =sgn(b(x u)) = sgn(ax) =F a( x) and 8u 2f;1 1g F b( I a u) =sgn(b(I a u)) =sgn(;u) =;u

In next lemma we build two vectors c and d in IR n+1

. Vector

d is such

that the function F

d is the projection over the

n+ 1th coordinate. Vectorc

denes the function F

c being the projection of the

n+ 1th coordinate from I

Rnf(I a

u)=u2f;1 1gg intof;1 1g and it considers only the sign of the nth coordinate of (I

a

u) for u 2f;1 1g.

In order to show how proceeds the proof of lemma 2 we give an example: 9

(13)

Let I a= ( ;1 ;1). We dene c= (;1 ;1 2; 1 2) = (;1 ;1 32) (25) and d= (;1 ;1 2; ;1 2 ) = (;1 ;1 52) (26) Then c(x u) for (x u)2f;1 1g 2x f;1 1g is given by: c(;1 ;1 ;1) =;c(1 1 1) = 12 c(;1 ;1 1) =;c(1 1 ;1) = 72 c(;1 1 1) =;c(1 ;1;1) = 32 c(;1 1 ;1) =;c(1 ;1 1) =; 3 2 so, cis strict,I c = ( ;I

a 1) = (1 1 1)-variable and satises F c( x u) =ux6= I a x6=;I a , F c( ;I a ;1) =;1 and F c( ;I a 1) = 1 i.e., F c( I a u) =.

By the other hand d(x u) is given by: d(;1 ;1 ;1) =;d(1 1 1) =; 1 2 d(;1 ;1 1) =;d(1 1 ;1) = 92 d(;1 1 1) = ;d(1 ;1;1) = 52 d(;1 1 ;1) =;d(1 ;1 1) =; 5 2 so,d is strict, (I a ;1)-variable and F d( x u) =u

Lemma 2.

ForI a 2IR n ( I a)n=

v there existcand d2IR n+1 such that: (a) I c = ( ;I a 1) I d = ( I a ;1) (b.1) 8(x u)2f;1 1g nx f;1 1g x6=I a x6=;I a F c( x u) =u (b.2) 8 u2f;1 1g F c( I a u) =. (c)8(x u)2f;1 1g nx f;1 1g F d( x u) = u

Proof

The construction of cand dis very similar. So, we give this

construc-tion in only a vector e(r) which will be appropriately evaluated in order to

obtain cand d. Let e(r) t= I a ( n; r 2) jr j= 1 belong to IR n+1. 10

(14)

For (x u)2f;1 1g nx f;1 1g we have e(r)(x u) = xI a+ u(n; r 2) (27)

It is easy to see thatx6=I a,

=;1 1 is equivalent to;n+2x I a

n ;2,

which applied to equation 27 implies:

u= 1 e(r)(x 1) ;n+ 2 +n; r 2 = 2; r 2 >1 (28) u=;1 e(r)(x ;1);n+ r 2 +n;2 =;2 + r 2 <;1 (29) i.e., sgn(e(r)(x u)) =u when x6=I a =;1 1 Let x=I a, = 1 ;1. Then e(r)(I a u) = I a I a+ u(n; r 2) = (+u)n; ur 2 So,je(r)(I a u)j=j(+u)n; ur 2 j j+ujn;j ur 2 j = j+ujn; 1 2 >

0. Hence, from equations 28 and 29 e(r) is strict.

We prove that e(r) is I

e(r )-variable with I t e(r ) = r(;I a 1). Compute the valuee(r)I e(r ): e(r)I e(r )= (;r+r)n+; r r 2 =; 1 2 moreover, e(r)(;I a ;1) = (;2n+ r 2) =;2n+ r 2 <;1 Since r(;I a 1) 6 = (;I a ;1) we obtain: (x u) 6= I e(r ) and e(r)(x u) < 0 i x 6= I a x 6= ;I a and u = ;1 or (x u) = (;I a

;1) and then for e(r)(x u)<0 we get

(x u)6=I e(r ) )e(r)(x u)<;1<e(r)I e(r )= ; 1 2 So, e(r) is I e(r )-variable.

Taking c=e(1) and d=e(;1) it is easy to see that (a), (b) and (c) are

satised.

(15)

The extension for a matrix A is given in the following theorem. As an

example of the construction consider the real matrixA given by: A = 1 ; 1 2 ; 1 2 1 = a a

wherea and a are given in equations 2 and 9.

Then from the analysis for a and a, A is strict and I A = ( ;1 ;1)-variable. Consider B given by B = 0 B @ b b c 1 C A = 0 B @ 1 ; 7 8 ;3 8 ; 1 2 5 8 ;3 8 ;1 ;1 3 2 1 C A where B, b and c are

constructed by equations 6, 10 and 25. Then B is strict and (1 1

1)-variable. Moreover, for x6=I A x6=;I A F B( x u) = (F b( x u) F b( x u) F c( x u)) t = (F a( x) F a( x) u) t= ( F A( x) u) t and F B( I A u) = (F b( I A u) F b( I A u) F c( I A u)) t = (;u ;u ) t = (;ue 2 ) t where e t 2 = (1 1). Now, let C = 0 B @ a 0 a 0 d 1 C A = 0 B @ 1 ; 1 2 0 ; 1 2 1 0 ;1 ;1 5 2 1 C A where d is

given by equation 26. Then C isstrict and (I A ;1)-variable. Moreover, F C( x u) = (F a( x) F a( x) F d( x u)) t = (F A( x) u) t

The last construction is generalized for any matrix in M n( IR) in the following theorem.

Theorem 1.

ForA 2M n(

IR), there exist B and C in M n+1( I R) such that (a) I B = ( ;I A 1) I C = ( I A ;1) (b) 8(x u)2f;1 1g nx f;1 1g F C( x u) = (F A( x) u) t (c)8(x u)2f;1 1g nx f;1 1g x6=;I A x6=I A F B( x u) = (F A( x) u) t 8 u2f;1 1g F B( I A u) = (;ue n ) t (30) 12

(16)

where e t n= (1 ::: 1)2 IR n.

Proof

Construction of matrixB : Since A 2 M n(

IR) we know that each row, a i

i = 1 ::: n belongs to I

R n

. Applying Lemma 1 to each row we nd vectors b i 2 IR n+1 which are (;I A 1)-variables, satisfying: 8(x u) 2 f;1 1g nx f;1 1g x 6= ;I A x 6= I A F b i( x u) = F a i( x) and F b i(I A u) =;u

By applying Lemma 2, for I =I

A, we obtain c2IR n+1, ( ;I A 1)-variable, such that 8(x u) 2 f;1 1g nx f;1 1g x 6= ;I A x 6= I A F c( x u) = u and F c( I A u) = DeneB t= ( b 1 b 2 ::: b n c). Since eachb i, for i = 1 ::: nbelongs toIR n+1 with I b i = (;I

A 1) one knows that

B 2 M n+1(

IR) and from lemma 1 and

conclusion (b) of lemma 2, B veries properties (a) and (c) of the theorem .

Construction of matrixC :

Let d be the vector given by lemma 2 which is (I a ;1)-variable. Let C be dened by: C ij = a ij 1 i j n C jn+1 = 0 1 j n C n+1j = d j 1 j n+ 1 Since d 2 IR n+1 with I d = ( I A ;1), C 2 M n+1. Moreover, C x u = Ax du and then F C is given by F C( x u) = (F A( x) F d( u)) t = ( F A( x) u).

So, C satises (a) and (b) in theorem.

3 Non equivalent neural networks

Consider the set P

n of the bijective functions on

f;1 1g

n. The following

property is shown in 2] for F 2P n. 8x2f;1 1g n 9s2IN F s( x) =x (31)

We dene the cycle of xby F,O F( x), forF 2P n by O F( x) =<x F(x) ::: F T F x ;1( x)> 13

(17)

where T F

x is the rst integer such that F T F x( x) = x. T F

x is called the period

of the cycleO F(

x). We say that y2O F(

x) i there exists s2 IN such that F s( x) = y. Taking A= 1 ; 1 2 ; 1 2 1 we have O F A(1 1) = <(1 1)> O F A( ;1 ;1) =;<(;1 ;1)> and O F A(1 ;1) =<(1 ;1) (1 ;1) > Then T F A (11)= 1 = T F A (;1;1) T F A (;11)= T F A (1;1) = 2

In order to show the power of the construction given in section 2 it is necessary to specify when two neural networks have dierent dynamics. For that we dene the following equivalence relation:

Given F and G in P

n we say that

F is equivalent to G i there exists a

function on P

n such that 8x2f;1 1g

n

F((x)) = (G(x)) (32)

This denition does not permit easily to prove that our construction builds non equivalent neural network. For that, given a functionF 2P

n, we dene

the characteristic ofF by a vector(F) inIN 2

n

, such that itsith component

gives the cycle numbers of period i of F and we prove the following lemma.

Lemma 4.

Two functions F and G inP

n are equivalent i

(F) =(G).

Proof

()) We prove the following equivalence for satisfying equation 32:

C F = <x F(x) ::: F L;1( x)> is a cycle forF i (33) C G = <(x) (F(x)) ::: (F L;1( x))>is a cycle for G.

Indeed, since F and G are equivalent we have that

(F i(

x)) =G i((

x)) for i= 0 ::: L;1

and then equation 32 is true. Hence for each cycle of size Lof F we have a

cycle of size L for G and conversely for each cycle of size L of G we have a

cycle of size L forG with which(F) =(G).

(18)

(() Since F and G belong to P

n, a vector

x2 f;1 1g

n can belong to only

one cycle. LetC i j i j j = 1 ::: n

i be the dierent cycles of size

iforF and G

respec-tively. We dene the function associating C i j to i j as follows: Let C i j = < x F(x) ::: F i;1( x) > and i j = < y G(y) ::: G i;1( y) > then we dene by: (G k( y)) =F k( x) 1k i;1 (y) =x

Making this process for any j and any i we dene completely satisfying

F =G.

Denition

We say that a real matrixA is a reverberation neural network if F A belong to P n.

Proposition 1.

Let A 2 M n(

IR) a reverberation neural network. Then B

and C given in theorem 1 are reverberation neural networks and the periods

of their cycles are determined in terms of the periods of the cycles of F A as follows: 8(x u)2f;1 1g nx f;1 1g T F C (xu)= T F A x (34) 8(x u)2f;1 1g nx f;1 1g I A ;I A = 2O F A( x) T F B (xu) = T F A x (35) if I A 2O F A(

x) for some =;1 1 then T F B (xu) = T F B (I A u) (36) whereT F B (I A u) = 8 > > > > > > < > > > > > > : 2T F A I A if e n 2O F A( I A) 2T F A I A if u=; and e n = 2O F A( I A) T F A I A if u= and e n = 2O F A( I A) (37)

Proof

Before giving the proof we analyze our example: From the denition of B and C in it is easy to see that:

O F B(1 ;1 ;1) =O F B( ;1 1 ;1) =<(1 ;1 ;1) (;1 1 ;1)> 15

(19)

O F B( ;1 1 1) = O F B(1 ;1 1) =<(;1 1 1) (1 ;1 1) > O F B( ;1 ;1 1) =;O F B(1 1 ;1) =<(;1 ;1 1) > O F B(1 1 1) = O F B( ;1 ;1 ;1) =<(1 1 1) (;1 ;1 ;1) > and O F C(1 ;1 ;1) =O F C( ;1 1 ;1) =<(1 ;1 ;1) (;1 1 ;1)> O F C( ;1 1 1) = O F C(1 ;1 1) =<(;1 1 1) (1 ;1 1)> O F C( ;1 ;1 1) =;O F C(1 1 ;1) =<(;1 ;1 1) > O F C(1 1 1) = ;O F C( ;1 ;1 ;1) =<(1 1 1)> and then 8(x u)2f;1 1g 2x f;1 1g T F C (xu) = T F A x 8x2f;1 1g 2 x6=I A x6=;I A T F B (xu)= T F A x T F B (I A 1) = T F B (;I A ;1)= 1 = T F A I A and T F B (;I A 1) = T F B (I A ;1)= 2 = 2 T F A I A

Note that we are in the case e 2= O F A( I A), ( I A 1) and ( ;I A ;1) satisfy the condition u = and (;I a 1) and ( ;I A

;1) satisfy the condition u = ;.

This proves the proposition in the our example.

Now we give the general proof. Firstly we prove that B and C are

re-verberation neural networks. Suppose that F C( x u) = F C( x 0 u 0). Since F C( x u) = (F A( x) u) we have that u = u 0 and F A( x) = F A( x 0). But A is

a reverberation neural network, so (x u) = (x 0

u 0) and

C is a reverberation

neural network. Now, suppose that F B( x u) = F B( x 0 u 0). Then if x 6=I A

we proceed as above. When x=I

A we have F B( x u) = (;ue n ). Since A is a reverberation neural network, F

A( y) =;ue n only for y =I A. Then x 0= 0 I A and from ( ;ue n ) = (;u 0 e n 0) we conclude that ( x u) = (x 0 u 0)

and B is a reverberation neural network.

Properties 34 and 35 follow from the fact that

8k 2IN F k C( x u) = (F k A( x) u) andF k B( x u) = (F k A( x) u) when F k A( x)6= ;I A I A. 16

(20)

When I A 2O F A( x) we have that O F B( x u) =<(x u) ::: (z t) (I A u) (;ue ) ::: (y w)> and sinceF B 2P n+1, O F B( I A u) is given by: O F B( I A u) =<(I A u) (;ue ) ::: (y w)(x u) ::: (z t)>

Observe the structure of O F B( I A u). Suppose that e2 O F A( I A) then since F A 2P

n the following sequence of transition is true: I A !;e!!;I A !e!!I A | {z } T F A I A

and then from Theorem 1 (I A )!(;e )!!(;I A )!(;e ;)!!(;I A ;) (;I A ;)!(e ;)!!(I A ;)!(e )!!(I A ) i.e.,T F B (I A ) = T F B (I A ;) = T F B (;I A ) = T F B (;I A ;) = 2 T F A I A If e2= O F A( I A) then (I A )!(;e )!!(I A ) i.e.,T F B (I A ) = T F A (I A ), moreover, (I A ;)!(e )!!(;I A ) (;I A )!(;e ;)!!(I A ;) i.e.,T F B (I A ;) = 2 T F A (I A

) and we have the conclusions.

Observe that in our example we have (F

A) = (2 2 0 0), (F

B) =

(2 3 0 0 0 0 0 0) and (F

C) = (4 2 0 0 0 0 0 0) which motives the

fol-lowing corollary which is a conclusion of lemma 4 and proposition 1.

Corollary 1

For matricesA,B and C in proposition 1 we have

(a) (F C)i = 2 (F A)i 1 i2 n (F C)i = 0 2 n <i2 n+1 17

(21)

(b)(F B)i = 2 (F A)i 1 i2 n (F B)i = 02 n <i2 n+1 i6=T A I A i6= 2T A I A If e2O F A( I A) then (c)(F B) T F A I A = 2((F A) T F A I A ;1) (F B) 2T F A I A = 1 + 8 < : 2(F A) 2T F A I A 2T F A I A 2 n 0 2T F A I A >2 n if e2=O F A( I A) then (d) (F B) T F A I A = 2((F A) T F A I A ;2) + 2 (F B) 2T F A I A = 1 + 8 < : 2(F A) 2T F A I A 2T F A I A 2 n 0 2T F A I A >2 n Since (F B) 2T F A I A

is odd, the neural network B and C are not equivalent.

Proof

Observe that since 2((F

A) T F A I A ;1) = 2((F A) T F A I A ;2) + 2 we could

join (c) and (d). For sake of clearness we prefer this form. From Proposition 1 one knows that from each cycleO

F A( x) we can obtain two cyclesO F C( x ;1) andO F C(

x 1) with the same period and that the cycle

numbers of a given size ofF

Ais doubled in F

C. This same argument is true for F

Bwhen the cycle O

F

A does not contain neither I Anor ;I A. When I Aor ;I A belongs to O F A( x) we know that if e 2 O F A( I A) then O F A( I A) = O F A( ;I A)

and the cycle, O F A( I A) which is of size T F A I

A , is transformed in the cycle O F B( I A ) of size 2T F A I A . This is described by (c). If e 2= O F A( I A) then O F A( I A) 6 =O F A( ;I

A) and both are transformed in the cycle O F B( I A ;1) of size 2T F A I A , cycle O F B( I A 1) of size T F A I A and cycle O F B( ;I A ;1) of sizeT F A F A.

The last observations is trivial from the denition of .

Proposition 2.

Let fA

i g

L

i=1 be a family of non equivalent reverberation

neural networks in M n( I R). Then fB i C i g L

i=1 is a family of non equivalent

reverberation neural networks in M n+1(

IR), where B i and

C

i are built from A

i in theorem 1.

Proof

Suppose that there exist two equivalent neural networks in fB i C i g L i=1.

Then, it is sucient to analyze the following cases: 18

(22)

(a) (F B

i) = (F B

j). Then we have that 81 k 2 n , k 6= T F A I A and k 6= 2T F A I A (F B i) k = (F B j) k )(F A i) k = (F A j) k

and from (c) and (d) in Corollary 1 one obtains (F A i) =(F A j) (b)(F C i) = (F C

j). Applying the same arguments as in (a) we conclude that (F

A

i) =(F A

j) so, (a) and (b) are in contradiction with the non equivalence

of A i and A j. (c)(F B i) =(F C j). Then (F B i) 2T A i A i = 1 + 8 < : even 2T A i I A i 2 n 0 2T A i I A i >2 n and (F C j) 2T A i A i = ( even 2T A i A i 2 n 0 2T A i A i >2 n

but, this is a contradiction too.

Theorem 2

For any n 2 IN there exist 2

n non equivalent reverberation

neural networks in M n(

I R)

Proof

We proceed by induction on n

For n= 2 the matricesA i : i= 1 2 3 4 given by A 1 = 1 ; 1 2 ; 1 2 1 A 2= ; 1 2 1 1 ; 1 2 A 3 = ;1 1 2 1 2 ;1 A 4 = 1 2 1 ;1 ;1 2 are in M

2, and have the following characteristics (A 1) = (4 0 0 0) (A 2) = (2 1 0 0) (A 3) = (0 2 0 0) (A 4) = (0 0 0 1)

and then are not equivalent. Accepting that there exists 2n non equivalent

neural networks for matrices of sizen we can apply proposition 2 in order to

obtains 2n+1 non equivalent neural networks for size

n+ 1.

By using corollary 1 we get the following result which is given in 1]:

Corollary 2

8n2IN there existsA 2M n(

I

R) whose characteristic is given

by:

(23)

(F A k) i = 0 for i6= 2 n and (F A k) 2 n = 1

Proof

Taking n = 2 we have that A

4 given by theorem 2 belong to M

2

and its characteristic is (0 0 0 1). Accepting that there exist A 2 M n(

IR)

with (A) = (0 ::: 1) then by corollary 1 we obtain B 2 M n+1( I R) with (B) = (0 ::: 1) because e n 2O F A( I A). 4 Conclusion

The results shown in this work permit us to obtain a wide variety of non equivalent dynamics when we consider the family of reverberation neural networks inM

n(

I

R). This kind of constructions can be applied for information

storage where the information is codied in the cycles of the neural network. It is desirable to extend our construction to any function in M

n(

IR). In

this case theorem 1 is true and we can build recursively neural networks in

M n(

I

R). Moreover, we can obtain an analogous result to proposition 1 which

permits us to know the behavior of neural networks of size n + 1 in term

of those of the neural networks of size n. But, the characterization given in

lemma 4 for the equivalence of two functions in M n(

I

R) is not longer true.

For that, it is interesting to nd an invariant in the general case.

References

1] S. A. Gustafson C.H., Haring D.R. and W.-S. T.G. Systhesis of counter with threshold elements. In Record of 6th Annual Symposium on Switch-ing Circuit Theory and Logical DesSwitch-ing, pages 25{35. IEEE conferance, 1965.

2] M. Sato and C. Tanaka. Characterization and constructions of reverber-ating networks. Mathematical Biosciences, 62:201{217, 1982.