Yingjun Cao, Lingchu Yu, Alade Tokuta Paul P. Wang
Department of Mathematics & Computer Science Department of Electrical & Computer Engineering
North Carolina Central University Duke University
Durham, NC 27707 Durham, NC 27708
ycao@nccu.edu, lyu@mail.nccu.edu, atokuta@nccu.edu ppw@ee.duke.edu
Abstract-In this paper, we propose a general network model, fuzzy logic network (FLN), and study its stability and conver-gence properties. The converconver-gence property was first deduced theoretically. Then a random approach was adopted to simulate the convergence speed and steady-state properties for a variety of fuzzy logical functions. The simulation results show that MV logi-cal function causes the system to be on the edge of chaos when the number of nodes increases. Thus this logical function is more use-ful to infer real complex networks, such as gene regulatory net-works.
I. INTRODUCTION
One of the most challenging problems in bioinformatics is to determine how genes inter-regulate in a systematic manner which results in various translated protein products and pheno-types. To find the causal pathways that control the complex biological functions, researchers have been modeling gene regulatory mechanisms as a network topologically in order to gain more detailed insight [1]. It, in return, arouses the need of novel network models. The importance of the networking model is that normal regulatory pathways are composed of regulations resulting from many genes, RNAs, and transcrip-tion factors (TFs). The complicated inter-connectranscrip-tions among these controlling chemical complexes are the driving force in maintaining normal organism functions. The simplest yet com-monly used model for gene regulatory networks is the so called NK Boolean network [2]. It is a directed graph to model the situation where gene A and gene B interact during some time intervals and their interactions will determine or regulate the status of another gene C through a Boolean logical function at the next step. If numerous genetic regulations occur simultane-ously, the participating genes with their unique logical func-tions form the components of a gene regulatory network. This network will be self-evolutionary and eventually reach certain final states. In the NK network nomenclature, N is the total number of genes in the network, and K denotes the maximum number or the average number of regulating genes. The NK Boolean network theory has been carried out in a variety of ways both in deduced mathematical approximation and com-puter simulations [2-4]. Due to the binary limitation inherent in Boolean values, however, the exact properties of gene regula-tion cannot be expressed in detail based on this model. Thus other approaches were adapted to model the gene regulation mechanism, such as differential equations [5], Bayesian
net-works [6], and genetic circuits [7]. These models, however, have stressed different aspects of the regulatory behavior, and each model has contributed good inference results in certain aspect of the issue. The ongoing research on those models has focused on non-linear data processing, noise tolerance, and model over fitting [8].
In this paper, we propose and study a general network model, the fuzzy logic network (FLN) which is believed to possess the capacity of modeling complex networks and self-organizable systems, such as biological or economical systems. In a sense, the FLN is the generalization of Boolean network, but is capa-ble of overcoming the unrealistic constraint of Boolean value (ON/OFF symbolically). Fuzzy logic has evolved as a powerful tool over 40 years, and its applications are widely available in scientific research and engineering literature. The proposed FLN is able to inherit all the good properties of Boolean net-works, especially the causal property in the dynamic network behavior. Additionally, it is also expected to be a more effec-tive model with the nuance of membership function adjustment and inference rules. The FLN also has numerous known advan-tages such as modeling the highly non-linear relationships and periodicity. With distinctive properties in processing real-life incomplete data and uncertainties, the gene regulation analysis based on fuzzy logic theory did emerge after 2000 [9] and some good developments have been documented since then [10-16].
The general study of FLN’s convergence and stability pre-sented in this paper is organized as follows. In section II, the FLN's definitions and their appropriate meanings are given.
Two important theorems concerning the evolutionary property of the FLN are proved. In section III, the simulation algorithm is illustrated. In the following section, the simulation results are presented and discussed in detail. Conclusions and future re-search are discussed in section V.
II. FUZZY LOGIC NETWORK
A. Definitions 1) Fuzzy logic network
Given a set of Nfuzzy variables (genes), N ],i , [ ],x ,x , ,x [x
X N it
t t t
t= 1 2L ∈ 01 ∈
r , index t represents
time; the variables are updated by means of dynamic
equa-T. Sobh et al. (eds.), Innovative Algorithms and Techniques in Automation, Industrial Electronics and Telecommunications, 17–21.
© 2007 Springer.
17
tions,xti+1=fi(xti1,xti2,L,xtiK) where fi is a randomly chosen fuzzy logical function.
In the FLN, the fuzzy logical functions can be constructed using the combination of AND, OR,←and COMPLEMENT.
The total number of choices for fuzzy logical functions is de-cided only by the number of inputs. If a node has
) 1
( K N
K ≤ ≤ inputs, then there are 2Kψdifferent fuzzy logi-cal functions. In the definition of FLN, each node xti has K inputs on average.
2) Fuzzy logical functions
Fuzzy logical function is a binary operation that satisfies the identity, commutative, associative and increasing properties. A fuzzy logical function usually has to satisfy the so called t-norm/t-co-norm. Table I is a list of commonly used fuzzy logical functions with the AND, OR and COMPLEMENT [17].
TABLE I
COMMONLY USED FUZZY LOGICAL FUNCTIONS Fuzzy Logical 3) Quenched update
If all the fuzzy logical functions,fi(i∈N), and their related system remain the same throughout the whole dynamic proc-ess, then the system is termed as quenched updated.
4) Synchronous update
If all the fuzzy variables,xti, are updated at the same time, then the system is called synchronously updated; otherwise, it is asynchronously updated. In this paper, the FLN is assumed to be synchronously updated.
5) Basin of attraction
It is the set of points in the system state space, such that ini-tial conditions chosen in this set dynamically evolve toward a particular steady state.
6) Attractor
It is a set of states invariant under the dynamic progress, to-ward which the neighboring states in a given basin of attraction asymptotically approach in the course of dynamic evolutions.
It can also be defined as the smallest unit which cannot be de-composed into two or more attractors with distinct basins of attraction.
7) Limit cycle
It is an attracting set of state vectors to which orbits or tra-jectories converge, and upon which their tratra-jectories are peri-odic.
B. Theorems
Theorems in this section have focused on the dynamical convergence process of the FLN. The reason is not all FLNs
have limit cycles or attractors as strictly as in the case of Boo-lean. Excellent work has been done in Boolean Network on the characteristics of the cycles [18-19], but it has been shown that power law appears when the system has exponentially short cycles locally. The length of cycles and the number of cycles are heavily affected by the chaotic property. This property arouses the motivation to simulate the convergence of ran-domly FLNs.
Theorem 1: Quenched FLN using the Max-Min logical function must reach limit cycles or attractors
Proof:
If the initial conditions of the network areXr1 [x11,x12,L,x1N]
= , and the Max-Min logical function is used, it is obvious that the possible values of any variable,xti, at any timet can be only selected from
So the state space initially includes maximally 2 possible N values (some values out of 2 may be the same so N 2 is the N upper limit). Since the FLN is quenched, the initial configura-tions will remain the same throughout the whole dynamic process. So the state space remains the same, which are all the possible iterations of 2 values on a N N×1vector space. Thus the state space includes maximally (2N)Ndifferent vectors.
After (2N)N updates at most, the network must have reached a state where it has already visited. So the network must have limit cycles or attractors.
This property is only valid for the quenched network using the Max-Min logical function. If other types of logical func-tions (GC, MV or Probabilistic shown in Table I) are used, then the network cannot be guaranteed to reach exact limit cy-cles or attractors. Take GC logical function as an example. A simple two variable network,{x1t,xt2}, has the following network will evolve through the following states:
L
As can be seen, it will never reach a previously visited state because the value of the first variable at the current time is al-ways different from any of its ancestors. However, one trend can be seen is that although some FLNs will not reach the ex-act steady state, the network can be thought as reaching a pseudo-steady state asymptotically. In this example, the pseudo steady state is(0,0.5). However, the convergence properties of FLNs based on different logical functions are unknown. We have found that given a precision, all FLNs we simulated con-verged. Fig.1 shows examples of convergence based on the four logical functions shown in Table I.
CAO ET AL.
18
2 4 6 8 1012
Figure 1. The selected convergence phenomena of FLNs based on the four logical functions: Max-Min, GC, MV, and Probabilistic. The x-axis represents
the numerically-numbered nodes in the system. There are 13 nodes in all four sub-figures. The systems were simulated for 100 updates (y-axis). The z-axis represents the states of the system after each update. The initial values were
randomly selected.
As can be seen, the convergence speed and the steady-states of the four logical functions are different. The phenomena are further illustrated in section IV.
Theorem 2: For a quenched FLN using the Max-Min logical function, the values of all variables at the end of the process has a lower bound of min{x11,1−x11,x12,1−x12L,x1N,1−x1N}and an upper bound of max{x11,1−x11,x12,1−x12L,x1N,1−x1N} Proof:
Suppose at timet, the system reaches steady state. Then for∀xti, we can trace it back to the initial configurations due to the quenched property,
M composite of Ktmembership functions applied on the initial conditions. For any Max-Min logical function, it can be de-composed as the conjunction of disjunctions (the same as minterm presentations in Boolean logic). Since the Max-Min logical function preserves its initial values, so each disjunction preserves its input values. From the definition of composite functions, the composite of those disjunctions will also pre-serve input values. Thus we have proved that the initial values will be channeled to the steady state.
So the values of all variables at the end of the process have a lower bound of min{x11,1−x11,x12,1−x12L,x1N,1−x1N}and an upper bound of max{x11,1−x11,x12,1−x12L,x1N,1−x1N}.
III. SIMULATIONS
To study how the FLN evolves according to different number of nodes and different functions, the convergence property of the FLN was simulated. We have focused on two parameters
that govern the stability and convergence speed of the FLN: the length of limit cycles and the number of updates before reach-ing a limit cycle. The number of updates is a measurement of how the system converges and with what speed. The length of limit cycles shows the steady-state behavior of the system as well as its stability. If the number of limit cycles appears to follow the power law, then the system is believed to be on the edge of chaos [19]. The simulation algorithm is illustrated as follows.
Input: N (number of variables), MaxUpdates (Maximum num-ber of iterations allowed), δ(Precision of the Hamming dis-tance)
Output: Length (limit cycle length), NumUpdates (the number of updates before reaching the steady state)
Algorithm 1
Input: N (number of variables) Output: Fr [f1,f2,L,fN]
STABILITY OF FUZZY LOGIC NETWORKS 19
=1
In the simulations presented in section IV, uniform random number generator was used. The number of nodes in a FLN was limited to be no more than 13. The precision used to com-pute the Hamming distance was 0.0001. The maximum num-ber of iterations was 100.
IV. RESULTS AND DISCUSSIONS
The algorithm was implemented with the number of nodes in the FLNs ranging from 2 to 13. All four logical functions in Table I were tested. Firstly, the number of updates a FLN needs to reach limit cycles is shown in Fig. 2.
2 4 6 8 10 12
Number of Updates (Max−Min)
2 4 6 8 10 12
Number of Updates (GC)
2 4 6 8 10 12
Number of Updates (MV)
2 4 6 8 10 12
Number of Updates (Probabilistic)
Figure 2. The average number of updates before randomly generated FLNs reach limit cycles or attractors. The logical functions tested were Max-Min, GC, MV, and Probabilistic. The x-axis shows the number of nodes in the ran-domly generated FLN, and the y-axis shows the average number of updates before the FLN reaches limit cycles. The average number of updates was
com-puted as the mean of 10 simulations. The variations among the 10 simulations were also presented as error bars in the figures.
As can be seen, the number of updates required for GC and Probabilistic logic functions declines rapidly after the number of nodes reaches 6. However, Max-Min and MV logical func-tions’ convergence speed slows down if there are more nodes in the network. The trend of variations on the number of up-dates in Max-Min and MV logical functions also confirms that systems using these two logical functions are becoming more unstable for a large number of variables.
Another important measurement on FLN’s stability is the length of limit cycles. If the length of limit cycles has greater variations as the number of nodes increases, then the system’s
stability is weakening because the possible outcomes of system behaviors are more diverse. As expected, the Max-Min and MV logical functions have a greater variety of cycle lengths as the system possess more nodes while GC and Probabilistic do not (Fig. 3).
Length of Limit Cycles (Max−Min)
2 4 6 8 10 12
Length of Limit Cycles (GC)
2 4 6 8 10 12
Length of Limit Cycles (MV)
2 4 6 8 10 12
Length of Limit Cycles (Probabilistic)
Figure 3. The average length of limit cycles for randomly generated FLNs. The logical functions tested were Max-Min, GC, MV, and Probabilistic. The x-axis shows the number of nodes in the randomly generated FLN, and the y-axis shows the average length of limit cycles. The average number was computed as
the mean of 10 simulations. The variations of the 10 simulations were also presented as error bars in the figures.
As shown in Fig. 3, when the number of variables is greater than 6, GC and Probabilistic logical functions always reach the steady states in the form of attractors. The Max-Min and MV logical functions have limit cycles with a wide range of lengths.
It is believed that a fit network should be on the edge of chaos when it is applied to infer gene regulatory network. It has been found that inference results using the MV logical function did not introduce as many false positives as that from using other commonly used fuzzy logical functions. Further-more, MV logical function causes the algorithm to be less sen-sitive to small variations of δ. These properties help to reduce the effects of noise from the microarray data [14]. The simula-tion results in this paper confirm that MV logical funcsimula-tion in-deed can generate a general chaotic phenomenon.
V. CONCLUSIONS AND DISCUSSIONS
In this work, the focus was on the convergence and stability of a randomly generated FLN. The simulation results not only show the properties of different logical functions, but also con-firm the assumption that the MV logical function is fit for in-ferring gene regulatory networks.
Regarding future research on the theoretical aspects of the FLN, we think that the dynamics and the steady-state proper-ties of the FLN should be mathematically deduced. Further-more, the time invariant constraint on the selection of fuzzy logical functions should be extended to be time variant in order to infer more accurate and more realistic complex networks.
CAO ET AL.
20
REFERENCES
[1] S. Strogatz, “Exploring complex networks,” Nature, vol. 410, pp. 268–
276, 2001.
[2] S.A.Kauffman, Origins of order: Self-Organization and selection in evo-lution, New York, NY: Oxford University Press, 1993.
[3] C.H. Yuh, H. Bolouri and E.H. Davidson, “Genomic cis-regulatory logic:
experimental and computational analysis of a sea urchin gene,” Science, vol. 279, pp. 1896-1902, 1998.
[4] T. Akutsu, S. Miyano and S. Kuhara, “Inferring qualitative relations in genetic networks and metabolic pathways,” Bioinformatics, vol. 16, pp.
727-734, 2000.
[5] T. Chen, H.L. He, G.M. Church, “Modeling gene expression with differ-ential equations,” Pacific Symposium on Biocomputing, pp. 29-40, 1999.
[6] N. Friedman, M. Linial, I. Nachman and D. Pe’er, “Using Bayesian net-work to analyze expression data,” Journal of Computational Biology, vol.
7, pp. 601-620, 2000.
[7] D. Sprinzak and M.B. Elowitz, “Reconstruction of genetic circuits,”
Nature, vol. 438, no. 24, 2005.
[8] Z.B. Joseph, “Analyzing time series gene expression data,” Bioinformat-ics, vol. 20, pp. 2493-2503, 2004.
[9] P.J. Woolf and Y. Wang, “A fuzzy logic approach to analyzing gene expression data,” Physiological Genomics, vol.3, pp. 9- 15, 2000.
[10] B.A. Sokhansanj, J.P. Fitch, J.N. Quong and A.A. Quong, “Linear fuzzy gene network models obtained from microarray data by exhaustive search,” BMC Bioinformatics, vol. 5, no. 108, 2004.
[11] Y. Cao, P.P. Wang and A. Tokuta, Gene Regulating Network Discovery Studies in Computational Intelligence, Verlag: Springer, vol. 5, pp. 49-78, Jul 2005.
[12] Y. Cao, P.P. Wang and A. Tokuta, “A study of two gene network - the simplest special case of SORE (Self Organizable & Regulating Engine),”
Proc. of 7th JCIS joint conference, pp. 1716-1720, 2003.
[13] P.P. Wang, Y. Cao and A. Tokuta, “SORE - an example of a possible building block for a ‘Biologizing’ control system,” in Proc. 4th Interna-tional Symposium on Intelligent Manufacturing Systems, Sajarya, Turkey, May 2006, pp. 42–48.
[14] Y. Cao, P. Wang, and A. Tokuta, “S. pombe regulatory network construc-tion using the fuzzy logic network,” Poster, LSS Computaconstruc-tional Systems Bioinformatics Conference, Stanford University, August 2006.
[15] G. Resconi, Y. Cao, and P. Wang, “Fuzzy biology,” in Proc. 5th Interna-tional Symposium on Intelligent Manufacturing Systems, Sajarya, Turkey, May 2006, pp. 29–31.
[16] Y. Cao, P. Wang, and A. Tokuta, Gene regulatory network modeling: a data driven approach, ser. Fuzzy Logic - A Spectrum of Theoretical &
Practical Issues. Springer-Verlag GmbH, accepted, to appear in 2007.
[17] C.A. Reiter, “Fuzzy Automata and Life,” Complexity, vol. 7, no. 3, pp.
19-29, 2002.
[18] Z. Somogyvari and S. Payrits, “Length of state cycles in random Boolean networks: an analytical study,” Journal of Physics, vol. 33, pp. 6699-6706, 2000.
[19] R. Sole and B. Luque, “Phase transitions and anti-chaos in generalized Kauffman networks,” Physical letters A, vol. 196, pp. 331-334, 1996.
STABILITY OF FUZZY LOGIC NETWORKS 21