Dali Wang and Ali Zilouchian
5.5 2-D FILTER DESIGN USING NEURAL NETWORK 5.5.1 Two-dimensional Signal and Digital Filters
N. Lease Square Subspace
ε2 1.39e-02 3.26e-2 1.45e-2
ε∞ 1.40e-2 2.46e-2 1.40e-2
Table 5.6: Error Norms of Example 5.2 under White Noise + Spike Noise Conditions
N. N. Lease Square Subspace
ε2 2.86e-2 1.01e-01 4.18e-2
ε∞ 2.69e-2 6.88e-2 4.67e-2
Example 5.3:
This example is provided in order to emphasize the effectiveness of the proposed model for multi-input and multi-output systems. The filter to be identified is a two inputs, two outputs system governed the following state space form as provided by Taylor [26]:
A= The corresponding transfer function matrix can be derived as:
T z system identified by NN is:
ANN = 0.8840 0.4722 1.0046 1.0047 1.1123 1.2534
CNN = The corresponding transfer function matrix is:
T z
The two error values measured for the first 50 samples of impulse response are calculated in vector form:
5.6.2 Two-dimensional Filters
Two numerical examples are provided for 2-D recursive filter, each emphasis different aspects of the proposed algorithm. The following L2 and L∞ norm [29]-[32] error criteria are defined for error analysis:
ε
MNR = Maximum Negative Ripple
where H(i,j) and HNN(i,j) are the impulse responses of the original system and identified system, respectively, Λ’={(i, j) | 0 ≤ i ≤ M’, 0 ≤ j ≤ N’} is the given region where the error norms are calculated.
Example 5.4: First Quarter Gaussian Filter
The prototype model used by Aly and Fahmy in [30] for designing a 2-D causal recursive filter is presented here. It is a first quadrant Gaussian 2-D scalar filter described by the following impulse response:
H i j( , )=0 256322. exp{ .−0103203⋅ −[(i 4)2+(j−4) ]}2
with most of its energy in the first-quadrant. The selected region for identification consists of Λ={(i, j) | 0 ≤ i ≤ 10, 0 ≤ j ≤ 10}. The same region was used for error norm calculation: Λ’=Λ.
The proposed NN consists of one input neuron, one output neuron, and two groups of hidden neurons, each with three neurons. After 80 epochs of training, the identified 2-D filter in Roesser’s LSS model of order (3,3) is given as,
ANN = 3.0059e0 -1.8840e0 2.2325e0 8.8268e -1 -3.1194e -1 -6.4903e -1 2.0812e0 - 9.6880e -1 2.0168e0 1.4857e0 - 5.2382e -1 -1.0898e0 - 9.1445e -1 6.9213e -1 -1.3040e -1 3.5259e -1 -1.2248e -1 - 2.5830e -1 -1.7607e -1 1.2241e -1 -1.5569e -1 4.8882e -1 - 2.7540e -1 -1.6637e -1 4.4500e -1 - 2.8219e -1 3.2107e -1 8.4165e -1 1.2692e0 4.7849e -1 - 4.5596e -1 3.1093e -1 - 3.8792e -1 - 8.6954e -1 - 8.7703e -1 1.4460e -1
[ ]
BNN = - 6.6533e - 3 -1.1209e - 2 - 2.6681e - 3 -1.0466e -1 6.0182e - 2 -1.4947e -1T
[ ]
CNN = 3.8730e + 1 - 2.9244e + 1 1.8910e + 1 -1.2433e0 4.3619e -1 9.1152e -1
Table 5.7 is presented to compare the error measurements of our design to that of Aly and Fahmy [30]. Notice that the total order realization of our design (3 + 3 = 6) is the same theirs (4 + 2 = 6).
Table 5.7: Simulation Experiments for Example 5.4
ε2 % ε∞ % MNR
Our Design 3.71 5.16 Always positive
Design [Aly and Fahmy] 10.78 9.19 0.04479
Example 5.5: A (2, 2) 2-D Digital Filter
This example is presented to illustrate the identification of a 2-D system using various responses. The random input response as well as the impulse response are utilized to identify the given 2-D filter. The 2-D filter to be identified is governed by the following state space model (D=0):
generated input/desired output pairs for 40 epochs. The identified filter is:
In the second phase, the same NN was trained with the impulse response defined in the region Λ={(i, j) | 0 ≤ i ≤ 9, 0 ≤ j ≤ 9} for 40 epochs. The state-space form of the identified filter is as follow:
ANN =
Design with a random response 0.166 0.115 -5.8e-03 Design with an impulse response 2.82 3.14 always > 0
5.7 CONCLUSIONS
In this chapter, a novel NN technique is introduced for the design of recursive digital filters in the state space form. Instead of using spatial representation of time by delayed input/output feedback, we use hidden neurons to encode the temporal properties of the system. Through the self feedback of hidden neurons as well as the interconnection between the neurons in the input, hidden, and output layers, the proposed NN structure mimics the dynamics of a linear discrete system or digital filter. The proposed method also provides flexibility in selection of various state-space forms such as controllability and observability canonical forms as an identification model.
The NN approach is also extended for the design of general 2-D recursive digital filters where an analytical solution is not necessarily available. An attractive feature of the proposed algorithm is that the LSS model structure to be identified could be predefined in the design stage. This feature not only provides us with flexibility in selection of the structure of a 2-D filter, but also facilitates analyses on several implementation issues of 2-D filter, such as computation efforts and memory requirement. Furthermore, the proposed method herein places no limitation on the type of response to be approximated. Namely, any
ANN =
1.0641e -1 7.7004e - 2 -1.2062e - 2 5.3092e - 2 9.1873e - 2 8.5081e - 2 2.9856e - 2 7.1189e - 2 - 3.8449e - 3 6.9885e - 2 8.1948e - 2 8.9764e - 2 - 6.8676e - 4 7.1595e - 2 8.3222e - 2 9.0390e - 2
[ ]
BNN = 1.2555e0 3.1189e -1 1.1109e0 4.4088e -1 T
[ ]
CNN = 1.0784e0 4.6248e -1 8.4716e -1 6.9503e -1
The region for error norm calculation is Λ’={(i, j) | 0 ≤ i ≤ 19, 0 ≤ j ≤ 19} for both of designed filters. In Table 5.8, a comparison of the error analysis of two different training results is shown. It is observed that a random input response provides a more accurate model in comparison to an impulse response. A similar conclusion is obtained based on other simulation results, due to the fact that the responses generated by a large amount of random inputs contains more information compared to the impulse responses.
Table 5.8: Results of Example 5.5
ε2 % ε∞ % MNR
(original -5.5e-3)
for filter identification.
The effectiveness as well as robustness of this method have been demonstrated by simulations experiments for both single input/single output and multi-input/multi-output digital filters.
REFERENCES
1. Hopfield, J.J., Neural Networks and Physical Systems with Emergent Collective Computational Abilities, Proc. Nat. Acad. Sci., Vol. 79, 2554 − 2558, April 1982.
2. Hopfield, J.J., Neurons with Graded Response have Collective Computational Properties Like Those of Two State Neurons, Proc.
Nat. Acad. Sci., Vol. 81, 3088 − 3092, May 1984.
3. Elman, J.L., Finding Structure in Time, Cognitive Science, Vol. 14, 179 − 211, 1990.
4. Pineda, F.J., Dynamics and Architecture for Neural Computation, J.
Complexity, Vol. 4, 216 − 245, 1988.
5. Pineda, F.J., Recurrent Back Propagation and the Dynamical Approach to Adaptive Neural Computation, Neural Computation, Vol. 1, 161 − 172, 1989.
6. Robinson, A.J. and Fallside, F. A Recurrent Error Propagation Network Speech Recognition System, Computer Speech and Language 5, 259 − 274, 1991.
7. Irwin, K., Warwick G.W. and Hunt, K.J., Neural Networks for Control and Systems, IEE Publication, 1992.
8. Wang, D. Identification and Approximation of 1-D and 2-D Digital Filters, Ph.D Dissertation, Florida Atlantic University, Boca Raton, FL, May, 1998.
9. Wang, D. and Zilouchian, A., Identification of Discrete Linear Systems in State Space Form Using Neural Network, Proc. of Second IEEE Int.
Caracas Conf. on Devices, Circuits and Syst., Venezuela, 338 − 342, March, 1998.
10. Rumelhart, D.E. and McClelland, J.L.(eds.), Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 1, MIT Press, Boston, MA, 1986.
11. Galvan, J.B. and Perez-Ilzarbe, M.J., Two Neural Networks for Solving the Linear System Identification Problem, Proc. of IEEE Conf. on Neural Networks, 3226 − 3231, 1993.
12. Cichocki, A. and Unbehauen, R., Neural Networks for Solving Systems of Linear Equations and Related Problems, IEEE Trans. on Circuits and Syst., Vol. 39, No.2, 124 − 137, Feb., 1992.
13. Mammone, R.J. and Zeevi, Y., Neural Networks, Theory and Application, Academic Press, NY, 1990.
14. Lippman, M.P. and Chua, L.O., Neural Networks for Nonlinear Programming, IEEE Trans. on Circuits and Syst., Vol. 35, No.5, 554 − 562, May 1988.
15. Narendra, K.S. and Parthasarathy, K., Identification and Control of Dynamic Systems Using Neural Networks, IEEE Trans. on Neural Networks, Vol. 1, No. 1, 4 − 27, March, 1990.
16. Poggio, T. and Girosi,F., Network for Approximation and Learning, Proc. of IEEE, 1481 − 1495, Sept., 1990.
17. Jamshidi, J. (ed.), Circuits, Systems & Information, TSI Press, Albuquerque, NM, 1991.
18. Horton, M.P., Real-time Identification of Missile Aerodynamics Using a Linearised Kalman Filter Aided by an Artificial Neural Network, IEE Proc. Control Theory Appl., Vol. 144, No. 4, 299 − 308, July, 1997.
19. Hampel, F.R., Ronchetti, E.M., Roussew, P., and Stahel, W.A., Robust Statistics - the Approach Based on Influence Functions, John Wiley &
Sons, NY, 1987.
20. Wang, D. and Zilouchian, A., Identification of 2-D Recursive Digital Filters in State-Space Form Using Neural Network, Int. J. of Intelligent Automation and Soft Computing .
21. Silverman, L.M., Realization of Linear Dynamic Systems, IEEE Transaction on Automatic Control, AC-16, 554 − 567, 1971.
22. Wang, D. and Zilouchian, A., Model Reduction of Discrete Linear Systems via Frequency Domain Balanced Structure, IEEE Trans. on Circuits and Syst. Vol. 47, No. 6 , 830−838, July 2000.
23. Moonen, M., Moor, B. D., Vandenberghe, L., and Vandewalle, J., On-and Off-line Identification of Linear State-Space Models, Int. J.
Control, Vol. 49, No. 1, 219 − 232, 1989.
24. Ljung, L., System Identification, Theory for the User, Prentice-Hall, Inc., Englewood Cliffs, NJ, 1987.
25. Wang, D. and Zilouchian, A., Model Reduction of 2-D Separable-in-Denominator Systems via Frequency Domain Balanced Realization, Proc. of 37th IEEE Conf. on Decision and Control, Tampa, FL, 2179 − 2184, 1998.
26. Taylor, F.J., Digital Filter Design Handbook, Marcel Dekker Inc., NY, 1983.
27. MATLAB Toolbox, The Mathwork Inc., Boston, MA, 1998.
28. Ramos, J., A Subspace Algorithm for Identifying 2-D Separable in Denominator Filters, IEEE Trans. on Circuits and Syst., Vol. 41, No. 1, 63 − 67, January, 1994.
29. Hinamoto, T. and Maekawa, S., Spatial-Domain Design of a Class of Two-Dimensional Recursive Digital Filter, IEEE Trans. on ASSP, Vol.
32, No. 1, 153 − 162, February, 1984.
Dimensional Recursive Digital Filters, IEEE Trans. on Circuits and Syst., Vol. 27, No. 10, 892 − 901, October, 1980.
31. Lashgari, B., Siverman, L.M., and Abramatic, J., Approximation of 2-D Separable in 2-Denominator Filters, IEEE Trans. on Circuits and Syst., Vol. 30, No. 2, 107 − 121, February 1983.
32. Hinamoto, T. and Maekawa, S., Design of 2-D Separable in Denominator Filters Using Canonic Local State-Space Models, IEEE Trans. on Circuits and Syst., Vol. CAS-33, No. 9, 922 − 929, September, 1986.
33. Lin,L., Kawamata, M., and Higuchi,T., Design of 2-D Separable-Denominator Digital Filter Based on the Reduced-Dimensional Decomposition, IEEE Trans. on Circuits and Systems, Vol. CAS-34, No. 8, 934 − 941, August, 1987.
34. Raymond, D.M., and Fahmy, M.M., Spatial-Domain Design of Two-Dimensional Recursive Digital Filters, IEEE Trans. on Circuits and Syst., Vol. 36, No. 6, 901 − 905, June, 1989.
35. Bose, T. and Chen, M., Design of Two-Dimensional Digital Filters in the Spatial Domain, IEEE Trans. on Signal Processing, Vol. 41 No. 3, 1464 − 1469, March, 1993.
36. Attasi, S., Modeling and Recursive Estimation for Double Indexed Sequences, in System Identification: Advances and Case Studies, Mehra, R.K., and Lainiotis, D.G., (eds.), Academic Press, NY, 1976.
37. Roesser, R.P., A Discrete State-Space Model for Linear Image Processing, IEEE Trans. on Automatic Control, Vol. AC-20, 1 − 10, February 1975.
38. Kung, S., Levy, B.C., Morf, M., and Kailath, T., New Results in 2-D Systems Theory, Part II: 2-D State-Space Models - Realization and the Notions of Controllability, Observability, and Minimality, Proc. of the IEEE, Vol. 65, No. 6, 945 − 959, June, 1977.
39. Shanks, J.L., Treitel, S., and Justice, J.H., Stability and Synthesis of Two-Dimensional Recursive Filters, IEEE Trans. on Audio Electro-Acoust., Vol. AU-20, 115 − 128, June, 1972.
40. Hinamoto, T., Realizations of a State-Space Model from Two-Dimensional Input-Output Map, IEEE Trans. on Circuits and Syst., Vol. CAS-27, No. 1, 36−44, Jan., 1980.