Presentation "Capabilities and Prospects of Inductive Modeling" (Prof., Dr. Sci. Volodymyr Stepashko) »»

Group Method of Data Handling
(GMDH)

A brief description of GMDH

The Group Method of Data Handling (GMDH) is developed by Academician of NANU A.G.Ivakhnenko and his school beginning from [1]. This is a typical method of the inductive modeling and appears to be one of effective methods of structure-parametric identification of complex objects, processes and systems from observation data under the conditions of incomplete information. This year it is fulfilled 95 years from the birthday of the GMDH originator as well as 40 years from the date of the method appearance.

In general the problem of structure-parametric identification consists in generating (forming) from observation data some set F of models of a diverse structures of the type

and searching optimum model according to the condition of minimization of a model quality criterion

with parameters in (2) being estimated for every model as solving another extreme problem of the type

where is called the complexity of a model f and is equal to the number of unknown parameters in the model f type (1), and Q is a quality criterion for solving parametric identification problem for every particular model being generated in the structural identification problem.

GMDH holds the certain variety of possibilities on all stages of the modeling process in comparison with other methods of model construction. It applies first of all to the generators of models and to the used criteria of structure quality and also to classes of models (basic functions). The method differs from others by active application of the following principles: (i) automatic generation of model variants; (ii) successive selection of best models; and (iii) using external criteria for construction of the model with optimum complexity. It possesses an original multilayered procedure of automatic generation of model structures imitating the process of biological selection with a pairwise consideration of successive features. Such a procedure in up-to-date terminology is called a Polynomial Neural Network (PNN) and here the final model structure appears to be explicit and is built automatically.

For comparison and choice of the best models, the external criteria based on the division of the given data sample in two or more parts are used provided that the parameter estimation and control of a model quality is carried out on different subsamples. It allows working without burdensome a priori assumptions because the sample division allows implicitly (automatically) to take into account diverse types of a priori uncertainties at the construction of the model. GMDH holds advantage in the case of small data samples due to the optimal choice of the model complexity with automatic adaptation to an unknown level of the data uncertainties.

Efficiency of the method was repeatedly confirmed by solving the great number of real world problems from fields of ecology, economy, hydrometeorology, etc.[2-4]. The theoretical aspects of GMDH are considered in [5, 6]. In particular, in [5]on the basis of analogy between the problems of both construction of a model from noisy experimental data and signal passage through a noisy channel, principles of the theory of noise-immunity modeling were built. The basic result of this theory is that complexity of the optimum forecasting model depends on the level of uncertainty in the data: the higher it is – the simpler (more robust) there must be the optimum model (the less number of parameters being estimated).

GMDH is well known and very actively developed both in Ukraine and abroad [7-9]. In particular, substantial contribution to the method development was made in [5, 10-12]where foundations of the theory of structural identification of models with minimum variance of forecasting error was built. The method of critical variances is the effective instrument of this theory allowing analytically to solve actual problems: comparative analysis of criteria of structural identification, planning of experiments, analysis of properties of methods and others, at that both for the limited sample and in asymptotic. By the method, the conditions of choice of the optimum model structure depending on noise variance (level), sample length, input influences (plan of experiment), and plant parameters are investigated, close interrelation between them is thus specified. By means of this theory it is established that GMDH is the method for construction of models with minimum variance of forecasting error and the comparison is executed with other methods [12].

As it follows from this, GMDH as a basic tool of the inductive modeling theory belongs to the most modern methods of Computational Intelligence and Soft Computing. This method is the original and effective tool for solving wide spectrum of problems of artificial intelligence, including identification and forecasting, pattern recognition and clusterization, data mining and discovery of relationships.

In the last decade the interest to GMDH actively increases in the whole world, that one can explain, in addition to the known efficiency of the method, also by growth of popularity of the artificial neural networks technology. The fact is that the GMDH structure can be interpreted as a neuronet, originality of which consists in selforganization both of its structure and parameters. It appears thus that to the obvious advantages of GMDH one may attribute an automatic forming of the network structure, simplicity and speed of the parameters learning, and also the possibility to «reduce» the adjusted network directly to an explicit mathematical expression.

International forums on the inductive modelling confirm the GMDH popularity. So, in the period from 23 to 26 of September 2007, the II International workshop on inductive modelling IWIM’2007 was held in Prague [13-15]. The first workshop took place in Kyiv in July 2005 as continuation of the International conference on inductive modelling ICIM'2002 in Lviv in May 2002. Series of such conferences and workshops are the international actions concentrated on the theory, algorithms, applications, realization and new developments of technologies of data analysis and knowledge extraction based on the GMDH methodology.

The inductive modelling approach built on principles of selforganization actively develops during 40 years, is used in many areas and is present in such widespread technologies of data analysis as polynomial neuron nets, adaptive and statistical learning nets. In new developments for construction of models from data, evolution and genetic algorithms, also ideas of active neurons and multilevel selforganization and others are used.