Alternatives to replace STATISTICA. Neural networks

STATISTICA Automated Neural Networks is the only neural network software product in the world that is completely translated into Russian!

Neural network methodologies are becoming increasingly widespread in a variety of fields from fundamental research to practical applications of data analysis, business, industry, etc.

is one of the most advanced and most effective neural network products on the market. It offers many unique benefits and rich features. For example, the unique capabilities of the automatic neural network search tool, , allow the system to be used not only by experts in neural networks, but also by beginners in the field of neural network computing.

What are the advantages of using ?

    Pre- and post-processing, including data selection, nominal coding, scaling, normalization, removal of missing data with interpretation for classification, regression and time series problems;

    Exceptional ease of use plus unrivaled analytical power; for example, a unique automatic neural network search tool Automated neural network (ANN) will guide the user through all the stages of creating various neural networks and select the best one (otherwise, this task is solved through a long process of “trial and error” and requires serious knowledge of theory);

    The most modern, optimized and powerful network training algorithms (including conjugate gradient methods, Levenberg-Marquardt algorithm, BFGS, Kohonen algorithm); full control over all parameters affecting network quality, such as activation and error functions, network complexity;

    Support for ensembles of neural networks and neural network architectures of almost unlimited size;

    Rich graphical and statistical capabilities that facilitate interactive exploratory analysis;

    Full integration with the system STATISTICA; all results, graphs, reports, etc. can be further modified using powerful graphical and analytical tools STATISTICA(for example, to analyze predicted residuals, create a detailed report, etc.);

    Seamless integration with powerful automated tools STATISTICA; recording full-fledged macros for any analysis; creating your own neural network analyzes and applications using STATISTICA Visual Basic challenge STATISTICA Automated Neural Networks from any application that supports COM technology (for example, automatic neural network analysis in a MS Excel spreadsheet or combining several custom applications written in C, C++, C#, Java, etc.).

  • A selection of the most popular network architectures, including Multilayer Perceptrons, Radial Basis Functions, and Self-Organizing Feature Maps.
  • Tool available Automatic Network Search, which allows you to automatically build various neural network architectures and regulate their complexity.
  • Preservation of the best neural networks.

    Support various types of statistical analysis and construction of predictive models, including regression, classification, time series with continuous and categorical dependent variable, cluster analysis for dimensionality reduction and visualization.

    Supports loading and analysis of multiple models.

  • Optional ability to generate source code in C, C++, C#, Java, PMML (Predictive Model Markup Language), which can be easily integrated into an external environment to create your own applications.

Code generator

Code generator STATISTICA Automated Neural Networks can generate source system program code for neural network models in C, Java and PMML (Predictive Model Markup Language). The code generator is an additional application to the system STATISTICA Automated Neural Networks, which allows users, based on the neural network analysis, to generate a C or Java file with the source code of models and integrate it into independent external applications.

    Code generator requires STATISTICA Automated Neural Networks.

    Generates a version of the neural network source code (as a file in C, C++, C# or Java).

    The C or Java code file can then be embedded in external programs.

STATISTICAAutomated Neural Networks in neural network computing

The use of neural networks involves much more than just processing data using neural network methods.

STATISTICA Automated Neural Networks (SANN) provides a variety of functionality to work with very complex tasks, including not only the latest Neural Network Architectures And Learning algorithms, but also new approaches to constructing neural network architectures with the ability to enumerate various activation and error functions, which makes it easier to interpret the results. In addition, software developers and users experimenting with application settings will appreciate the fact that after conducting specified experiments in a simple and intuitive interface STATISTICA Automated Neural Networks (SANN),neural network analyzes can be combined in a custom ,application. This can be achieved either using the COM function library STATISTICA, which fully reflects all the functionality of the program, or using code in C/C++, which is generated by the program and helps run a fully trained neural network.

Module STATISTICA Automated Neural Networks fully integrated with the system STATISTICA Thus, a huge selection of tools for editing (preparing) data for analysis (transformations, conditions for selecting observations, data checking tools, etc.) is available.

Like all tests STATISTICA, the program can be "attached" to a remote database using in-place processing tools, or linked to live data so that models are trained or run (for example, to calculate predicted values ​​or classification) automatically every time the data changes.

Data scaling and nominal value conversion

Before data is entered into the network, it must be prepared in a certain way. It is equally important that the output data can be interpreted correctly. IN STATISTICA Automated Neural Networks (SANN) it is possible to automatically scale input and output data; Variables with nominal values ​​can also be automatically recoded (for example, Gender=(Male,Female)), including using the 1-of-N coding method. STATISTICA Automated Neural Networks (SANN) also contains tools for working with missing data. There are data preparation and interpretation tools specifically designed for time series analysis. A wide variety of similar tools are also implemented in STATISTICA.

In classification problems, it is possible to set confidence intervals that STATISTICA Automated Neural Networks (SANN) is then used to assign observations to one class or another. In combination with a special implemented in STATISTICA Automated Neural Networks (SANN) the Softmax activation function and cross-entropy error functions provide a fundamental probability-theoretic approach to classification problems.

Selecting a neural network model, ensembles of neural networks

The variety of neural network models and the many parameters that need to be set (network sizes, learning algorithm parameters, etc.) can confuse some users. But this is why there is an automatic neural network search tool, , which can automatically search for a suitable network architecture of any complexity, see below. In system STATISTICA Automated Neural Networks (SANN) All main types of neural networks used in solving practical problems have been implemented, including:

    multilayer perceptrons (networks with direct signal transmission);

    networks based on radial basis functions;

    self-organizing Kohonen maps.

The above architectures are used in regression, classification, time series (with continuous or categorical dependent variable) and clustering problems.

In addition, in the system STATISTICA Automated Neural Networks (SANN) implemented Network ensembles, formed from random (but significant) combinations of the above networks. This approach is especially useful for noisy and low-dimensional data.

In the package STATISTICA Automated Neural Networks (SANN) Numerous tools are available to help the user select an appropriate network architecture. The statistical and graphical tools of the system include histograms, matrices and error graphs for the entire population and for individual observations, final data on correct/incorrect classification, and all important statistics, for example, the explained proportion of variance, are calculated automatically.

To visualize data in a package STATISTICA Automated Neural Networks (SANN) Scatterplots and 3D response surfaces are implemented to help the user understand the “behavior” of the network.

Of course, you can use any information obtained from the listed sources for further analysis by other means. STATISTICA, as well as for subsequent inclusion in reports or for customization.

STATISTICA Automated Neural Networks (SANN) automatically remembers the best network option from those that you received while experimenting with the task, and you can refer to it at any time. The usefulness of the network and its predictive ability are automatically tested on a special test set of observations, as well as by estimating the size of the network, its efficiency, and the cost of misclassification. Implemented in STATISTICA Automated Neural Networks (SANN) Automatic cross-validation and weight regularization procedures allow you to quickly determine whether your network is under- or over-complicated for a given task.

To improve performance in the package STATISTICA Automated Neural Networks Numerous network configuration options are presented. Thus, you can specify a linear output network layer in regression problems or a softmax activation function in probabilistic estimation and classification problems. The system also implements cross-entropy error functions based on information theory models and a number of special activation functions, including Identical, Exponential, Hyperbolic, Logistic (sigmoid) and Sine functions for both hidden and output neurons.

Automated neural network (automatic search and selection of various neural network architectures)

Part of the package STATISTICA Automated Neural Networks (SANN) is an automatic neural network search tool, Automated neural network (ANN) - Automated Network Search (ANS), which evaluates many neural networks of varying architecture and complexity and selects the networks of the best architecture for a given task.

When creating a neural network, considerable time is spent on selecting appropriate variables and optimizing the network architecture using heuristic search. STATISTICA Automated Neural Networks (SANN) takes over this work and automatically conducts a heuristic search for you. This procedure takes into account the input dimension, network type, network dimensions, activation functions, and even the required output error functions.

It is an extremely effective tool when using complex techniques, allowing you to automatically find the best network architecture. Instead of spending hours sitting in front of your computer, let the system STATISTICA Automated Neural Networks (SANN) do this work for you.

The success of your experiments to find the best network type and architecture depends significantly on the quality and speed of network learning algorithms. In system STATISTICA Automated Neural Networks (SANN) The best training algorithms to date have been implemented.

In system STATISTICA Automated Neural Networks (SANN) Two fast second-order algorithms have been implemented - conjugate gradient methods and the BFGS algorithm. The latter is an extremely powerful modern nonlinear optimization algorithm, and experts highly recommend using it. There is also a simplified version of the BFGS algorithm that requires less memory, which is used by the system when the computer's RAM capabilities are quite limited. These algorithms tend to converge faster and produce a more accurate solution than first-order accuracy algorithms such as Gradient Descent.

Iterative process of network training in the system STATISTICA Automated Neural Networks (SANN) is accompanied by an automatic display of the current training error and the error calculated independently on the testing set, and a graph of the total error is also shown. You can interrupt training at any time by simply pressing a button. In addition, it is possible to set stopping conditions, under which the training will be interrupted; such a condition may be, for example, the achievement of a certain error level, or a stable increase in the test error over a given number of passes - “epochs” (which indicates the so-called retraining of the network). If overfitting occurs, the user should not care: STATISTICA Automated Neural Networks (SANN) automatically remembers an instance of the best network obtained during the training process, and this network option can always be accessed by clicking the corresponding button. After network training is completed, you can check the quality of its work on a separate test set.

After the network is trained, you need to check the quality of its work and determine its characteristics. For this purpose in the package STATISTICA Automated Neural Networks (SANN) There is a set of on-screen statistics and graphical tools.

In the event that several models (networks and ensembles) are specified, then (if possible) STATISTICA Automated Neural Networks (SANN) will display comparative results (for example, plot response curves of several models on one graph, or present predictors of several models in one table). This property is very useful for comparing different models trained on the same data set.

All statistics are calculated separately for the training, validation and test sets or in any combination of them, at the discretion of the user.

The following summary statistics are automatically calculated: the root mean square error of the network, the so-called confusion matrix for classification problems (where all cases of correct and incorrect classification are summed up) and correlations for regression problems. The Kohonen network has a Topological Map window in which you can visually observe the activations of network elements.

Ready-made solutions (custom applications using STATISTICA Automated Neural Networks)

Simple and convenient system interface STATISTICA Automated Neural Networks (SANN) allows you to quickly create neural network applications to solve your problems.

There may be a situation where it is necessary to integrate these solutions into an existing system, for example, to make them part of a wider computing environment (these may be procedures developed separately and built into the corporate computing system).

Trained neural networks can be applied to new datasets (for prediction) in several ways: You can save the trained networks and then apply them to a new dataset (for prediction, classification, or forecasting); You can use a code generator to automatically generate program code in C (C++, C#) or Visual Basic and then use it to predict new data in any visual basic or C++ (C#) programming environment, i.e. embed a fully trained neural network into your application. In conclusion, all system functionality STATISTICA, including STATISTICA Automated Neural Networks (SANN), can be used as COM objects (Component Object Model) in other applications (for example, Java, MS Excel, C#, VB.NET, etc.). For example, you can implement an automated analysis created using STATISTICA Automated Neural Networks (SANN) into MS Excel tables.

List of learning algorithms

    Gradient Descent

    Conjugate gradients

    Kohonen training

    K-Means Method for Radial Basis Function Network

Network size restrictions

A neural network can be of almost any size (that is, its dimensions can be taken many times larger than is actually necessary and reasonable); for a network of multilayer perceptrons, one hidden layer of neurons is allowed. In fact, for any practical tasks the program is limited only by the hardware capabilities of the computer.

e-Manual

As part of the system STATISTICA Automated Neural Networks (SANN) there is a well-illustrated textbook that provides a complete and clear introduction to neural networks, as well as examples. A system of detailed, context-sensitive help is available from any dialog box.

Source code generator

The source code generator is an additional product that allows users to easily create their own applications based on the system STATISTICA Automated Neural Networks (SANN). This additional product creates the source system code of the neural network model (as a file in C, C++, C# or Java), which can be separately compiled and integrated into your program for free distribution. This product is designed specifically for enterprise systems developers, as well as those users who need to transform highly optimized procedures created in STATISTICA Automated Neural Networks (SANN), into external applications to solve complex analytical problems. (It should be noted that in order to obtain permission, users must inform sitessia employees about the distribution of programs using the generated code).

In STATISTICA, the continuous forecasting problem is represented as a regression problem. In the context of this problem, a neural network is considered as a nonlinear function, the complexity of which is controlled “semi-parametrically” - the number of elements in the network affects the complexity of the solution, but, of course, the analyst cannot see the explicit form of the regression function.

It is required to build a neural network that calculates the emission of lead into the atmosphere depending on the number and type of passing transport. The data is stored in the file Lead.xls.

Open the Svinets.xls file in the Statistica package. The Open File window appears.

Rice. 4. 33. Import window.

You must select the “Import selected sheet” option and select the name of the data sheet:

Rice. 4. 34. Selecting an Excel sheet for import into the Statistica package.

In the next window, you need to specify the real data parameters, which, as a rule, are determined and displayed automatically (except for the last three checkboxes).

Rice. 4. 35. Setting the import area.

After this, the imported data will be displayed in the window.

Rice. 4. 36. Import results.

Run the analysis package using neural networks. To do this, select “Neural Networks” from the “Analysis” menu.

Rice. 4. 37. Selecting a data processing method - “neural network”.

after which the STATISTICA Neural Networks package window will appear:

Rice. 4. 38. Start window for the “neural networks” analysis.

Go to the “Quick” tab, where you need to set the task type - Regression, and the tool - Network Designer.

Rice. 4. 39. Launching the neural network designer.

Next, by pressing the “OK” button, you will switch to the mode for selecting output (dependent) and input (independent) variables. We select “Lead” as the first one, and the number of cars of all categories as the last ones. The "No" and "Street" columns remain unused.

Rice. 4. 40. Selecting input and output data for a neural network.

By clicking “Ok” you will return to the “Quick” tab. Then, by clicking the “Ok” button again, you will be taken to the neural network formation window. On the “Fast” tab, you need to select the network type - multilayer perceptron,

Rice. 4. 41. Selecting the type of neural network.

and on the “Elements” tab you can specify the required number of layers, the number of neurons in each, as well as the type of activation function:

Rice. 4. 42. Setting the number of layers and types of neurons.

Rice. 4. 43. Choosing a method for training a neural network.

Here, by clicking on the “Samples” button, you can set the number of training, control and test examples. If you set the number of test and control examples to zero, then the network will be trained using all examples:

Rice. 4. 44. Determine data for training and testing.

Returning to the main training window, you can click on the “User” button and go to the “Interactive” tab, request that the training process be reflected in the form of a graph:

Rice. 4. 45. Specifying the type of graph to demonstrate the learning process.

Finally, by clicking on the “Ok” button, you will start the learning process, the result of which will be displayed on the graph:

Rice. 4. 46. Training a neural network.

By clicking on the “Ok” button, you will be taken to the results window, where you can study the various characteristics of the created network by moving through the window tabs:

Rice. 4. 47. Results of neural network modeling.

So, for example, on the “Advanced” tab there is a “Network architecture” button, by clicking on which you can see the topology of the constructed network:

Rice. 4. 48. View of the constructed neural network.

as well as the “User Observations” button, where you can give the network new initial data and receive a response from an already trained network.

What are the similarities and differences between the languages ​​of neurocomputing and statistics in data analysis? Let's look at a simple example.

Let's assume that we have observations and experimentally measured N pairs of points representing a functional relationship. If you try to draw the best straight line through these points, which in the language of statistics will mean using to describe the unknown dependence linear model

(where denotes noise during observation), then the solution to the corresponding problem linear regression will be reduced to finding the estimated values ​​of the parameters that minimize the sum of quadratic residuals.

If the parameters are found, then the value can be estimated y for any value x, that is, to implement interpolation And extrapolation data.

The same problem can be solved using a single-layer network with a single input and a single linear output neuron. Link weight a and threshold b can be obtained by minimizing the same value of the residual (which in this case will be called the root mean square mistake) during training networks, for example using the backpropagation method. Property of a neural network generalization will then be used to predict the output value from the input value.

Figure 25. Linear regression and the single-layer perceptron that implements it.

When comparing these two approaches, what immediately strikes the eye is that when describing their methods, statistics appeals to formulas And equations, and neurocomputing to graphical description of neural architectures.

1 If we remember that the left hemisphere operates with formulas and equations, and the right hemisphere with graphic images, then we can understand that in comparison with statistics, “ right hemisphere” neural network approach.

Another significant difference is that for statistical methods it does not matter how the discrepancy is minimized - in any case model remains the same, while for neurocomputing the main role is played by teaching method. In other words, unlike the neural network approach, the estimation of model parameters for statistical methods does not depend on the minimization method. At the same time, statisticians will consider changes in the type of residual, say by

How fundamental change in model.

Unlike the neural network approach, in which most of the time is spent training networks, in the statistical approach this time is spent on a thorough analysis of the problem. It uses the expertise of statisticians to select a model based on the analysis of data and information specific to the field. The use of neural networks - these universal approximators - is usually carried out without the use of a priori knowledge, although in some cases it is very useful. For example, for the linear model under consideration, the use of the root mean square error leads to obtaining an optimal estimate of its parameters when the noise value has a normal distribution with the same variance for all training pairs. At the same time, if it is known that these dispersions are different, then the use weighted error functions

can give significantly better parameter values.

In addition to the simplest model considered, we can give examples of other, in a sense, equivalent models of statistics and neural network paradigms

Table 3. Similar techniques

The Hopfield network has an obvious connection with data clustering and factor analysis.

1 Factor analysis used for studying structures data. Its main premise is the assumption of the existence of such signs - factors, which cannot be observed directly, but can be assessed by several observable primary characteristics. For example, signs such as volume of production And cost of fixed assets, can determine such a factor as scale of production. Unlike neural networks, which require training, factor analysis can only work with a certain number of observations. Although, in principle, the number of such observations should only be one greater than the number of variables, it is recommended to use at least three times the number of values. This is still considered less than the size of the training sample for the neural network. Therefore, statisticians point out the advantage of factor analysis in using less data and therefore leading to faster model generation. In addition, this means that the implementation of factor analysis methods requires less powerful computing tools. Another advantage of factor analysis is that it is a white-box method, i.e. completely open and understandable - the user can easily understand why the model produces a particular result. The connection between factor analysis and the Hopfield model can be seen by recalling the minimum basis vectors for a set of observations (memory images - see Chapter 5). It is these vectors that are analogues of factors that unite various components of memory vectors - primary characteristics.

1 Logistic regression is a binary classification method widely used in financial decision making. It allows you to estimate the probability of the realization (or non-realization) of some event depending on the values ​​of some independent variables - predictors: x 1,...,x N. In the logistic regression model, this probability has the analytical form: Pr( X) =(1+exp(-z)) -1, where z = a 0 + a 1 x 1 +...+ a N x N. Its neural network analogue is obviously a single-layer perceptron with a nonlinear output neuron. In financial applications, logistic regression is preferred over multivariable linear regression and discriminant analysis for a number of reasons. In particular, it automatically ensures that the probability belongs to the interval and imposes fewer restrictions on the distribution of predictor values. The latter is very important, since the distribution of values ​​of financial indicators in the form of ratios is usually not normal and is “highly skewed”. The advantage of neural networks is that this situation does not pose a problem for them. In addition, neural networks are insensitive to the correlation of predictor values, while methods for estimating regression model parameters in this case often give inaccurate values.

Annotation: Neural networks and statistics. Neural networks and fuzzy logic. Neural networks and expert systems. Neural networks and statistical physics.

Animals are divided into:

  1. belonging to the Emperor,
  2. embalmed,
  3. tamed,
  4. suckers,
  5. sirens,
  6. fabulous,
  7. individual dogs,
  8. included in this classification,
  9. running around like crazy
  10. countless,
  11. painted with the finest camel hair brush,
  12. others,
  13. broke a flower vase,
  14. from a distance resembling flies.

H. L. Borges, "The Analytic Language of John Wilkins"

Neurocomputing has numerous points of contact with other disciplines and their methods. In particular, the theory of neural networks uses the apparatus of statistical mechanics and optimization theory. The areas of application of neurocomputing sometimes strongly overlap or almost coincide with the areas of application of mathematical statistics, fuzzy set theory and expert systems. The connections and parallels of neurocomputing are extremely diverse and indicate its universality. In this lecture, which can be considered additional, since it requires somewhat more mathematical preparation, we will talk only about the most important of them.

Neural networks and statistics

Since neural networks are now successfully used for data analysis, it is appropriate to compare them with older, well-developed statistical methods. In the statistics literature, you can sometimes come across the statement that the most commonly used neural network approaches are nothing more than ineffective regression and discriminant models. We have already noted before that multilayer neural networks can actually solve problems like regression and classification. However, firstly, data processing by neural networks is much more diverse - remember, for example, active classification by Hopfield networks or Kohonen feature maps, which have no statistical analogues. Secondly, many studies concerning the use of neural networks in finance and business have revealed their advantages over previously developed statistical methods. Let's take a closer look at the results of comparing the methods of neural networks and mathematical statistics.

Are neural networks a description language?

As noted, some statisticians argue that neural network approaches to data processing are simply rediscovered and reformulated but well-known statistical methods of analysis. In other words, neurocomputing simply uses a new language to describe old knowledge. As an example, here is a quote from Warren Searle:

Many neural network researchers are engineers, physicists, neuroscientists, psychologists, or computer scientists who know little about statistics and nonlinear optimization. Neural network researchers are constantly rediscovering methods that have been known in the mathematical and statistical literature for decades and centuries, but often find themselves unable to understand how these methods work.

This point of view, at first glance, may seem reasonable. The formalism of neural networks can truly claim to be a universal language. It is no coincidence that already in the pioneering work of McCulloch and Pitts it was shown that a neural network description is equivalent to a description of propositional logic.

I actually found that with the technique I developed in the 1961 paper (...), I could easily answer all the questions that brain scientists (...) or computer scientists asked me. As a physicist, however, I knew well that a theory that explains everything actually explains nothing: at best it is a language. Eduardo Cayanello

It is not surprising, therefore, that statisticians often discover that concepts they are familiar with have their analogues in neural network theory. Warren Searle has compiled a small glossary of terms used in these two areas.

Table 11.1. Glossary of similar terms
Neural networks Statistical methods.
Signs variables
inputs independent variables
exits predicted values
target values dependent variables
error residual
training, adaptation, self-organization grade
error function, Lyapunov function evaluation criterion
training images (pairs) observations
network parameters: weights, thresholds. Estimated parameters
high order neurons interaction
functional connections transformation
supervised learning or heteroassociation regression and discriminant analysis
unsupervised learning or auto association data compression
competitive learning, adaptive vector quantization cluster analysis
generalization interpolation and extrapolation
What is the difference between neural networks and statistics?

What are the similarities and differences between the languages ​​of neurocomputing and statistics in data analysis? Let's look at a simple example.

Let us assume that we have made observations and experimentally measured N pairs of points representing the functional dependence. If we try to draw the best straight line through these points, which in the language of statistics will mean using a linear model to describe the unknown dependence , (where denotes the noise during the observation), then solving the corresponding linear regression problem will be reduced to finding the estimated values ​​of the parameters that minimize the sum of quadratic residuals.

If the parameters are found, then it is possible to estimate the value of y for any value of x, that is, to interpolate and extrapolate the data.

The same problem can be solved using single layer network with a single input and a single linear output neuron. The connection weight a and the threshold b can be obtained by minimizing the same amount of residual (which in this case will be called the root mean square error) during network training, for example, using the backpropagation method. The generalization property of the neural network will be used to predict the output value from the input value.


Rice. 11.1.

When comparing these two approaches, what immediately catches your eye is that when describing their methods, statistics appeals to formulas and equations, and neurocomputing refers to a graphical description of neural architectures.

If we remember that the left hemisphere operates with formulas and equations, and the right hemisphere with graphic images, then we can understand that in comparison with statistics, the “right hemisphere” nature of the neural network approach again appears.

Another significant difference is that for statistical methods it does not matter how the discrepancy is minimized - in any case, the model remains the same, while for neurocomputing it is the training method that plays the main role. In other words, unlike the neural network approach, the estimation of model parameters for statistical methods does not depend on minimization method. At the same time, statisticians will consider changes in the type of residual, say by

Like a fundamental change to the model.

Unlike the neural network approach, in which most of the time is spent training networks, in the statistical approach this time is spent on a thorough analysis of the problem. It uses the expertise of statisticians to select a model based on the analysis of data and information specific to the field. The use of neural networks - these universal approximators - is usually carried out without the use of a priori knowledge, although in some cases it is very useful. For example, for the linear model under consideration, the use of the root mean square error leads to obtaining an optimal estimate of its parameters when the noise value has a normal distribution with the same variance for all training pairs. At the same time, if it is known that these variances are different, then using a weighted error function

Can give significantly better parameter values.

In addition to the simplest model considered, we can give examples of other, in a sense, equivalent models of statistics and neural network paradigms

The Hopfield network has an obvious connection with data clustering and factor analysis.

Factor analysis used to study data structure. Its main premise is the assumption of the existence of such signs - factors that cannot be observed directly, but can be assessed by several observable primary signs. For example, such characteristics as production volume and the cost of fixed assets can determine such a factor as the scale of production. Unlike neural networks, which require training, factor analysis can only work with a certain number of observations. Although, in principle, the number of such observations should only be one greater than the number of variables, it is recommended to use at least three times the number of values. This is still considered less than the size of the training sample for the neural network. Therefore, statisticians point out the advantage of factor analysis in using less data and therefore leading to faster model generation. In addition, this means that the implementation of factor analysis methods requires less powerful computing tools. Another advantage of factor analysis is that it is a white-box method, i.e. completely open and understandable - the user can easily understand why the model produces a particular result. The connection between factor analysis and the Hopfield model can be seen by recalling the vectors minimum basis for a set of observations (memory images - see Lecture 5). It is these vectors that are analogues of factors that unite various components of memory vectors - primary characteristics.

Over a certain history of monitoring patients, an array of data has accumulated and is stored in a table in the STATISTICA system. The corresponding Data Table is shown in Figure 6.

Figure 6. Fragment of the source data table

The purpose of the study is to build a neural network model that, based on a given set of initial data (patient examination data, test results, treatment before admission), based on the treatment prescribed in the hospital, would produce a prognosis for his treatment (values ​​of admission to the hospital I-APFARA, BAB, BKK, diuretics, centrally acting drugs) with sufficient accuracy.

The fact that the problem is nonlinear is beyond doubt. Of course, one could try to solve the problem using the STATISTICA Nonlinear Estimation module, namely, using the iterative procedures offered by this module to “grope” for the type of function. However, there are a number of troubles here that significantly extend the procedure for finding a solution. The most important of them is the formulation of a hypothesis about the explicit form of the dependence being studied, which is not at all obvious.

Without additional research, it is difficult to say anything about the obvious type of dependence. Moreover, it should be mentioned that we did not take into account one more factor. In general, solving such a problem using nonlinear estimation methods can take a very long time, or may not lead to anything. In such critical situations, when it is known that

There is a relationship between the variables;

The relationship is definitely nonlinear;

It’s difficult to say anything about the obvious form of dependence,

Neural network algorithms help out. Let's consider a way to solve this problem in the STATISTICA Neural Networks module.

Unfortunately, there are no universal rules indicating which neural network topology should be followed to solve a particular problem. Therefore, a reasonable procedure for finding the right network is necessary.

The Neural Networks module of the STATISTICA system includes a procedure that organizes the search for the desired network configuration. This procedure consists of building and testing a large number of networks with different architectures and then selecting from among them the network that is best suited for solving the given problem. This tool is called Intelligent Problem Solver. To launch the Neural Networks module, you must use the command of the same name in the main menu of the STATISTICA - Statistics system. (Figure 7)

Figure 7. Launching the Neural Networks module

The following thesis is very common: “neural networks are a universal structure that allows you to implement any algorithm.” Let's try, blindly believing this statement, to build a neural network that would “catch” the proposed dependence immediately (meaning, without preliminary exploratory analysis).

One of the most important issues that has not yet been resolved by modern science is the question of the structure of a neural network that would be capable of reproducing the desired multidimensional nonlinear dependence. Indeed, Kolmogorov’s theorem on completeness, which he proved back in 1957, states that a neural network is capable of reproducing any (very important - continuous) function. However, it does not offer the researcher a recipe for creating such a network. In 1988, a number of authors generalized Kolmogorov's theorem and showed that any continuous function can be approximated by a three-layer neural network with one hidden layer and a backpropagation algorithm with any degree of accuracy. Thus, in our case, the positive aspect is the knowledge that the network must be three layers, but again there are no rules at hand that establish the relationship between “any degree of accuracy” and the number of neurons in the intermediate, so-called hidden layer.

Summarizing all of the above, we note that there are no universal rules indicating which neural network topology should be followed to solve a particular problem. Therefore, a reasonable procedure for finding the right network is necessary.

The Neural Networks module of the STATISTICA system includes a unique procedure that organizes the search for the desired network configuration. This tool is called Intelligent Problem Solver. Let's use this tool and search for a neural network that will be capable of solving our problem.

Figure 8. Neural Networks module launch pad

In the Quick tab of this dialog box, in the Problem Type section, it is proposed to select the class of problems that we are faced with. Our goal is to build a multivariate relationship or, in other words, multivariate nonlinear regression. This means that in the Problem Type section you should specify Regression.

Having decided on the class of tasks, it is necessary to specify the variables for carrying out the Analysis. To select variables, use the Variables button. When you click this button, the Select input (independent), output (dependent) and selector variables dialog box appears. In this dialog box, you need to specify two lists of variables. Continuous outputs, in our case, are the variables Admission to the hospital ACEI/ARB, Admission to the hospital BAB, Admission to the hospital BKK, Admission to the hospital of diuretics and admission to the hospital of centrally acting drugs. Continuous inputs, in our example, are from 1 to 61 initial parameters.

Figure 9. Selecting variables for analysis

In the Select analysis section, two options are available: Intelligent Problem Solver and Custom Network Designer. To automatically select neural network parameters, the first option is required, which is set by default. To continue the Analysis, click OK.

In the next step, the Intelligent Problem Solver setup dialog box appears.

The Quick section contains a group of options that are responsible for the execution time of the neural network search algorithm. On this tab, you need to specify the number of networks that need to be tested (to find out whether they are suitable for solving the formulated problem), and also indicate how many of these networks will be included in the final report.

In the Networks tested section we indicate 100, Networks retained - 10 (Figure 10)

The Types tab specifies what types of neural networks will be used in the testing process. To solve the problem of nonlinear regression, a multilayer perceptron is most suitable, so we choose the networks marked in Figure 11.

Figure 10. Setting the number of networks to test

Figure 11. Selecting a network architecture

Next, to organize the search and testing process, it is necessary to indicate the range of changes in the number of neurons in each layer of the neural network. The recommended values ​​for this task are shown in Figure 12.

Figure 12. Indication of the sizes of training, control and test samples

Now, after setting all the training parameters, to start the network search procedure, you must click OK.

The status of the search algorithm is displayed in the IPS Training In Progress dialog box.

While the search algorithm for a suitable neural network is running, this dialog box provides information about the execution time of the algorithm, as well as about the neural networks considered. The purpose of the search algorithm is to enumerate a number of neural network configurations and select the best one in terms of the minimum error at the network output and the maximum of its performance.

Did you like the article? Share it
Print this article
Top