Спросить
Войти
Запропоновано метод нейромережевог оброб-ки зашумлених цифрових зображень, як можуть м^тити викривлен фрагменти. Метод заснова-ний на використанн шумопригшчуючих автое-нкодерiв (ШАЕ). Для настроювання параметрiв ШАЕ та вибору його структури застосовано ней-роеволюцшний пiдхiд. Запропонована нейроеволю-цшна модель ШАЕ характеризуеться полтшени-ми апроксимуючими властивостями. Результаты моделювання свидчать про можлив^ть практичного використання запропонованого методу (зокрема, для обробки даних в геотформацшних системах)
Ключовi слова: обробка цифрових зображень, фшьтращя шумiв, еволющя, популящя, штучна нейронна мережа, генетичний алгоритм, автоен-кодер
Предложен метод нейросетевой обработки зашумленных цифровых изображений, которые могут содержать искаженные фрагменты. Метод основан на применении шумоподавляющих автоэн-кодеров (ШАЭ). Для настройки параметров ШАЭ и выбора его структуры использован нейроэво-люционный подход. Предложенная нейроэволюци-онная модель ИНС характеризуется улучшенными аппроксимирующими свойствами. Результаты моделирования свидетельствуют о возможности практического применения предложенного метода (в частности, для обработки пространственных данных в геоинформационных системах)
UDC 004.852
|öOI: 10.15587/1729-4061.2017.116134|
PROCESSING OF NOISY DIGITAL IMAGES WITH USE OF EVOLVING AUTOENCODERS
O. Bezsonov
Doctor of Technical Sciences, Professor* E-mail: oleksandr.bezsonov@nure.ua O. Ruden ko Doctor of Technical Sciences, Professor, Head of Department* E-mail: oleg.rudenko@ hneu.net S. Udovenko Doctor of Technical Sciences, Professor, Head of Department** Е-mail: serhiy.udovenko@hneu.net O. Dudinova Postgraduate student Department of Electronic Computers Kharkiv National University of Radio Electronics Nauky ave., 14, Kharkiv, Ukraine, 61166 E-mail: olga.dudinova@nure.ua *Department of information systems** **Department of informatics and computer technique*** ***Simon Kuznets Kharkiv National University of Economics Nauky ave., 9-A, Kharkiv, Ukraine, 61166
In recent years, considerable attention has been paid to the problem of computer analysis of digital images in information systems of various functional purposes, in particular, in geographic information systems (GIS). In general, computer processing of realistic cartographic images in GIS involves preliminary processing, segmentation, recognition and interpretation. The problems of object recognition in satellite images and aerial photographs are of particular practical interest. The main difficulties of recognition include changes in the visibility of objects caused by various internal and external factors (lighting, orientation, position, presence of distorted image fragments, etc.) [1].
Analysis and interpretation of aerial photographs (or aerospace images) are an important part of the implementation of many GIS applications. These include, in particular, topographic mapping, cadastral mapping, localization of contaminated areas, monitoring of changes in the edges of parts of images.
The problem of spatial data processing in GIS for environmental monitoring can be either image improvement
(restoration) by some criterion, or a special conversion that purposefully changes the image. In the latter case, image processing can be an intermediate step for further image recognition (for example, edge detection). Image processing methods can vary significantly depending on the way the image was obtained (synthesized by the computer graphics system, or by digitizing a black and white or color photograph or video frame). If images are obtained by digitizing, they are usually distorted by noise. There may be different sources of noise: camera or scanner errors, poor shooting conditions, noise in transmission over analog channels, etc. [2]. Denoising in the problems of spatial data processing in GIS serves to enhance visual perception of the analyzed images. This raises the need to improve sharpness during the edge detection, preprocessing and subsequent recognition of image fragments, etc. Promising options for the processing of noisy spatial data include the application of neural network methods that allow using parallel flowcharts of image processing by means of ANN [3].
One of the approaches widely used in the field of neural network image analysis is the use of conventional classification models that undergo supervised learning of a data
array containing learning and test samples. Among the many methods that implement this approach, the support vector machine (SVM), multilayer perceptron (MLP) and convolutional neural networks (CNN), being a variation of the MLP architecture, are the most common. Various modifications of MLP are widely applied in recognition of images of certain categories, such as natural language symbols, handwritten digits, handwriting, static realistic images. However, it is often difficult to determine the MLP structure and parameters. In many applications that use direct supervised learning for image recognition, the SVM that requires less computing resources at the learning stage is more effective. At the same time, the learned MLP is more preferable for recognition, since the SVM makes predictions much more slowly in cases where the number of vectors is large compared to the sample size [4]. The CNN, being very effective in recognition of static images, are poor at processing small objects and unable to deal with distortions, such as blur or strong noise.
A promising class of neural network models applied recently to solve the problems of recognition of noisy images are autoassociative memory models or autoencoders (AE) that have certain advantages over SVM, MLP and CNN. Thus, the development of a neural network method for image filtering and recognition in the presence of noise and distortions using the modified AE model is relevant.
Due to the significant progress in the field of deep learning of artificial neural networks (ANN), new scopes of the AE neural network autoassociative model have been recently investigated. The AE basic architecture has initially been proposed for solving problems of classification and allocation of features of the input data using unsupervised learning. In particular, the authors of [5] first have considered the denoising AE (DAE) - a robust modification of AE used for filtering noisy images. In [6], the possibility of using the DAE to restore partially damaged images has been considered. In [7], the AE has been proposed to use for pre-learn-ing of deep neural networks. In [8], the AE model, capable of generating fragments of images based on their spatial orientation has been presented. In [9], the AE modification that allows using the experience of previous observations to solve image reconstruction problems has been considered. In [10], the AE has been proposed to use for restoring texture information on missing image fragments. In [11], the possibility of using the convolutional AE for image reconstruction in invisible areas has been considered.
Note that the use of AE poses difficulties in choosing the network structure (the number of hidden layers and the number of neurons in each hidden layer), the parameters and the type of activation functions. Furthermore, in the known studies for the AE learning, gradient and probabilistic methods are usually applied, which greatly increases the network learning time. In this case, it is difficult to determine the optimum parameters of the algorithms providing their maximum rate of convergence, accuracy, robustness, etc. These disadvantages can be eliminated by using, in addition to traditional learning, another fundamental form of adaptation - evolution realized by applying evolutionary computation [12]. Studies on the use of the evolution principles for the DAE synthesis in noisy image processing systems in
the presence of local distortions have not yet been properly developed in existing scientific publications. First of all, this is due to certain learning difficulties of such DAE, caused by the need for a large initial structured data set with different types of noise and distortion. The imbalance of data in such a set decreases the efficiency of the entire image processing flowchart using the evolution mechanisms.
To solve the problem of processing noisy and distorted digital images with the help of DAE, it is proposed to use a promising neuroevolutionary approach, allowing not only to adjust the neural network parameters, but also to determine the structure.
The aim of the research is to develop a method for processing noisy and distorted digital images using the neural network evolving autoencoder. This will allow implementing an approach to noise filtering and restoration of irregular local distortions during image processing in environmental monitoring systems, which provides for a periodic scanning of landscape areas, allowing to use a sequence of similar frames for assessing current changes in the monitored objects using autoencoders.
To achieve this aim, it is necessary to accomplish the following objectives:
- to synthesize the structure of the evolving denois-ing autoencoder (EDAE), designed to suppress noise and restore distorted fragments in digital images;
- to develop a neuroevolutionary algorithm for the EDAE setting;
- to carry out a test simulation to assess the effectiveness of the proposed method for processing noisy and distorted images.
An autoencoder is a type of a multilayer neural network that attempts to perform an autoassociative approximation of a function so that its output signal corresponds the input one as nearly as possible. This identical correspondence is not trivial if we impose some specific restrictions on the autoencoder neural network. These can be, first of all, restrictions on the number of neurons on the hidden layer and imposition of sparsity criteria on the activation of these neurons.
It should be noted that the AE is generally characterized by the following features:
- has a symmetric structure;
- contains an odd number of layers;
- consists of an encoder and a decoder;
- contains a so-called "bottleneck" - the encoder output layer (the decoder input layer), in which linear activation functions are used;
- the number of neurons in the "bottleneck" should not exceed the dimension of the input data, or only partial activation of these neurons (sparse activation) should occur;
- Gaussian activation functions are usually used in the AE input and output layers.
The architecture symmetry greatly facilitates adjustment of the AE parameters, since it is necessary to determine only half of all network parameters.
Let us consider a classical AE with one hidden layer. We denote the network input signals as x(1), x(2),..., x(n).
First, the AE converts (encodes) the input signal x e[0,1] into some internal representation y e[0,1] using a conversion of the form
learning flowchart can be used in denoising applications and is a popular way the autoencoder learns more meaningful hidden data representations [13].
The DAE architecture is shown in Fig. 2.
y=s(Wx+b),
where W and b are general weight and shift matrices of the network; s is the nonlinear conversion function (e. g. hyperbolic tangent).
Then, the internal representation (code) of the input signal y is converted back (decoded) into the signal 2, which is the reconstruction of the input signal and has the same dimension. This conversion can be written as follows:
where W& and b& are the adjusted weight and shift matrices of the decoding conversion network.
The model parameters (W, Wb, b&) are adjusted so as to minimize the input signal reconstruction error, which can be realized using various loss functions, e.g. quadratic function or cross-entropy.
The AE classical architecture is shown in Fig. 1 [13].
Layer Li
Fig. 1. AE classical architecture
Here, the L1 layer is an input layer that is encoded in the L2 layer using the hidden representation, and restoration of the input signal occurs in the L3 layer. Since the number of hidden neurons is lower than that of inputs, this forces the AE to perform data compression. It should be noted that the data conversion procedure in the AE resembles the similar procedure in the principal component analysis (PCA). If the number of hidden neurons exceeds the number of inputs, the AE can still find some useful information in the input data, establishing certain sparsity restrictions.
The denoising autoencoder (DAE) is an AE version, in which inputs are combined with some distortions (such as additive noise or masking) and the system is learned to restore data without noise [6]. Thus, the DAE can be considered as a stochastic extension of the classical autoencoder, which forces the model to learn restoration of the input signal when its noisy version is fed to the input. The stochastic data corruption process randomly sets some of the input data to zero, forcing the DAE to predict missing (corrupted) values for random subsets of missing patterns. The DAE
Fig. 2. Denoising AE architecture
Fig. 3 illustrates the principle of restoration of partially damaged or noisy landscape images with the help of DAE. This principle is that a network is given a number of noiseless (clean) images for learning. During the learning, all the images presented are compressed and stored in the autoencoder memory (in the form of its parameters). After the network is learned, it can be used in actual practice. It should be noted that when processing spatial data in GIS, actual images sometimes turn out to be noisy or partially damaged, as shown in Fig. 3. After such an image is presented to the previously learned DAE, the closest image will be extracted from its memory and partial loss of information will be eliminated. Obviously, the DAE can only use images from a set of memorized samples, which poses the problem of generating a learning sample.
Extensive practical application of DAE related to image processing is often hindered by the lack of a learning sequence of really clean (noiseless) data necessary for tuning encoders. In addition, the DAE parameters are usually set using gradient algorithms, the implementation of which involves cumbersome matrix inversion operations [14].
Encoding Decoding
Fig. 3. Principle of processing noisy images with the help of DAE
In this paper, a DAE modification, which uses the neu-roevolutionary approach, allowing not only to adjust the network parameters, but also determine its structure is proposed. Further, this modification will be referred to as the EDAE (evolving denoising autoencoder).
One of the most common types of evolutionary algorithms are genetic algorithms (GA), proposed by J. Holland. The main stages of the GA are population initialization, population estimation (fitness function computation), selection, crossover, mutation. The criterion of the GA stopping is usually either the fulfillment of the maximum permissible number of iterations, or the achievement of some predetermined threshold by the fitness function of an individual (solution).
The main difference between the neuroevolving algorithm (NA) and the gradient algorithms implemented with the help of ANN is that all operations are performed not with one network, but with a number of them (population). This approach is associated with the increased amount of memory for storing information about all networks in the population. At the same time, this allows abandoning complex operations such as matrix inversion in the network learning.
This eliminates the need to define the network structure (the number of hidden layers), the number of neurons of the hidden layer, and the type of basis functions. In addition, the problem of determining the coefficients included in the learning algorithm and affecting the EDAE learning process duration is automatically solved.
At the initial stage of the NA, the population P0 with N individuals (AE) is randomly initialized:
Po =H H2,...Hn }.
Each individual in the population gets a unique description encoded in the chromosome
H = {K j, h2 j ,4},
consisting of L genes, where h, e[®min®max ] is the value of the i-th gene of the j-th chromosome ( wmin - minimum, and wmax - maximum allowable values, respectively). The chromosome format and correspondence between the AE genes and parameters are shown in Fig. 4. It should be noted that the chromosome length is limited to the maximum permissible number of neurons.
hidden layer. The first gene of each block (1/0) determines whether the corresponding neuron is present in the network structure, i.e. involved or not in the calculation of the network output response to the incoming input signal. It should be noted that due to the special autoencoder architecture, the structure of only its encoding part is determined, and the structure of the decoding part is its mirror reflection. As activation functions (AF) in the AE, we use sigmoidal AF of the form
f ( x ) = (1 + e)(3)
so the next gene in the chromosome (a) determines the form of the AF of the corresponding neuron.
It should be noted that the mutation mechanism in GA is the main way of introducing new information into the individual&s chromosomes. This can lead to entirely new gene values, which can later be added to the population gene pool. With the help of these new gene values, the GA is able to find better solutions. Mutation is an important part of genetic search and helps to prevent the population from entering local minima. As a rule, it occurs in the individual&s chromosomes obtained during the crossover with some usually predetermined probability. The mutation probability can also be tied to the value of the fitness function, i. e. the lower the fitness function value, the higher the mutation probability.
In the proposed synthesis of the EDEA evolution procedure, two standard methods for gene mutation can be used. In the first case, the value of the random offspring chromosome gene is replaced with a new value in the range of admissible values
Encoding
Decoding
Output layer
General network parameters
i-th neuron
Fig. 4. EDAE chromosome formats
As can be seen from Fig. 4, each chromosome consists of genes, which store information on the relevant network parameters. The high-order bits of the chromosome contain the genes with information about the general network parameters (the maximum number of hidden layers M, the maximum number of neurons in each layer Si,..., SM). The following genes encode information on the shift of neurons of the AE output layer. Then there are blocks of genes that encode the parameters of the corresponding neurons of the
hj = rand [^ hm
where rand [x, y] is a random number in the [x, y] interval distributed under the uniform law.
In the second case, random shift is added to the existing gene value
K = L + Ah,
M Si SM hi WH2 • • • 1/0 a, b, wi,i • • • 1/0 a, b, wii
where hj is the gene before the mutation; hj is the gene after mutation.
However, when setting the AE used to process noisy images, the learning process is prolonged, since the adjusted parameters are diverse and their values lie in different ranges. In this regard, it becomes necessary to use adaptive mutations, which allow determining mutation parameters for each gene separately. The mutation parameters are stored in the additional vector v.
In the developed method, the possibility of using three types of adaptive mutations was investigated.
In the first case, the adaptive change in the mutation step was carried out as follows:
v j ( k) =yv j ( k -1),
Y G [0,1].
Herewith, the change in the gene value occurred under the following rule:
K = K + v, (k)C (k, t),
where C(k, t) is a random variable distributed under the Cauchy law with the parameter t. The probability density of the random variable is described by the following function:
/(x) = if T
In the second case, the mutation step was first corrected, and then the value of the gene mutating in the chromosome was changed. This procedure can be described as follows:
v , ( k) = v , ( k - 1)exp [IN (0,1) + FN (0,1) ];
K =K + v; ( k) N (0,1),
l& = (2y[N )respectively.
In the second case, the mutation step was adjusted according to the rule (9), but the gene mutation was performed as follows:
K =h + v j (k)L(a),
where L is a random variable distributed under the Laplace&s law with the parameter a, the probability density of which has the form:
/ (x ) =
-e"1, if x < 0; 2
/ (xj)=m £1 y* (xj) - y j (xj )l,
where M is the sample size.
To simplify further population sorting operations, the fitness function is usually normalized
/N (x, )=/j£ /, (x, )
where N(0, 1) is a random variable distributed under the normal law with zero population mean and unit variance; l and l& are some parameters, the following optimal values of which were obtained in the work:
Thus, to determine the network fitness, it is simulated on the entire sample, and then the network response is compared to the actual output signal of the object. The average error is the desired value of the fitness function used.
A number of experiments were carried out during the research of the proposed method of image processing with the help of EDEA. The aim was to show the possibility of using the neuroevolving encoder for processing noisy or partially damaged images.
Experiment 1. In this experiment, the EDEA denoising properties were investigated on the example of restoring the following noisy function of 6 variables:
F (x4, x2, x3, x4, x5, x6 ) = sin (n x4 )■ sin (n x2 )■ sin (n x3) + +sin (n x4 )■ sin (n x5 )■ sin (n x6) +
+exp(-( x2 + x22)) + exp (-(x2 + 6x3 x4 + 10x2 )) + £. (15)
To do this, the EDEA population consisting of 150 individuals was used. Each autoencoder had 7 input and output neurons (x1,., x6, F), respectively. The network structure (the number of layers and neurons in them) was determined by means of the evolutionary process, considering the EDEA structural peculiarities discussed above. The input signal corresponding to the value of the F function was noisy with different distributions: Rayleigh, Laplace, and Gaussian. The results of the experiment are shown in Fig. 5-7, which show the noise histograms and the corresponding graphs of the learning error. As can be seen from the simulation results, the autoencoder shows satisfactory results with any type of noise.
After the initial population is formed, the fitness of each entering individual is estimated using the fitness function. Often, when solving optimization problems, an estimate of the possible final result of the evolution algorithm is unknown. However, when solving image identification problems, the desired network response y (k) and actual output signal y(k) are known. It is obvious that the network for which the difference y (k)- y(k) is minimum will be considered the fittest for solving object identification problems. In the case of off-line learning with the full sample of input-output object signals, the fitness function of the i-th individual can be written as follows:
Evolution t
Fig. 5. Results of the experiment for the Laplace distribution: a — noise histogram; b — learning error
from the simulation results, the evolving autoencoder is a fairly effective neural network model for solving problems of recovering partially damaged landscape images.
Fig. 6. Results of the experiment for the Rayleigh distribution: a — noise histogram; b — learning error
Evolution f b
Fig. 7. Results of the experiment for the Gaussian distribution: a — noise histogram; b — learning error
Experiment 2. The study of the autoencoder restoring properties on partially damaged landscape images was conducted. For this purpose, the population of autoencoders consisting of 300 individuals was used. Learning was carried out with a learning sample of 1,000 different landscape images, whose size was previously reduced. The size of the images presented to the autoencoder was 80x80 pixels. Learning was conducted over 30,000 epochs. The evolutionary algorithm yielded an optimum seven-layer network, allowing to compress original images by an average of 32 %. After learning, some foreign objects, which partially obscured original information were superimposed on some images. The results of restoration of damaged images by the autoencoder are shown in Fig. 8, 9. The left parts in Fig. 8, 9 show distorted images, and the right - restored from memory of the best autoencoder in the population. As can be seen
Fig. 8. Image restoration example 1: a — distorted image; b — restored image
Fig. 9. Image restoration example 2: a — distorted image; b — restored image
During the experiments, it was found that the process learning the proposed evolving autoencoder shows satisfactory results with different types of noise typical of the process of recognition of noisy images. In addition, the possibility of recovering distorted areas of processed images with the help of EDAE was experimentally confirmed.
The advantages of the proposed approach include:
- the ability to automatically determine the structure and parameters of the ANN used for pre-processing of noisy images (for different types of noise) and recognition of their individual fragments;
- reduced computing complexity of image processing with the help of EDAE due to the use of an intermediate procedure for data compression and elimination of matrix inversion operations.
The disadvantage of the proposed approach is the lack of sufficient theoretical justification, which does not allow concluding about the EDAE effectiveness in various data processing problems.
At the same time, the results confirm the EDAE effectiveness in the problems of processing noisy and distorted images in the presence of various types of noise. Such problems include, in particular, processing of landscape images in systems of environmental monitoring of natural resources. This provides for a periodic scanning of natural and industrial areas, allowing to use a sequence of similar frames for assessing current changes in the monitored objects with the help of EDAE.
It should be noted that the research can be considered a continuation of scientific papers [2-4] in the field of creating neural network data processing systems. Further, it seems promising to carry out additional studies on the functionality expansion and theoretical justification of the proposed approach.
part is its mirror reflection. The flowchart of the evolutionary learning algorithm with the help of GA is given, and the EDAE chromosome formats considering the features of the problem being solved are proposed. The EDAE setting provides for the application of adaptive mutations, the parameters of which are determined for each gene separately and stored in an additional vector.
The results of the experiments indicate the reduced complexity of processing noisy images (with various types of noise). In particular, this is achieved through the use of an intermediate procedure of data compression and elimination of matrix inversion operations in the automatic determination of the structure and parameters of the adjusted network.
References
