Developing A Fault Diagnosis Approach Based On Artificial Neural Network And Self Organization Map For Occurred ADSL Faults

الموضوعات :

Vahid Golmah ¹ , Mina Tashakori ²

1 - Department of Computer Engineering, Neyshabur Branch, Islamic Azad University, Neyshabur,Iran
2 - Computer Engineering Department Ferdowsi University of Mashhad Mashhad, Iran

تاريخ الإرسال : 23 السبت , ذو الحجة, 1437 تاريخ التأكيد : 14 الجمعة , رمضان, 1438 تاريخ الإصدار : 09 الثلاثاء , ذو القعدة, 1438

الکلمات المفتاحية: data mining, Self Organization Map (SOM), Fault Detection and Diagnosis (FDD), multilayer perceptron Artificial Neural Network (MLP-ANN),

ملخص المقالة :

Telecommunication companies have received a great deal of research attention, which have many advantages such as low cost, higher qualification, simple installation and maintenance, and high reliability. However, the using of technical maintenance approaches in Telecommunication companies could improve system reliability and users' satisfaction from Asymmetric digital subscriber line (ADSL) services. In ADSL systems, there are many variables giving some noise for classification and there are many fault patterns with overlapping data. Therefore, this paper proposes a multilayer perceptron (MLP) classifier integrated with Self Organization Map (SOM) models for fault detection and diagnosis (FDD) of occurred ADSL systems. The interest of this paper is to improve the performance of single MLP by dividing the fault pattern space into a few smaller sub-spaces using SOM clustering technique and triggering the right local classifier by designing a supervisor agent. The performances of this method are evaluated on the fault data of Iranian Telecommunication Company which develop ADSL services and then the proposed algorithm is also compared against single MLP. Finally, the results obtained by this algorithm are analyzed to increase user's satisfaction with reducing occurred faults for them with predicting before they face it.

المصادر:

1 McKay, H.: ‘Development of the contractual relationship between an ISP and its customers–Is a fairer deal in sight?’, Computer Law & Security Review, 2005, 21, (3), pp. 216-225
2 Carrera, أ., Iglesias, C.A., Garcأa-Algarra, J., and Kolaإ™أk, D.a.: ‘A real-life application of multi-agent systems for fault diagnosis in the provision of an Internet business service’, Journal of Network and Computer Applications, 2014, 37, pp. 146-154
3 Hanafizadeh, P., and Mirzazadeh, M.: ‘Visualizing market segmentation using self-organizing maps and Fuzzy Delphi method–ADSL market of a telecommunication company’, Expert Systems with Applications, 2011, 38, (1), pp. 198-205
4 Gao, Y., Liu, S., Li, F., and Liu, Z.: ‘Fault detection and diagnosis method for cooling dehumidifier based on LS-SVM NARX model’, International Journal of Refrigeration, 2015
5 Muralidharan, V., Sugumaran, V., and Indira, V.: ‘Fault diagnosis of monoblock centrifugal pump using SVM’, Engineering Science and Technology, an International Journal, 2014, 17, (3), pp. 152-157
6 Hakim, S.J.S., Razak, H.A., and Ravanfar, S.A.: ‘Fault diagnosis on beam-like structures from modal parameters using artificial neural networks’, Measurement, 2015, 76, pp. 45-61
7 Lashkari, N., Poshtan, J., and Azgomi, H.F.: ‘Simulative and experimental investigation on stator winding turn and unbalanced supply voltage fault diagnosis in induction motors using Artificial Neural Networks’, ISA Transactions, 2015
8 Lau, C.K., Ghosh, K., Hussain, M.A., and Hassan, C.R.C.: ‘Fault diagnosis of Tennessee Eastman process with multi-scale PCA and ANFIS’, Chemometrics and Intelligent Laboratory Systems, 2013, 120, pp. 1-14
9 Mnassri, B., and Ouladsine, M.: ‘Reconstruction-based contribution approaches for improved fault diagnosis using principal component analysis’, Journal of Process Control, 2015, 33, pp. 60-76
10 Huang, J., and Yan, X.: ‘Dynamic process fault detection and diagnosis based on dynamic principal component analysis, dynamic independent component analysis and Bayesian inference’, Chemometrics and Intelligent Laboratory Systems, 2015
11 Zhao, Y., Wen, J., and Wang, S.: ‘Diagnostic Bayesian networks for diagnosing air handling units faults–Part II: Faults in coils and sensors’, Applied Thermal Engineering, 2015, 90, pp. 145-157
12 Zhang, Z., and Dong, F.: ‘Fault detection and diagnosis for missing data systems with a three time-slice dynamic Bayesian network approach’, Chemometrics and Intelligent Laboratory Systems, 2014, 138, pp. 30-40
13 Eslamloueyan, R.: ‘Designing a hierarchical neural network based on fuzzy clustering for fault diagnosis of the Tennessee–Eastman process’, Applied soft computing, 2011, 11, (1), pp. 1407-1415
14 Rad, M.A.A., and Yazdanpanah, M.J.: ‘Designing supervised local neural network classifiers based on EM clustering for fault diagnosis of Tennessee Eastman process’, Chemometrics and Intelligent Laboratory Systems, 2015
15 Mendoza, O., Melأ­n, P., and Castillo, O.: ‘Interval type-2 fuzzy logic and modular neural networks for face recognition applications’, Applied soft computing, 2009, 9, (4), pp. 1377-1387
16 Sanchez, D., Melin, P., and Castillo, O.: ‘Optimization of modular granular neural networks using a hierarchical genetic algorithm based on the database complexity applied to human recognition’, Information Sciences, 2015, 309, pp. 73-101
17 Sanchez, D., and Melin, P.: ‘Optimization of modular granular neural networks using hierarchical genetic algorithms for human recognition using the ear biometric measure’, Engineering Applications of Artificial Intelligence, 2014, 27, pp. 41-56
18 Chicco, G.: ‘Overview and performance assessment of the clustering methods for electrical load pattern grouping’, Energy, 2012, 42, (1), pp. 68-80
19 Lopez, J.J., Aguado, J.A., Martin, F., Munoz, F., Rodriguez, A., and Ruiz, J.E.: ‘Hopfield K-Means clustering algorithm: A proposal for the segmentation of electricity customers’, Electric Power Systems Research, 2011, 81, (2), pp. 716-724
20 Guerrero, J.D.M., Marcelli, D., Soria-Olivas, E., Mari, F., Martinez-Martأnez, J.M., Bech, I.S., Martinez-Sober, M., Scatizzi, L., Gimez-Sanchis, J., and Stopper, A.: ‘Self-Organising Maps: A new way to screen the level of satisfaction of dialysis patients’, Expert systems with applications, 2012, 39, (10), pp. 8793-8798
21 Segev, A., and Kantola, J.: ‘Identification of trends from patents using self-organizing maps’, Expert systems with applications, 2012, 39, (18), pp. 13235-13242
22 Davies, D.L., and Bouldin, D.W.: ‘A cluster separation measure’, Pattern Analysis and Machine Intelligence, IEEE Transactions on, 1979, (2), pp. 224-227
23 Szymczyk, P., and Szymczyk, M.: ‘Classification of geological structure using ground penetrating radar and Laplace transform artificial neural networks’, Neurocomputing, 2015, 148, pp. 354-362
24 Prakash, M., Pradhan, S., and Roy, S.: ‘Soft computing techniques for fault detection in power distribution systems: A review’, in Editor (Ed.)^(Eds.): ‘Book Soft computing techniques for fault detection in power distribution systems: A review’ (IEEE, 2014, edn.), pp. 1-6
25 Chojaczyk, A.A., Teixeira, A.P., Neves, L.C., Cardoso, J.B., and Soares, C.G.: ‘Review and application of Artificial Neural Networks models in reliability analysis of steel structures’, Structural Safety, 2015, 52, pp. 78-89
26 Palanichamy, A., Jayas, D.S., and Holley, R.A.: ‘Predicting survival of Escherichia coli O157: H7 in dry fermented sausage using artificial neural networks’, Journal of Food Protectionآ®, 2008, 71, (1), pp. 6-12
27 Gosukonda, R., Mahapatra, A.K., Liu, X., and Kannan, G.: ‘Application of artificial neural network to predict< i> Escherichia coli</i> O157: H7 inactivation on beef surfaces’, Food Control, 2014, 47, pp. 606-614
28 Li, Y., Xia, J., Zhang, S., Yan, J., Ai, X., and Dai, K.: ‘An efficient intrusion detection system based on support vector machines and gradually feature removal method’, Expert Systems with Applications, 2012, 39, pp. 424–430
29 Esa Alhoniemi, J.H., Jukka Parviainen and Juha Vesanto: ‘Som toolbox for matlab’, in Editor (Ed.)^(Eds.): ‘Book Som toolbox for matlab’ (1999, edn.), pp.
30 Vesanto, J., Himberg, J., Alhoniemi, E., and Parhankangas, J.: ‘Self-organizing map in Matlab: the SOM Toolbox’, in Editor (Ed.)^(Eds.): ‘Book Self-organizing map in Matlab: the SOM Toolbox’ (1999, edn.), pp. 16-17
31 Kuzmanovskia., I., Trpkovskaa., M., and Soptrajanov, B.: ‘Optimization of supervised self-organizing maps with genetic algorithms for classification of urinary calculi’, Journal of Molecular Structure, 2005, 744–747, pp. 833–838

نص كامل:



Developing A Fault Diagnosis Approach Based On Artificial Neural Network And Self Organization Map For Occurred ADSL Faults

 

 

 

 

 

Abstract— Telecommunication companies have received a great deal of research attention, which have many advantages such as low cost, higher qualification, simple installation and maintenance, and high reliability. However, the using of technical maintenance approaches in Telecommunication companies could improve system reliability and users' satisfaction from Asymmetric digital subscriber line (ADSL) services. In ADSL systems, there are many variables giving some noise for classification and there are many fault patterns with overlapping data. Therefore, this paper proposes a multilayer perceptron (MLP) classifier integrated with Self Organization Map (SOM) models for fault detection and diagnosis (FDD) of occurred ADSL systems. The interest of this paper is to improve the performance of single MLP by dividing the fault pattern space into a few smaller sub-spaces using SOM clustering technique and triggering the right local classifier by designing a supervisor agent. The performances of this method are evaluated on the fault data of Iranian Telecommunication Company which develop ADSL services and then the proposed algorithm is also compared against single MLP. Finally, the results obtained by this algorithm are analyzed to increase user's satisfaction with reducing occurred faults for them with predicting before they face it.

Keywords— Fault Detection and Diagnosis (FDD), Data mining, Self Organization Map (SOM), multilayer perceptron Artificial Neural Network (MLP-ANN)

 

1-       Introduction 

The Internet has revolutionized the world of people life over recent years. Anyone who wishes to place a web page on the Internet, or undertake e-mail communications will invariably have to enter into a contract for Internet access [1]. Such Internet services could be provided by different telecommunications companies. Whilst there is a significant array of companies which can provide Internet services, it would appear that technical advancement may well reduce competition for the provision of such services. Should this trend persist, then it is likely that the remaining companies will face far more stringent regulation [1].

Fault Detection and Diagnosis (FDD) is one of the most important technical approaches in network management tasks for telecommunications companies. Traditionally, this process has been carried out by humans and software systems that work in a cooperative way. The constant increase in the size and complexity of the network makes fault diagnosis a critical task that should be handled quickly and in a reliable way. Highly skilled engineers are required to carry out this task, although even these individuals are not always able to deal with the increasing heterogeneity and complexity of the networks. Although automated fault diagnosis processes have been developed, such as surveillance systems for symptom detection in the core or backbone networks, fault diagnosis is mainly a manual process managed by human operators [2].

Telecommunication industry is a growing industry in Iran which has an enormous potential market for different telecommunications services [3]. Comparing to all the existing researches, there aren’t many researches on analyzing the Iran satisfaction improvement of the ADSL users’ with reducing faults. Moreover, many Iranian telecommunication companies do not fallow any clear strategies for fault diagnosis and solve it before users face them. Our main contribution is based on the real register ADSL faults to diagnosis them for improving users’ satisfaction. Fig. 1 shows the picture of the ADSL equipment and how ADSL’s work is illustrated in under of figures.

Fault Detection and Diagnosis can be seen as a supervised classification task whose objective is to assign new observations to one of the existing classes. Various supervised learning models such as support vector machines (SVM)[4, 5], Artificial Neural Networks (ANN) [6, 7], adaptive network-based fuzzy inference system (ANFIS)[8], Principal Component Analysis (PCA) [9, 10] and the Bayesian network classifier[11, 12] have been applied for fault classification of industrial processes. Among the mentioned methods, the artificial neural network of multilayer perceptron (MLP) type in Fault Detection and Diagnosis has received considerable attention over the last three decades [13] and high efficiency as non-linear classifier. However, its application in large scale industries is not without difficulties[13, 14].Despite simplicity and versatility of MLP networks, their application to Fault Detection and Diagnosis in wide scale systems is not without difficulties. One of the most important issues of employing MLP networks in Fault Detection and Diagnosis is related to its training. As the size and complexity of a system increases, the number of measured variables and fault patterns rise. This results in a bigger and more complex MLP network to diagnose the system faults properly. Sometimes the structure of the network becomes so large that it could not learn all fault patterns with an acceptable performance goal. To overcome this problem in artificial neural network, hierarchical and modular artificial neural networks (HANN) have been proposed, developed and applied by researchers [15-17] in different research areas. According to these research works, diagnosis of single faults and simultaneous double and triple faults are hierarchically divided among neural networks specifically trained for that purpose. The concept behind this technique is that, instead of using a single neural network for diagnosing all faults, whole pattern space of faults is divided in a hierarchical way into a number of smaller sub-spaces. For each set of faults, in a particular sub-space, a special diagnostic agent is trained. Clustering methods may be employed to divide the whole space of fault patterns into alike fault pattern sets based on a similarity criterion [13].

 

 

 

 

 

 

 

 

 

 

 

 

 

FIG. 1. (a) Main Distribution Frame (MDF), (b) ADSL Panel Blocks and (c) ADSL Line Cards. From the MDF panel, telephone line will go to an ADSL line card, each line card ties up and labeled according to the MDF panel which your line is attached to.

 

In ADSL systems, there are many variables giving some noise for classification and there are many fault patterns with overlapping data. As a result, the MLP classifier cannot learn all the fault patterns with an acceptable performance goal. Therefore, the aim of this article is to present a new method for improving the performance of MLP classifier for fault diagnosis of ADSL systems. This method is based on dividing the huge amount of fault pattern dataset into smaller fault subset using the Self Organization Map (SOM) clustering algorithm. At each specific fault pattern subset, a local MLP classifier is trained. For a new test observation, the SOM clustering algorithm acts as a supervisor agent by assigning the data to the cluster corresponding to the highest mean similarity.

 

2-       Methodology

The idea behind this approach is to provide mechanisms for improving customer relationship management for an Iranian telecommunication company which develop ADSL services through increasing user's satisfaction with reducing occurred faults for them. There are various methods related to solve Fault Detection and Diagnosis problem in various ways. Among the mentioned methods, the MLP is more popular due to its simplicity and high efficiency as non-linear classifier. However, it hasn’t suitable performance in large scale systems. In regard to high variables, numerous noise data and overlapping data in occurred ADSL fault, a single MLP cannot learn all the fault patterns with an acceptable performance goal. Therefore, proposed approach to this problem is based on dividing the occurred faults for ADSL's user dataset into smaller fault subset using the SOM clustering algorithm and training MLP for patterns of faults any cluster. FIG. 2 shows our model based on collecting, preparing, clustering and training process schematically for Fault Detection and Diagnosis in ADSL systems. This Fig. provides a general overview of our model, specifying the components and characteristics.

 

FIG. 2 Developed methodology diagram for detecting and diagnosing of occurred faults for ADSL’

 

2-1- Data gathering and preparing

The Collecting and preparing stage starts with an initial data collection and proceeds with activities in order to get familiar with the data, to identify data quality problems, to discover first insights into the data or to detect interesting subsets to form hypotheses for hidden information. This phase covers all activities to construct the final dataset (data that will be fed into the modeling tool(s)) from the initial raw data. Data preparation tasks are likely to be performed multiple times and not in any prescribed order.

 

2-2- Clustering phase

The clustering methodologies are generally used to perform load pattern grouping with grouping them in such a way that patterns in the same group are more similar (in some sense or another) to each other than to those in other groups. In particular, it is possible to identify unsupervised learning based techniques, such as supervised learning-based techniques (such as the ones adopting multilayer perceptron or Elman neural networks), the Kohonen’s self organising map (SOM), or vector quantization, fuzzy logic-based techniques, statistical techniques such as k-means (KM) and multivariate analysis, and hybrid techniques such as probability neural networks (PNN) and fuzzy k-means (FKM)[18, 19].

All these techniques have advantages include high speed, the possibility of re-allocations in successive iterations and the fact that they do not require previous training but their obtained solutions depend on the initial centroids and number of predefined clusters (k-means (KM) and fuzzy k-means (FKM)), the internal distance threshold (follow the leader (FDL) algorithm), the initial neuron weight (The Adaptive Vector Quantization (AVQ) method) and they suffer from non-reallocation of data in successive iterations (hierarchical algorithm) and lack of objective function (hierarchical algorithm) [18]. Among these methods, SOM has other important advantages with less disadvantages. The SOM modifies the search space to represent the results on a bi-dimensional map, but does not generate the final clusters directly. Hence, a post-processing stage is needed to form the clusters, with arbitrary assumptions, so that different numbers of clusters can be formed starting from the same SOM outcomes, by using a specific technique to identify the final clusters. Likewise, support vector clustering (SVC) requires a first stage in which the support vectors are formed, followed by a second stage in which the groups are formed for the desired number of clusters. SOM has able to learn both the distribution and the topology of the input vector training set, and it also allows to identify new patterns and the interpretation and visualization of the results with the unified distance matrix method or U-matrix. The most appealing characteristic of SOM is that the underlying mathematics ensure that the map is a faithful representation of the original data, e.g. two data points (two faults in our case) are represented close to each other in the resulting map when they have similar features (e.g. Row, Terminal and etc.). Therefore, the U-matrix can be used to discover otherwise invisible relationships in a high-dimensional data space and classify data sets into clusters of similar values.[20, 21].

In order to dividing the huge amount of fault pattern dataset into smaller fault subset we use SOM clustering algorithm and to confront the randomness of the solution obtained drawback in the SOM algorithms we can run the algorithm more. We used the Davies–Bouldin index (DBI) [22] to measure the performance because of its popularity for cluster evaluation. Low DBI values indicate high clustering performance.

 

2-3- Training phase

Classification is a problem of identifying to which set of categories a new observation belongs, on the basis of a training set of data. Containing observations whose category membership is known. An algorithm that implements classification is known as a classifier [23].

Each classifier has its own strengths and weaknesses. The most widely used classifiers are the neural networks. Neural networks have been selected for further consideration because they are simple, easy to build, efficient and reliable classifier and they are also give good results [23]. An artificial neural network is a system based on the operation of biological neural networks, in other words, is an emulation of the central nervous system in animals. Neural networks comprise of many artificial fundamental units, called neurons. An artificial neuron is simply a computational model of a biological neuron [24].

In the ANN, the neuron is a processing element with several inputs and one output. Each neuron m receives an input signal vector X =x1 + x2 + … + xn from n input channels. Next, the weighted sum of x is calculated by multiplying each element xk by a coefficient wmk demonstrating adequate importance of the input channel k. The m-neuron activation a m is given by [25]:

Where bm called bias, is a constant corrective term which allows having a non-negative activation am, when all elements of the input vector X are equal 0. The output signal value sm is calculated as a function of the activation, called the transfer function f(am). Sigmoid transfer functions are typically used for this purpose and a common choice is the hyperbolic tangent sigmoid transfer function [25]:

(1)

The architecture of a single neuron is shown in FIG. 3.

.

FIG. 3. Artificial neuron [25]

Many types of neural network models have been described in the literature [26]. A majority of applications to date use some form of Multi-Layer Perceptron (MLP) due to its universal approximation and the compactness of its representation. In this study, an MLP neural network model with feed forward architecture was used as it can better distinguish data that are not linearly separable [26].

The training of a Multi-Layer Perceptron is an iterative process and consists in obtaining the unknown weights wmk and biases bm required for the LSF approximation. The initial weights and biases are set to random values and are subsequently updated by the training algorithm. For this purpose, the training data set with input and target values must be previously prepared. The set is then divided into two sub-sets: (1) the training sub-set which is used for updating the weights and biases and (2) the validation sub-set, used for stopping the training when the network performance fails to improve for previously specified number of iterations or for checking the network approximating capacities. The iterative training algorithm performs an error minimization procedure that is repeated until the network outputs converge to the target values [27].

The default error function used for training feed forward networks is the mean squared error (MSE). The average squared difference between the network output values si and the target outputs ti, defined as:

(2)

 

2-4- Evaluation

Before proceeding to final deployment of the model, it is important to more thoroughly evaluate the model and review the steps executed to construct the model to be certain it properly achieves the business objectives. A key objective is to determine if there is some important business issue that has not been sufficiently considered. Accuracy, sensitivity, and specificity are the most important performance measures in evaluation of classification methods [14], which are commonly used in the literature. So for measuring the performance of used algorithms ANN alone and our proposed hybrid approach based on SOM and ANN, these measures are used. The computation of these measures is based on confusion matrix.

A confusion matrix is a table that allows visualization of the performance of an algorithm. In a k class problem (with classes ,  and ), the matrix has i rows and i columns that specifies the following parameters:

·       True positive (): The number of sample that is correctly classified into the ith class;

·       False positive (): The number of samples being wrongly classified into the ith class;

·       True negative (): The number of outer samples that is correctly classified;

·       False negative (): The number of ith class samples which is wrongly classified into the other classes;

Therefore, the Accuracy, sensitivity, and specificity measures are defined as follows:

(3)

On the other hand, classifier is evaluated with 10-fold cross validation, which is a technique for estimating the performance of a classifier. First, the original samples are randomly partitioned into 10 subsets. Secondly, one subset is singled out to be the testing data and the remaining 9 subsets are treated as training data. Afterwards, the cross validation process repeat 10 times and the estimation accuracy of the classifier can be evaluated by the average accuracy of the ten estimations. The 10-fold cross validation is more popular in the circumstances of huge data set, compared with the Leave-one-out cross validation. The latter is usually very time expensive according to the high complexity of training times [28].

 

3- Experimental setup

3-1- Data gathering and preparing

In this research registered ADSL faults of a reputable Iranian Telecommunication company was studied. North Khorasan Telecommunication Company is a governmental company which provides various telecommunication services, but ADSL services involve a major volume of its market. Telecommunication Company of Iran (TCI) offers three different types of ADSL services with different specifications to cover the various needs of its customers. These services differ in sharing type of bandwidth, total data transmission limitation and billing method.

Used data set includes 2239 occurred faults from 20 March 2012 in ADSL services which registered by users. For each ADSL fault registers 14 characteristics that are shown in TABLE 1.

 

TABLE 1. Description of features in ADSL faults dataset

(4)

ِDescription	Attribute
User's name	A1
User's telephone number.	A2
Service name	A3
Center name	A4
Row	A5
Terminal	A6
Port	A7
Port state	A8
User address	A9
User mobile number.	A10
Register date	A11
Registrar	A12
Fault title	A13
Fault description	A14

 

The data sources are first located, accessed, and integrated. Next, selected data is put into a tabular format in which instances and variables take place in rows and columns, respectively. A labeling is used for a better understanding of results. By means of this labeling, a number is assigned to each registered fault.

Since the input to the data mining model affects the choice of a data mining algorithm and the results, we attempted to remove polluted data such as incorrectly coded input (e.g. typos) or inconsistent input (e.g. outliers or anomalous answers) from the database by filtering out the Excel file. Effective (2192) faults were amassed other than 47 invalid pieces containing omissions and incomplete registered faults.

 

3-2- SOM Training and Evaluation

One batch SOM algorithm is used for designing SOM and training it. Because it is more stable than the online version, as its results do not depend on the order in which the input patterns are presented to it. Therefore, the integration of the batch SOM in an industrial environment is more feasible, since it is a deterministic and reproducible algorithm. In addition, the batch version is faster and can be parallelized to reduce computational time.

All the simulations were programmed using Matlab 2014a and the specific Matlab SOM Toolbox [29]. See [30] for further references on the design of SOM networks.

Before the training process begins, data normalization is often performed. The linear transformation formula to [0, 1] is used:

Where  and  represent the normalized and original data; and  represent the maximum values among the original data.

The determination of the size of SOM is not an easy task, because a network with a greater number of cells would have hindered the visualization of the labels in each neuron. In the same way, a smaller map than the one used by the authors would cause many labels to be overlapped. It is recommended [31] that the number of neurons in the map training set and the length and width of the map should be proportional to the magnitude of the first eigenvalues obtained by the decomposition of the training set . The ratio calculated by the first two calculated eigenvalues in this case is 1.96. Having that in mind (and also the heuristic method in SOM toolbar of MATLAB[29]) we started the search for the optimal size of the map. After several trials, we chose a map with a size 12×20. The used map had plain boundary conditions, a hexagonal grid, Gaussian neighborhood function, and linearly decreasing learning rate.

The SOMs were trained using the batch training algorithm in two phases: (1) rough training phase which lasted 1000 iterations with an initial neighborhood radius equal to 5, a final neighborhood radius equal to 2, While a learning rate starts at 0.5 and end at 0.1, and (2) fine training phase which lasted 500 iteration cycles, While a learning rate starts at 0.1 and end at 0.02, we do set the neighborhood partitions started at 2 and end with 0. 

 

3-3- MLP Training

The clustered failure data is transferred to MLP. In this section we use NN toolbar in MATLAB software to classify faults and to predict new faults. Before the training process begins, data normalization is often performed. The linear transformation formula to [0, 1] is used. The MLP is executed by different properties which the follow parameter settings result to better performance. Therefore we use them in this study.

·       Network input: fault description.

·       Network output: fault title.

·       Network type: Feed-forward back-propagation.

·       Training function: Levenberg–Marquardt algorithm

·       (TRAINLM).

·       Adaptation learning function: Gradient descent with

·       Momentum weight/bias learning function (LEARNGDM).

·       Performance function: Mean square error (MSE).

·       Number of layers: 3 (layer-1: five neurons and TANSIG

·       transfer function; layer-2: three neurons and LOGSIG

·       Transfer function; output layer: PURELIN transfer function).

 

4- Results and discussion

After topological relationships among neurons after completion of the learning process, the U-matrix is achieved (see FIG. 4). The main purpose of U-matrix is to visualize the distances between each neuron and all its neighboring neurons revealing the local cluster structure of the map through colorcoding. The 2129 occurred ADSL faults can be segmented to 4 follows clusters based on U-matrix that it is shown in FIG. 4.

a)       Faults that user’s location is important for them: These ADSL faults from specified tell center is reported and they have same first digit in their tell number. User that report these ADSL faults usually scatter in a specified geographic region based on tell center (cluster 1). Cluster 1 has the least number of 1052 (1052/2129=49.41%) faults. They have the highest complaint rating and it can damage provider ADSL service organization/ user relationship in the long-term. An implication for this extracted phenomenon is that provider ADSL service organizations can decrease complaints by transport their center to another one and replace new equipment to previous equipment and sending themselves detail of these changes to them. 

b)       Users that quality of services is important for them: Based on the information from U-matrix, Clusters 3 has the lowest number of defaults. It has 202 (202/2129=9.48%) defaults, while the highest monetary rating, 0.440, is gained from its users since their financial value of selected ADSL services is high. They choice an ADSL service as their data communications which has a broad band with high quality in developed services. Moreover, they are loyal both in attitude and behavior in the ADSL services. Loyal users are profitable as they would contribute positively to the success of their ADSL provider organization experience. Therefore, the ADSL provider organizations should focus on the occurred faults of this cluster by increasing loyalty through providing customized services to encourage their future purchase.

c)       Customers that style of usage caused to problem for them. For Cluster 2, the average of start date of their first ADSL services and their port number is above the total average and they are new users which are unfamiliar to use ADSL technology. Cluster 2 consists 875 (875/2129=41%) faults that their corresponding users have different attributes. They could have the highest loyalty rating if ADSL provider organization/ users relationship maintains in the long-term. In this regard, the users which have announced faults of Cluster 2 can attract new users. Since according to the so-called “20–80” rule, a dramatic business improvement is often achieved by identifying the 20% of core customers and by maximizing the attention applied to them, they will attract the 80% of customers. Therefore, satisfying existing users’ needs and build close relationships with them will be very imperative.

 

 

 

FIG. 4. The U-Matrix achieved by SOM

 

To validate the clustering ability of the SOM, we applied a 10-fold cross-validation for used clustering methods (KM, SOM, AVQ, SVC) using the 3 clusters. The cluster quality was measured using DBI validity; there were 10 DBI values over the 10-fold cross-validation procedure. We used the average DBI (shown in TABLE 2) for any clustering method as its quality.

 

 

TABLE 2. The DBI values to number of clusters 3 using the KM, SOM, AVQ AND SVC in used dataset 

(5)

Method	DBI
k-means	0.93
SOM	0.90
AVQ	1.23
SVC	1.08

 

The DBI value for these four methods indicate SOM has the lowest DBI and as it has highest cluster quality. In general, however, the overall performance of SOM is better from others in this dataset. Therefore, we divide the whole dataset of faults into alike fault pattern sets based on a similarity criterion by using of SOM method and feed them to MLP for more precise classification.

The designed MLP is trained based on specified parameters in previous section. By using of Eq. (3) and average results of 10-fold validation, MSE calculated as 0.04425. These low amounts for MSE can demonstrate the appropriateness and precise of our modeling and forecasting. Whereas, alone MLP results 0.09641 as MSE measurement. Therefore, the proposed approach based on SOM and MLP has better classification precision.

Second experiment is for detecting new types of fault during the monitoring process. 438 new observations of ADSL faults (1-6) have been monitored and classified as new type of fault. According to TABLE 3, 414 of 438 new observations are correctly classified as new faults, i.e., the classification error rate of the classifier is 5.48%. Therefore, the proposed hybrid approach based on SOM and MLP can detect new types of ADSL faults with high performance (correct classification rate of 94.52%) without reducing the performance of discriminating between known fault classes (misclassification rate has increased from 22.58% to 23.21%).

 

TABLE 3. Confusion matrix of proposed hybrid approach based on SOM and MLP for ADSL

67	2	1	2	0	1
1	69	0	1	1	1
1	0	70	0	0	2
2	1	0	68	0	0
0	1	2	2	71	0
3	0	0	0	1	69

 

 

 

 

 

5- Conclusion

We developed expert system based on Artificial Neural Network (ANN) use a MLP with clustered data by using of SOM for fault diagnosis in ADSL services. The 10-fold cross validation for DBI value and MSE value is used for evaluating of SOM and MLP respectively. Low amounts for resulted DBI value for SOM compared with other used clustering methods (KM, SOM, AVQ and SVC) demonstrate the appropriateness of SOM to divide ADSL faults dataset faults into alike fault pattern sets. Moreover, the lower MSE value for MLP on clustered data compared with whole data demonstrate better forecasting of our proposed hybrid approach.

In this study, we regard 6 ADSL’ faults, whereas variety of ADSL faults is high. Supposing more ADSL faults can be subject of future research. Moreover, we developed a new proposed approach based on MLP and SOM to data classify, but it is has evaluated by using it to diagnosis faults of ADSL. Whereas it is not sufficient to evaluate performance of a new proposed approach. Therefore, performance evaluation of proposed approach based on MLP and SOM by using it in other fields can regard as future researches.

 

6- References

1        McKay, H.: ‘Development of the contractual relationship between an ISP and its customers–Is a fairer deal in sight?’, Computer Law & Security Review, 2005, 21, (3), pp. 216-225

2        Carrera, أ., Iglesias, C.A., Garcأa-Algarra, J., and Kolaإ™أk, D.a.: ‘A real-life application of multi-agent systems for fault diagnosis in the provision of an Internet business service’, Journal of Network and Computer Applications, 2014, 37, pp. 146-154

3        Hanafizadeh, P., and Mirzazadeh, M.: ‘Visualizing market segmentation using self-organizing maps and Fuzzy Delphi method–ADSL market of a telecommunication company’, Expert Systems with Applications, 2011, 38, (1), pp. 198-205

4        Gao, Y., Liu, S., Li, F., and Liu, Z.: ‘Fault detection and diagnosis method for cooling dehumidifier based on LS-SVM NARX model’, International Journal of Refrigeration, 2015

5        Muralidharan, V., Sugumaran, V., and Indira, V.: ‘Fault diagnosis of monoblock centrifugal pump using SVM’, Engineering Science and Technology, an International Journal, 2014, 17, (3), pp. 152-157

6        Hakim, S.J.S., Razak, H.A., and Ravanfar, S.A.: ‘Fault diagnosis on beam-like structures from modal parameters using artificial neural networks’, Measurement, 2015, 76, pp. 45-61

7        Lashkari, N., Poshtan, J., and Azgomi, H.F.: ‘Simulative and experimental investigation on stator winding turn and unbalanced supply voltage fault diagnosis in induction motors using Artificial Neural Networks’, ISA Transactions, 2015

8        Lau, C.K., Ghosh, K., Hussain, M.A., and Hassan, C.R.C.: ‘Fault diagnosis of Tennessee Eastman process with multi-scale PCA and ANFIS’, Chemometrics and Intelligent Laboratory Systems, 2013, 120, pp. 1-14

9        Mnassri, B., and Ouladsine, M.: ‘Reconstruction-based contribution approaches for improved fault diagnosis using principal component analysis’, Journal of Process Control, 2015, 33, pp. 60-76

10        Huang, J., and Yan, X.: ‘Dynamic process fault detection and diagnosis based on dynamic principal component analysis, dynamic independent component analysis and Bayesian inference’, Chemometrics and Intelligent Laboratory Systems, 2015

11        Zhao, Y., Wen, J., and Wang, S.: ‘Diagnostic Bayesian networks for diagnosing air handling units faults–Part II: Faults in coils and sensors’, Applied Thermal Engineering, 2015, 90, pp. 145-157

12        Zhang, Z., and Dong, F.: ‘Fault detection and diagnosis for missing data systems with a three time-slice dynamic Bayesian network approach’, Chemometrics and Intelligent Laboratory Systems, 2014, 138, pp. 30-40

13        Eslamloueyan, R.: ‘Designing a hierarchical neural network based on fuzzy clustering for fault diagnosis of the Tennessee–Eastman process’, Applied soft computing, 2011, 11, (1), pp. 1407-1415

14        Rad, M.A.A., and Yazdanpanah, M.J.: ‘Designing supervised local neural network classifiers based on EM clustering for fault diagnosis of Tennessee Eastman process’, Chemometrics and Intelligent Laboratory Systems, 2015

15        Mendoza, O., Melأ­n, P., and Castillo, O.: ‘Interval type-2 fuzzy logic and modular neural networks for face recognition applications’, Applied soft computing, 2009, 9, (4), pp. 1377-1387

16        Sanchez, D., Melin, P., and Castillo, O.: ‘Optimization of modular granular neural networks using a hierarchical genetic algorithm based on the database complexity applied to human recognition’, Information Sciences, 2015, 309, pp. 73-101

17        Sanchez, D., and Melin, P.: ‘Optimization of modular granular neural networks using hierarchical genetic algorithms for human recognition using the ear biometric measure’, Engineering Applications of Artificial Intelligence, 2014, 27, pp. 41-56

18        Chicco, G.: ‘Overview and performance assessment of the clustering methods for electrical load pattern grouping’, Energy, 2012, 42, (1), pp. 68-80

19        Lopez, J.J., Aguado, J.A., Martin, F., Munoz, F., Rodriguez, A., and Ruiz, J.E.: ‘Hopfield K-Means clustering algorithm: A proposal for the segmentation of electricity customers’, Electric Power Systems Research, 2011, 81, (2), pp. 716-724

20        Guerrero, J.D.M., Marcelli, D., Soria-Olivas, E., Mari, F., Martinez-Martأnez, J.M., Bech, I.S., Martinez-Sober, M., Scatizzi, L., Gimez-Sanchis, J., and Stopper, A.: ‘Self-Organising Maps: A new way to screen the level of satisfaction of dialysis patients’, Expert systems with applications, 2012, 39, (10), pp. 8793-8798

21        Segev, A., and Kantola, J.: ‘Identification of trends from patents using self-organizing maps’, Expert systems with applications, 2012, 39, (18), pp. 13235-13242

22        Davies, D.L., and Bouldin, D.W.: ‘A cluster separation measure’, Pattern Analysis and Machine Intelligence, IEEE Transactions on, 1979, (2), pp. 224-227

23        Szymczyk, P., and Szymczyk, M.: ‘Classification of geological structure using ground penetrating radar and Laplace transform artificial neural networks’, Neurocomputing, 2015, 148, pp. 354-362

24        Prakash, M., Pradhan, S., and Roy, S.: ‘Soft computing techniques for fault detection in power distribution systems: A review’, in Editor (Ed.)^(Eds.): ‘Book Soft computing techniques for fault detection in power distribution systems: A review’ (IEEE, 2014, edn.), pp. 1-6

25        Chojaczyk, A.A., Teixeira, A.P., Neves, L.C., Cardoso, J.B., and Soares, C.G.: ‘Review and application of Artificial Neural Networks models in reliability analysis of steel structures’, Structural Safety, 2015, 52, pp. 78-89

26        Palanichamy, A., Jayas, D.S., and Holley, R.A.: ‘Predicting survival of Escherichia coli O157: H7 in dry fermented sausage using artificial neural networks’, Journal of Food Protectionآ®, 2008, 71, (1), pp. 6-12

27        Gosukonda, R., Mahapatra, A.K., Liu, X., and Kannan, G.: ‘Application of artificial neural network to predict< i> Escherichia coli</i> O157: H7 inactivation on beef surfaces’, Food Control, 2014, 47, pp. 606-614

28        Li, Y., Xia, J., Zhang, S., Yan, J., Ai, X., and Dai, K.: ‘An efficient intrusion detection system based on support vector machines and gradually feature removal method’, Expert Systems with Applications, 2012, 39, pp. 424–430

29        Esa Alhoniemi, J.H., Jukka Parviainen and Juha Vesanto: ‘Som toolbox for matlab’, in Editor (Ed.)^(Eds.): ‘Book Som toolbox for matlab’ (1999, edn.), pp.

30        Vesanto, J., Himberg, J., Alhoniemi, E., and Parhankangas, J.: ‘Self-organizing map in Matlab: the SOM Toolbox’, in Editor (Ed.)^(Eds.): ‘Book Self-organizing map in Matlab: the SOM Toolbox’ (1999, edn.), pp. 16-17

31        Kuzmanovskia., I., Trpkovskaa., M., and Soptrajanov, B.: ‘Optimization of supervised self-organizing maps with genetic algorithms for classification of urinary calculi’, Journal of Molecular Structure, 2005, 744–747, pp. 833–838