Search results

The purpose of this paper is to demonstrate the applicability of swarm and evolutionary techniques for regularized machine learning. Generally, by defining a proper penalty function, regularization laws are embedded into the structure of common least square solutions to increase the numerical stability, sparsity, accuracy and robustness of regression weights. Several regularization techniques have been proposed so far which have their own advantages and disadvantages. Several efforts have been made to find fast and accurate deterministic solvers to handle those regularization techniques. However, the proposed numerical and deterministic approaches need certain knowledge of mathematical programming, and also do not guarantee the global optimality of the obtained solution. In this research, the authors propose the use of constraint swarm and evolutionary techniques to cope with demanding requirements of regularized extreme learning machine (ELM).

To implement the required tools for comparative numerical study, three steps are taken. The considered algorithms contain both classical and swarm and evolutionary approaches. For the classical regularization techniques, Lasso regularization, Tikhonov regularization, cascade Lasso-Tikhonov regularization, and elastic net are considered. For swarm and evolutionary-based regularization, an efficient constraint handling technique known as self-adaptive penalty function constraint handling is considered, and its algorithmic structure is modified so that it can efficiently perform the regularized learning. Several well-known metaheuristics are considered to check the generalization capability of the proposed scheme. To test the efficacy of the proposed constraint evolutionary-based regularization technique, a wide range of regression problems are used. Besides, the proposed framework is applied to a real-life identification problem, i.e. identifying the dominant factors affecting the hydrocarbon emissions of an automotive engine, for further assurance on the performance of the proposed scheme.

Through extensive numerical study, it is observed that the proposed scheme can be easily used for regularized machine learning. It is indicated that by defining a proper objective function and considering an appropriate penalty function, near global optimum values of regressors can be easily obtained. The results attest the high potentials of swarm and evolutionary techniques for fast, accurate and robust regularized machine learning.

The originality of the research paper lies behind the use of a novel constraint metaheuristic computing scheme which can be used for effective regularized optimally pruned extreme learning machine (OP-ELM). The self-adaption of the proposed method alleviates the user from the knowledge of the underlying system, and also increases the degree of the automation of OP-ELM. Besides, by using different types of metaheuristics, it is demonstrated that the proposed methodology is a general flexible scheme, and can be combined with different types of swarm and evolutionary-based optimization techniques to form a regularized machine learning approach.

The sources of magnetic sensors errors are numerous, such as currents around, soft magnetic and hard magnetic materials and so on. The traditional methods mainly use explicit error models, and it is difficult to include all interference factors. This paper aims to present an implicit error model and studies its high-precision training method.

A multi-level extreme learning machine based on reverse tuning (MR-ELM) is presented to compensate for magnetic compass measurement errors by increasing the depth of the network. To ensure the real-time performance of the algorithm, the network structure is fixed to two ELM levels, and the maximum number of levels and neurons will not be continuously increased. The parameters of MR-ELM are further modified by reverse tuning to ensure network accuracy. Because the parameters of the network have been basically determined by least squares, the number of iterations is far less than that in the traditional BP neural network, and the real-time can still be guaranteed.

The results show that the training time of the MR-ELM is 19.65 s, which is about four times that of the fixed extreme learning algorithm, but training accuracy and generalization performance of the error model are better. The heading error is reduced from the pre-compensation ±2.5° to ±0.125°, and the root mean square error is 0.055°, which is about 0.46 times that of the fixed extreme learning algorithm.

MR-ELM is presented to compensate for magnetic compass measurement errors by increasing the depth of the network. In this case, the multi-level ELM network parameters are further modified by reverse tuning to ensure network accuracy. Because the parameters of the network have been basically determined by least squares, the number of iterations is far less than that in the traditional BP neural network, and the real-time training can still be guaranteed. The revised manuscript improved the ELM algorithm itself (referred to as MR-ELM) and bring new ideas to the peers in the magnetic compass error compensation field.

Forecasting of stock indices is a challenging issue because stock data are dynamic, non-linear and uncertain in nature. Selection of an accurate forecasting model is very much essential to predict the next-day closing prices of the stock indices. The purpose of this paper is to develop an efficient and accurate forecasting model to predict the next-day closing prices of seven stock indices.

A novel strategy called quasi-oppositional symbiotic organisms search-based extreme learning machine (QSOS-ELM) is proposed to forecast the next-day closing prices effectively. Accuracy in the prediction of closing price depends on output weights which are dependent on input weights and biases. This paper mainly deals with the optimal design of input weights and biases of the ELM prediction model using QSOS and SOS optimization algorithms.

Simulation is carried out on seven stock indices, and performance analysis of QSOS-ELM and SOS-ELM prediction models is done by taking various statistical measures such as mean square error, mean absolute percentage error, accuracy and paired sample t-test. Comparative performance analysis reveals that the QSOS-ELM model outperforms the SOS-ELM model in predicting the next-day closing prices more accurately for all the seven stock indices under study.

The QSOS-ELM prediction model and SOS-ELM are developed for the first time to predict the next-day closing prices of various stock indices. The paired t-test is also carried out for the first time in literature to hypothetically prove that there is a zero mean difference between the predicted and actual closing prices.

The purpose of this paper is to improve the control precision of the station-keeping control for a stratosphere airship through the feedforward-feedback PID controller which is designed by the wind speed prediction based on the incremental extreme learning machine (I-ELM).

First of all, the online prediction of wind speed is implemented by the I-ELM with rolling time. Second, the feedforward-feedback PID controller is designed through the position information of the airship and the predicted wind speed. In the end, the one-dimensional dynamic model of the stratosphere airship is built, and the controller is applied in the numerical simulation.

Based on the conducted numerical simulations, some valuable conclusions are obtained. First, through the comparison between the predicted value and true value of the wind speed, the wind speed prediction based on I-ELM is very accurate. Second, the feedforward-feedback PID controller designed in this paper is very effective.

This paper is very valuable to the research of a high-accuracy station-keeping control of stratosphere airship.

The purpose of this paper is to conduct a comprehensive review of the noteworthy contributions made in the area of the Feedforward neural network (FNN) to improve its generalization performance and convergence rate (learning speed); to identify new research directions that will help researchers to design new, simple and efficient algorithms and users to implement optimal designed FNNs for solving complex problems; and to explore the wide applications of the reviewed FNN algorithms in solving real-world management, engineering and health sciences problems and demonstrate the advantages of these algorithms in enhancing decision making for practical operations.

The FNN has gained much popularity during the last three decades. Therefore, the authors have focused on algorithms proposed during the last three decades. The selected databases were searched with popular keywords: “generalization performance,” “learning rate,” “overfitting” and “fixed and cascade architecture.” Combinations of the keywords were also used to get more relevant results. Duplicated articles in the databases, non-English language, and matched keywords but out of scope, were discarded.

The authors studied a total of 80 articles and classified them into six categories according to the nature of the algorithms proposed in these articles which aimed at improving the generalization performance and convergence rate of FNNs. To review and discuss all the six categories would result in the paper being too long. Therefore, the authors further divided the six categories into two parts (i.e. Part I and Part II). The current paper, Part I, investigates two categories that focus on learning algorithms (i.e. gradient learning algorithms for network training and gradient-free learning algorithms). Furthermore, the remaining four categories which mainly explore optimization techniques are reviewed in Part II (i.e. optimization algorithms for learning rate, bias and variance (underfitting and overfitting) minimization algorithms, constructive topology neural networks and metaheuristic search algorithms). For the sake of simplicity, the paper entitled “Machine learning facilitated business intelligence (Part II): Neural networks optimization techniques and applications” is referred to as Part II. This results in a division of 80 articles into 38 and 42 for Part I and Part II, respectively. After discussing the FNN algorithms with their technical merits and limitations, along with real-world management, engineering and health sciences applications for each individual category, the authors suggest seven (three in Part I and other four in Part II) new future directions which can contribute to strengthening the literature.

The FNN contributions are numerous and cannot be covered in a single study. The authors remain focused on learning algorithms and optimization techniques, along with their application to real-world problems, proposing to improve the generalization performance and convergence rate of FNNs with the characteristics of computing optimal hyperparameters, connection weights, hidden units, selecting an appropriate network architecture rather than trial and error approaches and avoiding overfitting.

This study will help researchers and practitioners to deeply understand the existing algorithms merits of FNNs with limitations, research gaps, application areas and changes in research studies in the last three decades. Moreover, the user, after having in-depth knowledge by understanding the applications of algorithms in the real world, may apply appropriate FNN algorithms to get optimal results in the shortest possible time, with less effort, for their specific application area problems.

The existing literature surveys are limited in scope due to comparative study of the algorithms, studying algorithms application areas and focusing on specific techniques. This implies that the existing surveys are focused on studying some specific algorithms or their applications (e.g. pruning algorithms, constructive algorithms, etc.). In this work, the authors propose a comprehensive review of different categories, along with their real-world applications, that may affect FNN generalization performance and convergence rate. This makes the classification scheme novel and significant.

The purpose of this paper is three-fold: to review the categories explaining mainly optimization algorithms (techniques) in that needed to improve the generalization performance and learning speed of the Feedforward Neural Network (FNN); to discover the change in research trends by analyzing all six categories (i.e. gradient learning algorithms for network training, gradient free learning algorithms, optimization algorithms for learning rate, bias and variance (underfitting and overfitting) minimization algorithms, constructive topology neural networks, metaheuristic search algorithms) collectively; and recommend new research directions for researchers and facilitate users to understand algorithms real-world applications in solving complex management, engineering and health sciences problems.

The FNN has gained much attention from researchers to make a more informed decision in the last few decades. The literature survey is focused on the learning algorithms and the optimization techniques proposed in the last three decades. This paper (Part II) is an extension of Part I. For the sake of simplicity, the paper entitled “Machine learning facilitated business intelligence (Part I): Neural networks learning algorithms and applications” is referred to as Part I. To make the study consistent with Part I, the approach and survey methodology in this paper are kept similar to those in Part I.

Combining the work performed in Part I, the authors studied a total of 80 articles through popular keywords searching. The FNN learning algorithms and optimization techniques identified in the selected literature are classified into six categories based on their problem identification, mathematical model, technical reasoning and proposed solution. Previously, in Part I, the two categories focusing on the learning algorithms (i.e. gradient learning algorithms for network training, gradient free learning algorithms) are reviewed with their real-world applications in management, engineering, and health sciences. Therefore, in the current paper, Part II, the remaining four categories, exploring optimization techniques (i.e. optimization algorithms for learning rate, bias and variance (underfitting and overfitting) minimization algorithms, constructive topology neural networks, metaheuristic search algorithms) are studied in detail. The algorithm explanation is made enriched by discussing their technical merits, limitations, and applications in their respective categories. Finally, the authors recommend future new research directions which can contribute to strengthening the literature.

The FNN contributions are rapidly increasing because of its ability to make reliably informed decisions. Like learning algorithms, reviewed in Part I, the focus is to enrich the comprehensive study by reviewing remaining categories focusing on the optimization techniques. However, future efforts may be needed to incorporate other algorithms into identified six categories or suggest new category to continuously monitor the shift in the research trends.

The authors studied the shift in research trend for three decades by collectively analyzing the learning algorithms and optimization techniques with their applications. This may help researchers to identify future research gaps to improve the generalization performance and learning speed, and user to understand the applications areas of the FNN. For instance, research contribution in FNN in the last three decades has changed from complex gradient-based algorithms to gradient free algorithms, trial and error hidden units fixed topology approach to cascade topology, hyperparameters initial guess to analytically calculation and converging algorithms at a global minimum rather than the local minimum.

The existing literature surveys include comparative study of the algorithms, identifying algorithms application areas and focusing on specific techniques in that it may not be able to identify algorithms categories, a shift in research trends over time, application area frequently analyzed, common research gaps and collective future directions. Part I and II attempts to overcome the existing literature surveys limitations by classifying articles into six categories covering a wide range of algorithm proposed to improve the FNN generalization performance and convergence rate. The classification of algorithms into six categories helps to analyze the shift in research trend which makes the classification scheme significant and innovative.

This paper aims to develop a novel algorithm and apply it to solve two-dimensional linear partial differential equations (PDEs). The proposed method is based on Chebyshev neural network and extreme learning machine (ELM) called Chebyshev extreme learning machine (Ch-ELM) method.

The network used in the proposed method is a single hidden layer feedforward neural network. The Kronecker product of two Chebyshev polynomials is used as basis function. The weights from the input layer to the hidden layer are fixed value 1. The weights from the hidden layer to the output layer can be obtained by using ELM algorithm to solve the linear equations established by PDEs and its definite conditions.

To verify the effectiveness of the proposed method, two-dimensional linear PDEs are selected and its numerical solutions are obtained by using the proposed method. The effectiveness of the proposed method is illustrated by comparing with the analytical solutions, and its superiority is illustrated by comparing with other existing algorithms.

Ch-ELM algorithm for solving two-dimensional linear PDEs is proposed. The algorithm has fast execution speed and high numerical accuracy.

The purpose of this paper is to study the problem of fashion flat sketches classification and proposed an integrated approach. It aims to propose a fast, reliable method to handle multi-class fashion flat sketches classification problems and lay the foundation for the garment style query in the next step.

The proposed integrated approach adopts wavelet Fourier descriptor (WFD), linear discriminant analysis (LDA) and extreme learning machine (ELM). First, the discrete wavelet and Fourier transform are adopted to extract the shape features of fashion flat sketches. Then, LDA is employed for multi-class classification to reduce dimensionality. Finally, ELM is used as the classifier.

The experimental results show that the classification accuracy of the integrated approach is obtained at about 100 percent. Contrary to the traditional approaches, efficiency and accuracy are the advantages of the present approach.

Fashion concept is conveyed often in the form of the fashion illustration or sketch. This type of sketch is useful to imply the style and overall feel of the design. However, this sketch gives no clue about the parts or sections that make up each garment. For this reason, this paper only studies the classification of flat sketches.

A new shape descriptor named WFD is proposed. The WFD acquires high classification accuracy comparing with Fourier descriptor (FD) and multiscale Fourier descriptor (MFD) without dimensionality reduction and nearly the same classification accuracy comparing with FD while MFD easily causes small sample size problem with dimensionality reduction using LDA. In addition, ELM is first used as the classifier in the textiles field related to the classification problem.

Road traffic emissions are generally believed to contribute immensely to air pollution, but the effect of road traffic data sets on air quality (AQ) predictions has not been fully investigated. This paper aims to investigate the effects traffic data set have on the performance of machine learning (ML) predictive models in AQ prediction.

To achieve this, the authors have set up an experiment with the control data set having only the AQ data set and meteorological (Met) data set, while the experimental data set is made up of the AQ data set, Met data set and traffic data set. Several ML models (such as extra trees regressor, eXtreme gradient boosting regressor, random forest regressor, K-neighbors regressor and two others) were trained, tested and compared on these individual combinations of data sets to predict the volume of PM_2.5, PM₁₀, NO₂ and O₃ in the atmosphere at various times of the day.

The result obtained showed that various ML algorithms react differently to the traffic data set despite generally contributing to the performance improvement of all the ML algorithms considered in this study by at least 20% and an error reduction of at least 18.97%.

This research is limited in terms of the study area, and the result cannot be generalized outside of the UK as some of the inherent conditions may not be similar elsewhere. Additionally, only the ML algorithms commonly used in literature are considered in this research, therefore, leaving out a few other ML algorithms.

This study reinforces the belief that the traffic data set has a significant effect on improving the performance of air pollution ML prediction models. Hence, there is an indication that ML algorithms behave differently when trained with a form of traffic data set in the development of an AQ prediction model. This implies that developers and researchers in AQ prediction need to identify the ML algorithms that behave in their best interest before implementation.

The result of this study will enable researchers to focus more on algorithms of benefit when using traffic data sets in AQ prediction.

The purpose of this paper is to make compliant training control of exoskeleton for ankle joint with electromyograph (EMG)-torque interface.

A virtual compliant mapping which is modeled by mass-spring-damper system is incorporated into the whole system at the reference input. The EMG-torque interface contains both data acquisition and torque estimator/predictor, and extreme learning machine is utilized for joint torque estimation/prediction from multiple channels of EMG signals.

The reference ankle joint angle to follow is produced from the compliance mapping whose input is the measured/predicted torque on healthy subjects. The control system works well with the desired angle to track. In the actuation level, the input torque to drive the ankle exoskeleton is less than the actual torque of the subject(s). This may have positive influence on diminishing overshoot of input torque from motors and protect the actuators. The torque prediction and final tracking control performance demonstrate the efficiency of the presented architecture.

This work can be beneficial to compliant training of ankle exoskeleton system for pilots and enhance current training control module in rehabilitation.