Search results

The aim of this study is to propose a deep neural network (DNN) method that uses side information to improve clustering results for big datasets; also, the authors show that applying this information improves the performance of clustering and also increase the speed of the network training convergence.

In data mining, semisupervised learning is an interesting approach because good performance can be achieved with a small subset of labeled data; one reason is that the data labeling is expensive, and semisupervised learning does not need all labels. One type of semisupervised learning is constrained clustering; this type of learning does not use class labels for clustering. Instead, it uses information of some pairs of instances (side information), and these instances maybe are in the same cluster (must-link [ML]) or in different clusters (cannot-link [CL]). Constrained clustering was studied extensively; however, little works have focused on constrained clustering for big datasets. In this paper, the authors have presented a constrained clustering for big datasets, and the method uses a DNN. The authors inject the constraints (ML and CL) to this DNN to promote the clustering performance and call it constrained deep embedded clustering (CDEC). In this manner, an autoencoder was implemented to elicit informative low dimensional features in the latent space and then retrain the encoder network using a proposed Kullback–Leibler divergence objective function, which captures the constraints in order to cluster the projected samples. The proposed CDEC has been compared with the adversarial autoencoder, constrained 1-spectral clustering and autoencoder + k-means was applied to the known MNIST, Reuters-10k and USPS datasets, and their performance were assessed in terms of clustering accuracy. Empirical results confirmed the statistical superiority of CDEC in terms of clustering accuracy to the counterparts.

First of all, this is the first DNN-constrained clustering that uses side information to improve the performance of clustering without using labels in big datasets with high dimension. Second, the author defined a formula to inject side information to the DNN. Third, the proposed method improves clustering performance and network convergence speed.

Little works have focused on constrained clustering for big datasets; also, the studies in DNNs for clustering, with specific loss function that simultaneously extract features and clustering the data, are rare. The method improves the performance of big data clustering without using labels, and it is important because the data labeling is expensive and time-consuming, especially for big datasets.

To overcome the shortcomings of traditional industry classification systems such as the Standard Industrial Classification Standard Industrial Classification, North American Industry Classification System North American Industry Classification System, and Global Industry Classification Standard Global Industry Classification Standard, the authors explore industry classifications using machine learning methods as an application of interpretable artificial intelligence (AI).

The authors propose a text-based industry classification combined with a machine learning technique by extracting distinguishable features from business descriptions in financial reports. The proposed method can reduce the dimensions of word vectors to avoid the curse of dimensionality when measuring the similarities of firms.

Using the proposed method, the sample firms form clusters of distinctive industries, thus overcoming the limitations of existing classifications. The method also clarifies industry boundaries based on lower-dimensional information. The graphical closeness between industries can reflect the industry-level relationship as well as the closeness between individual firms.

The authors’ work contributes to the industry classification literature by empirically investigating the effectiveness of machine learning methods. The text mining method resolves issues concerning the timeliness of traditional industry classifications by capturing new information in annual reports. In addition, the authors’ approach can solve the computing concerns of high dimensionality.

Vertical handover has been grown rapidly due to the mobility model improvements. These improvements are limited to certain circumstances and do not provide the support in the generic mobility, but offering vertical handover management in HetNets is very crucial and challenging. Therefore, this paper presents a vertical handoff management method using the effective network identification method.

This paper presents a vertical handoff management method using the effective network identification method. The handover triggering schemes are initially modeled to find the suitable position for starting handover using computed coverage area of the WLAN access point or cellular base station. Consequently, inappropriate networks are removed to determine the optimal network for performing the handover process. Accordingly, the network identification approach is introduced based on an adaptive particle-based Sailfish optimizer (APBSO). The APBSO is newly designed by incorporating self-adaptive particle swarm optimization (APSO) in Sailfish optimizer (SFO) and hence, modifying the update rule of the APBSO algorithm based on the location of the solutions in the past iterations. Also, the proposed APBSO is utilized for training deep-stacked autoencoder to choose the optimal weights. Several parameters, like end to end (E2E) delay, jitter, signal-to-interference-plus-noise ratio (SINR), packet loss, handover probability (HOP) are considered to find the best network.

The developed APBSO-based deep stacked autoencoder outperformed than other methods with a minimal delay of 11.37 ms, minimal HOP of 0.312, maximal stay time of 7.793 s and maximal throughput of 12.726 Mbps, respectively.

The network identification approach is introduced based on an APBSO. The APBSO is newly designed by incorporating self-APSO in SFO and hence, modifying the update rule of the APBSO algorithm based on the location of the solutions in the past iterations. Also, the proposed APBSO is used for training deep-stacked autoencoder to choose the optimal weights. Several parameters, like E2E delay, jitter, SINR, packet loss and HOP are considered to find the best network. The developed APBSO-based deep stacked autoencoder outperformed than other methods with minimal delay minimal HOP, maximal stay time and maximal throughput.

A cost-effective way to achieve fuel economy is to reinforce positive driving behaviour. Driving behaviour can be controlled if drivers can be alerted for behaviour that results in poor fuel economy. Fuel consumption must be tracked and monitored instantaneously rather than tracking average fuel economy for the entire trip duration. A single-step application of machine learning (ML) is not sufficient to model prediction of instantaneous fuel consumption and detection of anomalous fuel economy. The study designs an ML pipeline to track and monitor instantaneous fuel economy and detect anomalies.

This research iteratively applies different variations of a two-step ML pipeline to the driving dataset for hatchback cars. The first step addresses the problem of accurate measurement and prediction of fuel economy using time series driving data, and the second step detects abnormal fuel economy in relation to contextual information. Long short-term memory autoencoder method learns and uses the most salient features of time series data to build a regression model. The contextual anomaly is detected by following two approaches, kernel quantile estimator and one-class support vector machine. The kernel quantile estimator sets dynamic threshold for detecting anomalous behaviour. Any error beyond a threshold is classified as an anomaly. The one-class support vector machine learns training error pattern and applies the model to test data for anomaly detection. The two-step ML pipeline is further modified by replacing long short term memory autoencoder with gated recurrent network autoencoder, and the performance of both models is compared. The speed recommendations and feedback are issued to the driver based on detected anomalies for controlling aggressive behaviour.

A composite long short-term memory autoencoder was compared with gated recurrent unit autoencoder. Both models achieve prediction accuracy within a range of 98%–100% for prediction as a first step. Recall and accuracy metrics for anomaly detection using kernel quantile estimator remains within 98%–100%, whereas the one-class support vector machine approach performs within the range of 99.3%–100%.

The proposed approach does not consider socio-demographics or physiological information of drivers due to privacy concerns. However, it can be extended to correlate driver's physiological state such as fatigue, sleep and stress to correlate with driving behaviour and fuel economy. The anomaly detection approach here is limited to providing feedback to driver, it can be extended to give contextual feedback to the steering controller or throttle controller. In the future, a controller-based system can be associated with an anomaly detection approach to control the acceleration and braking action of the driver.

The suggested approach is helpful in monitoring and reinforcing fuel-economical driving behaviour among fleet drivers as per different environmental contexts. It can also be used as a training tool for improving driving efficiency for new drivers. It keeps drivers engaged positively by issuing a relevant warning for significant contextual anomalies and avoids issuing a warning for minor operational errors.

This paper contributes to the existing literature by providing an ML pipeline approach to track and monitor instantaneous fuel economy rather than relying on average fuel economy values. The approach is further extended to detect contextual driving behaviour anomalies and optimises fuel economy. The main contributions for this approach are as follows: (1) a prediction model is applied to fine-grained time series driving data to predict instantaneous fuel consumption. (2) Anomalous fuel economy is detected by comparing prediction error against a threshold and analysing error patterns based on contextual information.

Sampling taxpayers for audits has always been a major concern for policymakers of tax administration. The purpose of this study is to propose a systematic method to select a small number of taxpayers with a high probability of tax fraud.

An efficient sampling method for taxpayers for an audit is investigated in the context of a property acquisition tax. An autoencoder, a popular unsupervised learning algorithm, is applied to 2,228 tax returns, and reconstruction errors are calculated to determine the probability of tax deficiencies for each return. The reasonableness of the estimated reconstruction errors is verified using the Apriori algorithm, a well-known marketing tool for identifying patterns in purchased item sets.

The sorted reconstruction scores are reasonably consistent with actual fraudulent/non-fraudulent cases, indicating that the reconstruction errors can be utilized to select suspected taxpayers for an audit in a cost-effective manner.

The proposed deep learning-based approach is expected to be applied in a real-world tax administration, promoting voluntary compliance of taxpayers, and reinforcing the self-assessing acquisition tax system.

In the world of big data, data integration technology is crucial for maximising the capability of data-driven decision-making. Integrating data from multiple sources drastically expands the power of information and allows us to address questions that are impossible to answer using a single data source. Record Linkage (RL) is a task of identifying and linking records from multiple sources that describe the same real world object (e.g. person), and it plays a crucial role in the data integration process. RL is challenging, as it is uncommon for different data sources to share a unique identifier. Hence, the records must be matched based on the comparison of their corresponding values. Most of the existing RL techniques assume that records across different data sources are structured and represented by the same scheme (i.e. set of attributes). Given the increasing amount of heterogeneous data sources, those assumptions are rather unrealistic. The purpose of this paper is to propose a novel RL model for unstructured data.

In the previous work (Jurek-Loughrey, 2020), the authors proposed a novel approach to linking unstructured data based on the application of the Siamese Multilayer Perceptron model. It was demonstrated that the method performed on par with other approaches that make constraining assumptions regarding the data. This paper expands the previous work originally presented at iiWAS2020 [16] by exploring new architectures of the Siamese Neural Network, which improves the generalisation of the RL model and makes it less sensitive to parameter selection.

The experimental results confirm that the new Autoencoder-based architecture of the Siamese Neural Network obtains better results in comparison to the Siamese Multilayer Perceptron model proposed in (Jurek et al., 2020). Better results have been achieved in three out of four data sets. Furthermore, it has been demonstrated that the second proposed (hybrid) architecture based on integrating the Siamese Autoencoder with a Multilayer Perceptron model, makes the model more stable in terms of the parameter selection.

To address the problem of unstructured RL, this paper presents a new deep learning based approach to improve the generalisation of the Siamese Multilayer Preceptron model and make is less sensitive to parameter selection.

This study aims to propose a learning pattern analysis method which can improve a predictive model’s performance, as well as discover hidden insights into micro-level learning pattern. Analyzing student’s learning patterns can help instructors understand how their course design or activities shape learning behaviors; depict students’ beliefs about learning and their motivation; and predict learning performance by analyzing individual students’ learning patterns. Although time-series analysis is one of the most feasible predictive methods for learning pattern analysis, literature-indicated current approaches cannot provide holistic insights about learning patterns for personalized intervention. This study identified at-risk students by micro-level learning pattern analysis and detected pattern types, especially at-risk patterns that existed in the case study. The connections among students’ learning patterns, corresponding self-regulated learning (SRL) strategies and learning performance were finally revealed.

The method used long short-term memory (LSTM)-encoder to process micro-level behavioral patterns for feature extraction and compression, thus the students’ behavior pattern information were saved into encoded series. The encoded time-series data were then used for pattern analysis and performance prediction. Time series clustering were performed to interpret the unique strength of proposed method.

Successful students showed consistent participation levels and balanced behavioral frequency distributions. The successful students also adjusted learning behaviors to meet with course requirements accordingly. The three at-risk patten types showed the low-engagement (R1) the low-interaction (R2) and the non-persistent characteristics (R3). Successful students showed more complete SRL strategies than failed students. Political Science had higher at-risk chances in all three at-risk types. Computer Science, Earth Science and Economics showed higher chances of having R3 students.

The study identified multiple learning patterns which can lead to the at-risk situation. However, more studies are needed to validate whether the same at-risk types can be found in other educational settings. In addition, this case study found the distributions of at-risk types were vary in different subjects. The relationship between subjects and at-risk types is worth further investigation.

This study found the proposed method can effectively extract micro-level behavioral information to generate better prediction outcomes and depict student’s SRL learning strategies in online learning. The authors confirm that the research in their work is original, and that all the data given in the paper are real and authentic. The study has not been submitted to peer review and not has been accepted for publishing in another journal.

The purpose of this paper is to study the road segmentation problem of cross-modal remote sensing images.

First, the baseline network based on the U-net is trained under a large-scale dataset of remote sensing imagery. Then, the cross-modal training data are used to fine-tune the first two convolutional layers of the pre-trained network to achieve the adaptation to the local features of the cross-modal data. For the cross-modal data of different band, an autoencoder is designed to achieve data conversion and local feature extraction.

The experimental results show the effectiveness and practicability of the proposed method. Compared with the ordinary method, the proposed method gets much better metrics.

The originality is the transfer learning strategy that fine-tunes the low-level layers for the cross-modal data application. The proposed method can achieve satisfied road segmentation with a small amount of cross-modal training data, so that is has a good application value. Still, for the similar application of cross-modal data, the idea provided by this paper is helpful.

In novelty detection, the autoencoder based image reconstruction strategy is one of the mainstream solutions. The basic idea is that once the autoencoder is trained on normal data, it has a low reconstruction error on normal data. However, when faced with complex natural images, the conventional pixel-level reconstruction becomes poor and does not show the promising results. This paper aims to provide a new method for improving the performance of novelty detection based autoencoder.

To solve the problem that conventional pixel-level reconstruction cannot effectively extract the global semantic information of the image, a novel model with the combination of attention mechanism and self-supervised learning method is proposed. First, an auxiliary task, reconstruct rotated image, is set to enable the network to learn global semantic feature information. Then, the channel attention mechanism is introduced to perform adaptive feature refinement on the intermediate feature map to optimize the correspondingly passed feature map.

Experimental results on three public data sets show that the proposed method has potential performance for novelty detection.

This study explores the ability of self-supervised learning methods and attention mechanism to extract features on a single class of images. In this way, the performance of novelty detection can be improved.

The purpose of this study is to analyze the intelligent fault diagnosis method of rolling bearing.

The vibration signal data of rolling bearing has long time series and strong noise interference, which brings great difficulties for the accurate diagnosis of bearing faults. To solve those problems, an intelligent fault diagnosis model based on stacked denoising autoencoder (SDAE) and convolutional neural network (CNN) is proposed in this paper. The SDAE is used to process the time series data with multiple dimensions and noise interference. Then the dimension-reduced samples can be put into CNN model, and the fault classification results can be obtained by convolution and pooling operations of hidden layers in CNN.

The effectiveness of the proposed method is validated through experimental verification and comparative experimental analysis. The results demonstrate that the proposed model can achieve an average classification accuracy of 96.5% under three noise levels, which is 3-13% higher than the traditional models and single deep-learning models.

The combined SDAE–CNN model proposed in this paper can denoise and reduce dimensions of raw vibration signal data, and extract the in-depth features in image samples of rolling bearing. Consequently, the proposed model has more accurate fault diagnosis results for the rolling bearing vibration signal data with long time series and noise interference.

The peer review history for this article is available at: https://publons.com/publon/10.1108/ILT-11-2019-0496/