Search results

Current practices in data classification and retrieval have experienced a surge in the use of multimedia content. Identification of desired information from the huge image databases has been facing increased complexities for designing an efficient feature extraction process. Conventional approaches of image classification with text-based image annotation have faced assorted limitations due to erroneous interpretation of vocabulary and huge time consumption involved due to manual annotation. Content-based image recognition has emerged as an alternative to combat the aforesaid limitations. However, exploring rich feature content in an image with a single technique has lesser probability of extract meaningful signatures compared to multi-technique feature extraction. Therefore, the purpose of this paper is to explore the possibilities of enhanced content-based image recognition by fusion of classification decision obtained using diverse feature extraction techniques.

Three novel techniques of feature extraction have been introduced in this paper and have been tested with four different classifiers individually. The four classifiers used for performance testing were K nearest neighbor (KNN) classifier, RIDOR classifier, artificial neural network classifier and support vector machine classifier. Thereafter, classification decisions obtained using KNN classifier for different feature extraction techniques have been integrated by Z-score normalization and feature scaling to create fusion-based framework of image recognition. It has been followed by the introduction of a fusion-based retrieval model to validate the retrieval performance with classified query. Earlier works on content-based image identification have adopted fusion-based approach. However, to the best of the authors’ knowledge, fusion-based query classification has been addressed for the first time as a precursor of retrieval in this work.

The proposed fusion techniques have successfully outclassed the state-of-the-art techniques in classification and retrieval performances. Four public data sets, namely, Wang data set, Oliva and Torralba (OT-scene) data set, Corel data set and Caltech data set comprising of 22,615 images on the whole are used for the evaluation purpose.

To the best of the authors’ knowledge, fusion-based query classification has been addressed for the first time as a precursor of retrieval in this work. The novel idea of exploring rich image features by fusion of multiple feature extraction techniques has also encouraged further research on dimensionality reduction of feature vectors for enhanced classification results.

Traffic congestion is becoming a serious problem that has adverse consequences on the socio-economy, environment, and public health of various cities worldwide. The purpose of this paper is to contribute to the continuous search for new alternative solutions to prevent or alleviate these concerns. It particularly deals with the development of decision support system based on a data fusion for the management and control of traffic at signalized intersections. The role of such systems is to manage the existing infrastructure to ease congestion and respond to crises. The proposed system is based on multi-detector data fusion, a data processing function that combines imperfect information collected from systems involving several detectors. The developed system is then tested on a virtual junction, and the results obtained are reported and discussed.

This paper presents a new traffic light control based on multi-detectors data fusion theory. The system uses a new multi-detectors data fusion method for traffic data analysis. Moreover, the system integrates a method for the estimation of the reliability degree of different detectors taking into account their imperfection and the conflict between them. These estimated reliability degrees are combined using Dempster’s rule of combination.

The paper provides a decision support system for traffic regulation at intersection based on multi-sensors. It suggests the fusion of captured data by many sensors for measuring information. The system use the Belief Functions Theory for information fusion and decision making using combination and decision rules.

The paper proposes a new Adaptive Traffic Control System based on a new data fusion approach that include a method for the estimation of the reliability degree of different detectors taking into account their imperfection and the conflict between them. These estimated reliability degrees are combined using Dempster’s rule of combination.

This study aims to develop a data fusion method for powder-bed fusion (PBF) process monitoring based on process image information. The data fusion method can help improve process condition identification performance, which can provide guidance for further PBF process monitoring and control system development.

Design of reliable process monitoring systems is an essential approach to solve PBF built quality. A data fusion framework based on support vector machine (SVM), convolutional neural network (CNN) and Dempster-Shafer (D-S) evidence theory are proposed in the study. The process images which include the information of melt pool, plume and spatters were acquired by a high-speed camera. The features were extracted based on an appropriate image processing method. The three feature vectors corresponding to the three objects, respectively, were used as the inputs of SVM classifiers for process condition identification. Moreover, raw images were also used as the input of a CNN classifier for process condition identification. Then, the information fusion of the three SVM classifiers and the CNN classifier by an improved D-S evidence theory was studied.

The results demonstrate that the sensitivity of information sources is different for different condition identification. The feature fusion based on D-S evidence theory can improve the classification performance, with feature fusion and classifier fusion, the accuracy of condition identification is improved more than 20%.

An improved D-S evidence theory is proposed for PBF process data fusion monitoring, which is promising for the development of reliable PBF process monitoring systems.

Although numerous signal modalities are available for emotion recognition, audio and visual modalities are the most common and predominant forms for human beings to express their emotional states in daily communication. Therefore, how to achieve automatic and accurate audiovisual emotion recognition is significantly important for developing engaging and empathetic human–computer interaction environment. However, two major challenges exist in the field of audiovisual emotion recognition: (1) how to effectively capture representations of each single modality and eliminate redundant features and (2) how to efficiently integrate information from these two modalities to generate discriminative representations.

A novel key-frame extraction-based attention fusion network (KE-AFN) is proposed for audiovisual emotion recognition. KE-AFN attempts to integrate key-frame extraction with multimodal interaction and fusion to enhance audiovisual representations and reduce redundant computation, filling the research gaps of existing approaches. Specifically, the local maximum–based content analysis is designed to extract key-frames from videos for the purpose of eliminating data redundancy. Two modules, including “Multi-head Attention-based Intra-modality Interaction Module” and “Multi-head Attention-based Cross-modality Interaction Module”, are proposed to mine and capture intra- and cross-modality interactions for further reducing data redundancy and producing more powerful multimodal representations.

Extensive experiments on two benchmark datasets (i.e. RAVDESS and CMU-MOSEI) demonstrate the effectiveness and rationality of KE-AFN. Specifically, (1) KE-AFN is superior to state-of-the-art baselines for audiovisual emotion recognition. (2) Exploring the supplementary and complementary information of different modalities can provide more emotional clues for better emotion recognition. (3) The proposed key-frame extraction strategy can enhance the performance by more than 2.79 per cent on accuracy. (4) Both exploring intra- and cross-modality interactions and employing attention-based audiovisual fusion can lead to better prediction performance.

The proposed KE-AFN can support the development of engaging and empathetic human–computer interaction environment.

Multi‐sensor data fusion (MSDF) is defined as the process of integrating information from multiple sources to produce the most specific and comprehensive unified data about an entity, activity or event. Multi‐sensor object recognition is one of the important technologies of MSDF. It has been widely applied in the fields of navigation, aviation, artificial intelligence, pattern recognition, fuzzy control, robot, and so on. Hence, aimed at the type recognition problem in which the characteristic values of object types and observations of sensors are in the form of triangular fuzzy numbers, the purpose of this paper is to propose a new fusion method from the viewpoint of decision‐making theory.

This work, first divides the comprehensive transaction process of sensor signal into two phases. Then, aimed at the type recognition problem, the paper gives the definition of similarity degree between two triangular fuzzy numbers. By solving the maximization optimization model, the vector of characteristic weights is objectively derived. A new fusion method is proposed according to the overall similarity degree.

The results of the experiments show that solving the maximization optimization model improves significantly the objectivity and accuracy of object recognition.

The paper studies the type recognition problem in which the characteristic values of object types and observations of sensors are in the form of triangular fuzzy numbers. By solving the maximization optimization model, the vector of characteristic weights is derived. A new fusion method is proposed. This method improves the objectivity and accuracy of object recognition.

The purpose of this paper is to apply a hybrid neuro-fuzzy paradigm called self-organizing neuro-fuzzy multilayered classifier (SONeFMUC) to classify the operating faults of a hydraulic system. The main motivation behind the use of SONeFMUC is to attest the capabilities of neuro-fuzzy classifier for handling the difficulties associated with fault diagnosis of hydraulic circuits.

In the proposed methodology, first, the neuro-fuzzy nodes at each layer of the SONeFMUC are trained separately using two well-known bio-inspired algorithms, i.e. a semi deterministic method with random walks called co-variance matrix adaptation evolutionary strategy (CMA-ES) and a swarm-based explorer with adaptive fuzzified parameters (SBEAFP). Thereafter, a revised version of the group method data handling (GMDH) policy that uses the Darwinian concepts such as truncation selection and elitism is engaged to connect the nodes of different layers in an effective manner.

Based on comparative numerical experiments, the authors conclude that integration of neuro-fuzzy method and bio-inspired supervisor results in a really powerful classification tool beneficial for uncertain environments. It is proved that the method outperforms some well-known classifiers such as support vector machine (SVM) and particle swarm optimization-based SVM (PSO-SVM). Besides, it is indicated that an efficient bio-inspired method can effectively adjust the constructive parameters of the multi-layered neuro-fuzzy classifier. For the case, it is observed that designing a fuzzy controller for PSO predisposes it to effectively balance the exploration/exploitation capabilities, and consequently optimize the structure of SONeFMUC.

The originality of the paper can be considered from both numerical and practical points of view. The signals obtained through the data acquisition possess six different features in order for the hydraulic system to undergo four types of faults, i.e. cylinder fault, pump fault, valve leakage fault and rupture of the piping system. Besides, to elaborate on the authenticity and efficacy of the proposed method, its performance is compared with well-known rival techniques.

This study aims to evaluate a new fusion technique of visual and textual clusters of objects from a real multimedia data-driven collection to improve the performance of multimedia applications.

The authors focused on using multi-criteria for clustering texts and images. The algorithm consists of these steps: first is text representation using the statistical method of weighting, second is image representation using a bag of words feature descriptors methods and finally application of multi-criteria clustering.

As an application for event detection based on social multimedia data, in particular, Flickr platform. Several experiments were conducted to choose the appropriate parameters for a better scheme of clustering. The new approach achieves better performance when aggregate text clustering is done with image clustering for event detection.

Further researches would be investigated on other social media platforms such as Facebook and Twitter for a generalization of the technique.

This study contributes to multimedia data mining through the new fusion technique of clustering. The technique has its root in such strong field as the field of multi-criteria clustering and decision-making support.

Content‐based image retrieval (CBIR) is an important research area for automatically retrieving images of user interest from a large database. Due to many potential applications, facial image retrieval has received much attention in recent years. Similar to face recognition, finding appropriate image representation is a vital step for a successful facial image retrieval system. Recently, many efficient image feature descriptors have been proposed and some of them have been applied to face recognition. It is valuable to have comparative studies of different feature descriptors in facial image retrieval. And more importantly, how to fuse multiple features is a significant task which can have a substantial impact on the overall performance of the CBIR system. The purpose of this paper is to propose an efficient face image retrieval strategy.

In this paper, three different feature description methods have been investigated for facial image retrieval, including local binary pattern, curvelet transform and pyramid histogram of oriented gradient. The problem of large dimensionalities of the extracted features is addressed by employing a manifold learning method called spectral regression. A decision level fusion scheme fuzzy aggregation is applied by combining the distance metrics from the respective dimension reduced feature spaces.

Empirical evaluations on several face databases illustrate that dimension reduced features are more efficient for facial retrieval and the fuzzy aggregation fusion scheme can offer much enhanced performance. A 98 per cent rank 1 retrieval accuracy was obtained for the AR faces and 91 per cent for the FERET faces, showing that the method is robust against different variations like pose and occlusion.

The proposed method for facial image retrieval has a promising potential of designing a real‐world system for many applications, particularly in forensics and biometrics.

The paper aims to solve the problem of personnel intrusion identification within the limits of high-speed railways. It adopts the fusion method of millimeter wave radar and camera to improve the accuracy of object recognition in dark and harsh weather conditions.

This paper adopts the fusion strategy of radar and camera linkage to achieve focus amplification of long-distance targets and solves the problem of low illumination by laser light filling of the focus point. In order to improve the recognition effect, this paper adopts the YOLOv8 algorithm for multi-scale target recognition. In addition, for the image distortion caused by bad weather, this paper proposes a linkage and tracking fusion strategy to output the correct alarm results.

Simulated intrusion tests show that the proposed method can effectively detect human intrusion within 0–200 m during the day and night in sunny weather and can achieve more than 80% recognition accuracy for extreme severe weather conditions.

(1) The authors propose a personnel intrusion monitoring scheme based on the fusion of millimeter wave radar and camera, achieving all-weather intrusion monitoring; (2) The authors propose a new multi-level fusion algorithm based on linkage and tracking to achieve intrusion target monitoring under adverse weather conditions; (3) The authors have conducted a large number of innovative simulation experiments to verify the effectiveness of the method proposed in this article.

Critical thinking is considered important in psychological science because it enables students to make effective decisions and optimizes their performance. Aiming at the challenges and issues of understanding the student's critical thinking, the objective of this study is to analyze online discussion data through an advanced multi-feature fusion modeling (MFFM) approach for automatically and accurately understanding the student's critical thinking levels.

An advanced MFFM approach is proposed in this study. Specifically, with considering the time-series characteristic and the high correlations between adjacent words in discussion contents, the long short-term memory–convolutional neural network (LSTM-CNN) architecture is proposed to extract deep semantic features, and then these semantic features are combined with linguistic and psychological knowledge generated by the LIWC2015 tool as the inputs of full-connected layers to automatically and accurately predict students' critical thinking levels that are hidden in online discussion data.

A series of experiments with 94 students' 7,691 posts were conducted to verify the effectiveness of the proposed approach. The experimental results show that the proposed MFFM approach that combines two types of textual features outperforms baseline methods, and the semantic-based padding can further improve the prediction performance of MFFM. It can achieve 0.8205 overall accuracy and 0.6172 F1 score for the “high” category on the validation dataset. Furthermore, it is found that the semantic features extracted by LSTM-CNN are more powerful for identifying self-introduction or off-topic discussions, while the linguistic, as well as psychological features, can better distinguish the discussion posts with the highest critical thinking level.

With the support of the proposed MFFM approach, online teachers can conveniently and effectively understand the interaction quality of online discussions, which can support instructional decision-making to better promote the student's knowledge construction process and improve learning performance.