Search results

For robots to more actively interact with the surrounding environment in object manipulation tasks or walking, they must understand the physical attributes of objects and surface materials they encounter. Dynamic tactile sensing can effectively capture rich information about material properties. Hence, methods that convey and interpret this tactile information to the user can improve the quality of human–machine interaction. This paper aims to propose a visual-tactile cross-modal retrieval framework to convey tactile information of surface material for perceptual estimation.

The tactile information of a new unknown surface material can be used to retrieve perceptually similar surface from an available surface visual sample set by associating tactile information to visual information of material surfaces. For the proposed framework, the authors propose an online low-rank similarity learning method, which can effectively and efficiently capture the cross-modal relative similarity between visual and tactile modalities.

Experimental results conducted on the Technischen Universität München Haptic Texture Database demonstrate the effectiveness of the proposed framework and the method.

This paper provides a visual-tactile cross-modal perception method for recognizing material surface. By the method, a robot can communicate and interpret the conveyed information about the surface material properties to the user; it will further improve the quality of robot interaction.

With the rapid development of short videos in China, the public has become accustomed to using short videos to express their opinions. This paper aims to solve problems such as how to represent the features of different modalities and achieve effective cross-modal feature fusion when analyzing the multi-modal sentiment of Chinese short videos (CSVs).

This paper aims to propose a sentiment analysis model MSCNN-CPL-CAFF using multi-scale convolutional neural network and cross attention fusion mechanism to analyze the CSVs. The audio-visual and textual data of CSVs themed on “COVID-19, catering industry” are collected from CSV platform Douyin first, and then a comparative analysis is conducted with advanced baseline models.

The sample number of the weak negative and neutral sentiment is the largest, and the sample number of the positive and weak positive sentiment is relatively small, accounting for only about 11% of the total samples. The MSCNN-CPL-CAFF model has achieved the Acc-2, Acc-3 and F1 score of 85.01%, 74.16 and 84.84%, respectively, which outperforms the highest value of baseline methods in accuracy and achieves competitive computation speed.

This research offers some implications regarding the impact of COVID-19 on catering industry in China by focusing on multi-modal sentiment of CSVs. The methodology can be utilized to analyze the opinions of the general public on social media platform and to categorize them accordingly.

This paper presents a novel deep-learning multimodal sentiment analysis model, which provides a new perspective for public opinion research on the short video platform.

With the increasing abundance of network resources and big data, multi-modal information search (MMIS) has been paid more and more attention, but the research results of MMIS are relatively few. This paper attempts to put forward constructive suggestions for the design of multi-modal information system, so that the system can have a better user experience, help users improve the efficiency of obtaining information and optimize the information service mode.

A research model of influencing factors is established by using the TAM (technology acceptance model) theory. The influencing factors of users' multi-modal information search behavior (MMISB) are analyzed by using questionnaire, experiment and the structural equation model. On the basis of this, some suggestions are put forward to build the multi-modal search (MMS) system and improve the efficiency of MMIS.

The research shows that users' MMISB is directly related to their search intention, and the search intention can influence users' cognition of the usefulness and ease of MMIS through their own information search ability and system characteristics. The user's MMIS ability is affected by the demand expression ability and retrieval ability cognition; the user's cognition of system characteristics is affected by the system function and information quality. This shows that the user's MMISB is closely related to the user's cognitive situation, but due to the author's limited time and research ability, only the questionnaire survey method cannot be used to in-depth research and explore the influencing factors of MMIS. Therefore, in the future research, we should combine the interview method to further track the user's emotional factors and scene factors.

For the first time, TAM theory is combined with cross-modal retrieval behavior and the paper explores the influencing factors and evaluation indexes of users' MMISB. The second, the questionnaire was compiled to investigate the influencing factors of the MMISB of the university group, and the reliability analysis, validity analysis, correlation analysis and structural equation model analysis of the survey data are carried out . The survey data and analysis results are original, which can provide a theoretical basis for improving the service level of MMIS.

This paper aims to extend current understanding concerning the cross-modal correspondences between sounds and tastes by introducing new research tools and experimental data to study associations and their reflections between music and taste.

The experiment design addresses the multidisciplinary approach by using cultural, chemical and statistical analysis methods.

The paper provides further evidence that exposure to the “sweet” or “sour” musical pieces influences people’s food-related thinking processes and behaviors. It also demonstrates that sweet or sour elements in the music may reflect to actual sweetness (as measured by sugar content) and sourness (as measured by organic acid content) of foods developed in association with music carrying similar taste characteristics.

The findings should be replicated and expanded using larger consumer samples and wider repertoires of “taste music” and dependent variables. Also, the level of experimental control should be improved; e.g., the “sweet” and “sour” music were produced using different instruments, which may have an influence to the results.

Ambient “taste music” that is congruent with the basic flavors of the dishes can be played in restaurants to highlight guests’ sensory experience.

By carefully considering the symbolic meanings of the music used in different social situations, it is possible to create multimodal experiences and even subconscious expectations in people’ minds.

Cross-modal associations are made between the tastes and music. This can influence on perception of food and provide new ways to build multimodal gastronomic experiences.

Dyadic interactions are significant for human life. Most body sensor networks-based research studies focus on daily actions, but few works have been done to recognize affective actions during interactions. The purpose of this paper is to analyze and recognize affective actions collected from dyadic interactions.

A framework that combines hidden Markov models (HMMs) and k-nearest neighbor (kNN) using Fisher kernel learning is presented in this paper. Furthermore, different features are considered according to the interaction situations (positive situation and negative situation).

Three experiments are conducted in this paper. Experimental results demonstrate that the proposed Fisher kernel learning-based framework outperforms methods using Fisher kernel-based approach, using only HMMs and kNN.

The research may help to facilitate nonverbal communication. Moreover, it is important to equip social robots and animated agents with affective communication abilities.

The presented framework may gain strengths from both generative and discriminative models. Further, different features are considered based on the interaction situations.

The purpose of this paper is to study the road segmentation problem of cross-modal remote sensing images.

First, the baseline network based on the U-net is trained under a large-scale dataset of remote sensing imagery. Then, the cross-modal training data are used to fine-tune the first two convolutional layers of the pre-trained network to achieve the adaptation to the local features of the cross-modal data. For the cross-modal data of different band, an autoencoder is designed to achieve data conversion and local feature extraction.

The experimental results show the effectiveness and practicability of the proposed method. Compared with the ordinary method, the proposed method gets much better metrics.

The originality is the transfer learning strategy that fine-tunes the low-level layers for the cross-modal data application. The proposed method can achieve satisfied road segmentation with a small amount of cross-modal training data, so that is has a good application value. Still, for the similar application of cross-modal data, the idea provided by this paper is helpful.

The idea that criminal behavior is a function of the offender's personality, also called the Homology hypothesis, has a long history in forensic psychology and criminology. This assumption, however, has been decried as lacking empirical support. In spite of much social concern relative to sexual offenses, there is virtually no research looking at the stability of offending pattern in sex offenders of adult women. This paper aims to fill some of the gaps.

Latent structure analyses were conducted on a secondary dataset including 145 serial rapists. A cross‐sectional, discrete time‐series design was used including a sequence of three offenses.

Moderate support was found for the three main assumptions underlying the Homology hypothesis. Offenses tended to share a relatively similar underlying structure, with the victimology and aggression components being more prominent than the sexual dimension. The three primary profiles identified, labeled “Passive”, “Stranger‐aggressive”, and “Antisocial”, were found to be about 50 percent stable across the sequence. Finally, the presence of significant dysfunction in the family of origin predicted membership in the “Antisocial” class, as well as increased the specificity and stability of this profile. The presence of early maladjustment was not related to any of the states.

Based on the results, it is proposed that future research incorporates contextual‐environmental elements in order to increase the validity of the findings.

This study represents a unique attempt at documenting patterns of stability and variations across incidents of rape, using an institutional sample. Furthermore, it illustrates the use and potential benefits of latent structure models in criminological research.

This paper aims to study the cross-modal correspondence between a visual stimulus (i.e. glossiness), haptic perception and consumers’ reactions (internal responses and behavioral intentions).

Using an experimental design, three experimental studies have been conducted to test the effect of a glossy (versus matte) packaging upon the perception of haptic features of a packaging (roughness, thickness and lightness), internal reactions (perceived product quality and product attractiveness) and behavioral intentions (purchase intention and willingness to pay).

This paper evidences the significant impact that glossiness bears on the haptic perception of a packaging material as well as upon internal reactions and behavioral intentions. A new conceptual framework combining the SOR model and the cross-modal correspondences is validated.

The results encourage further research to explore the wide range of potential cross-modal correspondences between visual stimuli and haptic perception.

The results highlight the critical influence of visual cues for managers, especially for online shopping or advertising. Even if consumers cannot touch the product, it is possible to induce haptic perception through visual cues and to influence the internal reactions and behavioral intentions.

This research demonstrates that the packaging texture and weight can be visually induced through glossiness.