Search results

Student engagement is a key factor that connects with student achievement and retention. This paper aims to identify individuals' engagement automatically in the classroom with multimodal data for supporting educational research.

The video and electroencephalogram data of 36 undergraduates were collected to represent observable and internal information. Since different modal data have different granularity, this study proposed the Fast–Slow Neural Network (FSNN) to detect engagement through both observable and internal information, with an asynchrony structure to preserve the sequence information of data with different granularity.

Experimental results show that the proposed algorithm can recognize engagement better than the traditional data fusion methods. The results are also analyzed to figure out the reasons for the better performance of the proposed FSNN.

This study combined multimodal data from observable and internal aspects to improve the accuracy of engagement detection in the classroom. The proposed FSNN used the asynchronous process to deal with the problem of remaining sequential information when facing multimodal data with different granularity.

This paper presents a survey of research into interactive robotic systems for the purpose of identifying the state of the art capabilities as well as the extant gaps in this emerging field. Communication is multimodal. Multimodality is a representation of many modes chosen from rhetorical aspects for its communication potentials. The author seeks to define the available automation capabilities in communication using multimodalities that will support a proposed Interactive Robot System (IRS) as an AI mounted robotic platform to advance the speed and quality of military operational and tactical decision making.

This review will begin by presenting key developments in the robotic interaction field with the objective of identifying essential technological developments that set conditions for robotic platforms to function autonomously. After surveying the key aspects in Human Robot Interaction (HRI), Unmanned Autonomous System (UAS), visualization, Virtual Environment (VE) and prediction, the paper then proceeds to describe the gaps in the application areas that will require extension and integration to enable the prototyping of the IRS. A brief examination of other work in HRI-related fields concludes with a recapitulation of the IRS challenge that will set conditions for future success.

Using insights from a balanced cross section of sources from the government, academic, and commercial entities that contribute to HRI a multimodal IRS in military communication is introduced. Multimodal IRS (MIRS) in military communication has yet to be deployed.

Multimodal robotic interface for the MIRS is an interdisciplinary endeavour. This is not realistic that one can comprehend all expert and related knowledge and skills to design and develop such multimodal interactive robotic interface. In this brief preliminary survey, the author has discussed extant AI, robotics, NLP, CV, VDM, and VE applications that is directly related to multimodal interaction. Each mode of this multimodal communication is an active research area. Multimodal human/military robot communication is the ultimate goal of this research.

A multimodal autonomous robot in military communication using speech, images, gestures, VST and VE has yet to be deployed. Autonomous multimodal communication is expected to open wider possibilities for all armed forces. Given the density of the land domain, the army is in a position to exploit the opportunities for human–machine teaming (HMT) exposure. Naval and air forces will adopt platform specific suites for specially selected operators to integrate with and leverage this emerging technology. The possession of a flexible communications means that readily adapts to virtual training will enhance planning and mission rehearsals tremendously.

Interaction, perception, cognition and visualization based multimodal communication system is yet missing. Options to communicate, express and convey information in HMT setting with multiple options, suggestions and recommendations will certainly enhance military communication, strength, engagement, security, cognition, perception as well as the ability to act confidently for a successful mission.

The objective is to develop a multimodal autonomous interactive robot for military communications. This survey reports the state of the art, what exists and what is missing, what can be done and possibilities of extension that support the military in maintaining effective communication using multimodalities. There are some separate ongoing progresses, such as in machine-enabled speech, image recognition, tracking, visualizations for situational awareness, and virtual environments. At this time, there is no integrated approach for multimodal human robot interaction that proposes a flexible and agile communication. The report briefly introduces the research proposal about multimodal interactive robot in military communication.

With the rapid development of short videos in China, the public has become accustomed to using short videos to express their opinions. This paper aims to solve problems such as how to represent the features of different modalities and achieve effective cross-modal feature fusion when analyzing the multi-modal sentiment of Chinese short videos (CSVs).

This paper aims to propose a sentiment analysis model MSCNN-CPL-CAFF using multi-scale convolutional neural network and cross attention fusion mechanism to analyze the CSVs. The audio-visual and textual data of CSVs themed on “COVID-19, catering industry” are collected from CSV platform Douyin first, and then a comparative analysis is conducted with advanced baseline models.

The sample number of the weak negative and neutral sentiment is the largest, and the sample number of the positive and weak positive sentiment is relatively small, accounting for only about 11% of the total samples. The MSCNN-CPL-CAFF model has achieved the Acc-2, Acc-3 and F1 score of 85.01%, 74.16 and 84.84%, respectively, which outperforms the highest value of baseline methods in accuracy and achieves competitive computation speed.

This research offers some implications regarding the impact of COVID-19 on catering industry in China by focusing on multi-modal sentiment of CSVs. The methodology can be utilized to analyze the opinions of the general public on social media platform and to categorize them accordingly.

This paper presents a novel deep-learning multimodal sentiment analysis model, which provides a new perspective for public opinion research on the short video platform.

The purpose of this paper is to present a Big Data solution as a methodological approach to the automated collection, cleaning, collation, and mapping of multimodal, longitudinal data sets from social media. The paper constructs social information landscapes (SIL).

The research presented here adopts a Big Data methodological approach for mapping user-generated contents in social media. The methodology and algorithms presented are generic, and can be applied to diverse types of social media or user-generated contents involving user interactions, such as within blogs, comments in product pages, and other forms of media, so long as a formal data structure proposed here can be constructed.

The limited presentation of the sequential nature of content listings within social media and Web 2.0 pages, as viewed on web browsers or on mobile devices, do not necessarily reveal nor make obvious an unknown nature of the medium; that every participant, from content producers, to consumers, to followers and subscribers, including the contents they produce or subscribed to, are intrinsically connected in a hidden but massive network. Such networks when mapped, could be quantitatively analysed using social network analysis (e.g. centralities), and the semantics and sentiments could equally reveal valuable information with appropriate analytics. Yet that which is difficult is the traditional approach of collecting, cleaning, collating, and mapping such data sets into a sufficiently large sample of data that could yield important insights into the community structure and the directional, and polarity of interaction on diverse topics. This research solves this particular strand of problem.

The automated mapping of extremely large networks involving hundreds of thousands to millions of nodes, encapsulating high resolution and contextual information, over a long period of time could possibly assist in the proving or even disproving of theories. The goal of this paper is to demonstrate the feasibility of using automated approaches for acquiring massive, connected data sets for academic inquiry in the social sciences.

The methods presented in this paper, together with the Big Data architecture can assist individuals and institutions with a limited budget, with practical approaches in constructing SIL. The software-hardware integrated architecture uses open source software, furthermore, the SIL mapping algorithms are easy to implement.

The majority of research in the literature uses traditional approaches for collecting social networks data. Traditional approaches can be slow and tedious; they do not yield adequate sample size to be of significant value for research. Whilst traditional approaches collect only a small percentage of data, the original methods presented here are able to collect and collate entire data sets in social media due to the automated and scalable mapping techniques.

Several review articles have been published within the Artificial Intelligence (AI) literature that have explored a range of applications within the tourism and hospitality sectors. However, how efficiently the applied AI methods and algorithms have performed with respect to the type of applications and the multimodal sets of data domains have not yet been reviewed. Therefore, this paper aims to review and analyse the established AI methods in hospitality/tourism, ranging from data modelling for demand forecasting, tourism destination and behaviour pattern to enhanced customer service and experience.

The approach was to systematically review the relationship between AI methods and hospitality/tourism through a comprehensive literature review of papers published between 2010 and 2021. In total, 146 articles were identified and then critically analysed through content analysis into themes, including “AI methods” and “AI applications”.

The review discovered new knowledge in identifying AI methods concerning the settings and available multimodal data sets in hospitality and tourism. Moreover, AI applications fostering the tourism/hospitality industries were identified. It also proposes novel personalised AI modelling development for smart tourism platforms to precisely predict tourism choice behaviour patterns.

This review paper offers researchers and practitioners a broad understanding of the proper selection of AI methods that can potentially improve decision-making and decision-support in the tourism/hospitality industries.

This paper contributes to the tourism/hospitality literature with an interdisciplinary approach that reflects on theoretical/practical developments for data collection, data analysis and data modelling using AI-driven technology.

In this article, we develop and advance an understanding of institutions as multimodal accomplishments. We draw on social semiotics and the linguistic concept of metafunctions to establish the visual as a specific mode of meaning construction. In addition, we make semiotic modes conducive to institutional inquiry by introducing the notion of distinct “modal registers” – specialized configurations of linguistic signs within a particular mode that are adapted and applied in the reproduction of institutions or institutional domains. At the core of our article, we operationalize metafunctions to develop methodology for the analysis of visual registers. We illustrate our approach with data from Corporate Social Responsibility (CSR) reporting in Austria.

The purpose of this paper is to explore the formation, maintenance and disintegration of a fringe Twitter community in order to understand if offline community structure applies to online communities.

The research adopted Big Data methodological approaches in tracking user-generated contents over a series of months and mapped online Twitter interactions as a multimodal, longitudinal “social information landscape”. Centrality measures were employed to gauge the importance of particular user nodes within the complete network and time-series analysis were used to track ego centralities in order to see if this particular online communities were maintained by specific egos.

The case study shows that communities with distinct boundaries and memberships can form and exist within Twitter’s limited user content and sequential policies, which unlike other social media services, do not support formal groups, demonstrating the resilience of desperate online users when their ideology overcome social media limitations. Analysis in this paper using social networks approaches also reveals that communities are formed and maintained from the bottom-up.

The research data is based on a particular data set which occurred within a specific time and space. However, due to the rapid, polarising group behaviour, growth, disintegration and decline of the online community, the data set presents a “laboratory” case from which many other online community can be compared with. It is highly possible that the case can be generalised to a broader range of communities and from which online community theories can be proved/disproved.

The paper showed that particular group of egos with high activities, if removed, could entirely break the cohesiveness of the community. Conversely, strengthening such egos will reinforce the community strength. The questions mooted within the paper and the methodology outlined can potentially be applied in a variety of social science research areas. The contribution to the understanding of a complex social and political arena, as outlined in the paper, is a key example of such an application within an increasingly strategic research area – and this will surely be applied and developed further by the computer science and security community.

The majority of researches that cover these domains have not focused on communities that are multimodal and longitudinal. This is mainly due to the challenges associated with the collection and analysis of continuous data sets that have high volume and velocity. Such data sets are therefore unexploited with regards to cyber-community research.

The study aims to identify the status quo of artificial intelligence in entrepreneurship education with a view to identifying potential research gaps, especially in the adoption of certain intelligent technologies and pedagogical designs applied in this domain.

A scoping review was conducted using six inclusive and exclusive criteria agreed upon by the author team. The collected studies, which focused on the adoption of AI in entrepreneurship education, were analysed by the team with regards to various aspects including the definition of intelligent technology, research question, educational purpose, research method, sample size, research quality and publication. The results of this analysis were presented in tables and figures.

Educators introduced big data and algorithms of machine learning in entrepreneurship education. Big data analytics use multimodal data to improve the effectiveness of entrepreneurship education and spot entrepreneurial opportunities. Entrepreneurial analytics analysis entrepreneurial projects with low costs and high effectiveness. Machine learning releases educators’ burdens and improves the accuracy of the assessment. However, AI in entrepreneurship education needs more sophisticated pedagogical designs in diagnosis, prediction, intervention, prevention and recommendation, combined with specific entrepreneurial learning content and entrepreneurial procedure, obeying entrepreneurial pedagogy.

This study holds significant implications as it can shift the focus of entrepreneurs and educators towards the educational potential of artificial intelligence, prompting them to consider the ways in which it can be used effectively. By providing valuable insights, the study can stimulate further research and exploration, potentially opening up new avenues for the application of artificial intelligence in entrepreneurship education.

The goal of this study is to explore how content creators engage in critical data literacies on TikTok, a social media site that encourages the creation and dissemination of user-created, short-form videos. Critical data literacies encompass the ability to reason with, critique, control, and repurpose data for creative uses. Existing work on critical data literacies on social media has focused on understanding of personal data, critique of data use, and strategies to protect privacy. This work focuses on how TikTok content creators repurpose data to construct their own narratives.

Through hashtag search, the authors created a corpus of 410 TikTok videos focused on discussing environmental and climate action, and qualitatively coded the videos for data literacies practices and video features (audio, footage, background images) that may support these practices.

Content creators engaged in multiple practices to attach meanings to data and situate environmental and climate action discourse in lived experiences. While there were instances of no data practices, we found cases where creators compiled different data sources, situated data in personal and local contexts, and positioned their experiences as data points to supplement or counter other statistics. Creators further leveraged the platform’s technical features, particularly the ability to add original audio and background images, to add narratives to the collective discourse.

This study presents a unique focus on examining critical data literacies on social media. Findings highlight how content creators repurpose data and integrate personal experiences. They illustrate platform features to support data practices and inform the design of learning environments.

Educational data mining (EDM) and learning analytics, which are highly related subjects but have different definitions and focuses, have enabled instructors to obtain a holistic view of student progress and trigger corresponding decision-making. Furthermore, the automation part of EDM is closer to the concept of artificial intelligence. Due to the wide applications of artificial intelligence in assorted fields, the authors are curious about the state-of-art of related applications in Education.

This study focused on systematically reviewing 1,219 EDM studies that were searched from five digital databases based on a strict search procedure. Although 33 reviews were attempted to synthesize research literature, several research gaps were identified. A comprehensive and systematic review report is needed to show us: what research trends can be revealed and what major research topics and open issues are existed in EDM research.

Results show that the EDM research has moved toward the early majority stage; EDM publications are mainly contributed by “actual analysis” category; machine learning or even deep learning algorithms have been widely adopted, but collecting actual larger data sets for EDM research is rare, especially in K-12. Four major research topics, including prediction of performance, decision support for teachers and learners, detection of behaviors and learner modeling and comparison or optimization of algorithms, have been identified. Some open issues and future research directions in EDM field are also put forward.

Limitations for this search method include the likelihood of missing EDM research that was not captured through these portals.

This systematic review has not only reported the research trends of EDM but also discussed open issues to direct future research. Finally, it is concluded that the state-of-art of EDM research is far from the ideal of artificial intelligence and the automatic support part for teaching and learning in EDM may need improvement in the future work.