Search results

The paper deals with the design and creation of an intelligent user interface augmenting the user experience in everyday environments, by providing an immersive audio environment. We highlight the potential of augmenting the visual real environment in a personalized way, thanks to context modeling techniques. The LISTEN project, a system for an immersive audio augmented environment applied in the art exhibition domain, provides an example of modeling and personalization methods affecting the audio interface in terms of content and organization. In addition, the different evolution steps of the system and the outcomes of the accompanying user tests are here reported.

The purpose of this paper is to better understand communication between musicians in a free jazz improvisation in comparison to traditional jazz.

A cybernetic informative feedback model was used to study communication between musicians for free jazz. The conceptual model consists of the ears as sensors, an auditory analysis stage to convert the acoustic signals into symbolic information (e.g. notated music), a cognitive processing stage (to make decisions and adapt the performance to what is being heard), and an effector (e.g. muscle movement to control an instrument). It was determined which musical features of the co‐players have to be extracted to be able to respond adequately in a music improvisation, and how this knowledge can be used to build an automated music improvisation system for free jazz.

The three major findings of this analysis were: in traditional jazz a soloist only needs to analyze a very limited set of music ensemble features, but in free jazz the performer has to observe each musician individually; unlike traditional jazz, free jazz is not a strict rule‐based system. Consequently, the musicians need to develop their personal symbolic representation; which could be a machine‐adequate music representation for an automated music improvisation system. The latter could be based on acoustic features that can be extracted robustly by a computer algorithm.

Gained knowledge can be applied to build automated music improvisation systems for free jazz.

The paper expands our knowledge to create intelligent music improvisation algorithms to algorithms that can improvise with a free jazz ensemble.

The purpose of this paper is to examine a central need of students who are blind: the ability to access science curriculum content.

Agent-based modeling is a relatively new computational modeling paradigm that models complex dynamic systems. NetLogo is a widely used agent-based modeling language that enables exploration and construction of models of complex systems by programming and running the rules and behaviors. Sonification of variables and events in an agent-based NetLogo computer model of gas in a container is used to convey phenomena information. This study examined mainly two research topics: the scientific conceptual knowledge and systems reasoning that were learned as a result of interaction with the listen-to-complexity (L2C) environment as appeared in answers to the pre- and post-tests and the learning topics of kinetic molecular theory of gas in chemistry that was learned as a result of interaction with the L2C environment. The case study research focused on A., a woman who is adventitiously blind, for eight sessions.

The participant successfully completed all curricular assignments; her scientific conceptual knowledge and systems reasoning became more specific and aligned with scientific knowledge.

A practical implication of further studies is that they are likely to have an impact on the accessibility of learning materials, especially in science education for students who are blind, as equal access to low-cost learning environments that are equivalent to those used by sighted users would support their inclusion in the K-12 academic curriculum.

The innovative and low-cost learning system that is used in this research is based on transmittal of visual information of dynamic and complex systems, providing perceptual compensation by harnessing auditory feedback. For the first time the L2C system is based on sound that represents a dynamic rather than a static array. In this study, the authors explore how a combination of several auditory representations may affect cognitive learning ability.

The current golden standard for attention deficit hyperactivity disorder (ADHD) diagnosis is clinical diagnosis based on psychiatric interviews and psychological examinations. This is suboptimal, as clinicians are unable to view potential patients in multiple natural settings – a necessary condition for objective diagnosis. The purpose of this paper is to improve the objective diagnosis of ADHD by analyzing a quantified representation of the actions of potential patients in multiple natural environments.

The authors use both virtual reality (VR) and artificial intelligence (AI) to create an objective ADHD diagnostic test. Diagnostic and statistical manual of mental disorders, 5th Edition (DSM-5) and ADHD Rating Scale are used to create a rule-based system of quantifiable VR-observable actions. As a potential patient completes tasks within multiple VR scenes, certain actions trigger an increase in the severity measure of the corresponding ADHD symptom. The resulting severity measures are input to an AI model, which classifies the potential patient as having ADHD in the form inattention, hyperactivity-impulsivity, combined or neither.

The result of this study shows that VR-observed actions can be extracted as quantified data, and classification of this quantified data achieves near-perfect sensitivity and specificity with a 98.3% accuracy rate on a convolutional neural network model.

To the best of the authors’ knowledge, this is the first study to incorporate VR and AI into an objective DSM-5-based ADHD diagnostic test. By including stimulation to the visual, auditory and equilibrium senses and tracking movement and recording voice, we present a method to further the research of objective ADHD diagnosis.

Law and Film both enjoy the power to mediate the social imaginary. Here, we explore the resonance of this insight in the register of affect and intensity, movement, and change. This demands a different approach to doing theory. As Andrew (1976, pp. 66–67) argues, ‘film is not a product but an organically unfolding creative process in which the audience participates both emotionally and intellectually.’ Seeing a film is not just an exercise in imagining alternatives; it is an unfolding experience in time. It is an event shaded with particular embodied dimensions: one's heart races, pupils contract, skin shivers, muscles tense. Involuntary sensations of nausea or vertigo combine with cognitive responses to produce the lived experience of viewing a particular film that is incorporated into one's sensibility, sometimes very powerfully. It is not just that the mind has spent time in a darkened theatre. The body has also had an affect-laden auditory, visual, and tactile encounter. The affect-rooted experience of the film is a piece of the subject's past, its history, its self. This is another way to understand how film not only represents the world, but participates in its making.

Vision, audition, olfactory, tactile and taste are five important senses that human uses to interact with the real world. As facing more and more complex environments, a sensing system is essential for intelligent robots with various types of sensors. To mimic human-like abilities, sensors similar to human perception capabilities are indispensable. However, most research only concentrated on analyzing literature on single-modal sensors and their robotics application.

This study presents a systematic review of five bioinspired senses, especially considering a brief introduction of multimodal sensing applications and predicting current trends and future directions of this field, which may have continuous enlightenments.

This review shows that bioinspired sensors can enable robots to better understand the environment, and multiple sensor combinations can support the robot’s ability to behave intelligently.

The review starts with a brief survey of the biological sensing mechanisms of the five senses, which are followed by their bioinspired electronic counterparts. Their applications in the robots are then reviewed as another emphasis, covering the main application scopes of localization and navigation, objection identification, dexterous manipulation, compliant interaction and so on. Finally, the trends, difficulties and challenges of this research were discussed to help guide future research on intelligent robot sensors.