AI empowered context-aware smart system for medication adherence

Qiong Wu (Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly, Nanyang Technological University, Singapore, Singapore)
Zhiwei Zeng (Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly, Interdisciplinary Graduate School, Nanyang Technological University, Singapore, Singapore)
Jun Lin (Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly, Nanyang Technological University, Singapore, Singapore)
Yiqiang Chen (Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China)

International Journal of Crowd Science

ISSN: 2398-7294

Article publication date: 12 June 2017



Poor medication adherence leads to high hospital admission rate and heavy amount of health-care cost. To cope with this problem, various electronic pillboxes have been proposed to improve the medication adherence rate. However, most of the existing electronic pillboxes use time-based reminders which may often lead to ineffective reminding if the reminders are triggered at inopportune moments, e.g. user is sleeping or eating.


In this paper, the authors propose an AI-empowered context-aware smart pillbox system. The pillbox system collects real-time sensor data from a smart home environment and analyzes the user’s contextual information through a computational abstract argumentation-based activity classifier.


Based on user’s different contextual states, the smart pillbox will generate reminders at appropriate time and on appropriate devices.


This paper presents a novel context-aware smart pillbox system that uses argumentation-based activity recognition and reminder generation.



Wu, Q., Zeng, Z., Lin, J. and Chen, Y. (2017), "AI empowered context-aware smart system for medication adherence", International Journal of Crowd Science, Vol. 1 No. 2, pp. 102-109.



Emerald Publishing Limited

Copyright © 2017, Qiong Wu, Zhiwei Zeng, Jun Lin and Yiqiang Chen


Published in the International Journal of Crowd Science. Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at

1. Introduction

With rapid population aging and rising health-care cost, ensuring medication adherence is becoming increasingly important. The 2016 population report from the Department of Statistics Singapore shows that the proportion of residents in Singapore aged 65 years and over has reached 12.4 per cent, with an increase of 4 per cent in the past 10 years. The probability of suffering from multiple medical conditions (e.g. hypertension, heart failure and high cholesterol) increases with age, resulting in growing needs of proper medication management (Li et al., 2014). According to the World Health Organization (WHO, 2003), the average medication adherence rate in developed countries is only 50 per cent and is even lower in developing countries. Poor medication adherence can cause up to 69 per cent of medication-related hospital admission and lead to $100bn health-care cost in the USA per year (Ozok et al., 2011).

Poor adherence could be caused by various reasons, including forgetfulness, disruption of daily routines, complexity of regime and, sometimes, intentional nonadherence (Kaushik et al., 2008). Of these, forgetfulness has been reported to be the most common reason, which suggests that effective reminders could be helpful. However, most of the existing electronic pillboxes adopt time-based reminders, which may not successfully improve adherence if the time-triggered alerts occur at inopportune moments. For example, a reminder is triggered when the user is eating, whereas the medication should be taken after meal. In fact, most medication could be taken within a several hour window around the prescribed time and still be fully effective (Kaushik et al., 2008). Thus, incorporating context awareness into the reminders of smart pillboxes could increase the chance of successful medication adherence by alerting at opportune moments.

In this paper, we propose an AI-empowered smart pillbox system that provides context-aware reminders. Context information is acquired by multi-sensor system placed in a smart home environment. A novel argumentation based approach is adopted for solving the activity recognition problem (Fan et al., 2016) and reminder planning problem (Zeng et al., 2017). The activity recognition module is able to produce classification results comparable to pure machine learning approaches while significantly reducing the training time. Knowing the user’s current activities, e.g. sleeping, watching TV or not at home, the reminder planner module can adaptively plan the reminding time (e.g. delay to a later time) or the reminding device (e.g. display reminder on TV). Moreover, this argumentation-based approach can generate argumentative explanations to the classification results and reminding decisions, which makes the reasoning process more transparent and interpretable.

2. Related works

There are a number of electronic pill dispensers available in the market. For example, GMS MED-E-LERT automatic pill dispenser, E-Pill automatic pill dispenser and Philips medication dispensing service. These commercial electronic pill dispensers mainly focus on pill organization and prevention of incorrect dosage with locking functions and rely on build-in alarms for medication time notification.

Other designs focus on tracking long-term medication adherence data and reporting adherence in real time through IOT and ICT techniques. For example, an instrumented pillbox called MedTracker (Hayes et al., 2006) can track medication consumption data continuously by attaching sensors to the lids of a seven-day plaid pillbox and transmitting sensor triggering information using Bluetooth. Huang et al. (2014) pointed out that the transmitting range of Bluetooth is rather limited and proposed to use the Internet to send medication adherence information to any location in the world. Othman and Ek (2016) proposed to use pop up notifications on smart phones for alarming at the right medication time.

The abovementioned pillbox systems all adopt time-based reminders without considering the contextual information of the users. The MIT House_n project (Kaushik et al., 2008) highlighted the importance of adaptive reminders and implemented a context-aware reminder in their Placelab. Their case study showed that context-aware reminders have received positive feedback from the user, especially the reminders generated based on proximity detection. However, their focus is more on adaptive reminders without integrating with an electronic pillbox. In this work, we propose a holistic smart pillbox system with sensor-enabled context-aware reminders and medication adherence data tracking.

3. The proposed communication system

The conceptual design framework of the proposed context-aware smart pillbox system is illustrated in Figure 1. The conceptual design framework is composed of three main parts: activity recognition, reminder planner and medication intake tracking. The sensor data collected from the smart home environment is input into the activity recognition module to generate user specific context information. This context information, together with the medication schedules input prior by the user or caregivers, are input to the reminder planner module to generate appropriate reminders. Medication intake tracking is used to track medicine consumption and used for adapting the reminders.

3.1 Activity recognition

Multiple sensors, including two Grid-Eye infrared array sensors, two force sensors, one noise sensor and one electric current detector, are continuously collecting data from the smart home environment. Some of the sensors placed at our testing homes are shown in Figure 2(a). We adopt a computational abstract argumentation-based classifier (Fan et al., 2016) (computational abstract argumentation – CAA) to classify basic activities of daily living that may influence medication adherence, for example, sleeping, watching TV, cooking, eating, reading, having visitors, and not at home, etc.

CAA is able to seamlessly connect low level data processing with high level inference-based reasoning. A CAA framework consists of arguments and attacks in which each argument is a self-contained unit of computation and attack relationships are defined over arguments. The computation on sensory data is encapsulated within the individual arguments so that high level reasoning can be performed using well-established argumentation semantics. The advantage of the CAA-based activity detection approach is twofold. First, CAA is able to incorporate domain expert knowledge into problem solving, which significantly reduces the training time as required by machine-learning based model construction. Second, explanations for the classification results can be generated based on the argumentative structure of CAA.

3.2 Reminder planner

Based on the medication schedule input prior and the identified activities that the user is currently engaging in, the reminder planner module will intelligently decide when, where and how to issue the reminders. The goal of the reminder planner is to issue reminders accurately and provide pleasant user experiences. Before each reminder is issued, the reminder planner needs to make the following three decisions based on context:

  1. whether to delay the reminder: delay versus no_delay;

  2. where to display the reminder (if it is not delayed): on television or tablet or mobile phone; and

  3. whether to increase the reminding volume: normal or increase.

We adopt an argumentation-based approach (Zeng et al., 2017) for making these decisions. A graphical representation, named decision graph with context (DGC), is used model decision problems with context. The problem of making context-aware decisions in DGCs is converted into argumentation computation in well-studied argumentation frameworks. In addition, two forms of explanation can be generated to provide general and context-specific reasons for decisions made. We follow this approach because of the following advantages it has over other available decision-making approaches: first, DGC is able to capture the dynamic relationships among decision alternatives and goals, making it more expressive and flexible. Second, the explanations help to make decision-making process less opaque and more understandable to human users.

We illustrate the responses of the reminder planner module in different context as follows:

  • If user is sleeping and medication is not urgent, delay the reminding time till the user get up.

  • If user is eating and the medication should be taken half an hour later after food, remind medication half an hour after food consumption.

  • If user if cooking and pillbox reminding device is in the living room, increase the reminding volume.

  • If user is watching TV, display reminder on TV.

  • If user is not at home, display reminder on mobile devices.

An illustration of the implemented reminder display is given in Figure 2(b) and (c).

3.3 Medication intake tracking

The medication intake tracking is done through a tag-and-scan fashion implemented by radio frequency identification (RFID) tags and scanner. RFID tags encoded with the medication information (e.g. name, description, frequency of taking) are attached to medicine containers. The advantage of using RFID tags for identifying medication is that it can be easily attached to the containers of all kinds of medications, including pills, liquids and powder. To log medication intake, the user simply needs to scan the RFID tag on the medicine container, which is similar to supermarket check-out. With RFID tags, medication intake tracking can get the following information:

  • whether medication is taken;

  • the medication taken is right or wrong; and

  • what time the medication is taken, etc.

An illustration of the RFID scanning is given in Figure 2(d). Similar ideas have been used in some exiting design concepts. For example, Li et al. (2014) adopted RFID in smart pillbox system for medication adherence. The difference is that RFID tags are not attached to pillboxes but to the wristband of users. Hence, RFID is for detecting users when nearby rather than identifying correct pill dosage. Wu et al. (2015) proposed to use matrix barcode and a camera for checking the right medication before intake, which is similar to our design concept.

4. Use cases

In this section, we describe two use cases based on our proposed context-aware smart pillbox system.

4.1 Use Case 1: user sleeping

This case is illustrated in Figure 3(a), (b) and (c). According to the medication schedule, Ticlopidine Hydrochloride should be taken at 13:00 in the noon. After processing the raw sensory data, the activity recognition module has constructed the following arguments:

  1. sofa empty;

  2. TV off;

  3. bed occupied; and

  4. no person by dining table.

Argument (1) and (2) rule out the possibility that the user is watching TV and reading. Argument (4) rules out the possibility that the user is eating or having a friend visit. After argumentative computation, the output of the activity recognition module is sleeping and some explanations generated for analysis above are:

[…] it is not WatchTV as the Sofa is empty, TV is off and Bed is occupied” and “it is not Visit as the Bed is occupied and No person by Table […]

Knowing that the user is sleeping and Ticlopidine Hydrochloride can be taken within a 30 min window, the reminder planner decides to delay the reminder to 13:30 to achieve the goal of offering pleasant user experience. The delay is reflected in our Web-based reminding platform.

4.2 Use Case 2: user eating

Figure 3(d), (e) and (f) describes this case. According to the medication schedule, Ticlopidine Hydrochloride should be taken at 18:00 in the evening. The activity recognition module constructs the following arguments after processing the sensory data: sofa empty, TV off, bed empty and one person by dining table. The output of the activity recognition module is eating and explanations for this result include:

[…] it is not Sleep as Bed is empty and One person by Table and it is not Read as Sofa is empty and One person by Table […]

Considering that the user is eating and Ticlopidine Hydrochloride needs to be consumed after food, the reminder planner delays the reminder to 18:30, as is reflected in our Web-based reminding platform. The explanation for making such decision is:

[…] reminding without delay is neither pleasant nor accurate in current context. In the context when the user is eating and the medication is instructed to be taken after food, reminding on time does not follow the instructions.

5. Discussion and future work

By considering user’s context information, the proposed smart pillbox system is expected to produce more meaningful reminders that occur at opportune moments and on appropriate devices, which may improve medication adherence in comparison to pure time-based reminders. For future work, we are planning to study the effectiveness of context-aware reminders for medication adherence. With the current setting, we are able to track longitudinal medical adherence data, e.g. whether medicines are taken, whether medicines are taken on time, etc. This feature allows us to unobtrusively evaluate the effectiveness of the proposed context-aware smart pillbox system over an extended period of time.


Conceptual framework of the smart pillbox system

Figure 1.

Conceptual framework of the smart pillbox system

Implementation of the smart pillbox

Figure 2.

Implementation of the smart pillbox

Illustration of cases

Figure 3.

Illustration of cases


Fan, X., Zhang, H., Leung, C. and Miao, C. (2016), “A first step towards explained activity recognition with computational abstract argumentation”, Proceeding of IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, Baden-Baden, pp. 1-6.

Hayes, T.L., Hunt, J.M., Adami, A. and Kaye, J.A. (2006), “An electronic pillbox for continuous monitoring of medication adherence”, Proceeding of 28th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, New York, NY pp. 6400-6403.

Huang, S.-C., Chang, H.-Y., Jhu, Y.-C. and Chen, G.-Y. (2014), “The intelligent pill boxDesign and implementation”, Proceeding of IEEE International Conference on Consumer Electronics-Taiwan (ICCE-TW), Taipei, pp. 235-236.

Kaushik, P., Intille, S.S. and Larson, K. (2008), “User-adaptive reminders for home-based medical tasks”, Methods of Information in Medicine, Vol. 47 No. 3, pp. 203-207.

Li, J., Peplinski, S.J., Nia, S.M. and Farajidavar, A. (2014), “An interoperable pillbox system for smart medication adherence”, Proceeding of IEEE Engineering in Medicine and Biology Society, Chicago, IL, pp. 1386-1389.

Othman, N.B. and Ek, O.P. (2016), “Pill dispenser with alarm via smart phone notification”, Proceeding of the IEEE Global Conference on Consumer Electronics, Kyoto, pp. 1-2.

Ozok, A., Patel, A., Wu, H. and Gurses, A.P. (2011), “Medication adherence among community-dwelling older adults: current practices and potential technology solution”, Proceeding of the Human Factors and Ergonomics Society Annual Meeting, pp. 316-320.

Wu, H.-K., Wong, C.-M., Liu, P.-H., Peng, S.-P., Wang, X.-C., Lin, C.-H. and Tu, K.-H. (2015), “A smart pill box with remind and consumption confirmation functions”, Proceeding of 4th IEEE Global Conference on Consumer Electronics (GCCE), Osaka, pp. 658-659.

Zeng, Z., Fan, X., Wu, Q. and Leung, C. (2017), “Making context-aware and explainable decisions with assumption-based argumentation”, presented at the 5th International Conference on Ageless Aging.


This research is supported by the National Research Foundation, Prime Ministers’ Office, Singapore, under its IDM Futures Funding Initiative. This research is also partially supported by the NTU-PKU Joint Research Institute, a collaboration between Nanyang Technological University and Peking University that is sponsored by a donation from the Ng Teng Fong Charitable Foundation.

Corresponding author

Qiong Wu can be contacted at:

About the authors

Qiong Wu received her bachelor’s degree in Software Engineering from Shandong University, China, in 2010 and her PhD degree in Computer Engineering from Nanyang Technological University (NTU), Singapore, in 2015. Currently, she serves as a Research Fellow at the Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly (LILY). She has years academic experience in AI and her research interests include intelligent agents, artificial curiosity, machine learning and recommender systems.

Zhiwei Zeng received the BEng (Hons) degree in computer science and BBus (Hons) degree from Nanyang Technological University (NTU) in 2015, where she is currently pursuing a PhD degree at the NTU-UBC Joint Research Centre of Excellence in Active Living for the Elderly (LILY), Interdisciplinary Graduate School. Her current research interests include artificial persuasion, its computational modelling and applications of intelligent agents in health-care systems.

Jun Lin is a Research Fellow of the Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly (LILY), Nanyang Technological University, Singapore, and an Associate Professor of College of Software, Beihang University, China. He got the PhD degree in the School of Computer Science and Engineering, Nanyang Technological University, Singapore. His current research interests include crowd science, Internet of Things, LoRaWAN with block chain and AI technologies.

Yiqiang Chen received the BASc and MA degrees from the University of Xiangtan in 1996 and 1999, respectively, and the PhD degree from the Institute of Computing Technology, Chinese Academy of Sciences in 2002. In 2004, he was a Visiting Scholar Researcher in the Department of Computer Science at the Hong Kong University of Science and Technology (HKUST). He is now an Associate Professor of ICT and Vice Director-General of the Shanghai Branch of the Institute of Computing (ICT), The Chinese Academy of Sciences (CAS). His research interests include data mining and mobile multimedia.