Outpatient regulation system in health management: economic bene ﬁ ts of technological innovations

Purpose – The purpose of this paper is to evaluate the economic bene ﬁ ts of managing an outpatient appointmentssystem with technological innovations. Design/methodology/approach – This study uses a quantitative methodological procedures aiming to evaluate the cost-bene ﬁ t relation and also the payback of the management and operation of an outpatient appointmentssystem with technological innovations. Findings – This study found a great bene ﬁ t-cost relation of 30.6 showing the great economic value and social impact of managing an outpatient appointments regulation system with technological innovations. Research limitations/implications – This study presents contribution to the literature discussion about the economic evaluation of the bene ﬁ ts of managing and operating more effective outpatient appointmentssystemsbecause of importanttechnological innovations. Practical implications – This paper presents and discusses the most important and commonly used strategies and technological innovations to deal with and to manage an outpatient appointment regulation system aiming to reducethepatient no-show rates. Social implications – The ﬁ ndings of this study show a great bene ﬁ t-cost relation of about 30.6 which is being reverted to thesociety. Originality/value – There not exist many similar studies in the pertinent literature, mostly with the Braziliancontexts.


Introduction
One of the main roles of the appointment system for medical consultations is to guarantee and prioritize access to the health system for the individual looking for medical assistance. The appointment system of an outpatient department must be effective; that is, it must establish the most adequate health-care access to each patient considering his/her medical needs and health issues involved. An outpatient department must also consider social, demographic and logistical issues in addition to operating efficiently with low no-show rates, thus minimizing the waste of resources required by the health system. It is well known in pertinent literature that the nonattendance of patients to their outpatient medical consultations is a chronic problem in every health system in the world and also in Brazil (Cavalcanti, Cavalcanti, Serrano, & Santana, 2013;Dreiher et al., 2008;Izecksohn & Ferreira, 2014;Moore, Wilson-Witherspoon, & Probst, 2001;Silveira, Ferreira, Silveira, & Siqueira, 2018). The consequences of patients' absenteeism end up generating various types of problems, mostly related to the patients' health and also to adverse operational and economic consequences for society.
When a patient misses his/her medical appointment, he/she is harming his/her own medical treatment; that is, the patient is delaying a medical treatment that could have been initiated immediately to prevent health complications or to combat an illness. The postponement of a health treatment can harm a patient's health irreversibly, which complicates his/her medical condition. In addition to the harm that the patient may cause to him/herself, a no-show appointment ends up giving the outpatient department no opportunity to fill the appointment slot, which eliminates the ability for another patient to receive caresuch situation causes an increase of queues in outpatient departments. If we consider only the above-mentioned disadvantages taking into account solely the patients' health, the social impact is already expressive, but there are other disadvantages in the operational and economic spheres that may affect the society.
Higher patient no-show rates in outpatient departments end up wasting the utilization of the installed capacity and cause a decrease in the productivity of the health system. The problem involved in no-shows reduces the productivity of medical attendance and mostly the nonuse of resources, such as equipment, outpatient facilities, doctors, nurses, secretaries and other human resources involved in the operational routine of outpatient departments; another problem related to the no-show is the waste of investments and expenses involved. This is a serious problem for many countries, especially those with a high demand for health services, where effectiveness should be the priority in the operation of the health system. Some studies show that patient no-show in outpatient departments is a problem that affects the health system not only in Brazil, but also in the world. Some studies indicate patient no-show rates worldwide that may range from 5% or 6% to very high levels, such as 50 or 60% (Cohen, Dreiher, Vardy, & Weitzman, 2008;Deyo & Inui, 1980;Dreiher et al., 2008;Guy et al., 2012;Moore et al., 2001;Murray & Berwick, 2003). In Brazil, these rates may range from 19% up to 48.9% in some locations (Bender, Molina, & Mello, 2010;Cavalcanti et al., 2013;Izecksohn & Ferreira, 2014;Neves, 2017;Silveira et al., 2018).

Outpatient regulation system
In this context, this work aims to answer the following research question: What is the value of an appointment regulation system that uses technological innovations to reduce patient no-show rates in Brazil?
To address or to find answers to the research question, our work aims at analyzing the appointment system of outpatient departments operating in the state of São Paulo, Brazil), considering public health units, i.e. municipal and state-owned health units, which have used technological innovation to reduce the patient no-show rates. Such technological innovation was implement through machine learning algorithms with complex systems models, operational research with heuristic solution methods and optimality criteria to designate the most appropriate appointment for a determined patient considering the region in which the patient lives and the location of the outpatient department. This technological innovation is used by the state of São Paulo to manage citizens' access to public health care.
This study evaluates the benefits considering the benefit-cost ratio, a widely used indicator that evaluates the economic outcomes of, especially, public ventures that must often deal with resource allocation decisions because of the scarcity of resources. Economists and public managers usually use cost-benefits analysis to estimate the economic value of the benefits stemming from the implementation of a project in comparison to the costs and expenses incurred in the project (Alt et al., 2009;Canada, Sullivan, & White, 1996;Elixhauser, Luce, Taylor, & Reblando, 1993;Klarman, 1974;Lehoux & Blume, 2000;Robertson, Skelly, & Philips, 2019;Warner & Hutton, 1980;Williams & Giardina, 1993).
The results show that the annual economic benefits stemming from the operation of the outpatient regulation system in the state of São Paulo are approximately R$250m (in Brazilian currency), while the annual costs sum up to R$8.15m, which results in a benefit-cost ratio of 30.6 and payback of 12 days, which indicate the great value and an important social impact of the management and operation of the outpatient regulation system in the state of São Paulo.

Literature review
Section 2 presents and discusses concepts and theoretical issues on the appointment systems, the problem of patient no-show and also the main technological innovations created to deal with absenteeism and manage the reduction of no-show rates.

Appointment systems in outpatient departments
The regulation of the appointments system is very important for health management to fulfill its democratic role of providing health services equally and indistinctly to all citizens. From a managerial point of view, the appointment system ensures efficiency in the allocation of health resources to promptly meet the demands of the population with quality and fulfilling adequately medical treatments and needs (Cayirli & Veral, 2003;Guy et al., 2012;Ho & Lau, 1992;Murray & Berwick, 2003;Rocha, 2015).
There is a high demand from citizens to specialized health services and each citizen, when looking for some sort of health attendance, may require different services that can create a sequential and iterative flow of health care, consultations, examinations, hospitalization, among others (e. g., Cayirli & Veral, 2003;Cayirli, Veral, & Rosen, 2006;Ho & Lau, 1992;Kaandorp & Koole, 2007;Kemper, Klaassen, & Mandjes, 2014;Klassen & Rohleder, 1996). Such health services may be available at the same outpatient department or hospital at any given time, but the offer is not entirely homogeneous and some procedures, examinations, consultations and/or hospitalizations may not be available all at the same time in the same clinic or hospital. Thus, the appointment system plays a fundamental role in allocating the patientaccording to his/her medical urgencyto the most appropriate health center available within a reasonable period of time (Liu, 2016;Rocha, 2015;Souza, 2017). The health appointment system is an essential instrument to ensure the effectiveness of health management, which administers a large structure divided into three levels of hierarchical health care (primary, secondary and tertiary care)the conglomerate of health basic units, outpatient departments, hospitals, emergency rooms and large hospitals correspond to the tertiary level, i.e. facilities that deal with more complex cases. One of the most important functions of the appointment system is to instantly guarantee and prioritize access to health services for citizens who are in need of medical care. The appointment system must be effective in addition to operating efficiently; that is, it must establish the most adequate appointment to each patient considering his/her medical and health needs while considering other aspects, namely, social, demographic and logistical (Cayirli & Veral, 2003;Cayirli et al., 2006;Denton & Gupta, 2003;Kemper et al., 2014;Klassen & Rohleder, 1996;Liu, 2016;Murray & Berwick, 2003;Robinson & Chen, 2003;Souza, 2017).
The appointment system favors the organization and the control of health services, i. e. it brings clinics, health basic units, outpatient departments, hospitals and emergency rooms together, in an integrated, articulated and structured way, providing high levels of health-care quality to citizens, increasing efficiency and reducing resource waste. The appointment system integrates all health units and provides online information, in real time, of the capacities and availability of appointments for consultations and health exams, procedures and hospital bedsin addition, the system also integrates urgencies (prioritizations) and medical protocols (Deyo & Inui, 1980;Kemper et al., 2014;Liu, 2016;Murray & Berwick, 2003;Souza, 2017;Vissers, 1979).
Appointment systems should use heuristic and allocation criteria that guide the patient toward the most appropriate health unit to optimize the conditions of the health care, to provide the appropriate resolubility of the medical treatment and to consider the mitigation of possible risks involved. Appointment systems can also implement various technological innovations for scheduling appointments and provide clinical sequencing to maximize the utilization of the installed capacity of health units or minimize patient waiting time. To turn the operation of the health system or of an outpatient department into more effective, agile and intelligent actions, the appointment system can use some technological innovations, as machine learning algorithms, along with process optimization techniques and resource allocation. Some examples of these techniques are linear and nonlinear programming, mixed-integer programing, balancing and sequencing procedures, queueing theory, Monte Carlo simulation and Markov chains (used in reinforcement learning) (Bailey, 1952;Cayirli & Veral, 2003;Cayirli et al., 2006;Denton & Gupta, 2003;Ho & Lau, 1992;Kemper et al., 2014;Klassen & Rohleder, 1996;Liu, 2016;Robinson & Chen, 2003).
In addition, implementing technological innovations, such as machine learning algorithms, integration and optimization procedures, will enable a more modern and effective higher-quality management; such management tends to be faster and to provide more assertive responses to the demands of citizens and the appointment system will also allow a more efficient management of the resources involved in the provision of health-care services and also a deeper analysis of the restructuring capacity of the entire health system. Together with the primary functions of health-care regulation, the appointment system can also enable the monitoring of general indicators Outpatient regulation system of production, productivity of the resources used, equipment, employees, maintenance expenses and equipment failures, absenteeism of patients, as well as the generation of occupancy rates in outpatient departments, health basic units, hospitals, hospital beds and health examinations (Cayirli et al., 2006;Deyo & Inui, 1980;Kaandorp & Koole, 2007;Kemper et al., 2014;Klassen & Rohleder, 1996;Zacharias & Pinedo, 2014).
The appointment system also considers, when regulating the access to outpatient departments, the characteristics and needs of the patient. The appointment system should allocate the most appropriate and specialized medical appointment considering some aspects, like personal needs, demographic profile, proximity to the health-care unit and convenience. The appointment system, which considers the principle of priority of patients' treatment and important health protocols related to the complexity of the medical condition and consequent urgency, seeks to allocatewith the support of machine learning algorithmsthe best outpatient consultation appointment to a patient, to improve the chance of his/her attendance, i.e. the attendance of patient to the appointment, minimizing no-show rates (Cayirli & Veral, 2003;Cayirli et al., 2006;Kemper et al., 2014;Liu, 2016;Srinivas & Ravindran, 2018).

Problem of patient no-show in outpatient departments
Absenteeism of patients is a serious problem for the management of any health system, for the patient him/herself and for others involved. In addition to being harmful to the patient's own health condition, the absenteeism causes other problems, as it presents difficulties in managing the outpatient queue, hinders the access to consultations in a determined outpatient facility (considering that the queue is increased), besides creating an idle appointment schedule, which implies loss of time and resources of the management of the health unit and/or of the entire health system (Bender et al., 2010;Cavalcanti et al., 2013;Cohen et al., 2008;Deyo & Inui, 1980;Dreiher et al., 2008;Guy et al., 2012;Izecksohn & Ferreira, 2014;Moore et al., 2001;Murray & Berwick, 2003;Neves, 2017;Silveira et al., 2018). Deyo and Inui (1980), when developing a literature review on absenteeism in medical consultations, found a few studies that reported no-show rates between 15% and 30% in outpatient departments. The authors also cite papers that report lower rates, between 5% and 11%, in family health clinics and mentioned other studies that reported no-show rates of 15%-18% in prepaid health clinics. Guy et al. (2012), when carrying out their literature review, found several studies with reports of patient no-show rates in outpatient departments ranging from 10% up to 60%. Moore et al. (2001) report that some studies indicate no-show rates of 6%, 26.1% and even of 50% in some family health clinics. The authors conducted their study based on 4,055 visits to The Family Practice Center (FPC) of Palmetto Richland Memorial Hospital of the University of South Carolina and found a 24.4% no-show rate. Cohen et al. (2008) conducted a research on absenteeism at a pediatric dermatology clinic of the Clalit Health Service (CHS), an Israeli state-mandated health service organization. The authors analyzed a sample composed of 52,604 appointments over a 44-month period and found a 27.6% no-show rate. Dreiher et al. (2008) also conducted a survey on the absenteeism of patients in obstetrics and gynecology clinics of the Clalit Heath Service (CHS); they analyzed 8,883 appointments in 2003 and indicated a noshow rate of 30.1%.
Considering this specific matter, one finds several examples of no-show rates in Brazilian medical institutions. Bender et al. (2010) conducted a research on the Saco Grande Health Center (CSSG) in Florian opolis from January to July 2008 and analyzed 828 appointments of various outpatient specialties and discovered a no-show rate of 34.4%. Cavalcanti et al. (2013) studied patient absenteeism levels in outpatient departments in the city of João Pessoa between 2009 and 2010, and they found no-show rates between 32.9% and 39.8%. Izecksohn and Ferreira (2014) carried out an analysis of patient absenteeism, from July to December 2012, in family health clinics of the Manguinhos School of Health Center in the city of Rio de Janeiro, which was later incorporated into the School of Health Center Germano Sinval Faria. The authors analyzed 2,272 appointments and found a no-show rate of 48.9%. Neves (2017) studied the patient absenteeism in outpatient departments at the Antonio Pedro University Hospital (HUAP) in the city of Niter oi; this outpatient department belongs to the Fluminense Federal University (UFF). The author analyzed 357,251 appointments made in 2015 and 2016 and found a patient no-show rate of 28.2%. Silveira et al. (2018) conducted a study on patient absenteeism in a basic health unit supported by the Federal University of Pelotas from July 2016 to April 2017. The authors analyzed 3,131 appointments for various outpatient specialties and found a no-show rate of 19.2%. Table 1 summarizes some of the main characteristics of the studies that analyze patient no-show rates in outpatient departments in Brazil.

Technological innovation and reduction of no-show rates in outpatient departments
Patient absenteeism is a serious issue for medical treatment, for other patients waiting in the hospital queue and for the efficient functioning of outpatient departments; thus, such matter has been the focus of several regulation appointment systems. Literature points out some strategies that can be implemented through the employment of technological innovation by outpatient departments to reduce the patient no-show rates.
Among the main strategies, we mention the reduction of waiting time; that is, the time interval between the appointment and the consultation itself. Some studies found that noshow rates increase considerably when the patient faces longer waiting times. Technological innovations, as machine learning algorithms, optimization procedures, dynamic sequencing, Markov chains and others, have been implemented in appointment systems to automatically detect patients' characteristics, patterns and needs to turn the appointment process more assertive and effective and, consequently, reducing patient no-show rates (Benjamin-Bauman, Reiss, & Bailey, 1984;Denton & Gupta, 2003;Deyo & Inui, 1980;Kaandorp & Koole, 2007;Kemper et al., 2014;Klassen & Rohleder, 1996;Liu, 2016;Norris et al., 2014;Srinivas & Ravindran, 2018;Wang & Gupta, 2011).
Some appointment systems in outpatient departments use innovation to automatically create and to send alerts through short message service (SMS), voice mail or e-mails a few  (Almog et al., 2003;Deyo & Inui, 1980;Downer, Meara, Da Costa, & Sethuraman, 2005;Parikh et al., 2010;Robotham, Satkunanathan, Reynolds, Stahl, & Wykes, 2016;Teo, Forsberg, Marsh, Saha, & Dobscha, 2017). Another strategy for managing the problem of patient absenteeism in medical consultations is medical appointment overbooking (LaGanga & Lawrence, 2007;Muthuraman & Lawley, 2008;Vissers, 1979;Zacharias & Pinedo, 2014;Zeng, Turkcan, Lin, & Lawley, 2010). Overbooking is not an ideal solution considering it does not solve the problem definitively because it does not reduce the no-show rate itself; it is, however, a risk management that allows a greater utilization of resources, increasing the productivity of the health system. More recently, other technological innovations, as business intelligence management techniques that use data mining and artificial intelligence algorithms, make it possible to better understand the demographic and behavioral characteristics of patients who represent the majority in no-show rates; the use of these innovations aim at better understanding patient scheduling and better assisting patients to schedule their appointments, reducing no-show rates (Bean & Talaga, 1992;Defife, Conklin, Smith, & Poole, 2010;Lee, Heim, Srikandarajah, & Zhu, 2018;Liu, 2016;Nancarrow, Bradbury, & Avila, 2014;Samorani & LaGanga, 2015).

Research method
The objective of this work is to evaluate the economic and financial benefits of managing an appointment system in outpatient departments; such system operates with low patient noshow rates because of the implementation of technological innovation. Thus, we present herein a quantitative approach, with quantitative methodological procedures. Quantitative methods seek to either develop simulations to represent the functioning or behavior of a determined system or to analyze existing relationships between input factors and output variables or results. Thus, the researcher can use evaluations, simulations and quantitative techniques to analyze the impacts or behaviors of output variables as a result of variations in input factors (Babbie, 1998;Dane, 1990;Davis, Eisenhardt, & Bingham, 2007;Kerlinger, 1979).
To approach economic and financial benefits, a modeling is used specifically for the operationalization of the benefit-cost ratio as a consequence of the management and operation of an appointment system in outpatient departments with lower no-show rates. The cost-benefit analysis is widely used in economic and financial analyses, especially when evaluating the investment in projects carried out by the public sector. It is one of the many cash flow analysis used in economic and financial studies, but it can also be used to measure efficiency of resources allocation in the public sector (Alt et al., 2009;Canada et al., 1996;Elixhauser et al., 1993;Klarman, 1974;Lehoux & Blume, 2000;Robertson et al., 2019;Warner & Hutton, 1980;Williams & Giardina, 1993). "Hence, the B/C is sometimes defined as the ratio of the present or annual worth of benefits for the user public to the present or annual worth of the total costs of supplying the benefits" (Canada et al., 1996, p. 193).
After analyzing and obtaining the cash flow of the project, the researcher obtains the decapitalization of benefits and costs, separately, to access the benefit-cost ratio (BCR) of the project by dividing the current value of the benefits by the current value of the costs, as we can see by the mathematical equation (1): INMR 18,1 where: BCR = benefit-cost ratio obtained at the present date (t = 0, in cash flow); Bj = benefit (cash flow of a period obtained at instant j); J = total number of benefits presented in the project's cash flow; Ck = cost (cash flow of a period obtained at instant k); K = total number of costs presented in the project's cash flow; and (i) = discount rate or minimum acceptable rate of return (MARR), used for the decapitalization of benefits and costs.
The benefit-cost ratio (BCR) can, therefore, assume, for every project, any positive value and the manager's decision is based on this result. If the ratio is superior to the unit (1.0), then the management design is considered feasible and efficient because the current value of the benefits exceeds the current value of costs and investments. The economic evaluation of payback is also used to evaluate the time required for the return or recovery of the investments and expenses incurred with the management of appointment system in outpatient departments. Payback is the instant (t P ) in which the accumulated present value begins to present a positive value, showing the instant in which the accumulation of revenues or benefits, decapitalized in current value, overcome the expenses, also decapitalized to current value, as presented in mathematical equation (2).
where: t P = payback; Bt = benefit obtained in the project at instant t; Ct = cost or investment made in instant t; and (i) = discount rate or minimum acceptable rate of return (MARR), used for the decapitalization of benefits and costs.
Data were obtained from Transparency Portal of the Health Department of the state of São Paulo, from SEADE Foundation, and from CROSS, the organization that manages the appointment system in outpatient departments in the state of São Paulo.

Result analysis and discussion
Section 4 presents an estimation of patient no-show rates in outpatient departments and also the economic evaluation of the benefits of an appointment system in outpatient departments, with technological innovation, operating with lower no-show rates.

Estimation of patient no-show in outpatient departments
The Regulation Center of Supply of Health Services (CROSS -Central de Regulação de Oferta de Serviços de Saúde, 2019) manages the health appointments through appointment systems in the outpatient, hospital bed and emergency spheres in the state of São Paulo. It works in an integrated way with all public health units, i.e. all municipal and state-owned health units at the three levels of hierarchical health care (primary, secondary and tertiary care; the latter deals with complex care units and involves health basic units, outpatient departments, outpatient departments, hospitals, emergency rooms and large hospitals). With a request for consultation, the appointment system in outpatient departments presents an optimal scheduling Outpatient regulation system algorithm that pursues and identifies the best appointment for the patient taking into consideration the region where the patient lives and the location of the outpatient facility.
The appointment system that operates in the state of São Paulo has, as main priority, the fulfillment of requests, medical protocols and meets, by regulating the access to appointments, the needs of the patient regarding appointment scheduling and, at the same time, pursues the most adequate appointment so that the patient attends the medical consultation. To minimize the chances of patient no-show, the appointment system of the state of São Paulo uses heuristic and artificial intelligence algorithms that take into account the accumulated knowledge of no-show ratios considering some aspects, like personal needs, demographic profile, proximity to the health-care unit and convenience to regulate the appointment schedule, minimizing the chances of no-show.
In addition to the employment of these technological innovations, scheduling heuristics and artificial intelligence algorithmswhich identify and designate the most appropriate medical appointment from a medical protocol perspective while minimize patient no-showthe appointment system that operates in the state of São Paulo also presents internal modules responsible for the automatic alerts sent (e-mail, SMS and phone calls) to patients some days prior to the consultation to remind them of the appointment, thus, reducing the chances of patient no-show. If the patient cannot attend the medical consultation appointment, the system proceeds immediately by trying to schedule the appointment to another patient in the queue, hence, avoiding resource waste in the health system.
The outpatient regulation platform of the state of São Paulo schedules appointments, keeps all the information safely stored and relies on disaster recovery as a risk management strategy. The appointment system also providesin addition to data referring to scheduled appointmentsan integrated registration of users, which takes into account data related to the capacity of the health system and the availability of resources of every health unit. Such technological innovations have provided a reduction in patient no-show rates in the operation of the outpatient regulation system of the state of São Paulo, especially when comparing these rates to the ones found in other regions of Brazil, as we can see in Table 1 (in the literature review section) and Table 2, which demonstrates no-show rates across some Brazilian regions and, more specifically, in the state of São Paulo.
Tables 1 and 2 show absenteeism rates with averages of 33.4% and 18.2% in outpatient departments in medical facilities from, respectively, various locations in Brazil and in the  Table 2 indicates that the appointment system in the state of São Paulo also presents greater robustness regarding the response to the increasing growth in the demand for regulation over the past few years. The appointment system in outpatient departments has performed very efficiently, reducing no-show rates even when there was a strong growth in demand for appointments because of the high and continuous inclusion of citizens in the system over recent years, considering that the beginning of the operation of the appointment system in outpatient departments in the state of São Paulo was in 2011. Table 2 and Figure 1 show that, even with a stronger demand for appointments, the CROSS system was able to continuously reduce patient no-show rates until reaching the percentage of 17.39% at the end of 2019; in the same year, more than 11.5 million appointments were scheduled, which represents the double of the appointments scheduled in 2014.
This increase in the number of outpatient appointments, which we observe in Table 2, is the result of the continuous entryin addition to the regulatory system in the state of São Pauloof new outpatient departments, health basic units, clinics and hospitals, in addition to others that were continuously being included in the system. Simultaneously, there was an increase in new registrations of citizens in outpatient departments, which reinforces the need for regulation in the state of São Paulo. Table 3 and Figure 2 show the continuous increase in the number of appointments scheduled in outpatient departments, the population growth and the variation in population growth over the years. Thus, we can observe that the appointment system in outpatient facilities, in the state of São Paulo, performed efficiently and with robustness, reducing noshow rates, dealing with the increasing and continuous number of citizens registered for health case and managing the higher number of health units incorporated into the system.

Economic evaluation of the benefits of an appointment system with technological innovation and low patient no-show rate
To estimate the value of a medical consultation, we must resort to data provided by the Transparency Portal of the Health Department of the state of São Paulo. These data bring additional information on the budget provided by the state government for consultations in specialized medical outpatient departments (AME, in Portuguese). Table 4 shows the total budget appropriations, the total number of consultations and the expenses per consultation in AMEs in the state of São Paulo during 2018. This knowledge provides us with the average expense consultation in outpatient departments; approximately R$141.88 (in current Brazilian currency). Table 5 summarizes the main values obtained in our study that are important for assessing the economic value of the benefits stemming from the management and operation of the appointment regulation system in the state of São Paulo, that is, the amount saved (not wasted) because of a lower patient no-show rates. The average difference between noshow rates in the State of São Paulo and those in other regions of Brazil is 15.18%.  Considering the total number of almost 11.6 million appointments scheduled in the State of São Paulo in 2019, we estimate that almost 1.76 million consultations were not lost; that is, these appointments were attended by patients who benefited from the appointment system of the state of São Paulo. Considering the average expense (in Table 5) of a medical consultation in an outpatient department (AME) -R$141.88 in 2018we estimate an economy of around R$250m annually because of the appointment regulation system, in the state of São Paulo, which reduced patient no-show rates.
To evaluate the economic benefits, we used the BCR, which is widely used for economic analysis of many public projects. Our intention was to verify if the benefits (when the BCR is greater than 1) overcome the investments and costs involved in the project. Thus, considering the annual costs of the operation of the regulation appointments system (R$8,151,378.00, in current Brazilian currency), we have a BCR of 30.6, a very high value that indicates that the benefits of the operation involved in the appointment system in outpatient departments in the state of São Paulo presents a high economic and social impact.
We can also analyze the economic value of the appointment system through the payback method, which evaluates a project by measuring the time it will take to recover the initial investment and costs. Thus, payback represents the time required to accumulate revenues or benefits, in current values, which must be sufficient to exceed the accumulation of expenses, also in current values.
When we crosscheck the annual costs of the operation of the appointment system with the benefits, the resulting payback is very small (12-days payback period) considering the annual basis of the analysis. It means that the economy (benefits generated by the operation of the appointment system, with lower rates of patient no-show in the state of São Paulo) in the first 12 days of the year would already be enough to recover the costs of the entire year involved in the operation of the appointment system.

Conclusion
This study aimed to analyze the economic evaluation of the benefits resulting from the operation of an appointment system in an outpatient department that relies on technological innovation to reduce the rates of patient no-show. The nonattendance of patients to their medical consultations is a chronic problem in all health systems all over the world, including Brazil. The consequences of patient absenteeism end up generating various types of problems related to not only the patients' medical condition but also operational costs for the society as a whole.

Outpatient regulation system
The appointment system is essential for health management to take on the role of provider of health-care services equally and indistinctly to all citizens. From a managerial point of view, the appointment system guarantees efficiency in the allocation of health resources to promptly meet the demands of the population and effectively provide the correct and needed medical treatment with quality. When a patient does not attend his/her medical appointment, he/she might interfere negatively with his/her medical treatmentin addition, no-showers end up giving the outpatient department no opportunity to fill the appointment slot, which eliminates the ability for another patient to receive care; such situation causes an increase of queues in outpatient departments. These circumstances also entail an accentuated decrease in the productivity of the health system. It is a serious problem for countries, especially those with greater demand for health services, where effectiveness must be a priority in the operation of the health system. Our study evaluates the benefits arising from the management of the appointment system in the state of São Paulo, which has implemented several technological innovations to reduce patient no-show rates in medical consultations. Our results show a benefit-cost ratio of 30.6 and a payback of 12 days, which indicates a high economic relevance and social impact in managing and operating lower rates of patient no-show.
Thus, this work contributes to the discussion in literature about the economic evaluation and the benefits stemming from the operation of an appointment system with lower rates of patient no-show (Guzek et al., 2015;Kheirkhah et al., 2016;Moore et al., 2001). Our study found a high value and social impact in carrying out some technological innovations in the appointment systems to operate with lower patient no-show rates, which strongly reduces resource waste in the health system, boosting productivity and effectiveness.
Among the main managerial implications, this work presents some relevant issues related to the technological innovation used by the appointment system in the state of São Paulo, such as business intelligence, data mining, analytics and artificial intelligence. The use of these resources was important to recognize patterns or accumulation of knowledge about demographic profile, medical needs and behavioral characteristics of patients. We also mention the employment of heuristic criteria and scheduling algorithms for the designation of the most appropriate appointment to a determined patient. We also highlight the importance of automatic alerts created to remind the patient about his/her appointment to minimize the occurrence of patient no-show.
For future studies, we suggest the evaluation of the benefits of appointment systems in the health-care sector that aim at lower no-show rates in terms of quality of life and health gains for patients who have stopped not attending medical appointments. We also suggest the evaluation of the benefits in medical consultations carried out in outpatient departments according to each medical specialty. Another suggestion is the conduction of a study to evaluate the benefits arising from the effectiveness of the emergency appointment system and also the access to hospital beds for medical procedures in general.