Benford's law for integrity tests of high-volume databases: a case study of internal audit in a state-owned enterprise

Abstract

Purpose

The technical feasibility of using Benford's law to assist internal auditors in reviewing the integrity of high-volume data sets is analysed. This study explores whether Benford's distribution applies to the set of numbers represented by the quantity of records (size) that comprise the different tables that make up a state-owned enterprise's (SOE) enterprise resource planning (ERP) relational database. The use of Benford's law streamlines the search for possible abnormalities within the ERP system's data set, increasing the ability of the internal audit functions (IAFs) to detect anomalies within the database. In the SOEs of emerging economies, where groups compete for power and resources, internal auditors are better off employing analytical tests to discharge their duties without getting involved in power struggles.

Design/methodology/approach

Records of eight databases of an SOE in Argentina are used to analyse the number of records of each table in periods of three to 12 years. The case develops step-by-step Benford's law application to test each ERP module records using Chi-squared (χ²) and mean absolute deviation (MAD) goodness-of-fit tests.

Findings

Benford's law is an adequate tool for performing integrity tests of high-volume databases. A minimum of 350 tables within each database are required for the MAD test to be effective; this threshold is higher than the 67 reported by earlier researches. Robust results are obtained for the complete ERP system and for large modules; modules with less than 350 tables show low conformity with Benford's law.

Research limitations/implications

This study is not about detecting fraud; it aims to help internal auditors red flag databases that will need further attention, making the most out of available limited resources in SOEs. The contribution is a simple, cheap and useful quantitative tool that can be employed by internal auditors in emerging economies to perform the first scan of the data contained in relational databases.

Practical implications

This paper provides a tool to test whether large amounts of data behave as expected, and if not, they can be pinpointed for future investigation. It offers tests and explanations on the tool's application so that internal auditors of SOEs in emerging economies can use it, particularly those that face divergent expectations from antagonist powerful interest groups.

Originality/value

This study demonstrates that even in the context of limited information technology tools available for internal auditors, there are simple and inexpensive tests to review the integrity of high-volume databases. It also extends the literature on high-volume database integrity tests and our knowledge of the IAF in Civil law countries, particularly emerging economies in Latin America.

Citation

Morales, H.R., Porporato, M. and Epelbaum, N. (2022), "Benford's law for integrity tests of high-volume databases: a case study of internal audit in a state-owned enterprise", Journal of Economics, Finance and Administrative Science, Vol. 27 No. 53, pp. 154-174. https://doi.org/10.1108/JEFAS-07-2021-0113

1. Introduction

International Standards on Auditing (ISA) encourages auditors to use analytical procedures during the planning, execution and completion phases of the audit to identify, among others, the existence of unusual trends (ISA 300, 2009; ISA 315, 2016; ISA 330, 2006). Motivating this study is the fact that internal auditors operating in state-owned enterprises (SOEs) in emerging economies need to employ analytical techniques that allow them to discharge their duties without interfering with the political process that might ask to ignore rules, procedures and best practices in corporate governance. This study focuses on mitigating uncertainty by providing a test interpreted as an indicator of the confidence or alert of the possible inherent or pre-existing risk of the computerized data that are made available to the internal auditors in their routine tasks.

The inherent risk of information is determined by the characteristics of the entity and the information system under analysis. Auditors cannot change the inherent risk level, but if the auditor knows it, audit planning and execution can be better tailored. ISA 315 (2016) and ISA 330 (2006) define it as an a priori measure of risk that is independent of the applied controls. To properly measure the inherent risk level before evaluating the existence and effectiveness of internal controls, the auditor needs to find and investigate possible errors or material differences. International regulations urge internal auditors to act in computerized environments to evaluate the reliability of the data used by testing controls and applying substantive data tests at the transaction level (ISA 315, 2016; ISA 330, 2006). The literature documents the advantage of leaving proper audit trails (Okundaye et al., 2019) and propitiates the testing of large data sets to generate evidence of consistency in the data (Cleary and Thibodeau, 2005), one of which is Benford's law (Nigrini, 2019).

Benford's law considers certain digits to appear more frequently than others in a given data set, and it is used to verify whether or not the behaviour of a set of numbers conforms to expectations, assuming no interference or manipulation. It predicts that more than 30% of the numbers begin with digit 1, 18% with digit 2, and it descends successively to 9 with an incidence of less than 5%. This pattern of numbers behaviour responds to a logarithmic function (Benford, 1938). In the broad field of assurance, Hill (1995) is one of the first authors to suggest the presence of possible risks of irregularities or fraud when the behaviour of a representative set of data does not comply with Benford's distribution. This line was continued by Nigrini (1999, 2019), but in all cases, its application was circumscribed to transactions.

This study reports the results obtained from eight enterprise resource planning (ERP) modules of a large SOE in Argentina's energy sector. Each module contained 3–12 years of operational data. To test compliance with Benford's law, the number of records (size) of the tables of each module was used, including more than 1,900 tables with 4,500 million records. The results show that Benford's law is satisfied when all modules are considered, but the same does not hold when modules are considered individually. The decision to further investigate is political and outside the auditor's hand, but at least they can leave a paper trail showing the results of an unbiased analytical test.

The contributions of this study are twofold. First, it explores the usage of Benford's law at a dimension above the transactional level (i.e. at the meso level), something that has been unexplored in the literature. In an ERP system, the meso level is the tables that conform to each module and is an issue not covered in Nigrini's (2017) literature review, where five new perspectives on using Benford's law in auditing are presented. Analysing the meso level is deemed to be an important area to explore as (non-)compliance with Benford's law could potentially indicate an issue with the completeness of the modules that the internal auditor is provided for assessment. Second, by selecting a case from an emerging economy from Latin America that has institutions rooted in Civil law, this study expands our understanding of the internal audit function (IAF) (Kotb et al., 2020) particularly in emerging economies (Salcedo, 2021). Although Benford's law is frequently used in economics and finance, very little is available for applications in Latin American countries (Carrera, 2015). Furthermore, previous literature has documented that internal audit departments in emerging economies do not function effectively due to political interference (Emmanuel et al., 2013) and inadequate support from top management (Ahmad et al., 2009). As such, shedding light on a rarely discussed aspect of internal auditors working in SOEs in Latin America enhances our understanding of the profession outside Common law environments.

The remainder of this paper is structured as follows. The second section reviews the literature on IAF, focusing on digital analysis in the public sector, paying attention to the relevant discussions coming from emerging economies as well as on applications of Benford's law. The third section reviews the details of the company and country where the case is developed by providing evidence of its uniqueness and common traits that might be present in other countries or companies justifying a more extended use of the proposed tool by internal auditors. The next section describes the data and analyses performed together with a discussion and implications for practice. The write-up closes with a conclusion that highlights the key contributions.

2. Literature review

The roles and practices of internal auditors have not been as profusely studied as those of external auditors and public accountants. IAF size has been considered a predictor of information technology (IT) tools and responsibilities; the larger the IAF within the government unit, the more comprehensive the IT risk assessment and use of sophisticated audit technologies (Garven and Scarlata, 2021). There is a set of studies linking internal audit quality and external auditors work; lately, this stream of research has moved into big data and refined analytics tests (Boskou et al., 2019), and little by little studies done outside countries governed by Common law start to emerge (Oussii and Boulila Taktak, 2018). Another lens used to study the work and practices of internal auditors have been their role and place in corporate governance, particularly when the IAF reports to managers and the audit committee as well as to other powerful interest groups such as unions (Erasmus and Coetzee, 2018); but again, in this line of research, there is little if anything about the reality of non-Common law countries (Hay and Cordery, 2018; Kabuye et al., 2017; Kotb et al., 2020). Limited research has suggested that internal auditors working in SOEs of emerging economies are pressured to please diverse powerful groups (Young et al., 2008) to the point that a Latin American president needed to make an open call for more professional internal audits (Ruiz-Tagle, 1998). The remainder of this section identifies the existing disperse literature to show that no previous study has extended the testing of an existing tool, Benford's law, in a novel environment in Latin America, which increases the external validity of Benford's law tools to assess the inherent risk of large database information.

2.1 Digital analysis and IT audit

Some of the challenges that the audit profession is currently facing involve the increased use of Big Data and the application of advanced analytics by clients (Appelbaum et al., 2017). Big data is characterized by client data that exhibit enormous volume, high velocity and a large variety (Cukier and Mayer-Schoenberger, 2013). The emergence of big data provides a broad range of opportunities for auditors to utilize Audit Data Analytics (ADA) (Appelbaum et al., 2017). As defined by Byrnes et al. (2015, p. 92), “Audit data analytics (ADA) is the science and art of discovering and analyzing patterns, identifying anomalies, and extracting other useful information in data underlying or related to the subject matter of an audit through analysis, modeling, and visualization for the purpose of planning or performing the audit.” Due to the challenges that auditors face in evaluating financial and non-financial structured and unstructured data, auditing has begun to adopt and employ a variety of data analytics and artificial intelligence (AI) tools to gain insight into the auditee's performance (Kokina and Davenport, 2017). Evidence has emerged that AI can also be combined with robotic process automation to improve audit efficiency and effectiveness (Zhang, 2019). As a matter of fact, it has been shown that a variety of internal audit procedures can be automated (AICPA, 2015). To further support this assertion, recent literature notes that new technologies (e.g. data analytic tools) are being progressively implemented in internal audit departments (Betti and Sarens, 2021).

Various ADA tools have been proposed in the literature to perform analytical procedures at various phases of an audit (Appelbaum et al., 2016, 2017). These analytical models range from simple substantive tests to more advanced predictive techniques (Appelbaum et al., 2017). Among the ADA tools is the application of Benford's law for big data analysis (Lanham, 2019).

2.2 Benford's law

Frank Benford, a physicist at GE Research Laboratories, conducted a study of digit frequencies in tabulated data (Benford, 1938). Benford empirically tested the first digit frequencies of 20,229 observations (20 lists of relatively large numbers) and 2,968 observations (10 lists of relatively small numbers). His empirical results showed that 30.6% of the large numbers had the number 1 as the leading digit and 4.7% had the number 9 as the leading digit. He then hypothesized that, when ordered, naturally occurring data should form a geometric sequence. By applying integral calculus, Benford was able to formulate the expected frequencies for the first and second digits. The derived mathematical formula to compute the probability of observing the first leading digit is given by the following mathematical expression (Nigrini, 2012):

Prob(D1=d1)=log(1+1d1); d1∈{1,2,…,9}

where d₁ is the leading first digit that can take a value of 1–9.

In a naturally occurring data set, the number 1 is expected to occur with a frequency of 30.10%, the number 2 with a frequency of 17.61% logarithmically decreasing to 4.58% when the leading digit is 9. Benford's law could also be utilized to calculate the first-two, first-three, first-four and first-anything digits (Nigrini, 2012); the general formula is:

Prob(D1=d1,…,Dk=dk)=log[1+(1∑i=1kdi×10k−i)]

As mentioned before, this study is only concerned with applying Benford's law to the first leading digits as an integrity test for large databases.

2.3 Benford's law applications in accounting and auditing

The first studies using Benford's law in accounting emerged in the late 1980s following Varian's (1972) suggestion that Benford's law can be used as an honesty or validity test of supposedly random scientific data in a social science context. Two earlier studies relied on this analysis to detect revenue manipulation. Carslaw (1988) found that the earnings of New Zealand companies did not fit the expected distribution, and Thomas (1989) discovered a similar pattern in American companies' profits. Hill (1995) provided a test for Benford's law and demonstrated how it was applied to stock market data, census statistics and certain accounting data. He noted that Benford's distribution, similar to the normal distribution, is an empirically observable phenomenon.

Nigrini seems to be one of the first academics to apply Benford's law to accounting numbers to detect possible fraud. Nigrini (1996) used digit analysis to identify tax evaders based on earlier studies on earnings manipulation (Carslaw, 1988; Thomas, 1989) and added Benford's law as a test. Other earlier precursors were Busta and Weinberg (1998), who argued that Benford's analytical review procedure would allow the detection of whether the data set has been contaminated or not. Since then, there have been many different articles on practical applications for auditing purposes based on tests in sets of accounting numbers (Nigrini and Mittermaier, 1997; Nigrini, 1999, 2017, 2019), such as estimating the number of records omitted from a data set (Carreira and Gomes da Silva, 2016); explanations of why accounting data might not conform and how it is an indicator of data quality (Cleary and Thibodeau, 2005); a tool to discriminate between data sets that are naturally occurring and others that are not (Bhattacharya et al., 2011), visual analysis (Singh and Best, 2016) and the prevalence of cosmetic earnings management (Lacina et al., 2018).

However, academic literature is somewhat cautious in making claims about the adequacy of procedures based on Benford's law to detect fraud. In general, it is argued that if a data set is tested and it does not conform to Benford's law, it can only show operational inefficiencies or system failures, rather than pointing towards fraud. Under that view, it is understood as instructions on which direction to set the emphasis of control. Most studies that have tested Benford's law have focused on transactions or at the micro-level (Nigrini, 2017, 2019), while others using visualizations seem to be more at the macro-level (Singh and Best, 2016); however, little attention has been paid to the meso level of data. In an ERP system, the meso level is the tables that conform to each module and is an issue not covered in Nigrini's (2017) literature review, where five new perspectives on using Benford's law in auditing are presented. Analysing the meso level is deemed to be an important area to explore (non-)compliance with Benford's law, as this could potentially indicate an issue with the completeness of the modules that the internal auditor receives. As such, one of the primary objectives of the study is to verify whether Benford's statistical distribution is found in the set of numbers represented by the size (number of records) of the tables that make up a relational database. Subsequently, this can help deepen the analysis auditors use to determine the indicators of inherent risk.

2.4 Internal auditing in the public sector

Having identified Benford's law as a promising tool to assess the integrity of high-volume data sets, we explore its antecedents of use in the public sector. A search is EBSCOhost Business Source Complete databases using the keywords “internal audit” and “public sector” produces 15 results, of them only six are in English; four are based on eastern Europe cases from North Macedonia (Bozinoska, 2020), Bosnia and Herzegovina (Basic et al., 2015) and Romania (Emil and Mircea, 2008; Simona and Elisabeta, 2013), with the remaining two from Israel (Scwartz and Sulitzeanu-Kenan, 2002) and Australia (Chowdhury and Shil, 2019).

The scan of the existing literature reveals limited studies on the topic, with the majority focusing on descriptive approaches to internal audit regulation in the public sector. Despite the long tradition of studies on internal auditing in the public sector (Hay and Cordery, 2018; Kotb et al., 2020), none have applied Benford's law. Several studies describe the institutions and laws governing the IAF in the public sector, particularly the nature and effects of adopting the Public Internal Financial Control guidelines of the European Commission (Basic et al., 2015; Bozinoska, 2020; Emil and Mircea, 2008). Others present and discuss some elements related to risk assessment and management (Garven and Scarlata, 2021), either understood as an important activity that is seldom performed due to lack of resources (Simona and Elisabeta, 2013) or as a central activity of the IAF in the public sector (Chowdhury and Shil, 2019). Although the IAF in SOEs and in the public sector, in general, have experienced a formalization of rules and practices since the 1970s, it has been argued that not all promises were delivered (Kotb et al., 2020; Pilcher, 2014; Scwartz and Sulitzeanu-Kenan, 2002). Given the limited existing literature on the topic, this study intends to fill a void by documenting the use of a specific tool, Benford's law, in SOEs that is accepted for the IAF in the private sector.

2.5 Internal audit function in emerging economies

Latin America is not an environment that has informed much research, despite many professionals in these countries voluntarily adhering to the first Statement of Responsibilities of Internal Auditor (AIA, 1947). Argentina has been at the forefront of internal auditors' professionalism, as reported in early studies when it was the only South American country mentioned, while all the other regions surveyed in Vanasco (1996) included at least two countries. This notion of Argentina being among the most advanced environments for internal auditors in Latin America is reinforced by Burnaby and Hass (2011) who conducted a historical analysis of the IAF in the American continent. Their report shows that Argentina was the first country in Latin America to join the Institute of Internal Auditors in November 1960, followed by Colombia in 1971 and Brazil in 1977. Not only do internal auditors have a long tradition in Argentina, but the economy and the stock market are large enough to be used for comparative studies designed to elicit, among other elements, the determinants of corporate reporting. Argentina is also featured in international studies focused on emerging issues in internal audits, such as cybersecurity audits (Islam et al., 2018), consolidating the stature of a country with an audit profession relevant to study.

As seen in the previous section, IAF in the public sector has received some attention, but there are very few studies that explain or document the practices in countries not governed by Common law. The work of Abuazza et al. (2015) is a rare exception, but as they indicate “internal auditing remains under-researched in the context of developing countries” (p. 562). More under-researched are internal audit practices in SOEs of Latin America; therefore, shedding some light on what is done by internal auditors in this context is one of the main contributions of this study. It is of special interest to achieve a better understanding of the interplay of different tenures and powerful professional network expectations from the IAF in the public sector, particularly because “The preponderance of political appointments at senior agency levels and the short tenure of top management officials divert attention to short-term opportunities for political gain and away from the uncovering of administrative shortcomings that may cause political pain” (Scwartz and Sulitzeanu-Kenan, 2002, p. 212). A similar idea is studied and documented in Nigeria by showing the links between “political interference and local government performance and the moderating role of internal audit” (Usang and Salim, 2016, p. 111).

This study contributes to the literature on IAF practices by shedding light on a geographical area that has seldom been studied. The review revealed that no other study has documented the use of Benford's law to identify the inherent risk of information coming from large databases in the SOE of Latin America (Kotb et al., 2020). This study attempts to fill the void with preliminary results from a case study (Yin, 2017). As documented in previous sections, there are few studies exploring tools or techniques to elucidate the inherent risk of information contained in large databases that are available to internal auditors for its control, as per ISA 315 (2016) and ISA 330 (2006) (Nigrini, 2017, 2019; Singh and Best, 2016). It is worth mentioning that many economies of Latin American countries rely on large SOEs that either operate as monopolies or compete directly with a handful of private companies. Many of these companies are suspected of harbouring systemic fraud and corruption, limiting the IAF to its more basic tasks and putting internal auditors in a difficult position where compliance is mainly about form but not content similar to what has been observed in other contexts (Scwartz and Sulitzeanu-Kenan, 2002; Usang and Salim, 2016). Consequently, this study is not designed to build a theoretical framework or expand the existing theory in internal audits or IAF; instead, it is a test of current ideas in a novel environment with the intent of providing external validity to the use of Benford's Law to assess the inherent risk of information contained in large databases.

3. Case study: country and company

The geographical context was selected primarily because almost all previous studies were performed in countries that are based on a Common law environment, while Argentina offers the possibility of research and study in a G-20 country governed by Civil law. The research started as an exploration of issues of performance management interested in assessing the value an SOE delivers to its stakeholders. This fuzzy research design allowed researchers to make a series of organized but informal site visits and interact with multiple subjects (Yin, 2017) where soon emerged conversations about the role and tools available for internal auditors. This study focuses on internal audit tasks in an SOE that generates and distributes energy and documents a simple technique to measure ERP data's inherent risk. This study does not focus on detecting fraud or investigating auditing practices in the context of systematic public sector corruption (Neu et al., 2013). Instead, here is reported the use of a tool that allows internal auditors to discharge their duties without converting them by default in whistle-blowers of fraud and corruption practices and/or incidents, although case studies are seldom used to report on the IAF (Kotb et al., 2020). This limited exploratory case study makes explicit all steps followed in the application of an audit tool, allowing for replications (Yin, 2017).

The company where the test is applied is an organization owned and run by a provincial government in Argentina that generates, transports and distributes electric power. The origins of the company date back to smaller companies set up between 1896 and 1909, but the current organization and structure were established after the Second World War. It currently serves more than three million people, has more than one million paying customers and slightly more than 3,000 employees. The union representing its employees is considered among the most traditional, powerful and well-organized of the province, and in the last couple of years, the union has been in open confrontation with the management group that is implementing a plan to boost efficiency. As part of that plan, internal audits and investigations have been performed, and a handful of employees have been separated from their functions and fired [1] due to fraud, inappropriate access to information systems and asset misappropriation.

Internal auditors are placed in a difficult situation in which they are pressured to deliver results to two competing and powerful groups within the organization, managers and the audit committee on one side and their own union on the other (Young et al., 2008) and still need to discharge their professional duties. Following Länsiluoto et al. (2016) and their characterization of internal control effectiveness, the company oscillates between two of the four clusters along with its history, all depending on the political interference on its management. The argument for placing the company in Cluster 4 of “lowest compliance with laws and regulations” is mostly based on exacerbated political interference of appointed managers and directors as well as power struggles with strong union leaders. In times of limited political interferences, it can be characterized as a firm with “low effectiveness and efficiency of activities of internal controls” as described in Cluster 3. The differences in political influence over control systems in general and the IAF in particular are evident in this company. Internal auditors need to be objective in their evaluations to maintain their professional license and must also walk a fine line so as not to unsettle managers and union leaders who have a primary influence on the internal audit group (Gramling and Schneider, 2018). In this context, performing formulaic analytical tests allows them to be aware of pervasive control deficiencies across the organization or in certain units or departments. Given the work dynamics and power balances in the company, internal auditors collaborate with information system auditors (Bozkus Kahyaoglu and Caliyurt, 2018) to act as advisers in cybersecurity compliance for the most powerful clans and tribes while acting as enforcers with certain individuals not linked to known clans. This strategy provides longevity to employees' tenure.

3.1 Differences in legal systems

The legal theory asserts that a country's approach to market regulations and legislation is mainly defined by its legal tradition. All emerging economies have basic legal structures that are received and determined by their colonizers. Historically, there are two very different legal traditions that were spread in the world by colonizers: Common law and Civil law (La Porta et al., 2008). It is argued that the historical origin of a country's legal system has a direct influence on the regulation of individual contracts and markets, including corporate governance and the IAF's regulations and procedures. Institutions supporting the IAF can be based on two main models, Westminster and Napoleonic (Hay and Cordery, 2018), each of which is rooted in one type of legal system. Common law relies on contracts and litigation among the parties involved to solve a dispute. On the other hand, Civil law provides government and other recognized institutions, such as unions, direct intervention and supervision of markets and contractual relations and agreements between individuals. The literature has documented that countries based on Civil law have more stringent regulations for the entry of new firms and various types of dispute resolution in courts (Djankov et al., 2002, 2003; La Porta et al., 2008).

Significant differences have been documented between countries governed by Common and Civil law. Djankov et al. (2003) constructed an index of the procedural formalism of dispute resolution in 109 countries and grouped them according to legal tradition. Civil law countries show very high levels of court formalism when compared with Common law countries; high formalism is said to be associated with less fairness and impartiality in the process, less honesty in the parties involved, less consistency in process and outcomes and less confidence in the legal system by the public. The implantation of Civil law legal tradition in emerging economies implies “higher expected duration of judicial proceedings, less consistency, less honesty, less fairness in judicial decisions, and more corruption” (Djankov et al., 2003 p. 453). More details are provided by Djankov et al. (2002), who measured the differences in regulation of entry along with eight indicators (six of which are replicated in Table 1). Civil law countries exhibit a higher level of requirements to set up a company, and the entire process takes more time. It is argued that a more stringent entry regulation is associated with higher corruption and larger unofficial economies. The authors conclude their evidence “supports the public choice view that entry regulation benefits politicians and bureaucrats” (Djankov et al., 2002 p. 1).

A third set of evidence showing how regulations differ according to legal origin is provided by Botero et al. (2004). They demonstrate that countries of legal origin of Civil law have clearly higher levels of labour regulation than Common law countries and that in turn is associated with lower labour force participation and higher unemployment, especially among young people. The system of laws and institutions designed to balance power between workers and employers can be organized into three elements. Botero et al. (2004) built indexes for each of the three variables that are designed to measure the incremental cost to the employer of deviating from a hypothetical rigid contract (employment law), the power of unions (collective relations law) and the generosity and cost of benefits (social security law). Argentina is an interesting case to study because it is a country with extreme values in almost all aspects measured to date (see Table 1), placing it at the opposite extreme of Common law countries that have concentrated most of the research on IAF practices and tools.

In this context, it is relevant to mention that the origin of the IAF can be traced to countries governed by Common law. Regulations that guide professionals are built within the Common law framework in mind, but not being prescriptive and detailed, create some noise to internal auditors operating in environments shaped by Civil law (Hay and Cordery, 2018), particularly if they report to antagonist powerful groups (Young et al., 2008). When all aspects are considered, we can suggest that internal auditors of SOEs operating in an environment where (1) managers are politicians and change with each government, (2) unions and their leaders are very powerful and remain after changes in government and (3) courts operate with high formalism and governments are extremely bureaucratic, need to use depersonalized analytical methods to discharge their responsibility without affecting the vested interests of all parties that converge in large SOEs. This is well-aligned with Okundaye et al.'s (2019) observation of using information and communication technologies to provide accountability and proper audit trails.

4. Method

4.1 Data collection and variables

To empirically test whether Benford's law fits the table size of a database, data from a large company operating in Argentina's energy sector were used. The Oracle relational database consists of different functional modules according to the specificity of the process. For the purpose of the analysis, eight functional modules were considered including: Accounting (AC), Human Resources (HR), Inventory (IN), Operations Management (OM), Internal Requests (IR), Commercial Management (CM), Payroll (PR) and Records Management (RM). Each module comprises tables containing unique financial information. For example, the AC module contains individual tables ranging from data entities from the chart of accounts to the general ledger's transactions of the period as well as the opening and closing balances of each account, or in the CM module, from customer data to monthly service billing.

4.2 Analytical procedures

The data analysed are the number of records (size) of the different tables that comprise each module that conforms the database. These values ​​are obtained at a given time when a backup is executed, and they constitute a type of database profile. Null tables that did not have records were excluded from the analysis. The database for this case study was formed by selecting eight heterogeneous ERP functional modules. Heterogeneity arises when considering modules with different characteristics in terms of the type of processes they perform and the information volume they store. Thus, greater complexity is generated in the analysis and, consequently, greater robustness in the results achieved.

Table 2 describes the composition of each of the selected modules that constitute the database (volume in number of tables and records within these). The database contains a total of 1,923 tables with almost 4.5 billion records. The CM module is the most important, with 808 tables (42.02%) that reach 91.25% of the data with 4,093 million records, which implies a per-table average of more than 5 million records. On the other extreme is the HR module with 340 tables (17.68%), 0.03% of the data (1.5 million records) and an average of only 4,365 records per table.

To calculate the distribution frequency, the data were classified by counting the first digit of the number of records contained in each table. This task can be performed with different commands or tools for spreadsheets or with Audit Command Language (ACL Services, now Galvanize) audit software, which has a specific command for Benford's analysis. Table 3 describes the frequency of tables with records starting with digits 1 to 9.

5. Results

For the first digit's test, the bibliography recommends considering more than 1,000 observations. The total number of tables that comprise the database satisfies the requirements, but not each of the modules. However, in Wallace (2002), Benford's compliance was verified using four data sets with only 67 observations each. Only one module was below the threshold (OM), but three were well above (HR, CM and PR). Table 4 shows for each module the frequency listed in Table 3 transformed in percentage terms with the addition of the column indicating the expected Benford's law distribution for the first digit.

If the frequency of each module is compared with the expected ones, as determined by Benford's law (last column of Table 4), coincidences and deviations are observed. Some deviations appear to be significant; e.g. the accounting module (AC), where 6 is the first digit in 12.8% of the tables that compose it, when the expected according to Benford is 6.7%. This issue may be a consequence of the small sample size, where the AC module comprises only 78 tables in total. The visual comparison of the total values and the three largest modules with Benford's distribution shows differences, although not significant, to the naked eye (see Figure 1).

Visually, some deviations can be perceived, but it cannot be determined if the differences are material until a chi-square goodness-of-fit test is performed. Null hypothesis (Ho): actual data or observations follow the probability distribution expected by Benford's law. The chi-square formula (χ²) is as follows:

χ2=∑d=m9(pobs(d)−pt(d))2pt(d)

where P_t(d) is the expected frequency according to Benford (Table 4, last column), P_obs(d) is the observed frequency (Table 4) and m is the digit analysed. In this study, only the first digit (m = 1).

Table 5 lists the components of χ².

The critical value for χ² distribution with α = 0.05 (95% confidence) and eight degrees of freedom [(9 rows – 1) × 1 column] results in χ² (0.95, 8) = 15.51. As the statistic obtained in Table 5 is less than the critical value of 15.51, the null hypothesis, that the entire database conforms to Benford's law, cannot be rejected. The χ² test can then be applied to each of the ERP modules. Similarly, Table 6 shows the actual frequencies (obtained in Table 4) and the expected frequencies according to Benford's distribution. The values obtained for χ² in Table 6 are lower than the critical value of 15.51, so all modules conform to Benford's law.

It should be clarified that another alternative to execute χ² for the total database is the sum of all χ² values obtained for each module. This yields χ² = 64.28 for the entire database (see Table 6). In turn, the critical value χ² will be 56 degrees of freedom [(9 rows – 1) × (8 columns – 1)], which is 74.47. As the critical value is greater than the sum of the statistics obtained in Table 6, it is also confirmed that the behaviour of the actual frequencies for the entire database conforms to Benford's law, and there is no red flag to be raised by the internal auditor regarding the integrity of the data at the meso level.

Nigrini (2012) considered it appropriate to apply the goodness-of-fit analysis using the mean absolute deviation (MAD) test:

MAD=19∑d=19|Pobs(d)−Pt(d)|

where P_t(d) is the expected frequency of occurrence, according to Benford and P_obs(d) is the frequency of occurrence observed.

When this statistics is used for Benford's law, Nigrini (2012) argues that the conformity level can be determined according to ranges. The conformity level is close (high) when the MAD value is between 0.000 and 0.006, it is acceptable with MAD values between 0.006 and 0.012, and it is marginally acceptable with MAD values between 0.012 and 0.016. The data show no conformity with Benford's law when the MAD value is above 0.016 (Nigrini, 2012). Based on Table 6, the MAD calculations can be performed and qualify each module and the total database according to Nigrini's criteria (see Table 7). Based on this, the internal auditor can raise a couple of red flags that would benefit from further tests. Modules with more than 67 tables and MAD with non-conformity (low) are good candidates for further tests if approved by managers appointed by politicians and not objected by union representatives. In this case, modules HR and RM would be the first to consider, followed by IR, AC and IN.

6. Discussion

The discussion of results focuses on two aspects, namely test results and their implications for the IAF and profession in SOEs in Latin America. The results obtained by the χ² test allow us to assert that the first digit of the number of records contained in all the tables of the analysed database follows Benford's law, and no red flags are raised. The actual observations follow the probability distribution expected by Benford's law and the null hypothesis cannot be rejected. The observations distribution of each ERP functional module suggests that it resembles what is expected by Benford's law. These results confirm and extend the literature on applications of Benford's law tests to detect if data sets are contaminated (Busta and Weinberg, 1998); data not conforming and how it is an indicator of data quality (Cleary and Thibodeau, 2005); a proper tool to discriminate between data sets that are naturally occurring and others that are not (Bhattacharya et al., 2011) and a tool to estimate the number of records omitted from a data set (Carreira and Gomes da Silva, 2016).

The MAD test has a very favourable result for the entire database. When the MAD is carried out at the level of each module, it appears that only CM and PR modules allow us not to reject the null hypothesis, where the distribution of data resembles Benford's law. This is well aligned with Nigrini's (2012) goodness–of-fit analysis. In the remaining six modules, the null hypothesis can be rejected. A plausible explanation for these differences might be rooted in the module sizes, where those with the highest number of tables and the highest volume of information tend to conform better to Benford's law. It is generally observed that the lower the number of tables in the module, the higher the value of the MAD, which leads to nonconformity and rejection of the null hypothesis. This issue, as stated above, may correspond to possible limitations caused by a small amount of data and is not well aligned with Wallace (2002), who successfully tested Benford's law with 67 observations.

Using the MAD analysis, the auditor can inform that modules HR (MAD large and non-conforming) and RM (MAD larger than modules with fewer tables and records) raise red flags and merit further exploration. On the other hand, further investigation of the IR, AC, IN and OM (MAD large) may not result in a good allocation of constrained audit resources given their low number of tables. The MAD test shows that the adjustment with Benford is not met when the number of tables in the module is less than 350. The MAD reaches an average compliance with almost 400 tables and acceptable values ​​of 800, which is very similar to that of Nigrini (2017) and significantly higher than that of Wallace (2002). Because the number of tables in a module is of low elasticity, we suggest increasing the size by performing reruns in an iterative manner (Nigrini, 2019), the proposal is to consider not only an analysis at a given point in time but also add other moments to that starting point, replicating a moving average formula. The sum of each moment would double the number of tables but with a different number of records for the growth itself. This would allow the review of Benford's distribution according to the temporal evolution of the database. If acceptable results are achieved but red flags are raised, then it becomes possible to inquire into the feasibility of establishing an order of priority over the modules in which the attention of the audit or control area should be oriented.

The IAF in Latin America faces similar challenges compared to other jurisdictions but also presents some uniqueness. The complexity of ERP systems forces internal auditors to find approaches to process and extract information from the enormous amounts of data accumulated daily (Islam et al., 2018; Bozkus Kahyaoglu and Caliyurt, 2018). New and innovative methods to detect fraud are being developed and deployed, but not even more user-friendly visualization can do it all (Singh and Best, 2016). Benford's law applied to ERP tables suffers from the same shortcomings; it can sense something might not be right but cannot certify something is wrong. To determine if fraud exists, further investigations are needed, and at this level being in a Civil law context and in an SOE differs from what is normally expected in Common law countries (Erasmus and Coetzee, 2018). The situation experienced by the company studied fits quite well with Abuazza et al.'s (2015) observations, where the IAF is focused on the “traditional protective services”. However, the particularities of an SOE make internal auditors the watchdog of the most powerful group, whether they are managers and executives that respond to politicians or union leaders that facilitate their arrival into the company (Gramling and Schneider, 2018). This vulnerable position of internal auditors was properly reflected by Burnaby and Hass (2011), who documented that compliance was not expected, and professionals in Latin America give higher importance to statistical sampling as a key technical skill. In an environment where transparency and disclosure are low, the results reported by Fuertes-Callen et al. (2014) reinforce the call to avoid using the same tools and assumptions in the same manner in Common and Civil law countries (Areneke et al., 2019; Kotb et al., 2020).

Internal auditors in Latin American SOEs need to walk a fine line between performing their duties and not upsetting groups of power. Doing analysis to test the integrity of high-volume data sets using Benford's law is suitable for an environment with pervasive control deficiency related to the “tone at the top” (Gramling and Schneider, 2018). This study agrees with Kabuye et al. (2017) and Oussii and Boulila Taktak (2018), who find that internal audit organizational status is a better predictor of fraud management than internal audit activities. In the case studied, internal auditors raise red flags, which can be further investigated if management agrees. This study confirms that scant attention has been paid to the IAF that prevails in emerging economies based on Civil law (Kotb et al., 2020). Areneke et al.'s (2019) analysis of corporate governance and Okundaye et al.’s (2019) study of information technology adoption can be extended to IAF, where we find limited results coming from Civil law emerging economies or Latin America. This article intends to shed some light on this under-researched area (Salcedo, 2021), but more research is needed.

This study had some limitations. First, as a case study, the manuscript only assesses a single SOE. Future studies could examine whether the results of our study generalize to other organizations governed by Civil law in emerging economies and/or Latin America. Not only could the study be replicated in other SOEs, but it could also be extrapolated to financial data from public corporations whereby several groups (audit committee, external auditors, management, etc.) have the ability to exert pressure on the IAF. Examining alternative environments where the IAF could benefit from Benford's law would enable us to establish boundary conditions for our findings. Second, another limitation is that, based on our analysis, we cannot rule out the fact that the observed results are not, at least partially, due to pre-existing tampering of the database. As observed by Miranda-Zanetti et al. (2019), a forensic analysis aided by Benford's law ascertained that Argentinean national statistics on inflation were manipulated by the national government from 2006 to 2015. Having said this, our goal here is not to detect fraud but to illustrate how Benford's law could be utilized as additional supporting audit documentation to prevent the IAF from getting involved in confrontations between management and politicians.

7. Conclusion

This study focuses on mitigating the internal auditor's uncertainty by providing a test that can be interpreted as an indicator of the confidence or alert of the possible inherent or pre-existing risk of the computerized data available to internal auditors in their routine tasks. Useful is an analytical tool that can verify whether the behaviour of a set of numbers conforms to expectations. In the last 30 years, several analytical tools have been documented in academic studies, but among all Benford's law is one of the most widespread and simple to use (Sing and Best, 2016; Nigrini, 2017). Benford's law considers that certain digits appear more frequently than others in a data set. Uses of Benford's law in accounting are well represented by Nigrini (1999, 2017, 2019); however, in all cases, its application was circumscribed to transactions. One contribution of this study is to explore whether Benford's law can be utilized by auditors as a substantive analytical procedure to determine the inherent risk of data contained in large databases generated by diverse modules in the ERP system employed by companies.

Internal auditors operating in SOEs in emerging economies need to have access to and employ analytics techniques that allow them to discharge their duties without interfering with the political process of running the company. It has been suggested that rules, procedures and best practices in corporate governance and the IAF developed in a Common law environment might not operate similarly in emerging economies with systems rooted in Civil law (Hay and Cordery, 2018; Kabuye et al., 2017; Kotb et al., 2020). In emerging economies, SOEs are not always managed in a professional manner, and most frequently harbour intense disputes for resources. In this context, internal auditors are caught in the middle of power struggles where antagonist professional networks within organizations pressure for results favourable to their interests while upholding the appearance of pristine corporate governance policies (Young et al., 2008) and the framing of the IAF.

This study explores whether Benford's distribution applies to the number of records (size) that contain the different tables of an ERP relational database. The study reports the results obtained from eight ERP modules of a large SOE in the energy sector of Argentina. To test compliance with Benford's law, the number of records (size) of the tables of each module was used, including more than 1,900 tables with 4,500 million records. The results show that Benford's law is satisfied when all modules are considered (χ² and MAD conformity tests), but the same does not hold for all modules when considered individually (MAD conformity test of six smaller ERP modules). The decision to further investigate these non-conforming ERP modules is political and outside the auditor's hand, but at least they can leave a paper trail (Okundaye et al., 2019), showing the results of an unbiased analytical test.

According to the analysis performed and the results achieved, it is verified that Benford's law conforms to the size distribution (in the number of records) of the database tables. This study further contributes to the literature and tools designed to address the problem of uncertainty experienced by the auditor acting on an information system in a computerized context, regarding the trust or possible inherent risk of the information contained in the database used for audit protocols and controls. This second contribution also enhances the practice of internal auditing in ERP contexts of large SOEs in emerging economies, where internal auditors have limited resources at their disposal and are caught in between two powerful groups with divergent interests.

Finally, we consider this work to be novel and can contribute to mitigating auditor uncertainty regarding the inherent risk of the information that is determined by the characteristics of the entity and the information system under analysis. The use of this tool allows to infer if the distribution under review does not fit Benford, and there are indications of possible irregularities that require further analysis. We consider that this type of empirical study can also serve as a basis for deeper analysis in contexts where internal auditors are independent of management and other power groups. However, in contexts where internal auditors are not independent of management, this analysis would serve to raise red flags whose further investigation is decided by management, effectively limiting the IAF responsibility and liability in cases of fraud. This study contributes an analysis as a prior control or measure of risk that is independent of applied controls, designed to elucidate the so-called inherent risk of the information that is made available to the auditor for its control as per ISA 315 (2016) and ISA 330 (2006). This study was not about detecting fraud.

Figures

Figure 1

Actual and Benford's law expected frequency distributions

Table 1

Common and Civil law differences, identification of Argentina as an extreme case of Civil law

Study	Index	Common law	Civil law	ANOVA F-value (p-value)a	Argentina
Courts formalism (Djankov et al., 2003)	Formalism to collect a cheque	2.7564	4.2898	56.6533 (0.0000)	5.4
	Days to collect a cheque	176	272	8.6245 (0.0043)	300
	Formalism to evict a tenant	3.0200	4.3812	60.5628 (0.0000)	5.49
	Days to evict a tenant	199	266	2.6976 (0.1004)	440
Setting up a new company (Djankov et al., 2002)	Procedures	6.38	12.72	28.7391 (0.0000)	14
	Taxes	1.67	2.76	6.9657 (0.0110)	4
	Labour	1.3	2.52	10.2430 (0.0024)	5
	Screening	3.67	7.48	36.8632 (0.0000)	5
	Days	28.96	62.21	21.9800 (0.0000)	48
	Cost	0.4225	0.6376		0.1019
Regulation of labour (Botero et al., 2004)b	Employment law	0.2997	0.5470	35.9863 (0.0000)	0.3442
	Collective relations	0.3313	0.4914	31.3153 (0.0000)	0.5774
	Social security	0.4236	0.5454	6.0869 (0.0165)	0.7154

Note(s): ^aOne-way ANOVA was used to test the null hypothesis that the average index for each group of countries come from the same sample; the corresponding F-values (p-values) are shown, the results of the ANOVA indicate the null hypothesis can be rejected at the 0.10 in all but the cost index of setting up a new company (included is the marginal results obtained for days to evict a tenant); ^bCommon Law includes all English countries and Civil Law includes all French countries

Source(s): Own elaboration

Table 2

Relational database: selected modules

ERP functional module	Number of tables	Tables in the module as % of the total	Records within each module (in thousands)	Records of each module as % of the total	Average number of records per module
Accounting (AC)	78	4.06%	225,245	5.02%	2,887,757
Human Resources (HR)	340	17.68%	1,484	0.03%	4,365
Inventory (IN)	71	3.69%	14,916	0.33%	210,091
Operations Management (OM)	58	3.02%	381	0.01%	6,562
Internal Requests (IR)	87	4.52%	1,832	0.04%	21,063
Commercial Management (CM)	808	42.02%	4,093,412	91.25%	5,066,104
Payroll (PR)	381	19.81%	135,126	3.01%	354,662
Records Management (RM)	100	5.2%	13,417	0.30%	134,168
Total Database	1,923	100%	4,485,813	100.00%	2,332,716*

Note(s): *The average number of records per module at the level of the total database is calculated as the total records divided by the total of tables as follows: 4,485,813,693/1,923 = 2,332,716

Source(s): Own elaboration

Table 3

Frequency of occurrence of first digit in the tables of each ERP module

Digit	AC	HR	IN	OM	IR	CM	PR	RM	Total
1	22	94	21	12	23	254	114	26	566
2	11	74	17	13	17	135	70	24	361
3	11	35	11	12	10	89	62	13	243
4	6	38	6	2	10	89	39	12	202
5	5	34	7	4	9	54	33	6	152
6	10	20	2	6	2	60	23	7	130
7	4	16	3	5	5	39	14	7	93
8	5	15	3	3	5	45	14	3	93
9	4	14	1	1	6	43	12	2	83
Total	78	340	71	58	87	808	381	100	1,923

Source(s): Own elaboration

Table 4

Percentage of each first digit in each ERP module's table

Digit	AC	HR	IN	OM	IR	CM	PR	RM	Total	Benford
1	28.2%	27.6%	29.6%	20.7%	26.4%	31.4%	29.9%	26.0%	29.4%	30.1%
2	14.1%	21.8%	23.9%	22.4%	19.5%	16.7%	18.4%	24.0%	18.8%	17.6%
3	14.1%	10.3%	15.5%	20.7%	11.5%	11.0%	16.3%	13.0%	12.6%	12.5%
4	7.7%	11.2%	8.5%	3.4%	11.5%	11.0%	10.2%	12.0%	10.5%	9.7%
5	6.4%	10.0%	9.9%	6.9%	10.3%	6.7%	8.7%	6.0%	7.9%	7.9%
6	12.8%	5.9%	2.8%	10.3%	2.3%	7.4%	6.0%	7.0%	6.8%	6.7%
7	5.1%	4.7%	4.2%	8.6%	5.7%	4.8%	3.7%	7.0%	4.8%	5.8%
8	6.4%	4.4%	4.2%	5.2%	5.7%	5.6%	3.7%	3.0%	4.8%	5.1%
9	5.1%	4.1%	1.4%	1.7%	6.9%	5.3%	3.1%	2.0%	4.3%	4.6%
Total	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%

Source(s): Own elaboration

Table 5

Obtaining χ² from the actual and expected frequencies of the database

DIGITO	Actual	Benford	Pobs(d) − Pt(d)	[Pobs(d) − Pt(d)]2	[Pobs(d) − Pt(d)]2 Pt(d)
	Pobs(d)	Pt(d)*
1	566	578.88	−12.88	165.91	0.29
2	361	338.62	22.38	500.71	1.48
3	243	240.26	2.74	7.52	0.03
4	202	186.36	15.64	244.67	1.31
5	152	152.27	−0.27	0.07	0.00
6	130	128.74	1.26	1.59	0.01
7	93	111.52	−18.52	342.94	3.08
8	93	98.37	−5.37	28.80	0.29
9	83	87.99	−4.99	24.92	0.28
Total	1,923	1,923	Chi Squared (χ2)		6.77

Note(s): *Expected frequencies calculation: digit 1 is 578.88 = (1,923 × 0.301) and digit 2 is 338.62 = (1,923 × 0.176)

Source(s): Own elaboration

Table 6

Actual and Benford's expected frequencies for each module

Dig.	AC		HR		IN		OM		IR		CM		PR		RM
	Act.	Exp.*	Act.	Exp.	Act.	Exp.	Act.	Exp.	Act.	Exp.	Act.	Exp.	Act.	Exp.	Act.	Exp.
1	22	23.5	94	102.3	21	21.4	12	17.5	23	26.2	254	243.2	114	114.7	26	30.1
2	11	13.7	74	59.9	17	12.5	13	10.2	17	15.3	135	142.3	70	67.1	24	17.6
3	11	9.7	35	42.5	11	8.9	12	7.2	10	10.9	89	100.9	62	47.6	13	12.5
4	6	7.6	38	32.9	6	6.9	2	5.6	10	8.4	89	78.3	39	36.9	12	9.7
5	5	6.2	34	26.9	7	5.6	4	4.6	9	6.9	54	64.0	33	30.2	6	7.9
6	10	5.2	20	22.7	2	4.7	6	3.9	2	5.8	60	54.1	23	25.5	7	6.7
7	4	4.5	16	19.7	3	4.1	5	3.4	5	5.0	39	46.9	14	22.1	7	5.8
8	5	4.0	15	17.4	3	3.6	3	3.0	5	4.5	45	41.4	14	19.5	3	5.1
9	4	3.6	14	15.6	1	3.3	1	2.7	6	4.0	43	37.0	12	17.4	2	4.6
Total	78	78	340	340	71	71	58	58	87	87	808	808	381	381	100	100
χ2	6.09		9.48		6.15		10.98		5.17		8.55		11.34		6.51

Note(s): *Expected frequencies calculation: for AC module in digit 1 the expected frequency is 23.5 = (78 × 0.301)

Source(s): Own elaboration

Table 7

Modules ordered by lower MAD and largest number of tables and records

ERP module	N (tables)	Total records	MAD value	MAD conformity
Total	1,923	4,485,813,693	0.0049	Close (high)
CM	808	4,093,412,030	0.0102	Acceptable
PR	381	135,126,395	0.0130	Marginally acceptable
HR	340	1,483,944	0.0172	Non-conformity (low)
IR	87	1,832,473	0.0203	Non-conformity (low)
AC	78	225,245,009	0.0213	Non-conformity (low)
RM	100	13,416,811	0.0238	Non-conformity (low)
IN	71	14,916,459	0.0250	Non-conformity (low)
OM	58	380,572	0.0434	Non-conformity (low)

Source(s): Own elaboration

Note

1. Details provided on one of the news articles listed in https://www.lavoz.com.ar/categoria/temas-libres-10116 (last consulted on 20 June 2020).