This article aims to constructively critique the new global methodology for evaluating the effectiveness of anti-money laundering regimes against defined outcomes.
With surprisingly little discussion at the intersection of the money laundering and policy effectiveness and outcomes scholarship and practice, this article combines elements of these disciplines and recent peer-review evaluations, to qualitatively assess the Financial Action Task Force’s (FATF’s) anti-money laundering “effectiveness” methodology.
FATF’s “effectiveness” methodology does not yet reflect an outcome-oriented framework as it purports. Misapplication of outcome labels to outputs and activities miss an opportunity to evaluate outcomes, as the impact and effect of anti-money laundering policies.
If the “outcomes” of the “effectiveness” framework do not match the crime and terrorism prevention policy goals of nation states, the new “main” component for assessing the effectiveness of anti-money laundering regimes potentially detracts focus and resources from, rather than towards, intended policy objectives.
There is a dearth of scholarship whether the global anti-money laundering “effectiveness” framework is sufficiently robust to assess effectiveness as it purports. This article begins addressing that gap.
CitationDownload as .RIS
Emerald Publishing Limited
Copyright © 2018, Emerald Publishing Limited
The Financial Action Task Force (FATF) is responsible for developing and promoting global anti-money laundering/counter-financing of terrorism (AML/CFT) standards. FATF says that its “40 recommendations” represent a “comprehensive and consistent framework of measures which countries should implement to combat money laundering and terrorist financing” (FATF, 2013a, p. 7).
Scholars, however, have questioned AML/CFT controls based on adherence to standards not proven to be effective in preventing terrorism (Anand, 2011; Brzoska, 2016) or serious crime (Chaikin, 2009; Ferwerda, 2009; Harvey, 2008; Levi, 2002, 2012; Levi and Maguire, 2004; Levi and Reuter, 2006; Levi and Reuter, 2009; Naylor, 2002; Reuter and Truman, 2004, p. 9, Rider, 2002a, 2002b, 2004; Sharman, 2011; van Duyne, 2003, 2011).
Seemingly responsive to such criticism, FATF introduced a new methodology to evaluate the effectiveness of AML/CFT regimes against specified “outcome” measures (FATF, 2013b). Formerly assessed against technical compliance with FATF recommendations, “the main focus is now on effectiveness” (FATF, 2017b), determined by “the extent to which the country is achieving the defined set of outcomes” (FATF, 2013c, p. 3, emphasis added). Since December 2014, 49 “effectiveness” evaluations have been published. With a dearth of scholarship whether the new methodology is sufficiently robust to assess effectiveness, this article begins addressing that gap. It finds that FATF’s “effectiveness” methodology is not an outcome-oriented framework as it purports. Misapplication of outcome labels to outputs and activities miss an opportunity properly to evaluate outcomes, as the impact and effect of AML/CFT policies.
However, recognition that effectiveness matters, together with FATF’s tradition of continuously adjusting standards and the ubiquity of its network and persuasive power, suggest grounds for optimism. If the new methodology misses the mark, that it exists at all puts the prospect of effectiveness in closer reach than in other areas with less insight into the impact and effect of policy interventions.
2. Anti-money laundering effectiveness disconnect?
Distinguishing between “law-in-the-books” and “law-in-action” compliance (Deleanu and Ferwerda, 2014, p. 189), a line of enquiry looks beyond the presence of rules, whether they meet defined standards or whether firms comply with them. It asks whether the rules work. In that context, scholars have addressed the impact of AML/CFT policies on laundering (Chong and Lopez de Silanes, 2015), criminal financing (Cuéllar, 2003), terrorism (Anand, 2011; Brzoska, 2016) and crime detection and prevention (Alldridge, 2016; Chaikin, 2009; Ferwerda, 2009; Fisher, 2014; Halliday et al., 2014; Harvey, 2008; Levi, 2002, 2012; Levi and Maguire, 2004; Levi and Reuter, 2006; Naylor, 2002; Reuter and Truman, 2004, p. 9; Sharman, 2017; Unger et al., 2014).
Notwithstanding “significant gaps between what has been promised and what has been actually achieved” (Tavares et al., 2010, p. 4), surprisingly few commentators expressly connect AML/CFT scholarship with the policy effectiveness and outcomes discourse (exceptions include Fisher, 2014; Halliday et al., 2014; Levi and Reuter, 2006; McConnell, 2015; Sharman, 2011, 2017).
In one such exception, Halliday, Levi and Reuter found that in the third round of AML/CFT evaluations (before the fourth round “effectiveness” criteria), FATF failed to demonstrate that objectives were “more likely to be reached by compliance with FATF Standards”. The authors found that the AML/CFT system relies on an unproven assumption; the “prima facie plausibility of the claim that adherence to the Standards would help reduce money laundering and the financing of terrorism, and collaterally the reduction of serious crimes for gain and terrorism.” They concluded that the “net result” of focusing on compliance with standards was that “extensive efforts were expended with no demonstrable impact on money laundering or the financing of terrorism” and “very little emphasis, if any, […] [on] outcome effectiveness” (Halliday et al., 2014, pp. 5, 15).
This article joins the “does it work?” conversation and explicitly addresses the contemporary fourth round of “effectiveness” evaluations based on defined “outcomes”.
3. Outcomes matter
This section draws briefly from the public administration narrative linking policy-making with outcomes before application in the money laundering context.
3.1 New public management, precursor to ‘outcomes’
Widely adopted in Anglo-American jurisdictions (Kristensen et al., 2002; Pollitt, 2015; Tiernan, 2012), “new public management” (NPM) “almost completely dominated” (Dunleavy, 2013) public services administration between the 1980s and mid-2000s (Bouckaert et al., 2010; Bourgon, 2007; Denhardt and Denhardt, 2006; Tiernan, 2012). NPM performance metrics, which “often measure[d] ‘outputs’ rather than ‘outcomes’, what the organisation does, rather than what, if anything, it achieves” (Garland, 1996, p. 458, emphasis in original), heavily influenced public sector management (Mayne, 2001) and policing (Orde, 2012).
Deeply embedded even as NPM attracted “significant criticism” (Tiernan, 2012, p. 254), the emerging post-NPM environment remains unclear, yet is often characterised as shifting “from an emphasis on efficiency to […] effectiveness; from emphasizing outputs to outcomes” (De Vries and Nemec, 2013, p. 7, emphasis added). This conscious refocusing towards outcomes in contemporary public administration is consistent with the “outcomes” nomenclature of FATF’s effectiveness methodology.
Defining and measuring outcomes is, however, a “daunting task” (Cook, 2004, OECD, 2002) and differences between “outputs” and “outcomes” may be especially difficult to distinguish (Mulgan, 2008, p. 462). Confusingly, “outcomes” are sometimes defined as a form of “output”, with “impact” instead describing the ultimate effectiveness measure (Brzoska, 2016; Easton, 1965). It is therefore useful briefly to ground the narrative with definitional clarity before application in the money laundering context.
3.2 Outcomes define policy effectiveness
Outcomes may be the result of a combination of inputs, activities, outputs and external influences. A classic example involves success claimed for innovative crime reduction strategies in early 1990s New York (Kim and Mauborgne, 2003). Some scholars controversially claimed that external factors unrelated to crime prevention policy interventions, such as legalised abortion and lead paint, significantly contributed to falling crime rates (Blumstein and Wallman, 2006; Donohue and Levitt, 2001; François et al., 2014; Joyce, 2004; Levitt and Dubner, 2006; Reyes, 2007; Roeder et al., 2015).
A clear view of outcomes and causality is thus vitally important in evaluating effectiveness, because “outcomes alone do not provide information about causality – that the program inputs caused the outcome” (Productivity Commission, 2013). Relationships between elements of a simple results-chain model transforming resources into outputs to help achieve policy outcomes are illustrated in Figure 1.
Differences between each concept is also important, as outlined in Table I, with definitions distilled from multiple sources (Bourgon, 2007; Cook, 2004; Funnell and Rogers, 2011; Garland, 1996; Kristensen et al., 2002; Mayne, 2001, 2004; Mulgan, 2008; OECD, 2002; Paz-Ybarnegaray and Douthwaite, 2017; Productivity Commission, 2013, 2015).
The following section applies those concepts to FATF’s effectiveness methodology.
4. Anti-money laundering effectiveness: are we there yet?
No independent review of AML/CFT “effectiveness” evaluations has yet been conducted, but an “evaluation of evaluations” in a related area found a comprehensive failure to evaluate effectiveness. Despite many millions of dollars spent on anti-human trafficking interventions and claims of policy success, a review of 49 evaluations found that assessments mostly focused on processes and outputs rather than outcomes (Davy, 2016). Davy argues that without meaningful effectiveness insights, policies in effect still based on assumptions about what works may be ineffective or counterproductive.
A full review of the evaluations conducted in accordance with FATF’s “effectiveness” methodology may be similarly instructive. In the meantime, there is a dearth of scholarship whether the methodology itself is sufficiently robust to assess effectiveness as it purports. This section begins addressing that gap.
4.1 “Outcomes” apparent acme of effectiveness
Described as “fundamentally” different (FATF, 2013b, pp. 4, 14) from assessing technical compliance with FATF recommendations, the new methodology presents as the acme of policy effectiveness. Not whether rules are made, meet accepted standards, or are complied with, but whether they work. FATF’s hierarchy of outcomes also appears consistent with the results-chain concept of Figure 1. Illustrated in Figure 2, eleven “immediate outcomes” lead to three “intermediate outcomes” and the “high-level objective” (FATF, 2013b, p. 15).
The intention is clear. For policy effectiveness, outcomes matter. Its implementation, however, may not meet FATF’s intention, as detailed below.
4.2 Compliance primacy prevails?
Despite its “outcomes” terminology, the AML/CFT “effectiveness” framework inadvertently extends the “checklist mentality” (Cook, 2004) of FATF’s output-oriented technical compliance component into its effectiveness counterpart.
Connections between the effectiveness framework and FATF recommendations appear curiously pervasive. Table II illustrates links posited from each of the “immediate outcomes” to FATF recommendations (derived from FATF, 2013b, pp. 90-117). Common influences are unsurprising. Both are focused on a common objective. However, reversing their direction (in Table III) reveals that all 40 “technical compliance” (output) metrics track to the 11 “outcome” measures. The AML/CFT effectiveness framework may not be as outcome-oriented as it appears if, in effect, it operates as an abbreviated index to FATF’s 40 recommendations.
Moreover, FATF’s evaluation template (FATF, 2013b, pp. 120-142) reveals that assessors are encouraged to use the same evidence to evaluate “effectiveness” measures and compliance with FATF’s 40 recommendations. In Australia’s evaluation, for example, Immediate Outcome 3 (IO3) was only rated “moderate”, largely because of factors related to the assessment of recommendation 28, judged “non-compliant” (FATF and APG, 2015, pp. 94, 173).
Evidence for assessing “outcomes” may be relevant to evaluating compliance with FATF recommendations, and vice versa. However, overtly grouping them together and systematically using identical evidence to assess ostensibly different criteria suggests that the new methodology fails to assess effectiveness commensurate with the outcomes literature; instead compressing FATF’s 40 recommendations into an abbreviated equivalent list of 11. Using the same evidence to assess outcomes under a purported effectiveness framework as for “tick-the-box” technical compliance with output measures adds little new evaluative capability beyond another 11 boxes to tick.
4.3 When “outcomes” are not outcomes
This section suggests that FATF’s “immediate outcomes” – often ambiguous, unrealisable or lacking measurability – are mere process or output measures, not outcomes in the sense generally understood as the effect or impact of policy initiatives.
4.3.1 Unrealisable objectives are not outcomes.
“Immediate outcomes” 10 and 11 (IO10 and IO11) and the first part of IO5 are lofty goals. For terrorists to be prevented from raising, moving or using funds and for shell companies and other opaque corporate vehicles to be prevented from misuse, are laudable aspirations. It is, however, impossible to assess achievement of such goals. Outcome measures require “a much greater level of specificity than the very general objective” (Mulgan, 2008, p. 462). “Loose outcome descriptions” (Tanner, 2008, p. 4) defined in aspirational terms without a clear and specific statement of their impact and effect (ANAO, 2001, 2007; Mackay, 2011) are “so broad and general as to be virtually meaningless” (Tanner, 2008, p. 4).
4.3.2 Processes and activities are not outcomes.
The remaining “immediate outcomes” and the second part of IO5 are activities, not outcomes. For money laundering risks to be understood by officials (IO1), and for international cooperation to deliver appropriate information (IO2), for example, describe mere activities. Likewise, for supervisors to supervise, monitor and regulate (IO3) and financial and non-financial institutions to apply AML measures (IO4). For beneficial ownership information to be available (IO5) to authorities, and for them to use financial information (IO6), also describe processes and activities. In the typical results-chain construct they are not even outputs, let alone outcomes as they purport. They focus on processes, not the impact, effect or consequences of policy endeavours.
IO8 also illustrates unresolvable ambiguity. If all proceeds and instrumentalities of crime were confiscated, IO8 might be an outcome measure. The main reason it is not is the same as IO10 and IO11. The lofty goal to confiscate all criminal proceeds is incapable of objective, verifiable measurement or achievement.
The opposite of lofty goals, IO7 and IO9 are ineffective as outcome measures because they are easily attainable. Ironically, IO8 fails in this category too. It might be interpreted as confiscating all proceeds of crime (thus failing as an outcome measure as described above) or requiring the confiscation of any criminal proceeds and failing like IO7 and IO9, which require only that money laundering and terrorist financing offences are investigated, prosecuted and (some) criminal proceeds confiscated. These activities generate measurable metrics, as addressed below, but tell us little or nothing about outcomes; as the effect or impact of such activities.
4.3.3 Outputs are not outcomes either.
It might be countered that, whatever terminological faults afflict FATF’s “immediate outcomes”, assessors in practice make nuanced evaluations based on expert assessment, yet this reinforces rather than mitigates the problems described above.
In evaluating IO7, for example, FATF assessors consider factors such as the scale of money laundering investigations and prosecutions. Of 96 “cases” reported to Jamaican authorities, 25 led to charges, with “only three” convictions. Authorities focus mostly on predicate offences, and Jamaica received a “low” rating (CFATF, 2017, pp. 47, 53). Spain, by contrast, conducts many such prosecutions. Assessors detailed hundreds of convictions, and Spain was awarded a “substantial” effectiveness rating (FATF, 2014, pp. 51-58). Perhaps implicitly recognising that process or activity indicators mischaracterised as outcomes offer little meaningful indication of effectiveness, it is understandable if assessors use higher-order output measures to infuse some meaning into the so-called “immediate outcomes”. It does not, however, transform such output measures into outcomes, as the effect or impact of money laundering controls, illustrated in a practical example below.
4.3.4 Australia: outputs obscure outcomes.
FATF assessors recorded Australia’s “main policy objective” is “to disrupt and deter predicate crime, including, if necessary, through [money laundering] investigations”. Its primary focus in this regard is “predicate offences,[…] proceeds of crime, and disruption of criminal activity rather than the pursuit of convictions for [money laundering] offences” (FATF and APG, 2015, pp. 8, 14, emphasis added). “To achieve a substantial or even high level of effectiveness”, assessors recommended that Australian authorities pursue more money laundering investigations and convictions (FATF and APG, 2015, pp. 7-8, 15).
Officials should, of course, evidence the results of crime prevention efforts, but to “prove effectiveness with an output measure, such as the number of money laundering prosecutions, is misplaced. More such prosecutions might, or might not, reduce predicate crime as Australia seeks. Increasing prosecution numbers produces metrics readily presented to assessors, and may be an output indicator toward, but is not an outcome measure of, policy effectiveness.
With relatively few money laundering investigations, Australia was awarded only a “moderate” rating for IO7 as the assessed “effectiveness” of its investigation and prosecution of laundering offences. But what was the real outcome, as the effect or impact of Australia’s efforts to detect and deter serious crime? It is hard to tell. FATF’s “immediate outcomes” are mostly activity measures, so assessors have no framework to assess effectiveness on the scale of outcomes such as the impact or effect on crime. In practice, they add a veneer of output measures, yet assessors’ efforts to breathe life into the “immediate outcomes” shell is insufficient to transform them into the outcome measures they purport.
It might be argued that AML measures properly focus only on laundering, not the impact on criminal activity generating the proceeds for laundering. This issue is addressed in section 4.5.4. In the meantime, the following section records the severing of supposed causal links between parts of the model, further constraining the capacity to evaluate (or achieve) effectiveness.
4.4 Systemic disconnect: links administratively severed
Assessors may consider higher-order objectives when summarising overall effectiveness, but are required to assess only the 11 “immediate outcomes” (FATF, 2013b, pp. 14-15; FATF, 2017a). In practice, this may not matter much. Because the “intermediate outcomes” summarise the “immediate outcomes” it would serve little purpose to re-assess a pastiche of factors individually considered. Likewise, a “high-level objective” restricted to ill-defined perceived threats lacks the specificity required to evaluate impact and effect (ANAO, 2001, 2007; Mackay, 2011).
But, assuming the “intermediate outcomes” and “high-level objective” operate as part of a typical hierarchy of outcomes, as they purport, instructing assessors largely to disregard supposed indicators of policy effectiveness is inconsistent with outcome-oriented logic models (Moore, 2013). The first 49 country-level evaluations under the new “effectiveness” methodology generated considerably more data, but more performance indicators, more reporting and more paperwork does not necessarily evidence policy success. Emphasis on the “busyness” of reporting activity and output measures masquerading as outcomes leaves little room for evaluative focus on results and causality as the primary determinants of effectiveness (ANAO, 2007; Mackay, 2011).
The critical absence of evaluative capacity to assess the contribution to policy outcomes (Brzoska, 2016; Mayne, 2001) and the instruction largely to disregard higher-order objectives suggests a hope-based strategy that intended policy goals might be reached by some happy confluence of random events.
4.5 FATF’s new clothes: what do money laundering prosecutions really reveal?
This section draws distinctions between outputs and outcomes in a practical context. FATF’s evaluations of Australia and Canada illustrate why output metrics are poor indicators of AML/CFT policy effectiveness and may inadvertently distract governments from higher-order crime and terrorism prevention objectives.
4.5.1 Australia revisited: output measures disconnected from outcomes.
FATF evaluators say it “should be relatively easy” for Australia to achieve a “high” IO7 effectiveness rating (FATF and APG, 2015, pp. 15, 60). “Double-charging” challenges that limited the ability to sustain money laundering charges alongside those for predicate offences have been resolved (FATF and APG, 2015, p. 57), so it should be easier for Australian authorities to add such charges as appropriate. More such prosecutions would help meet expectations of upgrading Australia’s lowly “moderate” grade. A hypothetical example, however, illustrates practical differences between output and outcome measures, and why it matters for policy effectiveness.
Say that Australia currently detects and prosecutes 5 per cent of all profit-motivated crime, only some of which attracts concurrent money laundering charges. The following year, police detect the same proportion of profit-motivated crime and charge all offenders with money laundering offences. Australia’s FATF effectiveness rating would presumably increase, as evaluators anticipate, perhaps to the highest rating.
Now consider an alternative. Crime detection initiatives associated with Australia’s main policy objective, “to disrupt and deter predicate crime” (FATF and APG, 2015, pp. 8, 55), allow authorities to interdict 10 per cent of all serious crime but, with prosecutors overwhelmed, no concurrent money laundering charges are laid. Fewer money laundering offences than the current (low) measures bemoaned by assessors would dash FATF’s optimistic expectations, and Australia’s effectiveness rating on this count might fall to the lowest rating. Detecting and prosecuting twice as much serious crime might conceivably result in failure on this so-called “outcome” dimension of “effectiveness”.
The reality, of course, is more nuanced. There is some latitude for “outside-the-box” evaluation, and Australian authorities would promote a doubling of predicate crime detection as influencing results. Moreover, new money laundering investigations may well advance crime detection and prevention capability, as IO7 suggests. Following the money trails exposed by such investigations may reveal undetected predicate crimes, and prosecuting “criminalised” (Procunier, 2013) and wilfully blind professionals may deter others unwittingly enabling criminal enterprises. The scope and scale of money laundering prosecutions therefore remains a valid output indicator, but not an outcome measure of effectiveness, at least in relation to policy objectives to detect and deter profit-motivated crime. The number of money laundering prosecutions might, or might not, have a material effect or impact on Australia’s main policy objective to disrupt serious crime. To properly appraise such outcomes requires evaluative capacity beyond simplistic money laundering prosecution metrics.
Similar issues appear in many other evaluations. Canada offers an illustrative example, outlined in the next section.
4.5.2 Canada: better outcomes risk effectiveness ratings drop?
Canada’s reported shift in enforcement focus tackling drugs trafficking networks appears to reflect a move away from output metrics, towards outcomes. If FATF’s effectiveness framework is an outcome-oriented system as it purports, Canada’s re-orientation should be complementary. Counter-intuitively, however, re-orienting for outcomes might adversely affect Canada’s FATF “effectiveness” evaluations.
FATF records Canada’s “main law enforcement policy objective” is to “prevent, detect and disrupt crimes, including [money laundering], but in practice, most of the attention is focused on securing evidence in relation to the predicate offense”. FATF assessors highlighted “insufficient efforts” to pursue money laundering charges, adversely affecting Canada’s “effectiveness” ratings – including its meagre “moderate” rating for IO8 (FATF and APG, 2016, pp. 5, 40, 47-48, 52, 54, 56-57, 59).
Canada’s “dismantle” policing strategy (characterised by long-running investigations to arrest crime group leaders) is reportedly moving towards “disruption”, intended to make it increasingly difficult to conduct criminal enterprises (Desroches, 2016). Less resource-intensive and faster-moving, the new approach seeks to disrupt more criminal networks. The strategy includes seizing cash and assets, so money laundering prosecutions may increase. But, with crime disruption difficult to quantify, traditional metrics – such as the number of high-level arrests, the quantum of drugs and asset seizures and the number of money laundering prosecutions – might conceivably fall.
Like the hypothetical example in section 4.5.1 above, if Canadian authorities successfully disrupt more crime, the number of money laundering prosecutions may be a poor indicator even of lower-order objectives to curtail the capacity to launder, let alone higher-order crime prevention outcomes. As noted above, investigating, prosecuting and confiscating (so-called “outcomes” at the core of IO7 and IO8) are measures of activity, not outcomes. Assessors in practice overlay more advanced metrics, yet these record the number and scale of prosecutions (output measures), not their impact and effect on the ability to launder criminal proceeds or crime prevention.
FATF’s capacity to evaluate for such outcomes is conspicuous by its absence. If new strategies improve Canada’s crime detection and prevention effectiveness by disrupting more criminal activity, without necessarily increasing the number of money laundering prosecutions, its FATF “effectiveness” ratings might fall. An unintended consequence of a framework that fails to define or evaluate outcomes as it purports, such a result would not be Canada’s intention, or FATF’s.
4.5.3 FATF’s “high-level objective” misses the (crime and terrorism prevention) point.
Whether FATFs “high-level objective” to protect financial systems from the perceived threat of money laundering should be the peak policy goal generates a different debate. Scholars have questioned the premise that money laundering threatens the integrity of the financial system (Alldridge, 2003; Alldridge, 2016; Harvey, 2008, 2009; Harvey and Lau, 2009; Naylor, 2002; Tsingou, 2010; Van Duyne et al., 2005; Verhage, 2009). From an effectiveness perspective, however, FATF’s “high-level objective” arguably misses the mark in any event. Absent unintended consequences of inconsistency between domestic policy objectives to detect and prevent serious crime and terrorism and FATF “effectiveness” ratings based on activity and output measures, higher-order crime and terrorism prevention goals might better align with national criminal justice strategies. Moreover, such higher-order goals are already recognised by FATF [for example, “without a usable profit, the criminal activity will not continue” (FATF, 2015)] and by other supranational bodies such as the World Bank (Schott, 2006, p. II-7) and United Nations (UNODC, 2011, p. 131).
If there is any prospect of revisiting FATF’s “effectiveness” methodology, higher-order objectives consistent with member states’ criminal justice goals, and potentially offering a more powerful unifying self synchronisation enabler for coordinated action by independent state and private actors (Hayne et al., 2013), might also usefully be reviewed.
4.5.4 Whither FATF’s new clothes?
Before concluding, this section returns to the rejoinder in section 4.3.4, that FATF’s “effectiveness” measures properly focus only on laundering, not crime and terrorism prevention. If so, what is the point of the complexity and expense of a vast global AML/CFT industry? If the number of money laundering prosecutions matter more than genuine outcome measures reflecting their effect and impact on crime and terrorism prevention objectives, the global AML/CFT complex, and FATF, arguably serves little purpose.
Braithwaite and Drahos observed that the “global strategy” of establishing “a wider epistemic community of money-laundering compliance based on peer review [measured against FATF’s 40 recommendations] may not have worked in eliminating money-laundering, but it has worked as a strategy for globalizing an extremely expensive new regulatory order” (Braithwaite and Drahos, 2000, p. 106). Likewise, another layer of compliance adding 11 more tick-boxes may not have reduced the incidence of terrorism or serious profit-motivated crime but has certainly increased the complexity and expense of the modern regulatory environment.
The problem is not, however, “the expansion of [AML/CFT] measures as such, but the expansion of measures that are not based on the assessment of policy impacts” (Brzoska, 2016, p. 917). That is, outcomes, properly defined; unfortunately, largely absent from FATF’s “effectiveness” methodology.
With FATF’s “main focus” (FATF, 2017b) now on effectiveness, its methodology should focus on outcomes that matter. Money laundering prosecutions, for example, retain validity as output measures and a crime disruption tool, but not an outcome-oriented policy goal. Australian, Canadian and other authorities know this, evidenced by their primary criminal justice and security objectives. To meet the needs of member states, with genuine outcome measures better connected with detecting, deterring and preventing serious crime and terrorism, it may be time for FATF to consider reexamining the ruler against which it measures effectiveness.
4.6 Conclusion and grounds for optimism
Constructively critiquing FATF’s “effectiveness” methodology, this article contends that it does not yet fully reflect an outcome-oriented model as it purports. More interaction between the AML/CFT and policy effectiveness and outcomes literature and practice may help improve the implementation of a framework intended to advance better outcomes.
These findings are not critical of FATF or countries seeking to meet its strictures. The acknowledgement that evaluating for effectiveness was missing, and important, is a positive step. Likewise, that outcomes matter. If the current model fails to meet its intention, the intent is clear. It is not sufficient to establish rules, or for countries to implement them or to demonstrate that such rules meet a defined set of standards. They also need to work. Moreover, FATF has a long history of adjusting standards and guidance as circumstances change and new information emerges. If FATF accepts that some principles and practices underpinning its effectiveness framework can be improved, that tradition might reasonably be expected to continue.
|Inputs||Resources used to produce outputs||Resources applied to deliver outputs and influence outcomes
Example: Staff and consultant time and expenses to develop AML/CFT training program
|Activities||How inputs are used to produce outputs||Some activities and processes may be linked to specific outputs. Others, such as internal accountability systems, are not directly linked
Example: Preparing AML/CFT training sessions
|Outputs||Goods and services produced||Outputs that influence outcomes may need to be delivered before intended outcomes can be achieved
Example: AML/CFT training sessions presented
Example output targets (firm): Deliver 50 sessions per year Pass rates 90%+. Increase attendee numbers 10%.
Example output targets (country): “High” rating for FATF IO-7
|Outcomes||The effect or impact on the target population||The effects or consequences of the program may be directly or indirectly the result of inputs, activities, outputs and controllable and uncontrollable external influences. Immediate, intermediate and ultimate outcomes may be distinguished; as first-level effects and longer-term impacts|
“Immediate outcomes” linked to FATF recommendations
|Immediate outcome||FATF recommendation||FATF recommendation
|1 Risk, policy and coordination||1, 2, 33, 34|
|2 International cooperation||36-40||9, 24, 25, 32|
|3 Supervision||14, 26-28, 34, 35||1, 40|
|4 Preventive measures||9-23||1, 6, 29|
|5 Legal persons and arrangements||24, 25||1, 10, 37, 40|
|6 Financial intelligence||29-32||1, 2, 4, 8, 9, 34, 40|
|7 Money laundering investigation and prosecution||3, 30, 31||1, 2, 32, 37, 39, 40|
|8 Confiscation||1, 4, 32||30, 31, 37, 38, 40|
|9 Terrorist financing investigation and prosecution||5, 30, 31, 39||1, 2, 32, 37, 40|
|10 Terrorist financing preventive measures, sanctions||1, 4, 6, 8||14, 16, 30-32, 37, 38, 40|
|11 Proliferation financial sanctions||7||2|
FATF Recommendations linked to ‘immediate outcomes’
|FATF recommendation||IO||IO elements|
|1 Assessing risks and applying risk-based approach||1, 8, 10||3-7, 9|
|2 National cooperation and coordination||1||6, 7, 9, 11|
|3 Money laundering offence||7|
|4 Confiscation and provisional measures||8, 10||6|
|5 Terrorist financing offence||9|
|6 Targeted financial sanctions (terrorism)||10||4|
|7 Targeted financial sanctions (proliferation)||11|
|8 Non-profit organisations||10||6|
|9 Financial institution secrecy laws||4||2, 6|
|10 Customer due diligence||4||5|
|11 Record keeping||4|
|12 Politically exposed persons||4|
|13 Correspondent banking||4|
|14 Money or value transfer services||3, 4||10|
|15 New technologies||4|
|16 Wire transfers||4||10|
|17 Reliance on third parties||4|
|18 Internal controls and foreign branches/subsidiaries||4|
|19 Higher-risk countries||4|
|20 Reporting of suspicious transactions||4|
|21 Tipping-off and confidentiality||4|
|22 DNFBPs: Customer due diligence||4|
|23 DNFBPs: Other measures||4|
|24 Transparency, beneficial ownership (legal persons)||5||2|
|25 Transparency, beneficial ownership (legal arrangements)||5||2|
|26 Regulation and supervision of financial institutions||3|
|27 Powers of supervisors||3|
|28 Regulation and supervision of DNFBPs||3|
|29 Financial intelligence units||6||4|
|30 Law enforcement/investigative (responsibilities)||6, 7, 9||8, 10|
|31 Law enforcement/investigative (powers)||6, 7, 9||8, 10|
|32 Cash couriers||6, 8||2, 7, 9, 10|
|34 Guidance and feedback||1, 3||6|
|36 International instruments||2|
|37 Mutual legal assistance||2||5, 7-10|
|38 Mutual legal assistance: freezing and confiscation||2||8, 10|
|39 Extradition||2, 9||7|
|40 Other forms of international cooperation||2||3, 5-10|
Updated, at 12 April 2018: Armenia, Andorra, Australia, Austria, Bahamas, Bangladesh, Barbados, Belgium, Bhutan, Botswana, Cambodia, Canada, Costa Rica, Cuba, Denmark, Ethiopia, Fiji, Guatemala, Honduras, Hungary, Iceland, Ireland, Isle of Man, Italy, Jamaica, Macao, Malaysia, Mexico, Mongolia, Nicaragua, Norway, Panama, Portugal, Samoa, Serbia, Singapore, Slovenia, Spain, Sri Lanka, Sweden, Switzerland, Thailand, Trinidad & Tobago, Tunisia, Uganda, Ukraine, United States, Vanuatu and Zimbabwe.
Alldridge, P. (2003), Money Laundering Law: Forfeiture, Confiscation, Civil Recovery, Criminal Laundering, and Taxation of the Proceeds of Crime, Hart Publishing, Oxford.
Alldridge, P. (2016), What Went Wrong with Money Laundering Law?, Palgrave Macmillan, London.
Anand, A. (2011), “Combating terrorist financing: is Canada’s legal regime effective?”, University of Toronto Law Journal, Vol. 61 No. 1, pp. 59-71.
Anao (2001), Performance Information in Portfolio Budget Statements: Performance Audit Report 18, Australian National Audit Office, Canberra.
Anao (2007), Application of the Outcomes and Outputs Framework, Performance Audit Report 23, 2006-07, Australian National Audit Office, Canberra.
Blumstein, A. and Wallman, J. (2006), “The crime drop and beyond”, Annual Review of Law and Social Science, Vol. 2 No. 1, pp. 125-146.
Bouckaert, G., Peters, B.G. and Verhoest, K. (2010), The Coordination of Public Sector Organizations, Palgrave Macmillan, Basingstoke.
Bourgon, J. (2007), “Responsive, responsible and respected government: towards a new public administration theory”, International Review of Administrative Sciences, Vol. 73 No. 1, pp. 7-26.
Braithwaite, J. and Drahos, P. (2000), Global Business Regulation, Cambridge University Press, Cambridge, MA.
Brzoska, M. (2016), “Consequences of assessments of effectiveness for counterterrorist financing policy”, Administration & Society, Vol. 48 No. 8, pp. 911-930.
CFATF (2017), Mutual Evaluation Report: Jamaica, Caribbean Financial Action Task Force, Port of Spain, Trinidad and Tobago.
Chaikin, D. (2009), “How effective are suspicious transaction reporting systems?”, Journal of Money Laundering Control, Vol. 12 No. 3, pp. 238-253.
Chong, A.E. and Lopez de Silanes, F. (2015), “Money laundering and its regulation”, Economics & Politics, Vol. 27 No. 1, pp. 78-123.
Cook, A.-L. (2004), “Managing for outcomes’ in the New Zealand public management system”, New Zealand Treasury Working Papers, NZ Treasury, Wellington, NZ.
Cuéllar, M.-F. (2003), “The tenuous relationship between the fight against money laundering and the disruption of criminal finance”, The Journal of Criminal Law and Criminology, Vol. 93 Nos 2/3, pp. 311-466.
Davy, D. (2016), “Anti-human trafficking interventions: how do we know if they are working?”, American Journal of Evaluation, Vol. 37 No. 4, pp. 486-504.
De Vries, M. and Nemec, J. (2013), “Public sector reform: an overview of recent literature and research on NPM and alternative paths”, International Journal of Public Sector Management, Vol. 26 No. 1, pp. 4-16.
Deleanu, I.S. and Ferwerda, J. (2014), “Effectiveness: threat and policy response”, in Unger, B., Ferwerda, J., Van Den Broek, M. and Deleanu, I. (Eds) The Economic and Legal Effectiveness of the European Union’s Anti-Money Laundering Policy, Edward Elgar, Cheltenham.
Denhardt, J.V. and Denhardt, R.B. (2006), New Public Service: Serving, Not Steering, ME Sharpe, New York, NY.
Desroches, F. (2016), “Dismantle or disrupt: strategies targeting higher-level drug traffickers”, RCMP Gazette, Vol. 78 No. 1, pp. 26-28.
Donohue, J.J. and Levitt, S.D. (2001), “The impact of legalized abortion on crime”, The Quarterly Journal of Economics, Vol. 116 No. 2, pp. 379-420.
Dunleavy, P. (2013), “Joined-up public services”, in Kippin, H., Stoker, G. & Griffiths, S.. (Eds), Public Services: A New Reform Agenda, Bloomsbury, London.
Easton, D. (1965), A Systems Analysis of Political Life, John Wiley, New York, NY.
FATF (2013a), The FATF Recommendations: International Standards on Combating Money Laundering and the Financing of Terrorism & Proliferation, Financial Action Task Force, Paris.
FATF (2013b), Methodology for Assessing Compliance with the FATF Recommendations and the Effectiveness of AML/CFT Systems, Financial Action Task Force, Paris.
FATF (2013c), Procedures for the FATF Fourth Round of AML/CFT Mutual Evaluations, Financial Action Task Force, Paris.
FATF (2014), Mutual Evaluation Report: Spain, Financial Action Task Force, Paris.
FATF (2015), FAQ: Money Laundering [Former Webpage], Financial Action Task Force, Paris, available at: www.fatf-gafi.org/pages/faq/moneylaundering/ (accessed 25 May 2015).
FATF (2017a), An Effective System to Combat Money Laundering and Terrorist Financing, Financial Action Task Force, Paris, available at: www.fatf-gafi.org/publications/mutualevaluations/documents/effectiveness.html (accessed 20 July 2017).
FATF (2017b), Mutual Evaluations, Financial Action Task Force, Paris, available at: www.fatf-gafi.org/publications/mutualevaluations/documents/more-about-mutual-evaluations.html (accessed 20 July 2017).
FATF and APG (2015), Mutual Evaluation Report: Australia, Financial Action Task Force, Asia/Pacific Group on Money Laundering, Paris & Sydney.
FATF and APG (2016), Mutual Evaluation Report: Canada, Financial Action Task Force, Asia/Pacific Group on Money Laundering, Paris & Sydney.
Ferwerda, J. (2009), “The economics of crime and money laundering: does anti-money laundering policy reduce crime?”, Review of Law & Economics, Vol. 5 No. 2, pp. 903-929.
Fisher, J.Q. (2014), “Regulators must show firms that anti-money laundering rules are helping to tackle crime”, Legal Week, Vol. 23.
François, A., Magni-Berton, R. and Weill, L. (2014), “Abortion and crime: cross-country evidence from Europe”, International Review of Law and Economics, Vol. 40, pp. 24-35.
Funnell, S.C. and Rogers, P.J. (2011), Purposeful Program Theory: Effective Use of Theories of Change and Logic Models, Jossey-Bass, San Francisco, CA.
Garland, D.W. (1996), “The limits of the sovereign state: strategies of crime control in contemporary society”, The British Journal of Criminology, Vol. 36 No. 4, pp. 445-471.
Halliday, T.C., Levi, M. and Reuter, P. (2014), Global Surveillance of Dirty Money: Assessing Assessments of Regimes to Control Money-Laundering and Combat the Financing of Terrorism, Center on Law and Globalization, Champaign.
Harvey, J. (2008), “Just how effective is money laundering legislation?”, Security Journal, Vol. 21 No. 3, pp. 189-211.
Harvey, J. (2009), “The search for crime money. Debunking the myth: facts versus imagery”, Journal of Money Laundering Control, Vol. 12 No. 2, pp. 97-100.
Harvey, J. and Lau, S.F. (2009), “Crime-money, reputation and reporting”, Crime, Law and Social Change, Vol. 52 No. 1, pp. 57-72.
Hayne, S.C., Troup, L.J. and Li, K. (2013), “Commander’s Intent and distributed collaborating teams”, 46th Hawaii International Conference on System Sciences, pp. 86-95.
Joyce, T. (2004), “Did legalized abortion lower crime?”, The Journal of Human Resources, Vol. 39 No. 1, pp. 1-28.
Kim, W.C. and Mauborgne, R. (2003), “Tipping point leadership”, Harvard Business Review, April 2003, pp. 60-69.
Kristensen, J.K., Groszyk, W.S. and Bühler, B. (2002), “Outcome-focused management and budgeting”, OECD Journal on Budgeting, Vol. 1 No. 4, pp. 7-34.
Levi, M. (2002), “Money laundering and its regulation”, The Annals of the American Academy of Political and Social Science, Vol. 582 No. 1, pp. 181-194.
Levi, M. (2012), “How well do anti-money laundering controls work in developing countries?”, in Reuter, P. (Ed.), Draining Development? Controlling Flows of Illicit Funds from Developing Countries, Washington, DC, World Bank.
Levi, M. and Maguire, M. (2004), “Reducing and preventing organised crime: an evidence-based critique”, Crime, Law and Social Change, Vol. 41 No. 5, pp. 397-469.
Levi, M. and Reuter, P. (2006), “Money laundering”, Crime and Justice, Vol. 34 No. 1, pp. 289-375.
Levi, M. and Reuter, P. (2009), “Money laundering”, in Tonry, M.. (Ed.) The Oxford Handbook of Crime and Public Policy, Oxford University Press, Oxford.
Levitt, S.D. and Dubner, S.J. (2006), Freakonomics: A Rogue Economist Explores the Hidden Side of Everything, Penguin, London.
McConnell, A. (2015), “What is policy failure? A primer to help navigate the maze”, Public Policy and Administration, Vol. 30 Nos 3/4, pp. 221-242.
Mackay, K. (2011), “The performance framework of the Australian government, 1987 to 2011”, OECD Journal on Budgeting, Vol. 11 No. 3, pp. 1-48.
Mayne, J. (2001), “Addressing attribution through contribution analysis: using performance measures sensibly”, The Canadian Journal of Program Evaluation, Vol. 16 No. 1, pp. 1-24.
Mayne, J. (2004), “Reporting on outcomes: setting performance expectations and telling performance stories”, The Canadian Journal of Program Evaluation, Vol. 19 No. 1, pp. 31-60.
Moore, M.H. (2013), Recognizing Public Value, Harvard University Press, Cambridge, MA.
Mulgan, R. (2008), “The accountability priorities of Australian parliamentarians”, Australian Journal of Public Administration, Vol. 67 No. 4, pp. 457-469.
Naylor, R.T. (2002), Wages of Crime: Black Markets, Illegal Finance and the Underworld Economy, Cornell University Press, Ithaca.
OECD (2002), Governing for Results. Modernising Government: How Well Have Public Administration Policy and Practices Transferred from Developed to Transitional Countries?, Organisation for Economic Co-operation and Development, Paris.
Orde, H.S.R. (2012), “Foreword”, in Eterno, J.A. and Silverman, E.B. (Eds), The Crime Numbers Game: Management by Manipulation, CRC Press, Boca Raton, FL.
Paz-Ybarnegaray, R. and Douthwaite, B. (2017), “Outcome evidencing: a method for enabling and evaluating program intervention in complex systems”, American Journal of Evaluation, Vol. 38 No. 2, pp. 275-293.
Pollitt, C. (2015), “Towards a new world: some inconvenient truths for Anglosphere public administration”, International Review of Administrative Sciences, Vol. 81 No. 1, pp. 3-17.
Procunier, M. (2013), “The best of the worst: criminalized professionals create new challenges”, RCMP Gazette, Vol. 75 No. 1, pp. 7-8.
Productivity Commission (2013), On Efficiency and Effectiveness: Some Definitions. Staff Research Note, Productivity Commission, Melbourne.
Productivity Commission (2015), “Approach to performance reporting”, Report on Government Services, Productivity Commission, Melbourne.
Reuter, P. and Truman, E.M. (2004), Chasing Dirty Money: The Fight against Money Laundering, Peterson Institute for International Economics, Washington, DC.
Reyes, J.W. (2007), “Environmental policy as social policy? The impact of childhood lead exposure on crime”, The BE Journal of Economic Analysis & Policy, Vol. 7 No. 1, pp. 1-41.
Rider, B.A.K. (2002a), “The weapons of war: the use of anti-money laundering laws against terrorist and criminal enterprises, part 1”, Journal of Banking Regulation, Vol. 4 No. 1, pp. 13-31.
Rider, B.A.K. (2002b), “The weapons of war: the use of anti-money laundering laws against terrorist and criminal enterprises, part 2”, Journal of Banking Regulation, Vol. 4 No. 2, pp. 145-161.
Rider, B.A.K. (2004), “The war on terror and crime and the offshore centres: the ‘new’ perspective?”, in Masciandaro, D. (Ed.), Global Financial Crime: Terrorism, Money Laundering and Offshore Centres, Ashgate, Aldershot.
Roeder, O., Eisen, L.-B. and Bowling, J. (2015), What Caused the Crime Decline?, Brennan Center for Justice at New York University School of Law, New York, NY.
Schott, P.A. (2006), Reference Guide to Anti-Money Laundering and Combating the Financing of Terrorism, World Bank, Washington, DC.
Sharman, J.C. (2011), The Money Laundry: Regulating Criminal Finance in the Global Economy, Cornell University Press, Ithaca.
Sharman, J.C. (2017), The Despot’s Guide to Wealth Management: On the International Campaign against Grand Corruption, Cornell University Press, Ithaca.
Tanner, L. (2008), Operation Sunlight: Enhancing Budget Transparency, Commonwealth of Australia, Canberra.
Tavares, C., Thomas, G. and Roudaut, M. (2010), “Money laundering in Europe”, Methodologies and working papers, European Commission, Luxembourg.
Tiernan, A. (2012), “Delivering public policy”, in Smith, R., Vromen, A. and Cook, I.. (Eds), Contemporary Politics in Australia: Theories, Practices and Issues, Cambridge University Press, New York, NY.
Tsingou, E. (2010), “Global financial governance and the developing anti-money laundering regime: what lessons for international political economy?”, International Politics, Vol. 47 No. 6, pp. 617-637.
Unger, B., Ferwerda, J., Van Den Broek, M. and Deleanu, I.S. (2014), The Economic and Legal Effectiveness of the European Union’s Anti-Money Laundering Policy, Edward Elgar Publishing, Cheltenham.
UNODC (2011), Estimating Illicit Financial Flows Resulting from Drug Trafficking and Other Transnational Organized Crimes, United Nations Office on Drugs and Crime, Vienna.
Van Duyne, P.C. (2003), “Money laundering policy: fears and facts”, in van Duyne, P., Von Lampe, K. and Newell, J.L. (Eds), Criminal Finances and Organising Crime in Europe, Wolf Legal Publishers, Nijmegen.
Van Duyne, P.C. (2011), “Transnational organised crime, laundering and the congregation of the gullible”, Valedictory by Professor Petrus C Van Duyne, Tilburg University, Tilburg.
Van Duyne, P.C., Groenhuijsen, M. and Schudelaro, A.A.P. (2005), “Balancing financial threats and legal interests in money-laundering policy”, Crime, Law and Social Change, Vol. 43 No. 2-3, pp. 117-147.
Verhage, A. (2009), “Between the hammer and the anvil? The anti-money laundering-complex and its interactions with the compliance industry”, Crime, Law and Social Change, Vol. 52 No. 1, pp. 9-32.