TUNING!

The word “Tuning” may bring to mind the idea of tuning an instrument. As we all know, if your instrument is not tuned properly, no one will appreciate all the time and effort the musician puts into playing it.  If all the instruments in an orchestra are not tuned to the chosen frequency, for example, “A” at 440 Hz, the music will be extremely unpleasant, and the audience will never pay to see another performance from that orchestra again.   

What does that have to do with BSA/AML/OFAC systems? 

Like a fine instrument, your system will work better when it is tuned properly. 

As we all know, BSA/AML/OFAC systems range from relatively non-complex spreadsheet-based review of transactions to complex systems that use artificial intelligence to learn the characteristics of waived or cleared alerts and use that learning to help minimize future non-productive alerts. It does not matter whether you think of your BSA/AML/OFAC system or process as a model, series of models, an application, a series of rules, a series of spreadsheets, or a series of queries, or a bunch of algorithms.  If it is not tuned properly, the system output – while it doesn’t have a sound – may be totally unreliable and will open the financial institution to regulatory enforcement actions!  

What is “out of tune” in terms of a poorly tuned BSA/AML or OFAC system? 

Well, a poorly tuned BSA/AML/OFAC system may cost your bank in terms of false negatives leading to missed SARs or missed true OFAC hits and the resulting potential for regulatory fines or the expense of addressing a supervisory enforcement action.  It may also be manifest as an excessive and perhaps unnecessary number of OFAC false positive hits which will cost your bank in terms of resources devoted to investigating, documenting, and dispositioning unnecessary alerts and/or hits.  

Remember, you do not want to tune a system to simply reduce unproductive alerts or false positive hits. Regulators will not accept such reasoning.  While this may be a welcome outcome, the intent of a Tuning Project is to enhance or optimize your rules’ view into potentially suspicious or violative behavior. You want to make your enabled rules more effective. If saving resources through a reduction in alerts occurs in the process, so much the better.  

It is important to note that in a Tuning Project there may be recommendations that will potentially increase the alert/hit output but may identify previously unidentified potentially reportable activity.  An example of this might be a very narrow range of transaction dollar amounts to be considered for a Structuring Rule.  Increasing the size of that range may result in additional alerts but also may provide a better view into potentially reportable activity.  

Is there a specific frequency required for BSA/AML/OFAC system tuning? 

Interestingly, there is no explicit regulatory requirement to “tune” your BSA/AML/OFAC model, system, rules, etc. However, regulators’ expectations have developed, and grown, over time and BSA/AML/OFAC tuning is a normal expectation from regulators. Some banks tune annually, or every two years according to internal policy or only upon a triggering event, but periodic tuning should be required as a matter of internal policy. We often find that a requirement for tuning is included in an enforcement action, examination report, or an independent model validation. It is sound practice to cover this subject in your bank’s policies and procedures. 

Triggers for Tuning Projects 

Aside from requiring a tuning on an annual or a multi-year basis, a sound practice is to include potential triggering events in policy/procedure.  Some common triggering events are:  

• Significant changes in the bank’s customer base and/or product-services 
• Significant changes in the various rules’ alert and/or case production – or – notable changes in the alert-to-case-to-SAR statistics 

• The introduction of new rule(s), particularly if they had not been tuned done prior to implementation. 

When conducting a Tuning Project, should all rules be tuned? 

Not necessarily. 

• Rules established by management as “control rules” need not be tuned.  A control rule would be one that management has established to highlight a specific activity or event. An example of a control rule would be unusually large transactions of which management might wish to be aware.  
• Rules which search for transactional activity that no longer occurs and thus generates no alerts, may not need to be tuned and may be recommended for retirement if no longer necessary. (However, it is possible that the lack of observable transactional activity in the TMS may be the result of a mapping issue). 

• Rules (that are not control rules) that produce only a minimal amount of alerts/cases/SARs, may have thresholds set too high and be missing potentially reportable activity – or – could be a symptom of a mapping issue.  

Always ask yourself, “Do I have all the rules I need; and do I need all the rules I have? Are the settings/parameters appropriate for my bank’s risks?” A Tuning Project will help you answer these questions. 

What does a tuning look like? 

Tuning can take a number of forms – from simple adjustments of the specific thresholds used in a rule, to excluding transaction types that are not BSA/AML relevant but might nonetheless trigger alerts. Based on the features included in a BSA/AML/OFAC system and the complexity of the system in question, there may be specific features that facilitate tuning.  

A BSA/AML/OFAC Tuning Project will commonly include some or all of the following activities: 

• Determination of a review period 
• Review of Bank’s documentation supporting currently enabled rules and their parameters/settings 
• A review and analysis of currently enabled rules – rule types versus BSA/AML/OFAC risks identified in their most recent Risk Assessment 
• Keywords lists are reviewed. Are the list entries old and outdated or no longer relevant to the bank’s business activities? Has the bank periodically reviewed and if necessary, updated them? 
• Analysis of productivity by each rule including Alert-to-Case-to-SAR statistics 

• Determination of rules to be tuned in consultation with the bank 
• Analysis of the tunable rules using all or a combination of Above-the-Line (ATL) testing, Below-the-Line (BTL) Testing, and percentile analysis 
• Analysis will lead to recommendations for adjustments to rules’ threshold/parameter settings. This may also include recommendations to retire a rule that is inconsistent with the business activity of the bank, or non-productive rule. 
• Depending upon the system(s) in use, the initial recommendations may be tested and the results analyzed. The test results may lead to modification of recommendations and a second iteration of testing, or they may simply lead to final recommendations to the bank. 

Additionally, the judicious implementation of “exclude lists” or “good guy lists” (for both BSA and OFAC alerts) – if they are not already in use, and, if they are, a careful review of the entries on such lists, can also be a useful tool in tuning. The testing and analysis conducted, and the recommendations made must be documented.  

A Tuning Project can be very important to a financial institution.  It can help mitigate BSA/AML/OFAC risks and also help minimize the risk of regulatory enforcement actions, fines, and costly look-backs or other regulator directed remediation projects.  Regardless of the system in use and its technical complexity, a Tuning Project can be extremely beneficial.