Beyond the regulatory requirements (Federal Financial Institutions Examination Council, 2010), testing is conducting in different departments for different purposes. In the IT department you are focusing on software and system quality, bug identification and integration accuracy. The audit and compliance departments are validating the continued accuracy and compliance to policies and procedures. It is the continued monitoring that audit and compliance do that is addressed in this article.
Random sampling is likely to reduce the effectiveness to
identify risk or emerging issues according to (Hyde, G., 2007). It is these factors that drive monitoring testing. Targeted testing replaces random sampling based on risk and impact. Tests should be focused on risk and impact of compliance to policies and procedures. You should leverage year-over-year metrics to baseline, trend, and focus testing focus.
The purpose of testing is to validate that what was presented and what was delivered is the same. Testing is generally broken down into two types, event testing and monitoring. Event testing as the nomenclature represents is in response to an event, such as new system launch or product upgrade.
Examples of events:
- Proposal for change;
- new product;
- services;
- change management;
- introduction of new process;
- new systems; or
- monitoring control.
In this type of testing you are verifying the conditions stated in the business requirements have been meet. You may rely on a traceability metrics to satisfy the testing results. As an example, you are planning on launching a system to support the travel rule to detect names within the transaction that appears on the sanction lists from OFAC. In doing so you require that the algorithm considers name misspelling, normalization, the removal of noise words, and the consideration of synonyms. You would create test cases to validate that these requirements have been met. If not you would need to track the failure to meet these requirements to understand the confidence in the test delivery result set and the remediation rates. Your reporting from testing drives your costs for events and track closure of outstanding defects. In repeatable events you are able to compare these metrics to previous events and provide more in-depth reporting on controls. This type of testing generally is conducted by IT and QA groups.
The second type of testing; monitoring is the ongoing testing to validate that controls and processes are being completed correctly, timely, and accurately. Monitoring testing also identifies emerging challenges that that require changes or tuning to current process or controls. Unlike events these tests spot check or sample system outputs and continue to meet the requirements. An example would be a loan application that continues to collect the data required for compliance with institutions policies and procedures. Another would be to validate a new product or a service offering that is added into the existing AML system for monitoring.
Examples of monitoring:
- Reconciliation;
- new product offer;
- new service offer;
- new transaction code;
- KYC data collection process;
- log file reviews; or
- system controls.
It is in these testing situations that reporting is most important. You need repetitive testing processes with controls and tracking to demonstrate your institutions due diligence. Demonstrable metrics would identify satisfactory results, unexpected changes, remediation rates, identified issue rate, open issues, and testing frequency. This type of testing is conducted by audit (annually) and compliance (continuously).
Testing software today is developed to assist IT and QA in development testing as part of the Software Development Life Cycle (SDLC). This type of testing software requires adaptation for monitoring testing. Monitoring software should include templates for monitoring testing. These templates should support frequency of testing, automated scheduling, notification of testing required, segregation of duties, and trending reports. Consider software that specifically addresses process monitoring.
Questions to consider are: how are you storing the test results (spreadsheets, word documents, software)? Are the test cases reusable? Does the system provide automation? Does the previous test case results support trending? Does the system support segregation of duties (IT, Compliance, Audit)?
What is your institution doing to ensure you stay compliant?
References
Federal Financial Institutions Examination Council (2010). Bank Secrecy Act/ Anti-Money Laundering Examination Manual. Retrieved June 21, 2013: http://www.ffiec.gov/bsa_aml_infobase/documents/BSA_AML_Man_2010.pdf.
Hyde, G. (2007). Enhanced audit testing. The Internal Auditor, 64(4), 65-68,8. Retrieved from http://search.proquest.com/docview/202736076?accountid=458
If you would like to know more about ARC Risk and Compliance, and our tuning and false positive remediation, please contact us.
