ECB Banking Supervision Comprehensive Assessments (CAs)—Lessons learned

LISTEN TO AUDIO VERSION:

Critical Success Factors (CSF´s)

In our previous article – link, we provided an insight into the topic of Comprehensive Assessments from the perspective of its concrete results, experiences and future outlook. The aim of this article, however, is to provide more details on the practical Critical Success Factors (CSF´s) and lessons learned, from the assessed bank’s perspective. Questions like – How should we best prepare for the assessment? How should we communicate with the auditors? Do we need to prepare the physical files of our clients or just the digital ones? Which source of data should we take? Which collateral appraisal methodology will be applied by the auditors? etc., can all arise during the preparation phase of the assessment. In this article, we will try to provide at least some of the answers to those questions. The answers will be based on our methodological understanding of the topic as well as experiences obtained during multiple Comprehensive Assessment projects on which zeb was in a supportive role during 2014 and 2015.

2015 ECB Comprehensive Assessment in a nutshell

The 2015 exercise consisted of an asset quality review (AQR), a stress test and a “join-up” of AQR and stress test outcomes. It was similar to the 2014 Comprehensive Assessment in terms of its scope and depth, given that AQR and stress test were performed based on the methodologies applied in 2014. The AQR was a point-in-time assessment of the carrying values of banks’ assets as of 12/31/2014 and the AQR results served as a starting point for the stress test, which projected the evolution of banks’ capital positions over three years (2015-2017) under a baseline scenario and an adverse scenario. Following the completion of the AQR and stress test, the stress test was adjusted in the join-up to take into account the AQR findings. As in the 2014 exercise, banks were required to address remaining shortfalls in a timely manner by issuing capital instruments or undertaking other eligible measures to restore their capital positions to the required levels. The main difference between 2014 and 2015 assessment was in terms of the join-up exercise. In 2015, the banks had to run the join-up on their own, either using a “home-made” tool or the one provided by the ECB. It then only played the quality assurance role.

 What have we learned so far?

Given the very short timeline, ECB usually relies on the local auditing authority responsible for executing the assessment itself (in this context ECB provides only quality assurance function), one of the Critical Success Factors is to start with the preparation activities on the bank side in all of the assessment areas (incl. project management, client files, data, etc.) as soon as possible and definitely before the start of the on-site audit. In doing so, not only the assessment itself will be much more operationally efficient and communication bottlenecks will be avoided, but also the bank will be able to position itself much better against the auditors’ requests, resulting in overall better results of the assessment.

In the following paragraphs, we will provide a deeper insight and guidance into specific aspects of preparation and key lessons learned.

Abbildung 1: Key lessons learned from ECB Comprehensive Assessments

A) Prepare for the demanding organizational requirements

Due to its high complexity and underlying risks for the bank, the CA exercise needs to be tackled seriously and in a highly structured manner. This means not only to identify and mobilize the key bank resources to deal with the upcoming challenges, but also to prepare the organization in a best possible manner in all organizational aspects, including the definition of the processes, communication principles, templates and infrastructure to be used during the audit. In this context, we also recommend to establish a formal project organization and application of best project management practices. A strong project management function is as such one of the critical success factors for the overall exercise and its role is also to ensure that the bank is prepared organizationally for the upcoming audit as well as that the audit itself is handled in a professional manner. Our experiences show that not only the quality of the portfolio of the bank, its financial soundness and stability have an impact on the final result of the assessment. Moreover, “soft facts” like speed and quality of providing requested information to the auditors or attitude and professionalism of bank´s staff, play an often underestimated role.

Training the project team members (especially the staff responsible for direct contact with the auditors) and making sure that the principles and practices introduced in the preparation phase are applied for the whole duration of the audit lies in the responsibility of the whole project management team. In this context, the bank can expect that the auditor will usually welcome a proactive approach and will in most cases accept and apply the proposed concept in practice (with possibly some minor amendments). Further, based on the results of the sampling and selected client portfolio (its geographical diversity as well as pure numbers of files), it is worth considering to set up a local PMO organization in subsidiaries where assessments will take place and staffing the required roles with the most appropriate staff. This does not only streamline the communication process but also assures the bridging of any language-related / cultural differences that otherwise may arise. Having all this in mind, the following organizational key points can be derived:

• Definition and implementation of streamlined communication processes, making sure all the inputs from the auditors are collected centrally, setup of a SPOC and centralized collection of all requests from auditors and respective answers
• Setting up a proper communication infrastructure to support the defined communication processes, for the whole duration of the assessment – e.g. dedicated SharePoint site for providing credit files requested by the auditors in digital form, common e-mail account for direct communication with auditors, etc.
• Definition and implementation of the appropriate governance structure to support the effective information flow vertically and horizontally (e.g. weekly status meetings with subsidiaries, daily stand-up meetings, etc.)
• Ensuring appropriate knowledge gains in the bank by engaging into internal and / or external trainings.

B) Consider external support in case methodological knowledge is not available within the bank

Having a methodological knowledge of the Comprehensive Assessment topic is definitely one of the major leverages for the reduction of the capital add-on after the CA exercise. In order to use that leverage, the bank needs to consider engaging into external consultancy support, if the required know-how or resources are not available within the bank. Based on the specific knowledge gaps of the bank and the underlying risks, the bank should approach one or more possible vendors to support the most critical areas of the assessment, or the overall exercise itself. In practice such an investment will most likely pay off since vendors bring a combination of required methodological knowledge as well as practical experiences. Paired with in-depth know-how on the internal policies, procedures and client files from the bank side, a strong foundation for the successful preparation and conduction of the CA itself will be built up.

C) Put extra effort into quality assurance, consolidation and centralization of data

In the context of the sampling exercise, one of the key aspects of success is having a solid client segmentation logic in place. In this context, it is necessary for the bank to understand the overarching impact of any possible issues identified in this area, since the erroneous client segmentation not only results in huge impacts regarding the ECB focus portfolios, with the result that:

• ECB portfolio coverage in preparation phase can be dropped below 50% RWA threshold (e.g. huge companies which changed to state owned)
• Changes from retail to non-retail and vice versa (e.g. different exposure retail threshold defined by national supervisory)
• Changes between ECB focus portfolios,

but can also have consequent impact on some other areas of the assessment, e.g. stress testing. Additionally, erroneous NPE/PE decisions can also lead to changed risk levels and exposure buckets, which mean that a selection of main and reserve sample based on correct segmentation data needs to be repeated again.

When it comes to the Data Integrity Validation (DIV) exercise, practice often revealed a slow increase in data quality and that multiple cycles of data quality checks were necessary to reduce the error rate below the allowed threshold. This is even more emphasized in cases when no centralized source of data was available due to a different error history. Since data is fairly sparse and sub-segments are often times thinly populated, even small changes to the data base may have a significant impact on the risk parameters to be estimated and hence the ECB challenger provisions at a later time. A prerequisite for a successful DIV exercise was the implementation of the automatic error detection process (based on approximately 400 ECB checks) as well as a deep understanding of ECB data fields, error rates as well as the willingness for corrections on the bank side. For some data fields, an overruling of the local data was needed due to the application of the central logic (e.g. rating scores, Basel segmentation, etc.). At the beginning of the data preparation exercise, it is also highly recommended to take a decision on a data single point of truth. Two options are available:

1. Subsidiary source systems, in cases where banks do not have a harmonized IT landscape on the group level or
2. Flat table from the central Risk Management department.

The decision needs to be taken based on the comparison of both scenarios considering risk and overall data quality and availability aspects in a specific bank environment. In that context, it is also worth considering putting an extra effort into creating a centralized, group wide source of data, i.e. loan tape, based on the portfolio selection. This should assure avoidance of numerous pitfalls in data preparation and quality assurance area by ensuring the consistency of group-wide data as well as speeding up the creation process itself.

D) Prepare the credit files upfront based on available input and build up strong “defense strategies” of the cases

Once the preparation of the credit files starts, bank should not just focus on structuring and preparing the handover of the outgoing files to the auditors, but should also focus on building up the strong arguments and “defense” lines with the objective of justifying the current risk and provisioning levels. One of the biggest challenges is to keep debtors performing under AQR perspective due to the fact that triggers for financial difficulties are stricter than under IAS 39, e.g. DSCR > 1.1.

Experience shows, if presented and discussed with the auditors in the right way (e.g. starting with a brief executive summary, where auditor can easily find all necessary information at a glance and where the bank has the opportunity to present first strong arguments), the bank can expect these arguments to be considered, thus resulting in a reduced amount of additional provisions being calculated initially.

Smart argumentation of cases during credit file review can also lead to reduced capital impact during the join-up exercise (e.g. if a case is a new default during the ongoing year, it could be better to argument that the case was already defaulted under AQR perspectives in the previous year, since it can lead to a reduction of PD).

E) Analyze and challenge the methodologies and concepts applied by the auditors and underlying results

In general, practice shows that taking a more proactive approach and showing initiative in the analysis of methodologies and concepts applied by the auditors can significantly improve the overall assessment results. For example, a centrally aligned and optimized DSCR calculation can assure consistency within the whole bank and reduce the effort on the bank’s side enormously. In this way, the transparency of the calculation increases which can avoid the possible higher NPE classification. Further on, it is worth mentioning that in terms of the validation of the challenger model (Collective Provisions Analysis area – CPA), the analysis of the parameter deviation and in-depth discussion of methodology can in practice also lead to a correction and adaption of the parameter. The recommendation for the bank is to calculate the parameters in parallel to have a good basis for comparison and impact analysis.

F) Take early preparatory steps for the conduction of the stress test and join-up

Not only can the AQR part of the CA be prepared beforehand; there are also measures that can facilitate a smooth stress testing and join-up phase.

The first and most important point refers to data consistency. As elaborated previously, a specific AQR segmentation logic has to be applied during the loan tape preparation. Most often, this new segmentation logic has to be implemented during the AQR preparation. While doing so, one has to be aware that later during the Comprehensive Assessment, the auditor will carry out checks between stress testing and loan tape data. Therefore, an important preparatory step for the stress test can and should be conducted in advance, which consists of the assurance of data consistency, especially in terms of segmentation, between the data delivered during AQR and the stress testing base data set.

The second preparatory step refers to the join-up phase. During the join-up, the AQR results have to be incorporated into the stress test. This is not done via a complete re-run of the stress test using AQR results, but rather using a join-up tool. This tool joins AQR and stress testing data via several assumptions to achieve an easy prediction of the influence of AQR results on stress testing outcome. In 2015, the banks were allowed to use a home-made rather than an ECB-provided tool. Developing such a tool could help the banks to keep direct control during the join-up phase and applied underlying assumptions and methods can be shaped more easily. When deciding on the development of such a tool, one has to keep in mind that the results of the join-up calculations will have to be defended before the ECB. Therefore, it is crucial to be able to track the underlying assumptions and calculation steps of such a tool to make them visible to the ECB. As a join-up tool is not developed within days; an early focus on this topic is advised.

Conclusion

When facing the upcoming Comprehensive Assessment challenge, each bank needs to make a decision how it will approach and prepare in the best possible way. We believe that good preparation is the key, and that putting an extra effort in the preparation activities, while considering the above mentioned lessons learned can significantly influence the overall result of the assessment in a positive way.