Models are all around us — integral and important to operational efficiency — but the risks that they sometimes pose can materially impact the financial well-being of even the most well-structured organizations. In order to understand the risks, we must first define what a model is and what the inherent risks are when operating a model.
A model refers to a quantitative method, system or approach that applies statistical, economic, financial or mathematical techniques and assumptions to process data into quantitative estimates. In other words, it’s a methodical way to process and sort data. A model consists of three components:
- An input
- A processing element
- An output
To build on that idea, model risk is the potential for the misuse of models to adversely impact an organization. Model risk primarily occurs for three reasons: a) data, operational or implementation errors; b) prediction errors; and c) incorrect or inappropriate usage of model results.
Models exist and assist in the identification, assessment and evaluation measurement. It can also assist in monitoring both nonfinancial risks like behavioral analysis and financial risk like credit risk.
It therefore makes sense to warehouse these different models under a central Model Risk program. Such a holistic program can offer a consistent approach in addressing the challenges outlined below.
Challenges of a model risk program
Model risk programs can have the following inherent challenges:
- A lack of appreciation and understanding of the total number of models (inventory) being utilized by the organization
- Lack of a valid method to update the model inventory on a regular basis
- Inability to maintain the independence of the model validation department
- A model validation process that fails to demonstrate effective challenge, review and independence
- Lack of suitable data used in the model development process to support review/validation
- Lack of developmental evidence to substantiate model assumptions
- Lack of an explanation to support the application of expert judgment and model overrides
- Lack of validation or failure to re-evaluate at regular intervals
- Failure to maintain comprehensive and up-to-date model documentation
What to include in your model risk program framework
In order to address these challenges, I would recommend incorporating the following nine model risk documentation components that enable the framing of your model risk program.
9 risk documents that help frame the model risk program
- A clear and consistent organizational narrative documented in writing covering principles, objectives, scope, model risk program design (including standards) model risk appetite, model risk taxonomy, controls and industry regulations.
- A policy describing the oversight and governance which will include the roles and responsibilities of all stakeholders and participants.
- Well-thought-out and actionable model risk management policies and procedures to include:a. Data management policyb. Model validation policy and requirementsc. Model documentation requirements for both in-house developed models and third-party vendor models.
- Policy and procedures to include the completeness of the current model inventory and process of updating on an ongoing basis.
- Policy and procedures which will risk rate the model’s materiality to the function of the organization.
- A model development and implementation policy which incorporates the following considerations:a. Integration into new products b. Planning for model updates and changesc. Planning for additional uses of existing models
- A model validation policy which incorporates the following considerations:a. Evaluation of conceptual soundness, methodology, parameter estimation, expert and other qualitative datab. Assessment of data inputs and qualityc. Validation of model outcomesd. Assessment of ongoing monitoring metrics and performancee. Model risk scoring
- A documented model issue management and escalation process which will describe the issues, cataloguing of issues, issue remediation and action plans.
- Disaster and contingency planning policy for approved models describing the Plan B in case of model failure, corruption or cyberattack.
The payoff of so much documentation
Model risk is very real, and I’ll be honest — it requires a heavy lift in documentation. But by covering your bases, the documentation will provide a consistent set of standards, which articulate guiding principles that cover the model process and provide comprehensive guidance for practice and standards on an enterprise-wide level.