Thursday, June 26, 2014

Model Validation, Control and Governance Workshop

Finance is a Risky Business

While no financial institution can afford to ignore the risks inherent in this industry, Model Risk is still an uncharted territory for many organizations.  Even the regulators, until recently, have not placed much emphasis on this area of risk management.  
On May 30, 2000, Office of the Comptroller of the Currency (OCC) had issued its first guidance, OCC 2000-16, dealing with model validation and risk management.  More than 10 years later in 2011, the Fed and OCC jointly issued updated and vastly expanded guidance on the topic.   Though the reasons for this unusual joint effort are not entirely known, it is clear that reliance by financial institutions on mark-to-model versus mark-to-market approach has become more norm than exception.  The new Fed-OCC guidance aims to disseminate best industry practices to a wider audience in hopes of making model validation activity more robust and standardized.  Regulators in other regions, Asia and APAC, are also increasing efforts to mitigate this growing exposure.  Yet many institutions remain woefully unprepared when it comes to model risk, regarding it as compliance more so than risk management issue.

What is Model Risk?

Ask 10 people what a model is and you are bound to get 7 replies, none agreeing with each other. Words like "model", "product" or "payoff" are often (mis)used interchangeably when discussing the pricing and risk of financial products.  However, in  the context of model validation , the difference is important.    A "product" is a legal agreement between a collection of (usually) two parties.  The terms of the
agreement are expressed in a document, the term sheet, which outlines rights and obligations of each party.  
The definition of such rights and obliagations is the "payoff" or "payout".  It is often expressed as a function or algorithm.  For example, the payoff of a vanilla call option is defined as (S - K) where S is the underlying price at contract exercise time and K is the strike price.

A model is typically a mathematical or statistical expression which aims to approximate reality.  It often "lives" as a software library, algorithm or a spreadsheet.  The more accurate the model, the more observed features are explained by it.  Yet no model, however elaborate, is completely accurate in all circumstances.  There is always a gap.

Unlike most living creatures, financial models do not necessarily get wiser with age.  Dynamics of financial markets, regulatory changes and variations in investors' preferences affect the validity and accuracy of all financial models.  Models always follow the markets.
Model Risk arises when there exists "the risk of significant difference between the mark-to-model value of a complex and/or illiquid instrument, and the price at which the same instrument is revealed to have traded in the market." (Riccardo Rebonato, Theory and Practice of Model Risk Management, 2003). This risk is directly proportional to the complexity of the financial instrument.  

Complexity refers to the relation between the present value of financial product and the price of its underlying.  The more opaque this relation, the more complex the product.  Complex products are usually bought for hedging purposes or to express a view on the market.  They are sold because they most often carry attractive margins for the seller and differentiate the seller from the competition.

Stages of Model Risk

There is a close link between product's complexity and its liquidity.  In general, new custom financial instruments designed for niche markets are traded over-the-counter with few market participants.  Implications are low liquidity and poor price visibility which raise the model risk.  

The "true" model is never known and a challenge to estimate.  Using a model which aims to closely approximate the "true" model is known as the Value approach.  The model in this case is attempting to predict "what should happen" rather than to reflect "what is happening".  This approach is often the sole option in a new market with little or no price visibility.  
The Price approach refers to a model that is trying to match the market data.  This is possible if sufficient market price data is available and is regularly updated.
Early, new markets lack the necessary price information forcing the participants to rely on the Value approach.  As the market matures and more price information becomes available, the Price approach can be used to reduce the model risk.  
In practice a combination of the two approaches is required - what is everyone else doing and what happens if they should change their minds.  You can have the most accurate watch in the world but if everyone's watch is 30 minutes fast, you are still going to be late for dinner.

Model Validation Approaches

Model validation is a multi-stage process involving cross-functional domain expertise.  The first stage, Validation, is to determine if the model is appropriate for its intended purpose.  Is the model a realistic and a reasonable descripton of reality?  This "design review" questions the validity of assumptions, the maths, model construction and outputs to ensure no gross omissions, naive approximations or inappropriate maths are involved.   How is the probability or correlation being estimated?  Is stochastic or local volatility the right approach?  Is the model amenable to the firm's risk management methods?  Is the model appropriate for the locality?  What worked in London does not always transfer well to Singapore.  The complexity of the model should also be reviewed with the more parsimonious but equal solution being preferred.
Verification, the second stage, focuses on testing the behaviour to determine model's robustness and performance.  How will it behave under extremes?  Stressing the model under different scenarios helps to define the operating limits and prevent future discrepancies.  Verifying the model's computational performance should not be overlooked here.  A 5% increase in the time to price an option can mean overnight reports not ready till the following afternoon.
Application, the 3rd stage, focuses on operational aspects.  How and how often will the model be calibrated?  What are the operational risks?  How should the model be deployed?  Is the correct infrastructure in place to use the model in production? Is required data readily available to use the model?

Finally, model validation should be an ongoing activity, done periodically to ensure what worked before is still performing to expectations.  Changes to the model need to be controlled via an appropriate sign-off and approval loops to prevent drift.  Models deployed across a large organization with multiple locations can and will stray from its original purpose and application.  Periodic reviews help to stem the drift, control  ageing, discover errors and maintain validity.   

Mitigating Model Risk

Model risk governance is unlikely to succceed if treated as an ad-hoc effort done on the fringes of the firm's risk management activities.  Chances of success increase when this process becomes integral and visible part of firm-wide risk management.  To ensure this,  model risk governance requires:
  • Established standards, processes and procedures
  • Updated model inventory
  • People, processes and technology aligned to business needs and initiatives
  • Ownership of models, processes and priorities
For a large organization with extensive inventory of models in use, there is often a trade-off between resource allocation and risk mitigation.  Scoring the model inventory helps to focus on the highest exposures while minimizing the resource burden.
One effective method of scoring the model inventory is to assign Risk Priority Number (RPN) for each model in use.  Failure Mode and Analysis (FMEA) method is a systematic way to calcuate the RPN scores.  The RPN is a product of 3 factors, each evaluated on a scale of 1 - 10: 
          What is the impact of a given failure?  What is the exposure?
          How likely is this failure to happen?
          How likely is the failure to be detected?

RPN = Severity x Occurrence x Detection

This can be extened to include other factors depending on organization's needs.  Models with higher RPN numbers pose greater risks and should receive first priority.  
FMEA exercise is usually done under the guidance of risk management team with participants from front office, quant, technology, operations and back office.


No comments:

Post a Comment