May 6, 2016 •Anthony Curcio
In my last post, I reviewed the Model Risk Management guidance (AB 2013-07) set out by the Federal Housing Finance Agency (FHFA). This week, let’s consider examples of good and bad model governance in action.
In previous posts, we learned about three aspects of model risk management:
- The importance of robust model development, implementation, and use ("Why Is Model Governance Important in Federal Government Right Now?")
- Sound model-validation practices ("New Supervisory Guidance for Model Governance")
- A solid governance framework ("What Does the Guidance on Model Governance Say? A Closer Look at the New Supervisory Guidance from OCC/FRB" and "Model Governance Implications of FHFA Risk Management Bulletin AB 2013-07)
How does this translate to the Federal space? The two examples below illustrate the importance of model risk management.
Example 1: Poor Model Governance
Ken works for a big Federal agency in downtown Washington, DC. He is a talented, highly-trained quantitative analyst and the sole owner and operator of a model in his program office. Ken is the only person in the office who really understands how the model works, how to operate it, and the sensitivity of its results. His coworkers think of the model as a black box and do not understand how it works. Furthermore, if a coding fix were needed, Ken could perform this without approval and without a changelog. Unfortunately, Ken has not created comprehensive model documentation or user training, and what he does have is spotty and outdated. He is also the only one who has the knowledge and insight to interact with OMB and the auditors.
Now consider: What if Ken quits? If he found a coding error, would he be tempted to quietly fix it and not tell anyone? How do we know if Ken’s methods reflect best practices?
Solution: The organization should implement robust documentation and change management procedures, as outlined by the guidance.
Read on for a better example.
Example 2: STrong Model Governance
Alice is a model owner in a program that has appointed a Model Risk Management Board (MRMB). This board reviews any changes to the model code, model documentation, and model functionality annually. The MRMB is engaged, conversant in the operation of the model, and encouraged to challenge the model owner. The roles and responsibilities of all parities (model users, owners, and oversight committee) have been established and understood.
Alice and the MRMB are all equipped with the knowledge to discuss the model with OMB and the auditors. As a result, Alice's program office has a well-defined model governance framework that helps reduce model risk.
What Kind of Model does your program have?
Does your model feel like a black box that is only understood by one or two people? Is your model user overseen by an informed committee/board? Can a third party read your documentation and readily understand the model and its sensitivities? If your answers worry you, better model risk management may help.
If you would like to learn more about this topic, check out Summit's past posts about Federal credit or our other entries on model risk management.
Join Our Free Seminar on Model Risk Management
Don’t forget about Summit's upcoming (and free) seminar devoted to this topic on Tuesday, May 24, 2016. Click the button below to request an invitation.
