Practical Examples of Model Risk Management

Posted by Anthony Curcio on 5/6/16 9:30 AM

Find me on:

Read more about me: Biography

--------------------------------------------------------

Anthony_Curcio_Feb_2016.jpgIn my last post, I reviewed the Model Risk Management guidance (AB 2013-07) set out by the Federal Housing Finance Agency (FHFA). This week, let’s consider examples of good and bad model governance in action. 

In previous posts, we learned about three aspects of model risk management:

How does this translate to the Federal space? The two examples below illustrate the importance of model risk management.

Example 1: Poor Model Governance

Ken works for a big Federal agency in downtown Washington, DC. He is a talented, highly-trained quantitative analyst and the sole owner and operator of a model in his program office. Ken is the only person in the office who really understands how the model works, how to operate it, and the sensitivity of its results. His coworkers think of the model as a black box and do not understand how it works. Furthermore, if a coding fix were needed, Ken could perform this without approval and without a changelog. Unfortunately, Ken has not created comprehensive model documentation or user training, and what he does have is spotty and outdated. He is also the only one who has the knowledge and insight to interact with OMB and the auditors.

Now consider: What if Ken quits? If he found a coding error, would he be tempted to quietly fix it and not tell anyone? How do we know if Ken’s methods reflect best practices?

Solution: The organization should implement robust documentation and change management procedures, as outlined by the guidance.

Read on for a better example.

Example 2: STrong Model GovernanceModel_Governance_Blog_Images_1.jpg

Alice is a model owner in a program that has appointed a Model Risk Management Board (MRMB). This board reviews any changes to the model code, model documentation, and model functionality annually. The MRMB is engaged, conversant in the operation of the model, and encouraged to challenge the model owner. The roles and responsibilities of all parities (model users, owners, and oversight committee) have been established and understood.

Alice and the MRMB are all equipped with the knowledge to discuss the model with OMB and the auditors. As a result, Alice's program office has a well-defined model governance framework that helps reduce model risk.

What Kind of Model does your program have?

Does your model feel like a black box that is only understood by one or two people? Is your model user overseen by an informed committee/board? Can a third party read your documentation and readily understand the model and its sensitivities? If your answers worry you, better model risk management may help.

If you would like to learn more about this topic, check out Summit's past posts about Federal credit or our other entries on model risk management.

Join Our Free Seminar on Model Risk Management

Don’t forget about Summit's upcoming (and free) seminar devoted to this topic on Tuesday, May 24, 2016. Click the button below to request an invitation.

Request Your Invitation: Model Risk Management Seminar 


 

Topics: federal credit, model risk management

About the Summit Blog

Complexity simplified.

Summit is a specialized analytics advisory firm that guides clients as they decode their most complex analytical challenges. Our blog highlights the strategies and techniques we use, as well as relevant topics in current events.

Subscribe to Email Updates

Recent Posts

Posts by Topic