Predictive Analytics in Enforcement: Building User-Friendly Predictive Tools

*This is the sixth (and final) installment in our blog series: Predictive Analytics in Enforcement. See our previous posts: What is Predictive Analytics?, Searching for Regulatory Violations, Technical Challenges with Non-Random Investigative Data, Population Uncertainty and Lack of Information, and Why the IRS Does Random Audits.*

In previous posts, we discussed the increasing importance of predictive analytics in enforcement and as well as common technical challenges and possible solutions. In this post, we discuss how predictive analytics can be applied in civil enforcement.  

Which areas of enforcement can predictive analytics support?  

Predictive analytics can be used to create risk management, monitoring, and business intelligence systems that help agencies conduct all aspects of enforcement (including investigating, assessing penalties, correcting violations, and collecting fines) more effectively and efficiently. Agencies use predictive analytics to identify organizations that are most likely to violate regulations and/or most likely to require careful monitoring or additional assistance to correct violations or pay assessed fines. In this way, predictive analytics can help ensure that agencies' enforcement efforts are more effective at improving regulatory compliance. For instance, the Mine Safety and Health Administration could leverage predictive analytics to identify the compliance violators who are at high risk of fine payment delinquency and target their limited enforcement resource to increase the violators' timely payment, ultimately incentivizing all mines and mining operators to better comply with all the mine safety and health standards.  

What kinds of tools are needed to apply the results of predictive analytics to civil enforcement?   

In order to apply predictive analytics to enforcement activities, agencies must have business intelligence tools that can be implemented by staff in the field. To begin, the results of the predictive model should be converted into a scoring system or graphical template that can be easily used and interpreted by field staff. For example, an output of a predictive model might be that Firm A has a 12% chance of violating the law and Firm B has an 82% chance of violating the law. That score might then be converted into a color-coded system or other risk management framework—for example, a firm with high probability of violation (Firm B) might be receive a red score, while a firm with low probability of violation (Firm A) might receive a green. This type of tool could allowing investigators to easily assess which organizations are at high risk for violation. A well-built business intelligence tool should allow users to: 

• view various metrics of the predictive model (e.g. probabilities of violation, coefficients for specific predictors, etc.) on demand,  
• drill down to get more detailed information on these metrics, and  
• sort or filter the data as needed to better identify groups of organizations with similar characteristics. 

Finally, the predictive model should be set up in a platform that agency staff can easily use to update with new data, run analyses on new data, and maintain independently. This kind of business intelligence tool can be implemented in whichever platform is most cost effective to clients and easily accessible to the agency staff. This can range from a simple spreadsheet to a sophisticated IT system. Summit has helped our clients implement predictive analytics and build corresponding business intelligence tools. For one client, Summit created an interactive spreadsheet which generates and ranks risk scores to provide clients instant feedback on high risk violators. For another client, we implemented a suite of predictive models in a business intelligence platform that automatically turns the client's data into information usable by analysts. Such tools allow agency analysts to quickly identify the high-risk organizations while saving time and money, and ultimately enables agencies to manage their limited resources while still effectively achieving their mission. 

What should agencies keep in mind when applying predictive analytics to their enforcement efforts? 

Users of predictive models must always keep in mind that even the best model in the world will not be right all the time. Predictive models are intended to be probabilistic, not deterministic. The goal is not to identify criminals or firms with absolute certainty before they violate a law, a la Minority Report, but to estimate how likely an organization is to violate a law, based on the characteristics that the model controls for. Agencies can use predictive analytics to help them target their enforcement efforts, but predictive analytics should be just one tool available to guide agencies' enforcement activities. Predictive analytics should be used to supplement, not supplant, more traditional methods of enforcement targeting such as investigator judgement, experience, and on-the-ground knowledge.