What Are the Challenges of Using My Administrative Data for Analysis and Evaluation?

In the previous post, we discussed the many types of analyses that can employ administrative data. Once your organization decides to use its administrative data for analysis and evaluation, you might wonder about any potential obstacles for using the data for analysis. Despite the many advantages to using administrative data, there are a few specific ways in which using the data is more challenging than using primary data.

Data Quality

Administrative data can have serious data quality issues, including: invalid values, incorrect formatting, internal inconsistencies, outlier values, and missing information. These kinds of data quality issues must be resolved to ensure the accuracy and reliability of the analysis. To resolve this challenge, you should perform extensive data quality review and cleaning of the data before analysis.

Disparate Sources

Administrative data is often distributed across multiple tables or databases. All the information needed to conduct an analysis or evaluation may not be in one, single-format dataset. To resolve this challenge, your organization should integrate the separate administrative data sources.

External Data

Administrative data are usually single-purpose data. The data is collected primarily for case management and generally without input from researchers. For this reason, your administrative data may not include all the information needed to conduct a comprehensive analysis of the program. To resolve this challenge, you should consider incorporating external data sources into the analysis.

Analytic Methods

Administrative data are generally observational rather than generated from an experimental design. As such, the program participants are not representative of the entire, target population. In observational data, participants are not randomly assigned to the program. They sometimes self-select into the program or are selected by the program for specific reasons. An analysis using this data may produce biased estimates of program impacts or other effects meant to be attributable to the entire population. To resolve this challenge, your organization should employ statistical methods to produce unbiased analysis results using observational data.

As outlined above, there are several specific challenges to using administrative data for analysis. With preparation and forethought, these challenges can be resolved, allowing your organization to reap the full advantages of using its data for analysis and evaluation.

Be sure to check out our panel, Data Integration for Program Effectiveness, at this year’s Association for Public Data User’s (APDU) conference.