National Crime Victimization Survey Subnational Program - An Update
Michael Planty, Chief, Victimization Statistics, and Lynn Langton, Statistician, Bureau of Justice Statistics
The Bureau of Justice Statistics (BJS) has been using the National Crime Victimization Survey (NCVS) to produce national-level estimates of crime since the early 1970s. Local stakeholders, however, would find the survey data more useful if statistics could be produced at the local level as a means to reflect local crime conditions and as a tool to assess police and criminal justice services. Local social and economic conditions, often thought to be related to crime levels and types, may not reflect national condi- tions, suggesting that the national crime trend is of little relevance to local areas. Most local areas typically rely on data from official statistics generated from police activities, such as calls for service, recorded crimes, and arrests. However, various social and policy filters are associated with police records and, most importantly, a large portion of crime is not reported to the police.
Based on recommendations stemming from a 2008 National Academy of Sciences (NAS) review of the current victimization survey, BJS is engaged in a large-scale redesign of the NCVS to enhance its quality and value. Among NAS's recommendations was the suggestion that BJS enhance NCVS's utility by developing a subnational program for state and local victimization estimates. The challenge with any subnational program is to address issues with precision and representativeness. Subnational estimates often cannot be generated because the specific state, city, or other small area does not have sufficient sample, or any sample, to produce reliable estimates. In other cases, the sample sizes may be sufficient, but not allocated in the right areas to be completely representative of the particular population. For example, a victimization estimate based on a sample only taken from New York City is not likely to represent the entire state of New York. To address this need for subnational victimization estimates, BJS has taken a multipronged approach to developing a portfolio of both direct and indirect estimation procedures.
Direct estimations use current data or current data with increased sample cases ("sample boosts") to generate estimates directly from the NCVS responses. The benefit to direct estimation is that particular subnational areas are identifiable and rely on direct observation. Direct estimates are more easily replicable since they are derived from actual survey data. For areas with small or nonexistent sample sizes, direct estimates are problematic and provide unreliable estimates. Given the rare nature of crime, even relatively large and costly boosts to the sample may not produce measures with reliable precision. Another approach is to use indirect methods that involve modeling techniques to create estimates with ancillary information related to victimization at the local level. This approach uses information from direct sample cases when available, but primarily relies on secondary indicators to produce the local victimization rate.
BJS is examining a range of specific subnational estimate strategies, including: 1) direct estimates from existing data to create rolling averages across several years; 2) generic area estimates; 3) direct estimates with sample boost and reallocation; and 4) indirect, modeled estimates. The overall goal is to create a portfolio that produces reliable, valid, and timely estimates and affords the Bureau the flexibility to expand or contract efforts in a data-demanding, but fiscally uncertain environment.Direct Estimates from Current Data
BJS is assessing the feasibility of producing precise and reliable victimization estimates from the current NCVS sample in seven large states, 20 cities, and 20 metropolitan statistical areas (MSAs) where sample sizes should be sufficient to produce subnational estimates. BJS is assessing the estimates' precision by looking at the margin of error around various estimates of violent crime for one-year, three-year and five-year averages. However, even if the precision is satisfactory, it is likely that the current sample used for national estimation may not be representative of the local area. To assess sample representativeness, weighted NCVS population estimates for these areas are compared to known population estimates published by the American Community Survey (ACS). If necessary, subnational estimates will be reweighted and calibrated to the ACS population totals. The major limitation to this approach of using current sample is that many areas have few or no NCVS sample cases available.Generic Area Estimates
A generic area typology is created using available geographic, demographic, economic, or other identifiers to create "like places." As the term implies, these categories are not specific to any one state, city, or place; rather, these generic areas represent places that share the same general geographic characteristics. For example, a "city with a population of 250,000-500,000 in the northeast" represents places such as Buffalo, NY, and Pittsburgh, PA. The generic areas approach is limited by the need for sufficient sample sizes, the assumption of homogeneity shared within categories (are places really alike?), and the question of whether sample cases properly represent the targeted place. Like the use of current sample, this is an inexpensive option, requiring no change in the survey design, and it takes advantage of existing data. However, a drawback to this approach is that annual estimates can be prone to relatively lower statistical precision due to small sample sizes and possibly lower levels of external validity. In addition, estimates are not specific to a unique area, so specific places such as states, cities, or counties are not identifiable.Direct Estimates with Sample Boost and Reallocation
This approach involves increasing the current sample sizes in particular areas and reallocating the sample so that it is more suitable for subnational rather than national estimates. Beginning in July 2013, the Census Bureau initiated a trial boost of the NCVS sample in the 11 largest states: California, Texas, New York, Florida, Illinois, Pennsylvania, Ohio, Michigan, Georgia, North Carolina, and New Jersey. With the trial boost, BJS maintains the current NCVS national sample design, but has the ability to assess the costs and quality of the data produced in each of the states. This study will provide the necessary knowledge for understanding how sample boosts can be implemented in a responsible manner, taking into consideration factors such as interviewer workloads, response rates, and the precision of victimization estimates. Boosting the sample is an attractive option since it increases the precision and representativeness of the estimate through direct observation. However, this is the most expensive option for subnational estimation, as it involves increases in sample sizes.Model-Based or Indirect Estimates
Model-based or indirect techniques use statistical modeling to capture information from the current NCVS sample and leverage auxiliary information related to victimization to produce subnational estimates. This approach is used to develop estimates for areas without sufficient sample for direct estimates. The relationship between ancillary data on areas and NCVS estimates in the areas with data is used to predict victimization rates. A critical concern is identifying proper covariates that perform well in the modeling efforts. In addition, since these estimates are partially or completely based on modeling rather than direct observation, the value to local stakeholders is a concern. Currently BJS is developing initial model-based estimates for all 50 states and the District of Columbia for 15 measures of violent and property crime. Work is being conducted to assess the validity of these estimates compared to direct estimates, to explore the feasibility of expanding the model-based approach to produce city- and MSA-level estimates, and to develop and maintain a data dissemination strategy for releasing model-based subnational estimates.Fear, Safety, and Police Performance
In addition to producing victimization estimates for subnational areas, BJS is considering the development of additional questions for victims and nonvictims to produce a more comprehensive set of community-level crime indicators. These indicators can be organized into three groupings: 1) measures of nuisance crimes and disorder; 2) citizen's perceptions of fear and safety; and 3) citizens’ perceptions of police performance and legitimacy. These indicators are independent from police statistics and provide a perspective from the community.Conclusion
BJS's multipronged approach to generating subnational estimates of victimization and correlates of crime will increase the utility of NCVS data for stakeholders. A major goal of this work is to develop a more robust understanding of patterns and trends in criminal victimization. For example, in the short term, BJS could begin producing reports examining the relationship between NCVS rates of unreported crime and Uniform Crime Report crime rates in large cities; looking at rates of intimate partner violence and mandatory arrest policies; and examining the relationship between victimization and various community-level characteristics, such as changes in demographic composition, that could theoretically be related to variations in state, MSA and city victimization rates. BJS anticipates a large demand for these types of subnational estimates and is developing a strategic plan for how each of the different types of estimates will be disseminated. Timely, accessible, and transparent production of reports, tables, maps, figures, data files, and other products is critical to ensuring the utility of the estimates, and the success of the NCVS subnational program.