Skip to main content
U.S. flag

An official website of the United States government

6b: Developing a System wide Scorecard

Introduction

Historically, criminal justice agencies and their allied partners have developed independent methods to describe and measure their performance. Police agencies report on crime trends, arrests made, and the elapsed time between calls to dispatch and the arrival of patrol cars on the scene of a crime, for instance; courts report on case processing, fines imposed and collected, and cases settled by plea, bench, and jury trial; probation agencies report on numbers of individuals supervised, assessments conducted, and cases closed by successful termination. Rarely if ever do justice systems report on their progress in achieving their harm reduction goals and objectives. Examples of system wide harm reduction goals and objectives include (but are not limited to)

  • reduced justice system costs as a result of a combination of activities that reduce the demand for jail beds and correctional staff, and the time associated with judicial processing. These activities may include conducting pretrial screening and diversion at police substations, establishing alternative responses to the acutely mentally ill, and addressing probation violators administratively rather than through the court system; and
  • increases in the success rate of offenders, as a result of: improving adherence to the risk principle at the arrest, pretrial, plea, sentencing, and supervision decision points; “matching” offenders to appropriate services (e.g., by prosecutors and defenders in plea negotiations, by judges at sentencing, jailers operating risk reduction programs, probation officers making referrals to community treatment programs); and employing professional skills to positively influence defendant and offender behavior.

Purpose

The purpose of developing a system wide scorecard is to “measure what matters.” While the measurement of “activities” (e.g., pre-plea assessments of defendants, quality assurance to determine whether risk tools are completed properly) and “outputs” (e.g., percent of professionals trained in the use of a new tool or methodology, percent of sentence conditions informed by risk/needs assessments) in the system logic model is important, these are means to an end, not the end themselves. Articulating the ends we seek to achieve—and measuring those—focuses attention on the work that is critical to achieving a jurisdiction’s vision of the justice system. It also equips leaders with statements of intent they can use to clearly communicate with community members and other stakeholders about the purposes and goals of the justice system.

Participants

This document was developed to assist EBDM policy teams in identifying the harm reduction goals they seek to achieve through their policy change work.  All policy team members should be involved to some extent in the development of your harm reduction goals and scorecard.

Instructions

  1. Working as a team, identify the evidence-based decision making changes that are under consideration.[1]  Using the logic model template, identify the “impacts” you want to achieve through these policy change initiatives. These impacts are your jurisdiction’s harm reduction goals.[2]
  2. List the goals on a flip chart. As a team, determine whether you have consensus around the importance of each goal. If not, work to achieve consensus.
  3. Examine the examples of scorecards contained in this kit. As a team, agree to adopt a design for your scorecard, either by selecting one of the templates provided or by creating your own. Include your “identity” on your scorecard. [3]
  4. Next, discuss and agree with your team how you will measure the system’s performance in regard to each of these harm reduction goals. These discussions may be lengthy and may require expert consultation from those within your agencies and system—particularly your research, planning, and information technology staff—and perhaps outside expertise.[4]
  5. Once the methods to collect and assess performance on your harm reduction goals are determined, be sure to collect baseline data.[5] Baseline data indicates your “starting place,” or basis of comparison.
  6. Finally, discuss how and when the scorecard data will be collected and used. Be clear and specific about this; there is no sense in establishing goals that will not be measured or in collecting data that will not be analyzed and examined for its implications. Perhaps the policy team will task specific individuals with collecting and analyzing performance measurement data and reporting this information back to the policy team on a quarterly basis. Results may be included in agencies’ annual reports or in periodic press briefings. Most importantly, if reported results are less than expected, it is critical that the policy team reexamine the conditions, assumptions, resources, activities, outcomes, and outputs related to the implemented policy and practice changes to determine why the expected results have not occurred, and that the team make appropriate modifications so that results do, in fact, improve over time.

Tips

  • Don’t attempt to develop a lengthy list of scorecard items. Agreeing on two, three or four significant goals that everyone is in full agreement with is superior to a laundry list of less significant accomplishments, or goals that do not have full support of the full team. In addition, as a part of your communications strategy, you won’t want the scorecard to be too lengthy, or to lack support of the full team.
  • Be clear regarding your definitions for key words. For example, “recidivism” is often defined in multiple ways. Refer to the starter kit on Measuring Your Performance for a list of definitions that you might choose to draw from, or at least use as a starting place for the development of your own definitions. Whether you use the provided definitions, or definitions of your own making does not matter; what matters is that you are clear on what you mean by these terms, and that your team is in agreement on these definitions.
  • Follow the SMART principle when developing goals for your scorecard:
  1. Be Specific
  2. Make them Measurable (i.e., quantifiable)
  3. Be Action-oriented
  4. Be Realistic
  5. Articulate a Time in which the change will occur
  • When you’ve completed your list of harm reduction goals/scorecard items, it should elicit a reaction of satisfaction. Ask your team, “Would you feel proud to have been a part of the achievement of these goals?” When everyone responds in the affirmative, chances are you’ve succeeded in the development of your scorecard. 

Example:
Eau Claire County, Wisconsin, System Scorecard

ebdm score card

 

Example:
Charlottesville-Albemarle County, Virginia, System Scorecard

ebdm scorecard nc

Example:
Mesa County, Colorado, Systemwide Scorecard

ebdm scorecard az

Additional Resources/Readings

NIC. (2010). Achieving, measuring, and maintaining harm reduction and advancing community wellness. A Framework for Evidence-Based Decision Making in Local Criminal Justice Systems (pp. 22).
Retrieved from https://cepp.com/wp-content/uploads/2018/10/A-Framework-for-Evidence-Based-Decision-Making-in-State-and-Local-Criminal-Justice-Systems.pdf

Minnesota Department of Administration. (2002.) Minnesota milestones: Measures that matter. Retrieved from https://mn.gov/admin/demography/data-by-topic/minnesota-milestones/

 

[1] For more information, see: 3e: Prioritizing Your Team’s Targets for Change.

[2] See 5a: Building Logic Models and 6a: Measuring Your Performance.

[3] For more on developing an identity, see 7a: Developing a Communications Strategy; Building Stakeholder and Community Engagement.

[4] See 6a: Measuring Your Performance.

[5] See 3d: Gathering Baseline Data.