An evaluation roadmap is a graphical summary outlining how you plan to evaluate your TJC initiative. Each step of the roadmap must be completed before the next step can occur. Here, the old adage applies: “If you don't know where you're going, you'll probably end up somewhere else.”

A theory of change model, often referred to as a logic model, is one of the most common tools used to map out your specific community's approach.
It will not only document your theory of change for the TJC initiative but articulate to your TJC partners the rationale for actions or strategies. It guides your self-evaluation by identifying how resources and activities or programs are linked to desired outcomes.

Your theory of change model will describe your TJC strategies and activities, and also help identify the data that should be collected by stating what measures will be used for its short- and long-term outcomes. A reentry council or criminal justice policy council, as well as each partnering agency, will typically develop its own theory of change model.

The flow of the theory of change model begins with identifying the target population and ends with long-term outcomes. We've listed the headings most often used for each column.

Figure 1. Theory of Change Column Headings

A Theory of Change Model:

• Offers a written rationale or a theory for selecting various strategies, programs, or intervention activities to implement the TJC initiative.

• Identifies the objective or change you hope will occur by implementing those various strategies, programs, or interventions to the population targeted.

• States the strategies, programs, and activities to be implemented (Activities in figure 2).

• Identifies the immediate, short-term outcome of the strategies, programs, or intervention activities (Outputs in figure 2).

• Identifies the intermediate outcomes of the strategies, programs, or intervention activities (Outcomes in figure 2).

• Identifies the final outcomes (long-term outcomes) of the strategies, programs, or intervention activities (Impact in figure 2).

Figure 2. Theory of Change Model

(click for larger version)

Developing a theory of change model is a group activity; it requires input from multiple stakeholders central to your community's TJC initiative. Crafting a theory of change model begins by convening key stakeholders from the community and the jail to discuss and define the initiative's goals and objectives, and the interventions or activities your community views as essential to achieving these goals and objectives. As discussed in module 4, a data-driven understanding of local reentry issues should inform many of these early discussions.

A theory of change model, however, is only a first step. You will still need to define your key outcome measures, determine what data will be needed to measure these outcomes, decide how to collect and analyze the data, then do so, and report the results. See module 4 for more information on data collection and analysis.

Below, we have provided several examples of theory of change models, focusing on transitioning people back to the community, provided by the Criminal Justice Research and Evaluation Center at John Jay College.

Figure 3. Collaborative Initiative Program Plan
Criminal Justice Research and Evaluation Center, John Jay College.

(click for larger version)

Figure 4. Reentry Committee Community Capacity Building—Process Logic Model
Criminal Justice Research and Evaluation Center, John Jay College

(click for larger version)

Figure 5. Mother and Child, Jail and Community Connections
Criminal Justice Research and Evaluation Center, John Jay College

(click for larger version)

Performance Measures

The development of TJC performance measures is the next step after developing your theory of change model. Performance measurement refers to the “regular and systematic collection of quantitative information that will empirically demonstrate results (outcomes) of activities (e.g., modified policies, practices, new program activities). Performance measurement connects indicators (i.e., quantitative measures) with a specific agency or jurisdictional objectives (i.e., expected outcomes).” ¹

Performance measures determine the type of data you must collect to measure your short, intermediate, and long-term outcomes and are thus directly connected to your theory of change model. Reports of these data and outcomes should be readily available to assist decision makers in resource allocation and implementation strategy.

David Osborne and Ted Gaeble, in Reinventing Government: How the Entrepreneurial Spirit Is Transforming the Public Sector ² identify the power of performance measures:

• What gets measured gets done.

• If you don't measure results, you cannot tell success from failure.

• If you can't see success, you can't reward it.

• If you can't reward success, you're probably rewarding failure.

• If you can't see success, you can't learn from it.

• If you can demonstrate results, you can win public support.

Innovating agencies use performance measures to improve performance but they also recognize that for desired behaviors to replace old habits or behaviors, they must be reinforced. Below are several examples of actions or rewards to reinforce desirable organizational behaviors or actions:

• Paying for performance: Use performance measures as a basis to determine and reward effectiveness.
  
  The New York City Department of Corrections, for example, employs performance-based contracts with its transitional service providers. Service providers bill the NYC DOC for individual clients, and the billing structure relies on a pay scale that ties increased fee amounts with continued client involvement. Performance-based contracts provide strong incentives for service providers to be aggressive in their efforts to maintain client engagement.

• Managing for Performance: Use performance measures to improve outcomes through the following steps: identify problems, analyze them, locate the root cause, and develop and implement the solution.

• Budgeting for Results: Use performance measures and information to allocate resources. As table 1 indicates, results-based budgeting is outcome driven, promoting ongoing evaluation and interagency collaboration.

Table 1. Traditional Budgeting vs. Budgeting for Results ³

Begin by developing TJC performance measures for each of your theory of change model's short-term, intermediate, and long-term outcomes. This needs to be done before you develop your data collection plan to make sure your strategy for collecting data supports each performance measure used in your self-evaluation. There must be a clear and compelling link between your initiative's objectives, outcomes, performance measures, and indicators, and the data you plan to regularly collect and analyze.

Many people believe that rearrest, reconviction, and reincarceration rates are the only ways to measure successful transition strategies. Although long-term public safety is paramount, there are other important process and system outcomes to measure to enhance public safety and efficiency.

The TJC initiative has developed a menu of performance measures that reflect and support the initiative's broad goals to increase public safety, improve reintegration outcomes, and effect systems change. To help you with this process, we have identified the following system-level, public safety, and reintegration TJC initiative outcomes and performance indicators:

• System-Level Outcomes

  • Screening, assessment, referrals, engagement, service utilization, and completion

• Public Safety Indicators and Outcomes

  • Reduced reoffending, reduced jail stays, and increased time between jail stays

• Reintegration Indicators and Outcomes

  • Reduced substance use, reduced homelessness and increased housing stability, increased employment and employment stability, and improved physical or mental health

The menu of TJC performance measures located at the end of this module offers several indicators (actual measures) for each above measure. The process measures, although difficult to track, should be a priority for each jurisdiction as they will allow progress to be monitored on an ongoing basis. Depending on the agreed upon definitions (e.g., indicators selected, specified time frame for measurement), the public safety and reintegration measures can take a long time to demonstrate progress and success. The process measures represent intermediate outcomes that should be monitored closely, keeping in mind that if the associated activities are targeted and implemented correctly, they should positively affect reintegration and public safety outcomes.

¹ Rossman, Shelli B., and Laura Winterfield. 2009. Coaching Packet: Measuring the Impact of Reentry Efforts. Silver Spring, MD: The Center for Effective Public Policy, p. 8.

² Osborne, David, and Ted Gaeble. 1992. Reinventing Government: How the Entrepreneurial Spirit Is Transforming the Public Sector. Reading, MA: Addison Wesley.

³ Office of the Governor. 2005. The FY 2005-06 Executive Budget . Columbia, SC:
Author.

System-Level Performance Measures

This menu of system-level performance measures helps you identify those that are most important to your initiative. You may want to collect and review these data monthly to support internal monitoring but report on them quarterly to the broader stakeholder group to track outputs and short-term and intermediate outcomes.

System-Level Performance Measures

Public Safety Performance Measures

Reintegration Performance Measures

Data Collection

While the TJC initiative does not require a fully developed or collaborative management information system (MIS) in place for self-evaluation or inter-agency information-sharing, your agency should have the basic protocols and capacity to collect and access information relevant to operations and outcomes. An MIS is needed to measure performance and program success along desired outcomes through an external evaluation and a self-evaluation. This system will allow for each community to determine the who, what, where, and when of targeted service delivery. Refer back to Module 4: Data-Driven Understanding of Local Reentryfor Step 2: Data Collection.

Resources

1. Burke, Peggy. B. 2008. TPC Reentry Handbook: Implementing the NIC Transition from Prison to the Community Model. Chapter 6, “TPC Performance Measurement Framework."

2. Kent County, MI Sheriff’s Office. TJC mission-driven system-wide performance measures and outcomes.

3. Miami-Dade County, FL. 2008. Breaking the Cycle: Rehabilitation and Job Training in County Jails. Miami-Dade County Blue Ribbon Advisory Committee. A comprehensive report recommending initiatives to facilitate the reentry of inmates into the community and reduce recidivism.

4. Demographic characteristics the Montgomery County, Maryland, Correctional Facility uses to understand more fully the needs of its population.

5. University of Wisconsin Cooperative Extension. 2002. Enhancing Program Performance with Logic Models.

6. Urban Institute. TJC Performance Management Worksheet: Core. A detailed chart of TJC baseline measures of jail inmate population characteristics in Excel format and a memo providing guidance to assemble the initial TJC performance indicators.

7. Urban Institute. 2009. TJC Initiative Theory of Change Model. Brief explaining Theory of Change models.

8. W.K. Kellogg Foundation. Logic Model Development Guide.