Why Do This Step?

The successful implementation and sustainability of dosage probation (and any program, intervention, or practice, for that matter) rely on data-driven decision making. By continuously collecting and analyzing relevant and meaningful data, those impacted can identify which desired outcomes are being achieved and which are not. When desired outcomes are not achieved, data can guide policy, practice, and resource improvements to achieve success.

To make data-driven decisions, authorities must have the necessary infrastructure (i.e., an information system, policies, and procedures) to collect, store, process, report, and use that data effectively. These activities are known as performance measurement and management.

Once your jurisdiction’s dosage probation logic model is developed, your probation agency can begin creating a performance measurement and management plan for dosage probation. A plan will help ensure your agency is well-positioned to collect data, evaluate and report on or share information about dosage probation’s outcomes and impacts, and make any necessary policy, practice, or resource adjustments after implementation.

This section guides the people responsible for the performance measurement and management of dosage probation in your agency or jurisdiction (e.g., logic model subcommittee, probation leadership, support staff, data analysts at the county or state level, or others) to create an infrastructure that supports data-driven decision making. It is recommended that those overseeing and assisting with these efforts review this section before beginning their activities.

Establish Your Performance Measurement and Management Team

The first step in mobilizing your probation agency’s performance measurement and management efforts is determining who will oversee and complete these activities for dosage probation.

If your agency established a logic model data subcommittee during the second (preparation) phase of implementation, it is recommended that the subcommittee review this section and add relevant activities or tasks not yet included in its action plan. Depending on the issues or challenges being addressed, the subcommittee may also need to consider who else to involve in its efforts. For example, agencies that want to introduce or modify an electronic information system may involve executive leadership, county or state administrators, IT specialists, or the system vendor to acquire the necessary approval, funding, and technical support.

Alternatively, you will need to decide who will be on your dosage probation performance measurement and management team. Your team may be led by one, a few, or a variety of people, depending on your agency’s operational structure and available resources. They may include executive leaders, mid-level managers or supervisors, specialized employees who handle probation data or its data sources (e.g., probation support staff, county or state-level data analysts, IT specialists), and even outside experts who can help with your agency’s data collection and evaluation needs (e.g., technical assistance providers, university researchers, electronic information system vendors). You can always add people or adjust team membership as needed.

Your team may find it useful to develop a dosage probation performance measurement and management work plan based on the guidance provided in this section and additional needs your agency may have (e.g., acquiring approval and funding to hire outside assistance). Your team will want to agree on the work plan and their roles and responsibilities to meet your agency’s needs and expectations.

Review Your Jurisdiction’s Dosage Probation Readiness Assessment Rating Form 

Your performance measurement and management team will want to begin by reviewing the Dosage Probation Readiness Assessment: Jurisdiction Rating Form completed by probation leadership during the readiness assessment. The completed responses regarding data management will give your team a foundational understanding of the strengths, opportunities for improvement, or significant challenges to be addressed through the implementation process.

Determine How You Will Track Dosage

Tracking dosage and collecting that data is critical to implementing dosage probation, evaluating its results, and making policy, practice, and resource adjustments for long-term success. Experience with the dosage probation pilot sites suggests the person(s) responsible for performance measurement and management will be tasked with deciding and implementing your agency’s method for recording dosage information. However, you will want to identify those with decision-making authority and final input to ensure they are involved in the decision-making process.

Supervision staff must record accumulated dosage hours (and possibly other relevant information, such as the type of intervention used and criminogenic needs addressed) for every person on their caseload. Recording enables staff to track and motivate people’s progress toward their dosage target of 100, 200, or 300 hours. It also indicates when early release is earned for those eligible through dosage probation (and possibly for those using an alternative path to early discharge) and when to begin discharge planning.

Dosage information should be documented consistently and stored for easy access to measure your jurisdiction’s dosage probation logic model outcomes and impacts and answer other critical data-related questions (e.g., the number of people who earned early discharge by achieving the 100-, 200-, and 300-hour dosage targets or who remained conviction-free following supervision after completing 100, 200, or 300 hours). The following activity, "Develop a Performance Measurement Plan," can help determine what dosage information to document and where.

Agencies can use either electronic or manual methods to track dosage information, depending on resources. The most efficient way to record dosage information and track performance measures is by using an electronic information system, which is typically integrated into a case management system. Although storing and processing data electronically for dosage probation is not mandatory, having an electronic method can provide the most efficient and accurate results.

Questions to Consider for Documenting Dosage

Consider the following questions to help your probation agency determine how to record dosage information for all people on probation. You may need to address other issues specific to your probation agency or jurisdiction.

• What dosage information must be documented to track those hours efficiently and measure your jurisdiction’s dosage probation logic model outcomes and impacts, for example, intervention date, criminogenic need(s) addressed, intervention type (e.g., cognitive worksheet, structured skill building, community-based cognitive programming, in-house cognitive programming), dosage target, dosage minute or hour increments earned, etc.?
• If your agency uses an electronic information system, does it have mechanisms for recording and generating reports on dosage data, or must it be adapted? If adaptation is required, how feasible is it to make the changes based on the resources needed (e.g., personnel, approval, funding, time for modification and beta testing, end-user training, etc.)? If an outside vendor manages the system, do they allow changes to be made? Will the vendor need to make the changes, or can internal personnel, such as an IT specialist, make them?
• If your agency is interested in implementing an electronic information system, how feasible is it to execute one based on the resources needed (e.g., personnel, hardware/software, approval, funding, contracts, implementation and beta testing time, end-user training, etc.)? Does the system include mechanisms for recording and generating reports on dosage data, or will you need to work with the vendor to incorporate those?
• If your only resource is to track dosage manually, even for a relatively short period, how will staff document this information consistently? Where will the information be stored for easy access? How will you ensure the data can be analyzed?

Develop a Performance Measurement Plan

A performance measurement plan is a blueprint that outlines a systemic approach for collecting, storing, and analyzing or processing quantitative data—in this case, key success indicators for dosage probation. A plan is crucial for ensuring accuracy, consistency, and transparency in data measurement and reporting over time, which can ultimately help foster confidence in dosage probation among stakeholders.

Those responsible for developing a plan can begin by familiarizing themselves with the Dosage Probation Performance Measurement Plan Template (.xls). The template contains three spreadsheets that include the elements of a performance measurement plan for your jurisdiction’s dosage probation outcomes, impacts, and other metrics of interest. You may modify the spreadsheet to meet your needs. For example, you may add columns to indicate whether the metric is feasible to measure, or if you need to reduce the number of performance measures due to resource constraints, you may indicate suggested priority levels.

To get your plan off to a good start, it can be helpful to get a clear understanding of the metrics your agency can and cannot measure based on the data that is accessible or can otherwise be reasonably collected (e.g., through methods like surveys). The Dosage Probation Policy Team and Dosage Probation Workgroup outlined the outcomes and impacts in the logic model but may not have explicitly addressed their measurability. As a result, you may come across some performance measures for which data cannot be collected, or you may need to redefine/reword some of them to make them measurable.

Start by filling in columns A to C and E. In columns I and J, note any concerns, questions, or action steps to gather additional information, clarification, or guidance. If other columns can help you understand which outcomes, impacts, and other metrics (if any have already been identified) are practical to measure, go ahead and fill those out as well. You may or may not need to reach out to others for assistance.

Developing a performance measurement plan requires a collaborative and iterative approach that closely coordinates with various stakeholders. You will need to meet with the policy team, workgroup, and others responsible for managing probation data or its sources. This could include, for example, local or state probation leadership, support staff, data analysts, and/or IT specialists.

Gather Input and Complete Your Plan

After determining which metrics can be realistically measured, meeting with the Dosage Probation Policy Team and Dosage Probation Workgroup for further insights, clarification, and guidance is essential to finalizing your jurisdiction’s dosage probation performance measures.

Those meeting with the policy team and workgroup may prepare by reviewing the Dosage Probation Performance Measures Presentation Template (.ppt). The presentation proposes a format for gathering input on the measurability of outcomes, impacts, and additional metrics of interest. The guidance in the template will help you create two presentations: one for the Dosage Probation Policy Team and one for the Dosage Probation Workgroup, depending on the input needed from each. You must customize the slides with [bracketed] information and may make further adjustments to meet your needs.

Consider preparing the presentations before contacting the chairpersons to schedule meetings with the policy team and workgroup. This information will help you and the chairpersons agree on the information to be shared, flow, and presentation timing. For example, the policy team may be most interested in the impacts and may or may not also provide input on some or all outcomes.

Once you have gathered input, proceed with filling out the Dosage Probation Performance Measurement Plan Template to the best of your ability. You may have to schedule another meeting with the policy team or workgroup or work with other individuals, such as local or state probation leadership, support staff, data analysts, and/or IT specialists, to finalize the performance measures and ensure that the necessary infrastructure is established for collecting, storing, and processing the data.

Collect Baseline Data and Set Benchmarks

After finalizing the performance measures, gathering baseline data for each is essential. Baseline data provides a snapshot of current performance status, allowing you to establish a quantitative benchmark or performance goal (e.g., an actual number or percentage) for each measure. Ultimately, progress can be objectively measured and tracked by comparing the data collected before and after implementation.

Use the baseline data to set benchmarks for each performance measure. Benchmarks typically represent the ideal or exemplary level of achievement that your agency and jurisdiction will strive to reach through the implementation of dosage probation. Be sure to seek feedback, guidance, or approval from the policy team and workgroup as necessary. You can also research national or state benchmark standards or trends to help guide decisions.

After the benchmarks have been agreed upon, update your performance measurement plan and your jurisdiction’s dosage probation logic model accordingly. When updating the logic model, you may, for example, need to eliminate or rework specific outcomes or impacts and relevant footnotes to match your performance measurement plan.

Share the updated version of the logic model with the policy team, workgroup, and other interested parties, such as probation leadership at the state level (if applicable). It is important to remain flexible and open to adjusting benchmarks based on changing circumstances, feedback, and new insights that may arise after implementation.

Develop a Performance Management Plan

A performance management plan describes the policies or processes for ensuring data quality, reporting on performance measures, and using performance measurement data to guide relevant policy, practice, and resource decisions to enhance success—in this case, to ensure the effectiveness of dosage probation.

Those responsible for developing a performance management plan may begin by reviewing the Dosage Probation Performance Management Checklist (.pdf). The checklist provides brief descriptions of the components of a performance management plan and essential considerations for creating one. Use the checklist to develop a written performance management plan that meets your agency’s and jurisdiction's needs. You may need to join additional meetings with the policy team, workgroup, or others to gather input and feedback and develop a plan that works for everyone.

Get Ready for Next Steps

Once dosage probation launches in your jurisdiction, start following your performance measurement and management plans. After implementation, regularly review these plans to ensure they align with the evolving needs and goals of dosage probation and stakeholders.

It is also essential to foster a culture of accountability between the policy team, workgroup, and others overseeing the effectiveness of dosage probation. Encourage open communication channels to address accountability issues effectively and create a safe environment where stakeholders feel comfortable discussing challenges and proposing solutions.