Creating a Data Tracking System

Colleges are being asked to demonstrate student success while dealing with budget reductions and significant increases in enrollment. The need to do more with less is confounded by the institutions’ efforts to provide supports and services to prepare low-income and non-traditional students for a career that will lead them to economic stability. As colleges look for ways to improve practices while providing a holistic approach to preparing our future workforce, they frequently are asked to provide evidence of the effectiveness of their education and training services. Coupled with an increasing number of state legislatures moving towards performance-based budgeting—which applies a formula to student outcomes (instead of enrollment) in order to allocate funding — institutions need to be able to document student progress towards meeting outcomes. To do so, colleges must develop measures and indicators for monitoring student progress towards the attainment of a degree or occupational credential.

The design and implementation of a data collection process can be cumbersome for colleges, but data collection is necessary to understand the impact the strategy has on the students and the institution. In addition, data can be the tool used to communicate to external stakeholders, partners and funders on why they should invest additional resources in the institution.

Data collection should be thought of as a two-tiered process:

• Participant-level data are necessary to establish a baseline for students and track their progress towards goals.
• Program-level data is collected to identify specific interventions and monitor and evaluate institutional and community impacts to identify challenges and improve practices.

Establishing appropriate outcomes for individuals and for colleges

The approach used to implement the core strategies of the network vary by institution and thus, the collection and analysis of data may vary by institution. Participant-level and program-level outcomes are important in designing interventions and determining the practices that work best at the college. The Working Families Success Network has identified a list of outcomes that are important to assessing the impact of the WFS strategy. These are a broad set of outcomes focused on the results a student achieves as they connect to the various pillars of the strategy. The outcomes established by AECF capture results on the individuals who connect to WFS services and data on implementation of the program. The list below shows the outcomes AECF uses to monitor the WFS strategy at community colleges.

Common Outcomes for students receiving Working Families Success services:

• Completion of (or persistence towards) a degree or occupational credential
• Increased Income
• Debt reduction

Common Outcomes for colleges using the Working Families Success strategy:

• Increased retention rate for students receiving WFS services
• Increased completion rate for students receiving WFS services
• Improved employment outcomes for students receiving WFS services
• Efficient and effective service delivery (bundling)

The outcomes identified above are just a starting point. There likely will be other outcomes that are better measures of the impact on WFS participants. As services evolve to fit student needs, colleges may need to revisit the outcomes the institution has chosen to determine if they still fit the goals of the participants and provide enough information to assess the strategy.

While examining the effects of the WFS strategy at the institution, consider additional outcome measures up front and begin to set indicators to monitor progress.

Additional outcomes for consideration include:

Tracking Progress towards Outcomes

Tracking data for individual begins with the intake process. During intake staff should collect enough information to set a baseline for tracking progress towards meeting the individual outcome measures while providing the staff with enough information to determine the type, and sequence, of services the student will require in the short-term.

The indicators a college collects should be based on the outcomes it decides are most appropriate for the services provided to students and the information needed to monitor and evaluate the WFS strategy. A college needs to decide whether to track every student or a particular cohort of student to assess the effectiveness of the strategy. The table below displays a sample list of indicators and related services that can be used to measure progress towards achieving the chosen outcomes.

Self-reporting of data

In many cases, participant data will be self-reported, meaning that students will need to be educated or at least given clear instruction as to what they are expected to report back to the college. Measures of decreasing debt or increasing income require student’s honest and clear reporting of where they stood before and after a WFS intervention. In certain cases it may be possible to track the impact of WFS services after a student has left the institution. Some colleges, for instance, are able to track aggregate wage data for students once they graduate. However, it is unlikely that the data can be ascribed to particular students, meaning this may not be a fruitful exercise for colleges that only serve a small cohort of students through their WFS. Of course, not all data is self-reported; student completion and retention can be tracked by the college

Strategies for data collection

Data collection should be integrated into the service delivery model. Since the WFS strategy can take many forms within a college, the first place to start when designing a data collection process is deciding when a student is considered a participant in your institution’s WFS. Taking the time up front to understand when to count a student as a WFS participant will reduce confusion about who is in the “denominator” when assessing the strategy’s impact on participant- and college-level outcomes

Deciding how to track data on WFS service delivery is often a function of resources. Some colleges use case management software products, such as Efforts-To-Outcomes (ETO) by Social Solutions. Case management software allows staff to enter in every interaction with a student and results towards meeting intermediate goals and longer-term outcomes. A case management tool allows colleges to provide timely and more accurate reports on the percentage of students receiving the different types of services.

Many software products, like ETO, require a significant investment and may be difficult for colleges to justify when just beginning to implement the WFS strategy. Some colleges use paper and pen to track student services and create either a spreadsheet or database, which uses the same unique identifier to identify each student in a school’s data system, to store data on services and analyze progress toward outcomes. Involving the college’s IT staff in the design of the data collection process up front will help determine which tool will best fit the college’s needs.

Finally, tracking data on services received and outcomes is time-intensive and is not likely to be the role of one person. Design a workflow process that is flexible but still allows for the timely delivery of information. For example, a financial coach may use a paper-based form to record information from their coaching sessions. Then the form is later given to an office assistant to input into ETO or a Microsoft Excel spreadsheet or ACCESS database.

One additional item to consider is how external partnerships impact data collection. If some of the service provider staff are not employees of the college, or if data is shared with a community based organization, then the college will need to develop a data-sharing agreement.

Adapting existing data systems

Regardless of the tool used to input, aggregate and analyze the data, the college must put this information into existing student data systems such as Banner, Datatel or Peoplesoft to see how WFS services affect education, employment and economic outcomes for participants. While data from ETO, MS Excel and MS ACCESS can be uploaded to college data systems, some colleges have chosen to integrate tracking of WFS services into their existing data systems. See the case study on Des Moines Area Community College in the Center for Working Families at Community Colleges: Clearing the Financial Barriers to Student Success report.

Determining the audience for data collection

When determining the data to be collected, colleges should carefully consider their audiences and their strategic interests and ensure that the right data points are included. The numbers that resonate most strongly with a student may be different for a funder or community partner. The table below offers some suggestions on the different types of data that should be presented for different parties.