Define the Measurement Purpose

Once you know what guiding questions you are trying to answer, you need to solidify your purpose in collecting and analyzing your data. Drilling down to a specific purpose is important because it will help you determine what data you collect, how you collect it, how you analyze it and how you make the results meaningful in supporting your decision-making.

What is the Need?

Your guiding questions will likely indicate the type of need you have for collecting data. Some options include:

  • Refining models and methods: e.g., I have several different program models that work to achieve the same outcomes, and I want to know which is most effective.
  • Making programming decisions: e.g., I am budgeting for next year and I need to decide how much to allocate to each of my programs, and I want to base this on evidence of success.
  • Establishing a baseline: e.g., I want to establish a baseline so I can understand how my programs are impacting individuals over time.
Example

What is the Need?

As part of a strategy to increase the number of high school students taking a gap year before college to volunteer, a signature program gives teens the opportunity to volunteer at a summer camp for low-income families. Teens can choose one of two program models: a 2-week or 4-week volunteer position. To assess these options, your purpose would be to determine if one program model is more effective and why. This would fall under the category of refining models and methods.

Management or Strategy?

Another consideration as you define what data to collect is whether you are measuring for management or for strategy:

Measuring for management focuses on process, efficiency and program or initiative execution. Data to support management decisions is typically output data (e.g., number of participants, number of email clicks) and collected and analyzed more frequently (e.g., monthly). This type of measurement might be referred to as “monitoring.”

Measuring for strategy focuses on medium to long-term effects and information needed to support planning for the future. Data to support strategic decisions is typically outcome and impact data (e.g., effects of programs on participant behaviors) and collected and analyzed less frequently (e.g., semi-annually). This type of measurement might be referred to as “evaluation.”

For more on the distinction between monitoring and evaluation, check out this primer from Knowhow Nonprofit.

Example

Management or Strategy?

As part of a strategy to increase the number of high school students taking a gap year before college to volunteer, a signature program gives teens the opportunity to volunteer at a summer camp for low-income families. To understand whether the program is successful at influencing teens’ decisions about volunteering, the purpose would be to focus on measuring for strategy.

Qualitative or Quantitative?

A final consideration is what type of data to collect—quantitative (raw numbers, like how many times a day someone brushes their teeth) or qualitative (narratives, like why someone chooses to brush their teeth twice a day). While quantitative data is important for simple calculations and statistical analysis, qualitative data provides important context to help turn your data into a story—in fact, during early stages of a program or initiative, personal stories may be all that is available to collect.

As your programs aim to change attitudes about volunteering and also increase the likelihood that high school juniors and seniors go on to participate in a gap year volunteering program, you will need to collect quantitative data about attitudes and feelings toward volunteering. Because you want to understand the nuance of the efficacy of one program model over another, it would be helpful to collect some qualitative data about the experience of participants.

For more on the differences between qualitative and quantitative data (and a tutorial on how to make qualitative data more quantitative), check out The Qualitative Debate from Research Methods Knowledge Base.