Identify Resources

The first decision to make about collecting data is whether you want to use internal resources (e.g., use existing or hire new staff), external resources (e.g., a consultant) or a combination of both.

Internal v. External Resources

There is great value in building and using internal resources to support ongoing data collection and analysis—having data and analysis expertise at your fingertips can help you make informed decisions in real time. However, all organizations need to determine what works best for their unique needs when it comes to deciding how to allocate resources for data collection, analysis and evaluation.

  • Capacity: Do you have, or do you want to create, the internal capacity to support this project?
  • Expertise: Does your staff have the expertise to effectively support this project? If not, is it worthwhile to build that expertise?

The table below outlines several considerations when deciding between internal and external resources:

Internal Resources External Resources
    Less expensive

    Potential for faster completion of project (no need to wait for consultants who may be working on other projects)

    Potential to build in-house capacity for data analysis, learning and evaluation

    Better understanding of organization and program culture and nuance

Cost-effective, if extra capacity is needed for only a short time

Impartial perspective on the program or work being assessed

Contract with consultant can include training to build internal capacity

Fresh eyes can provide important insight and lead to new ideas and interpretation of results

Building Internal Capacity

You can build internal capacity in two ways: reallocate existing staff or hire new staff. What type of skills and experience will help someone be successful in a role that focuses on data and analysis?

  • Being “data-minded”: Someone who currently has or can build skills for creating charts, graphs, pivot tables and Excel formulas; someone who is interested in building surveys, exploring data and helping to make meaning out of numbers.
  • Facilitation skills: Someone who can work with others to define needs and questions, collect data, translate the results and then work with different individuals or teams to use it.
  • Communication and storytelling skills: Someone who understands the many audiences of your organization, both internal and external, and who can help you craft the right messaging and stories with your data.

Start Inside

When deciding how to build internal capacity for data and analysis, first consider whether you are able to reallocate existing staff.

  • Who in the workplace has the skills you need or the aptitude to learn?
  • Who has the capacity to accept some additional responsibility?
  • Who may be eager for a new challenge?

If you think you can identify one or several individuals internally, consider how you would shift their current responsibilities to make time for new duties. In addition, understand there will likely be a learning curve for new activities related to data and analysis.

Source New Staff

If you do not have the right people to do this work internally, consider recruiting a full-time or part-time employee. The person you are looking for does not need a degree in data science—perhaps they studied economics, statistics, finance, accounting, business or psychology. You might be looking for someone who can crunch numbers and put together charts and graphs; depending on the level of sophistication for evaluation, you might be more interested in a person who can both understand the data and also interpret it for key stakeholders and other people in the organization.

Do some research on job descriptions others are using to source “data-minded” staff. Look for a variety of backgrounds and experiences. An ideal candidate may indicate experience using keywords like “metrics,” “evaluation,” “data,” “analyst” or “measurement.”

When you are ready to interview candidates, here are a few example questions that you can customize to your specific job search:

  • “You’re going to be running a program and need to develop a post-program survey. What steps would you take?”
  • “We just finished collecting a post-program survey. Now that we have all the data, what would you do to make that data useable for us?”
  • “Let’s say I have data for this (insert a real example), how would you go about presenting it to the board?”

The more real the scenarios, the better you will be able to understand the candidate’s thought process and approach to working with data.

Choosing an External Partner

If you are planning to use external resources, it is important to choose the right partner—one who will not only support you in your efforts to answer your strategic questions, but who will also challenge you to think in new and different ways about your questions and your answers.

Developing a Request for Proposals (RFP)

In order to secure the right partner, it is important to write a RFP that effectively conveys your need and what you are hoping to gain from those submitting proposals. We recommend including the following sections:

  • Background: What contextual information is important for the respondent to know?
  • Need: Why are you looking for an external consultant?
  • Scope: What specifically are you looking for a consultant to do for you?
  • Timeline: When are you expecting the project to begin and end? Are there important dates (e.g., board meetings, staff retreats) where you would like to see interim deliverables?
  • Deliverables: What exactly do you want this consultant to provide for you? Paper or electronic presentations? A project plan and timeline? Interim and final reports?
  • Estimated Budget: How much would you like to spend? This could be a budget range or a maximum.
  • RFP Timeline: When are proposals due? When do you expect to hold interviews? When do you expect to make a decision about who to hire?

Here are a few tips for developing an RFP that will yield strong and clear responses:

  • Clarify: Be as specific as possible about your objectives and why you are seeking an external partner. The more clarity, the better responses you will receive.
  • Budget: If in reading the list above, you hesitated about including the budget, strongly consider including at least a range or ceiling. It can be frustrating for both you and the respondent to receive a great proposal that is outside your price range.
  • Review criteria: Be transparent about how you will be evaluating the proposals so the respondents will speak directly to them. This minimizes the need for follow-up questions and potential partners can decide not to submit if they are not able to meet the criteria.
  • Assumptions and constraints: Communicate any parameters or constraints, especially with regard to technology tools and services. For example, if you need to import data into Salesforce upon the completion of the project, tell them upfront so that they can ensure their deliverables are compatible.

Reviewing Proposals

It is helpful to put together a set of questions to guide reviewing proposals, particularly if you have multiple reviewers. You can use these questions in a group discussion when reviewing proposals or put the questions into a checklist or chart and ask each reviewer to score each proposal.

  • Experience
    • Have they done this type of work before?
    • Do they have the subject matter expertise needed?
    • Within the firm or agency, who specifically will be working on the project?
    • What do previous clients say about working with them?
  • Process
    • Will they challenge assumptions to ensure delivery of the best results?
    • Will they help you understand and effectively communicate the results?
    • Have they demonstrated a firm understanding of the questions and goals?
    • Do they have a good “vibe”? Will they work well with the internal project staff?
  • Budget and Timeline
    • Is their budget reasonable and affordable?
    • Can they deliver what I need within the necessary timeframe?