Management
  Alternatives Pty Ltd
  ABN 23 050 334 435

A. Thinking about
    evaluation
B. Framing an
     evaluation
C. Evaluation
     report outline
D. Model of service
E. Community
     profile
F. Peer review
G. Focus groups
H. Telling stories
I. Staff time use
J. Questionnaires

B. Framing an evaluation

There are many ways to develop a plan for a project evaluation. Here are some questions that may be useful in focusing a project evaluation plan.

1. What is the table of contents of the report?

See the next section for an sample table of contents which is a starting point for consideration, not a fixed evaluation report framework.

2. What are the focal questions for the evaluation?

A key to focusing the evaluation process and report will be having specific evaluation questions.

These questions can often usefully be linked to the original purpose of the project, ie. What is the purpose of the project? Within this context what do we really want to know about the project?

Answering one or two questions well is much preferable to answering many questions poorly.

3. What is the service model?

Being able to describe the service model for a project is a key step in thinking about the evaluation of the project

Ideally the service model should map the process of the service (or community development process) and identify the outcomes to be achieved at key points in the process.

See Section D for an example of a service models.

4. What are the key ideas underpinning the model and what are they based on? What’s the evidence for these ideas?

Each service or community development process is usually based on a small number of key ideas. For example:

• Play groups are a way for parents to make connections with other parents.
• Children will do better at school if their family is well connected with other people in the community.

It is important to identify the key ideas that underpin the service model and the evidence for these ideas.

The evidence could be a relevant journal article, a book, a research report, etc.

5. What are the values that underpin this service model?

Each community organisation (and project) has their own purposes and values.

It is important to name the core values for the project - and whose values they are (eg funding body, service providers, parents, community members).

6. What are the pointers to progress?

If the project was working well what would we notice?

For example if a transition to school project was working we would notice:

• children coming to school with more of the skills they need
• parents feeling comfortable to take their children to school
• teaching staff noting children are ready for kindergarten

7. How will we gather and record the ongoing story of the evolution of the Project?

Projects evolve. Part of the evaluation of the projects will be telling the story of the evolution of the project. A key question is: how will this story be gathered and recorded.

Some possibilities are:

• interview the project coordinator for 1.5 hours each six months and get her/him to tell the story of the evolution of the project - tape record the interview and have it available when the project evaluation is being written up.
• project coordinator to keep a project diary with monthly entries (not daily entries as these will focus too much on the day to day)
• steering committee meeting discussion once a quarter summarising progress over the previous quarter - tape record the discussion and have it available for the project evaluation write up.

8. How will we gather the data to show the extent of the project (numbers of clients, group sessions, etc, over time)?

Every project will need some numbers that reflect the extent of the project - how many people, how many groups, how many group participants, etc

This data needs to be systematically collected.

9. What projects compare with this project? How do they compare with this project?

A useful question to ask is: Are the other projects like ours? If so how do we compare with them?

10. How will we gather data to show what the project meant in the lives of the clients?

Human service projects are about making a difference for people. The evaluation needs to be able to describe what this 'difference' means in the lives of people. Often this is qualitative.

Some possible ways of gathering this data include:

• interviews with clients
• focus group of clients

11. How will we reflect on all this during the life of the project to improve it?

It is important to use the evaluation process to improve the project as it proceeds. How will the project reflect on what is being learnt so that the project can be improved?

This could happen at:

• staff meetings
• committee meetings
• supervision

12. How will reflect on this at the end of the project to crystallize what we have learnt?

How will everything that has been learnt be drawn together?