Evaluation is the process we use to find out whether a project can meet, or has met its aims. It will be led by the core values of the organisation or project.  It should be informed by a ‘theory of change’, that is, an understanding of how an activity may be linked with its outputs and outcomes, or why what is done has the effect you observe that it does.

It may be useful to think about what evaluation is not.  It involves more than just collecting and reporting monitoring information about a project. Although evaluation findings can be powerful when advocating for a project, it is important not to confuse the two, because evaluation will ask and answer questions and reveal impacts that may be both positive and negative. And evaluation and research are also different.  Research usually seeks to generate new knowledge, while evaluation generally judges an existing service.

We evaluate for various reasons.  Top amongst them has to be a desire to find out whether what we are doing has an effect and whether that effect is the one we thought it would.  Without this knowledge we cannot develop or refine a service or respond effectively to the needs of those people who hold a stake in the work – such as participants, project funders or commissioners.

Do any of the following apply to you? You want to know how many people took part in a project and whether they were the people you had hoped to reach. You are interested to know about how the artistic outputs of a project have been displayed or received.  You are concerned to know whether participants enjoyed the experience of taking part.  You want to establish what practical challenges were involved in delivering the project and how they were overcome, what did not go well and what you could improve on next time.  You are interested in unexpected outcomes, and whether the project worked in ways you had not anticipated.  You want to understand whether the project represented a worthwhile financial investment.  You are interested in understanding the benefits the project has delivered for participants in terms of their health, wellbeing or quality of life. Evaluation can help in all of these areas.

It is important to ask yourself this question before you begin an evaluation, and to answer it honestly. Is the evaluation intended for your own reflective use? Are you fulfilling the requirements or responding to the needs of a funder or commissioner? Are you primarily interested in evidence that will help to advocate for your work, or make your service more attractive to service users? Would you like to contribute to the wider evidence base for the arts and health sector? A clear understanding of who the evaluation is for will help you to make decisions about what to evaluate, what resources to allocate to the job, whether you might want to work with someone else to do it, and how you will disseminate your findings when you have them.

Evaluation can naturally inform a cycle of reflection and development. It can help arts and health practitioners establish what is good about what they do, to improve upon it, and to ensure that they are doing nothing that is harmful to participants. In addition, it can help organisations to develop and refine services based upon an informed understanding of the impact they have on participants, and in relation to the needs of commissioners and funders. It can also help to advocate for arts and health projects by showing positive benefits for health and wellbeing

It is worth looking carefully at the way you currently reflect on or measure the work you do. It may be that you can simply tweak and build on what you do now. For example, some evaluation questions can be very effectively answered through simple, accurate monitoring. But you may find that you are spending a lot of valuable time collecting data that you simply cannot or do not use. It is always best to collect less data, and to make sure that what you do collect will answer the questions that are most important to you.

In arts and health evaluation practice, ethical best practice relates to three main areas: respect for individuals; social responsibility; and, the maximising of benefit and minimising of harm. We ensure that our responsibilities in these areas are met through mechanisms such as having an informed consent process when collecting data, respecting people’s need for confidentiality and anonymity, considering potential risks to participants in an evaluation and ensuring that no individual or group is stigmatised or mis-represented through it.  Unless you are conducting research, you are unlikely to need to go through a formal ethical approval process and therefore there is no single set of formal rules to adhere to, but specific ethical considerations will apply to particular data collection methods.

Demonstrating your understanding of ethical good practice will, in itself, go a long way to delivering a trustworthy evaluation. You also demonstrate your credibility through use of a well-designed evaluation: one that uses appropriate methods and tools, does not gather information that is not used, and which does not place unnecessary burdens on your participants. In addition it is important to demonstrate that you have considered and attempted to avoid bias wherever possible.

Evaluation will take time at almost every stage of a project – from identifying appropriate aims at the outset, to collecting data while the project is underway, to analysing data, reporting and disseminating findings and then ensuring that learning is fed back into the next phase at the end. It will help if you appoint an ‘Evaluation Lead’ to take responsibility for the process.  The good news is that if you have very little time and resources available, you can shape your evaluation accordingly by focusing your evaluation aims and questions ruthlessly and minimising data collection and analysis.  Being realistic about what you can achieve will help you deliver stronger results and make it more likely that you answer the questions that are most important to you.

Arts and health organisations often look outwards when evaluating because they feel they don’t have the skills or the capacity to evaluate internally. External evaluation can appear more credible, avoiding issues around areas where bias may creep in.  An external evaluator provides specialist skills and knowledge and may be able to disseminate findings more widely. However, these benefits may come at a significant financial cost or mean that evaluation is ‘one off’ and that learning is not embedded within the project team. In reality, many successful projects develop by using an iterative process involving both internal and external evaluation. It may be useful to think through carefully where you need to be on a continuum between simple monitoring and academic research and the organisation AESOP has developed a number of tools designed to help think through this process. And sometimes, it is useful to accept that your evaluation just needs to be ‘good enough’ for the context in which you are working.

Arts organisations and practitioners may be contracted by local hospitals, care homes, GP surgeries and community and third sector organisations to support a range of health and wellbeing outcomes for participants. In this context, ‘co-production’ of both projects and evaluation with commissioners, partners and service users becomes particularly important. Taking a collaborative approach to evaluation with your stakeholders, can help ensure that the outcomes you want to evidence are the right ones. Working with independent evaluators or academics can also help to make sure that the language and evaluation methods you use are appropriate.