Words by Eman Bhatti, Impact Lead.
In December, Hayley (Creative Lead) and Eman (Impact Lead) had the pleasure of attending a sandpit session for Culture Forward, a forum created by the University of Birmingham that draws together researchers, cultural organisations, and local government representatives to share ideas and form meaningful partnerships. Spectra had been asked to give a short presentation on our approach to creative evaluation at this session – this is a slightly condensed version of that presentation.
As with all areas of our practice, our evaluation approach is always shifting as we regularly reflect on what is working well and where we could do better. Like Kate wrote in our January blog, we like to ‘hold space for chaos and the wisdom that springs from it’ – that includes being nice and flexy with evaluation.
OUR EVALUATION FRAMEWORK – IN PRACTICE
Our guiding principles run through every stage of a project, from planning to evaluation.
True Collaboration
We’re curious about contributions in all forms, providing equitable support and developing agency. When participants respond to our questions and invitations, we want them to feel that they have the power to make a difference; that their answers hold value.
Our most recent project is the creation and installation of a community garden space at the brand new Midland Metropolitan University Hospital in Smethwick. Since the beginning of last year we have been collaborating with our community partners to co-design and co-create this space.
The diversity of people that we work with (including learning disabled adults, South Asian Family Service, Brushstrokes, Sandwell General Hospital Paediatric ward, museums, libraries and schools) requires an equally diverse approach to our engagement offer, our ways of connecting, and our ways of evaluating.
How we do this…
- We offer a warm welcome, hold space for open conversation, ask open ended questions, give people time for creative processing, encouraging each of them to share their thoughts and ideas.
- We think carefully about who holds the power in the room – if the facilitator is seen as a leader or teacher, or someone who holds knowledge, how does that affect how people respond to their questions? We use various practices to create a non-hierarchial space where power is shared and distributed as equally as possible.
- We offer many entry points into creativity and self expression. The paper landscapes we created to explore garden designs do this beautifully – side by side, open ended, imaginative
Sensory Beings
We recognise that our bodies experience the world through our senses, so to use only verbal response/language in our conversations with participants would be leaving out a key aspect of communication. We value body intelligence and we take time in the sessions to listen to the body so that its knowledge contributes to our creative evaluation.
Many of the people we work with are non verbal, or speak a language other than English. Making connections and having conversations in ways that are not language based, but instead value the body-felt experience that people have in our sessions is so important. We may ask after an activity:
- How did this make you feel?
- Where do you feel this?
- How is your body feeling right now?
Noticing and observing body language is also key – what does it look like when people are engaged or disengaged, what does it look like when a person is expressing themselves fully, what does it look like when people feel a sense of belonging. This takes time, and a safe space, to develop.
Nourishing Spaces
We seek playful, brave and meaningful ways of being together that nurture a sense of belonging.
Spaces that are playful are also more likely to be brave, as play makes us more open to taking risks and trying something new. Though, being brave is easier when you feel like you belong in the room; that you have a place there and are valued in the space as you are.
Play is a leveller, and brings about feelings of connection and common ground, therefore creating the space for communication – and one that is more authentic and genuine in its response. Our evaluation activities often involve games and playful activities for just this reason. In the spirit of reciprocity, we’ll often offer food in return for the thoughts and opinions of our audience members.
OUR EVALUATION FRAMEWORK – IN THEORY
How have we organised our evaluation framework?
In order to formalise our framework, we created a theory of change – we pulled together all the reporting we have committed to doing for funders along with the information that we want to know in order to make sure we are tailoring our activity to be responsive and co-created.
After defining our indicators and evidences, we pulled together a set of lines of enquiry – we made use of the questions used in the impact and insight toolkit (which are in the form of statements to which level of agreement is indicated on a sliding scale, which other Arts Council funded organisations may be aware of). We tweaked the language to suit our audiences and our practice. The questions chosen reflect our mission and guiding principles closely. We then mapped these onto each of our projects. After the first year, we reflected on the process and further refined our lines of enquiry for Year Two, because, in practice, it became clear that some questions were really difficult to translate to our audiences and others ended up being less relevant than we thought they may be.
For example, the question ‘Creative Expression’ (It allowed me to express myself creatively) proved difficult to translate to our participants and Hayley realised that often our participants did not see themselves as creative and so struggled to connect to the idea of self expression through creativity. We’ve suggested instead for Year Two to try re-wording this to something like: I tried a new creative activity (assigning creativity to the activity rather than the person) / This was a creative activity / The activity gave me a chance to express myself in an interesting or new way.
Sometimes we decide not to collect data – for example, when considering demographic data, we try to do as little as possible because it is often invasive and difficult to collect in a creative way because it can’t be related to the activity that we’re doing. We recognise the importance of tracking demographics to ensure our work is genuinely inclusive so, when we need this information, we can get an idea of the demographics of participants based on the partners we work with as they often serve a particular community. We want everyone to feel welcome and empowered in the spaces we create together, so this sort of data collection feels extractive in a way that we are not comfortable with and has the potential to create barriers in our work.
There are a number of ways we then conduct evaluation and data collection. This includes creative activities and partner, facilitator and staff observations in the sessions, as described above.
In addition, where it is appropriate, we will do traditional digital questionnaires. Our Annual staff survey is one of these, and it offers an opportunity to engage staff members, the board and freelance artists in evaluation, with support where needed. We are able to get more detailed qualitative and quantitative responses to understand how people feel about working at Spectra. We ask about wellbeing, how valued and connected people feel, amongst other things. This survey is also an opportunity for consultation on specific topics, so contributions have the possibility to influence change at Spectra on an operational or project level.
Finally, we do extensive consultation. We conduct various stakeholder consultations to ensure that our community is influencing the work. For example, at our 10th anniversary celebration in 2024, we had consultation tables, each with a different creative activity and an opportunity to have a conversation about a particular question with a member of the team. Engaging in play and craft was a great way to connect with people and ease into these conversations, making the experience less intimidating and more accessible.
Creative evaluation to us means using a range of methods and communication styles, to ensure that everyone who wants to engage is able to, and employing our creative agility to co-create alternative ways of evaluating when traditional methods won’t work.
One of our aims in evaluation is to help participants reflect on their engagement and for them to develop a dialogue about their experience, rather than using pre-determined language to talk about their experience. Communicating in their own way gives them a deeper understanding about their creative experience.
In turn, this allows us as a cultural organisation to have a clearer and more personalised view on how participants are engaging with our work, and perhaps more importantly, where they are not.