Auditing complex community-based projects

Big noise logoAs we move into increasingly uncharted territory of more preventative, place-based public service delivery, reflecting the post-Christie agenda, there’s a growing need for audit and inspection to adapt. This will help us to better understand what difference public services (and their third and private sector partners) are making in giving Scotland’s communities the opportunity to fulfil their full potential.

For example, my Audit Scotland colleague Aileen Campbell recently finished an 18-month secondment with the Glasgow Centre for Population Health (GCPH) on a project to evaluate Sistema Scotland’s Big Noise programme in the Raploch area of Stirling and the Govanhill neighbourhood in Glasgow. This was an innovative piece of work, which Aileen has previously blogged about. I was involved in the programme board that oversaw the evaluation, and the experience was a real eye-opener.

It confirmed for me the importance of drilling below the surface and giving a voice to the people who actually use public services. Yes, it’s important that we check whether services are well led and effectively managed, given the public money involved. But, we should take this approach a step further: Are services being delivered in a joined-up way? Have local people been consulted on the potential for change? At a time of increasingly people-centred and localised public sector reform we need to think more deeply about how users’ experiences inform our judgements about how successful services are.

In a similar vein, the project made me reflect quite carefully on how the more intangible aspects of organisations impact on performance and outcomes. One of the strong messages from the Big Noise evaluation was about how valued the young people (and their families) felt, and the positive impact that had on their self-confidence and optimism. This result was as much to do with the values, attitudes and behaviours of Big Noise staff as it was to do with them being excellent musicians and teachers. A difficult achievement to measure, in traditional audit terms, but clearly a valuable asset to the programme and no less deserving of recognition than an impressive statistic. We need to make more room for identifying and reporting on the intangible, and its impact, than perhaps we have done in the past.

The experience also highlighted the value of theory building in areas where the evaluation evidence base is underdeveloped, such as prevention and effective partnership working. That shouldn’t be abstract theorising; it must focus on answering the key questions of what really does make a difference when delivering improved public services, and what are the real – not imagined – barriers to public services working together to redesign services.

The GCPH’s initial research findings are just the start of the evaluation of Big Noise project. I’m looking forward to staying involved with this exciting and innovative work, which undoubtedly has great value for all of the different agencies contributing to the process.

About the author

aclarkAntony Clark, Assistant Director, held a variety of public sector posts in England before joining Audit Scotland in 2003. Since then he has developed the Accounts Commission’s Best Value 2 audit approach in local government, led on Audit Scotland’s work on community planning and managed a national programme of Best Value audits in fire and rescue.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s