May 2, 2007
Interim Evaluation of the Community Voices Network
This report presents the findings of an interim evaluation of the Community Voices Network.
Interim Evaluation of the Community Voices Network
The study has been undertaken when it is still very early in the life of the CVN, which is useful in terms of reviewing progress and making productive adjustments, but less so in attempting to identify tangible impacts on individuals, their localities or national community regeneration policy.
It has proved difficult to collect evidence which moves beyond individual member’s enjoyment of CVN activities and a perceived increase in confidence, to obtaining evidence of how their involvement has led to direct or indirect impacts in their communities. This remains the challenge for future evaluation.
Below we outline our conclusions and make recommendations in relation to specific aspects of the structure, operation and future development of the CVN.
CVN STRUCTURES, MEMBERSHIP AND STRATEGIC LINKAGES
· There is a perception amongst some CVN members that it is a top down initiative, and the membership has limited opportunity to input into or lead its development. This persists, despite considerable evidence that members are consulted on future network priorities at events, and through other consultative mechanisms
· The national dimension of the network is viewed positively by members but there is a lack of connection with local structures and community engagement approaches
· The CVN has not made inroads into the area of equalities and this is an area which requires to be given due consideration. There is however, a need to be realistic about the CVN’s role here and the resources which would be necessary. Collaborative working with equalities networks and representative bodies will be crucial.
1.The CVN should establish a process for involving the membership base in forming the agenda and promoting and developing the Network. This could include the establishment of issue groups (possibly short life) which could look at topics such as promotion and member services in addition to the suggested annual review group
2.The CVN should review how to improve linkages between the CVN as a national body with local community planning structures and local engagement processes
3.The CVN should embark on an exercise to develop its position in relation to equalities in partnership with key representative groups
Membership and activity levels
· Whilst a 200% growth in the first year is a reasonable achievement, as a national network, the membership level of the CVN across Scotland needs to be increased
· Just under half of the network members are living in deprived communities, which, given its overall focus, is too small a percentage
· Activity levels amongst the membership are too low, with a core group of around 60 members being active on a regular basis
· The database is limited in terms of what it can tell us about the membership, particularly in terms of equal opportunities
4. The CVN needs to embark on a recruitment drive, targeted at the most deprived 15% data zones in Scotland
5. The CVN should conduct an exercise to re-engage with its non-active members to examine why they are not participating in events and to find ways of increasing activity levels amongst this group
6. The CVN should review its monitoring arrangements to ensure that system reflect the need for adequate monitoring of equalities. The current database should be amended to enable more detailed information collection and analysis in terms of age, gender, black & minority ethnic status and disability
· The CVN now needs to establish stronger linkages with related national and local networks. These have not advanced as first anticipated, and represent a missed opportunity to advance network development in a range of ways. Some group focused national networks express an ongoing willingness to work with the CVN, and these linkages could be very beneficial to advancing the CVN equalities agenda.
· Links with local structures – especially those linked to CPPs – are at best patchy. Some good practice is apparent in some areas, but more commonly connections are weak. This needs to be urgently revisited to ensure connections are maximised between local and national community engagement activity.
7.The CVN should seek to proactively develop stronger linkages with key national networks, linked in particular to advancing its equalities agenda. This should involve working with and utilising the specialist expertise in these forums to engage specific groups. High profile joint events may be a useful way of launching new working links
8.The CVN should revisit linkages with all local Community Panning Partnerships updating them on: developments to date; local membership levels; and forthcoming events. The CPP contacts should also be invited to make suggestions on further network development
· The CVN is delivering a good range of activities which have been well received by participants but only a core group of around 60 members are actively participating in events.
· Members are particularly positive about the opportunity to meet with people from other areas and to learn about their experiences.
· Activities designed to achieve the aim of “influencing policy” are less well developed and there is no clear process in place although recent approaches could be more effective in enabling members to input into the policy process.
· If the CVN is to make advances in the area of equalities, its events programme and other activities will have to be developed to reflect that.
9. Develop measures to increase the involvement of the membership in setting the programme of activities and to increase member activity levels
10. Review activities (e.g. learning opportunities and events) in relation to equalities and develop approaches in partnership with relevant equalities organisations and networks
11. Further develop the website as a mechanism for encouraging member participation in CVN development and activities
CVN IMPACT AND MEASUREMENT
Impact and measurement
· Measuring the impact of a development such as the CVN is always likely to be difficult and will require a number of differing information gathering mechanisms. Observations on impact are further restricted at this stage by the relatively early stage of network development.
· Discussions and surveys with members are nevertheless positive in terms of the way they perceive the network supporting them – most notably in terms of knowledge, understanding, contacts and confidence building. But in terms of what they have tangibly done with these supports to advance further regeneration activity, the message is less easy to articulate. In addition, an analysis of active membership – as measured by attendance at events – suggests the CVN has a relatively small number of core activists many of whom are very experienced. Consequently, the impact is more likely to be to deepen rather than widen involvement.
· Evidence of influencing national policy is weaker. This is not surprising at this stage, but we have concerns that the concept is not commonly understood, and open to varying interpretations. This leads to confusion, and the danger of creating false expectations in some members on what this can achieve – particularly given current membership levels and profile. We suggest “understanding and inputting” to policy may be a better future definition.
· An ongoing evaluation process for the CVN is required, and we have detailed a framework for this. This will require that Communities Scotland has access to appropriate information from the delivery contractor. It is also suggested that an annual review group should consider a composite report on performance.
12. Communities Scotland should review the second aim of the CVN considering whether “understanding and inputting” to national policy is a more realistic and understandable aspiration than “influencing”.
13. The CVN should adopt the suggested future performance management framework detailed in section 5, and convene an annual advisory group of members to consider an annual report from this.