June Lennie, Andrew Skuse, Bikash Koirala, Nirmal Rijal and Michael Wilmore

Download 582 b.
Hajmi582 b.

Developing a participatory impact assessment approach and action research culture within a communication for social change organisation in Nepal

  • June Lennie, Andrew Skuse, Bikash Koirala,

  • Nirmal Rijal and Michael Wilmore

  • Paper presented at the Perspectives on Impact Evaluation conference, Cairo, Egypt, 2 April 2009


  • The 1st Chief Investigator of the Assessing Communication for Social Change project is

  • Assoc Prof Jo Tacchi from Queensland University of Technology, Brisbane, Australia

Outline of presentation

  • Benefits of a participatory approach to evaluation capacity building (ECB)

  • Potential of ethnographic action research (EAR) for creating a research culture

  • Overview of the Assessing Communication for Social Change (AC4SC) project

Outline of presentation (cont.)

  • Context of the AC4SC project

  • Strategies used to develop stronger M&E systems

  • Challenges and issues that emerged

  • Key outcomes and learnings

Benefits of a participatory approach to ECB

  • Process is flexible and responsive to change

  • Uses inclusive processes that aim to be empowering

  • Can generate trust, and a shared vision and objectives

  • Can enable better decision making, program improvement and sustainability

  • Can be a cost-effective method

Potential of ethnographic action research (EAR)

  • Used to build the research and evaluation capacity of media initiatives

  • Aims to be a rigorous, holistic approach

  • Combines participatory techniques with an ethnographic approach in an ongoing action research framework

Potential of EAR (cont.)

  • Uses methods such as participant observation, in-depth interviews, and immersion in research sites

  • Focus is on community capacities to engage and take action to facilitate social change

  • EAR researchers aim to foster a research culture

The Assessing Communication for Social Change project: partners and funders

Overview of AC4SC

  • Aims to design and trial a participatory methodology for evaluating the impacts of two popular Communication for Social Change (CFSC) radio programs made by Equal Access – Nepal (EAN), and related outreach activities

  • Aims to develop locally-defined indicators of social change

  • Is developing and adapting the EAR approach and setting up systems that integrate an EAR-like approach into the operations of EAN

Radio programs the impact assessment is focussed on

  • Saathi Sanga Man Ka Kura (SSMK) (Chatting with my best friend) - youth-oriented; mixes drama, information and listeners feedback. Aims to improve life skills and empower young people

  • Naya Nepal (New Nepal) - aimed at peace-building and reconciliation

SSMK listener club members at World AIDS Day rally

Evaluation context in Nepal

  • Lack of community of practice in evaluation

  • Lack of M&E understanding at all levels of a program

  • Data manipulated to please donors

  • Lack of understanding by donors of the complexity of measuring social change impacts, and the time and resources required

Contextual challenges and issues in Nepal

  • Extreme poverty

  • Communication and travel problems

  • Gender and power issues – low status of women, caste-based culture

  • Sensitivity of issues such as sexuality & reproductive health

  • Political instability – frequent strikes

Maoist demonstration in Bharatphur, Central Terai, during AC4SC field visit, September 2007

Context in Equal Access – Nepal before AC4SC project

  • Lack of leadership in evaluation

  • Poor feedback systems; use of unrealistic indicators of program effectiveness & impact; uncoordinated approach to data analysis

  • Reliance on ‘success stories’ from listener letters and other feedback

  • Limited training in all aspects of M&E and EAR

  • Unable to demonstrate the usefulness of EAR

  • Staff wanted to improve their skills in a wide range of M&E methods

Strategies to develop stronger M&E systems

  • Critically reviewed existing M&E capacities, needs, systems and practices

  • Conducted relationship and team building exercises

  • Developed better data management, reporting and feedback systems

Strategies to develop stronger M&E systems (cont.)

  • Held capacity-building workshops with EAN staff and practiced skills with community groups

  • Identified and trained eight community researchers (CRs)

  • Developed basic manual and toolkit for CRs

Some key challenges and issues

  • Initial methods and framework seen as too complex - led to confusion; problems dealing with large volumes of qualitative data

  • Resistance from SSMK team to changing its M&E systems; lack of cooperation with M&E team

  • Human resources and implementation problems

Key project outcomes

  • Impact assessment methodology streamlined and immediate re outcomes

  • More effective M&E systems

  • Collaborative relationships between M&E and content team improved

  • Content team more aware of importance of M&E and enthusiastic about learning from CR data

Some learnings from AC4SC project

  • Building capacities and setting up effective M&E systems takes time and effort but is essential before good quality data can emerge

  • Need to understand organisational culture before collaborative relationships can be improved

  • Can train community members to collect participatory data but they need continuous mentoring & good feedback systems

  • Regular critical reviews needed to identify & address problems & encourage critical approach

Some learnings about creating an evaluation culture in development organisations

  • Involve all relevant staff in ECB activities

  • ECB initiatives need to be flexible and open to change

  • Communication and feedback systems need to be effective and timely

  • Benefits of evaluation needs to be actively demonstrated to program staff

  • Keep participatory methodologies simple and practical, esp. in initial phase, but ensure that methods and data are rigorous and trustworthy

Thank you!

  • Questions?

  • Comments?

  • For more information contact

  • June Lennie at j.lennie@bigpond.net.au or

  • Bikash Koirala at bikashkoirala@gmail.com

  • For more information about Equal Access go to http://www.equalaccess.org

Download 582 b.

Do'stlaringiz bilan baham:

Ma'lumotlar bazasi mualliflik huquqi bilan himoyalangan ©fayllar.org 2020
ma'muriyatiga murojaat qiling