Programme monitoring and evaluating for empowerment

Programme monitoring and evaluating for empowerment

by Dang Ngoc Quang

22 Dec 2007

Only recently when RDSC started engaging in advocacy and lobbying on agricultural and rural development policy at the local level, the organisation has come to an understanding that good evaluation system is not only an effective way for learning, but also a powerful instrument for community empowerment and efficient policy dialogues with other development stakeholders, such as governments and influential donors, such as World Bank, and ADB, which are active in RDSC project areas. This note, which should be seen as work-in-progress, is to highlight some efforts made in evaluation programme at RDSC so far, and to explain about the evaluation framework that guides RDSC in its monitoring and evaluation.

State-of the art

Before 2004, most of major evaluations of RDSC projects were made by external evaluators. RDSC staffs played a secondary role as data numerators or facilitators for the evaluators to enter the communities. Communities had minimal role of providing information in this learning exercise. Evaluation results were used for solely for project management purpose, for reporting to the project donor, and sometimes for reporting to local government. This evaluation approach has little empowerment impact on RDSC staffs and communities.

Since 2004, on-the-job capacity building efforts have been made to consistently enhance RDSC staffs’ skills in evaluation. The organisation aims to equip research-based evaluation skills to as many staffs as possible. This is a major shift compared to the approach in previous phase, when only very few persons have research and evaluation skills. Two middle level managerial staffs have been attending a two-year on-the job training programme on policy research, offered by Anti-poverty Policy Study Groups (APPS). In short, but regularly spaced courses, other staffs have been trained by internal trainers on evaluation planning, data collection (qualitative, quantitative), and reporting.

To have data for comparison, in 2005 at the project start, in addition to PRA/PLA in project communities, RDSC designed and conducted a comprehensive survey on the development status. Data were collected using indicators for the project outcome and impacts. Samples were 20% of the poorest population in the communities participating in the project. This survey involved little participation of the local community, but project staffs.

In 2006, the second survey was conducted among project participants and non-participants in new and veteran communes to measure project impacts. The staffs and leaders of community-based organisations and activists were encouraged to narrate their stories as case studies for qualitative evaluation. The result of this exercise was intensively used for programme review and instrumental for awareness raising purpose when the project enters a new community. These results are to be used in a series of workshops with local agriculture authority in the project districts and the province in 2007.

Measurable indicators of empowerment feasible for RDSC and its programs

There are some readily measurable indicators of empowerment which are feasible for RDSC and its programs. Major measurement of empowerment can be indicated by 1) changes in power relation (level of influence) in favour for the disadvantaged or marginalised groups, and b) capacity to make effective decision at various levels in addressing the external forces, which can be market, government ruling, climate changes, foreign culture, and natural calamities.

The first group of indicators is relevant to ownership and control of power, either by weight or by number in voting, or by occupancy in key positions. The sense of ownership of public and private institutions is important indicator of power, and as a result the changes in this sense of ownership indicate the empowerment. In the context of competition for power this can be the level of democracy, or level of decentralisation. Measuring power, we should keep in mind different types of power, for example, political, economic, social, and cultural.

Changes in capacity of effective decision making and of carrying out those decisions are another indication of empowerment. This capacity is expressed in economic, social, human, financial and natural assets that a group possess or have access to. Furthermore, this capacity includes skills of long- and short-term planning and organisational skills for delivering the plans to its ends.

Here are few examples of practical indicators for measuring empowerment. Our organisation uses the number of our partners and beneficiaries, who rise up and are elected to the Local Governing Councils as a good indicator for empowerment. Also, we record the participation of the poor, women in community activities, and in the interaction with local government for revealing empowerment. We also count action independently initiated and implemented by villagers to address their needs and concerns as an indicator of the highest level of empowerment. Economically, we observe systematically the villagers’ coping patterns for food shortage as an indicator for economic empowerment.

How to measure community empowerment in a way that can be empowering

At the community level, as an example, the changes in capacity of communities in making their own decision for their future, their safety, and security can be used as indicators of empowerment. At this level, the measurable indicators can be the analytical skills of community and their leaders in democratically making internal and external analysis, priority setting skills, the planning skills and the organisational skills for implementing community inspirations. At this level, the methods to be used for measurement of both knowledge and skills can be various survey questionnaires, qualitative listing, and story-telling by community members. For the skills and community actions, a better measurement method is observation with various forms of data recording, such as note-taking, photography, video-filming.

In our practice, to involve community members in the measurement process in a way that can be empowering for them, our organisation uses various participatory and empowering methods. It is our practice to partner with communities, and their organisation in monitoring and evaluating our collaborative actions that we feel has strong empowering element. By sharing and learning together, our capacity has been improving and build-up on lessons arrived from both successes and errors.

Another method for empowerment is to assist villagers and their leaders to records, take notes or take photographs of their achievements, or stories to tell or explain to other communities, or government authorities as a way of using evaluation data to influence the local government’s policies. Exchange the evaluation results with other neighbouring or same-ethnic but-far-away communities also demonstrates as a powerful tool for building self-confidence and self-esteem of the communities about their capabilities to make changes.

Finally, our organisation work with local government to promote involvement of local villager-evaluators in using the results of our monitoring and evaluation system in local socio-economic planning. Often, the leaders of managers of the community-based organisations, which are assisted by our projects, are promoted to various positions in local power systems, where they may make plans or approve them. These seem a powerful process of promoting people’s power, but there is always a possibility of co-option of this power by the elite. Up-to-date, we have not yet looked into this issue; hence this is an area for our improvement.

How to measure women's empowerment in a way that can be empowering

For measuring women’ empowerment, the changes in general relation, most importantly ownership and role in decision making, between men and women at household, community and societal levels are used as indicators. At the household level, capacity of women in effective decision making at family or household social and economic businesses is measured in our evaluation survey. Also, the women’s entitlement to various major household properties, such as land, house or building, big animals (e.g., water buffaloes) is an indicator for ownership. In our questionnaire and interview check-list, we give special attention to the veto power of men or women as a key indicator for decision making.

At the community level, for measuring women’s empowerment, we look into the participation of men and women in community meeting, the place they take seats, their interventions or ideas expressed in the meetings. Also, we take records of managerial positions that women take in community organisations and local institutions, such as party, people’s councils, and paid positions of local government. At this stage, while assisting women’s organisations in operating micro-finance services, we have missed measuring the number of women entrepreneurs, compared to men, as economic indicator of empowerment for women as result of our micro-finance programme.

In addition to approach and method for empowering community described in the previous section, our organisation considers the participation of women themselves and their organisations as an important approach for involving women in the measurement process in a way that can be empowering for them. By showing our example of having women in our team in key positions that manage our programmes and monitoring and evaluation, we indicate that women are equal to men. Furthermore, a quota for women participation in various project institutions, such as local project management committees, is a helpful way to promote women’s empowerment. In selecting participants for management training courses, women’s participation is tentatively monitored. Evaluation results achieved by female managers and leaders of women-operated community based organisations, such as credit unions, seed-producers’ groups, are widely informed and communicated to various levels of government and media.

Direction to empowering evaluation

At the moment, we think domestic violence and conflicts are strong indicators for women’s empowerment. In 2006, we initiated a small pilot research in this area, using female staffs to collect data on this culturally sensitive topic. So far, we have not achieved good success in finding reliable data on these issues as both men and women were reluctant in informing outsiders about these “internal” home-affaire issues. We also found that the records maintained by local government on these issues are under-reported. We plan to change to a community-owned (observational) monitoring system to explore an evaluation approach for these new topics.

  Return    Print    Tell friends    Go Top