Evaluation Consultancy

ToR for Final Independent Evaluation of ‘EqualiTea' Project


Centre for Education and Communication (CEC) seeks independent consultants to do an end of the project evaluation of ‘Sustainable Livelihoods for Small Tea Growers’ designed to assess the project against its own stated goals and objectives and to derive learning for CEC and its partners

Interested consultants are invited to submit detailed CV and Expression of Interest (EOI) marked “Final Evaluation - EqualiTea” to CEC, 173-A, Khirki Village, Malviya Nagar, New Delhi - 110017 or e-mail application to cec@cec-india.org on or before 28 February 2016.


Table of Contents

1. Background Information 

2. Purpose of the Final Evaluation 

3.  Specific Objectives 

4. Scope  

5. Final Evaluation - Key Questions 

6. Methodology for Evaluation

7. Deliverables      

8. Roles and Responsibilities  

9. Time Frame and Reporting 

10. Budget 

11. Evaluation Ethics

12. Qualification and Experience

13. Competencies 

14. Submission of Applications

Annex 2 : List of the Documents to be Consulted

Annex 3 : List of Key Project Informants 

Annex 04: Risk Assessment Matrix 

Annex 5: Achievement Rating Scale 

Annex 06: Format for the Evaluation Report 


1. Background Information

The European Union, represented by the European Commission and Centre for Education and Communication, with its office at Khirki Village, 173 A, Malviya Nagar, New Delhi 110017, India, signed the contract for implementation of the project 'Sustainable Livelihoods for Small Tea Growers' on March 28, 2011.. 

The Project is implemented by Centre for Education and Communication (CEC) along with Traidcraft Exchange UK (TX) for a period of five years from 01 April 2011 to 30 June 2016. In India, the project is being implemented in the states of Assam, West Bengal, Tripura, Arunachal Pradesh and Mizoram. In Bangladesh, the project is implemented in Tetulia region of Panchgarh district. Traidcraft is directly involved in the implementation of the project in Bangladesh.

The overall objective of the project is to reduce poverty among small tea growers (STGs) in India and Bangladesh. The specific objective is to enable STGs and their representative bodies to realise fairer terms of trade.

The project intends to reach 51,000 small tea growers (STGs), with focus on those below 5 acres of land. It is also expected to reach 10,000 tea workers (mainly women) who are employed by STGs. The project will engage with state level associations of small tea growers, bought leaf factories (BLFs), who purchase green leaves from the STGs, the Tea Boards of India and Bangladesh, technical and financial institutions, local NSAs and the private sector.

The project follows five key approaches: (i) creating a collective voice and identify for STGs setting up Primary Producer Societies (PPS) of STGs; (ii) capacity building and institution building for STG representative bodies by providing them trainings on technical, organisational, market issues; (iii) formation and strengthening of a national representative body of small tea growers for national and international advocacy; (iv) building sustainable supply chains for STGs in both countries, through transparent linkages with bought leaf factories and encouraging setting up their own factories wherever possible); and (v) ensuring sustainability through policy protection for STGs, collective bargaining, access to resources and technical support to enhance productivity and access to markets. In Bangladesh, the project provides saplings and credit to potential growers and link them up with a bought leaf factory through a by-back agreement.

The methodology is based on the assumption that if a dispersed group is organised into collectives, trained in technical, negotiation and governance skills, they will be able to achieve confidence to deal more effectively with other players in the supply chain. They will also achieve economies of scale thereby reducing their cost of production, access to finance and resources, increase volumes of tea leaf to support better price negotiations, command the attention of BLFs and receive better returns through embedded services and fairer contracts, get involved in value addition, and livelihood diversification measures. This will increase their confidence; reduce vulnerability to natural shocks, leading to further empowerment to achieve policy change.

The project is in the final year of its implementation to be concluded on 30 June 2016. A final evaluation is due.

2. Purpose of the Final Evaluation

This is an independent end of project evaluation designed to assess the project against its own stated goals and objectives and to derive learning for CEC and its partners.

3.  Specific Objectives

The specific objectives of the evaluation are to:

•Identify the outcome and the impact and ways it can be sustained

•Provide an analysis of the extent to which project funds have been used efficiently and effectively

•Develop recommendations for further developments of the project or the guidance of similar projects in the future

•Draw key lessons learned to contribute to the organisational learning of CEC and its partners

4. Scope

The independent evaluation commissioned by CEC is expected to cover project staff in CEC, Traidcraft and Bikash Bangladesh; the small tea grower societies and growers who have been benefited from the project in West Bengal, Assam, Tripura, Mizoram, Bangladesh, Small Tea Grower Associations; the Tea Boards of India and Bangladesh; the project associates and state / district level small tea grower associations; and bought leaf factories; estate factories; European Union and other relevant stakeholders.

The report of the independent evaluation will be shared with all stakeholders mentioned above.

5. Final Evaluation - Key Questions

Independent Final Evaluator will examine (i) Relevance, (ii) Effectiveness, (iii) Efficiency, (iv) Impact and (v) Sustainability. The evaluator should write a short synopsis of key points from each section for inclusion in the executive summary.

The broad key questions to be answered by the evaluator against the project log frame include:

5.1 Quality and Relevance of Project Design

Given that the project context, please assess the appropriateness and relevance of the project design. Specific questions include:

  • To what extent did the project respond to the identified problems? What was the quality of the problem analysis and the project’s logical framework?
  • Have the problems originally identified changed? How flexible was the project to changes in circumstances?
  • How relevant was the project to the needs of the different stakeholders’
  • What else has changed in the external context? For example, what has been happening in the tea sector in India and other tea producing and consuming countries?  What impact has this had on our work / future work in this area?  What else has supported / hindered our work in this area?
  • How did the design of the project take other interventions into account? Were there synergies or duplications?

5.2 Effectiveness

Assess the achievements of the project in relation to its stated objectives and intended outcomes/ results or are expected to be achieved, taking into account their relative importance. This should be assessed against the final version of project logframe as agreed with the lead funder (see Annex 01). Data already provided by the project’s monitoring and reporting systems should provide much of the basic information. Quantitative and qualitative data should be referred to.

  • To what extent will the objectives of the intervention be (most likely) achieved?
  • To what extent were the originally defined objectives of the intervention realistic? To what extent do they still meet the most recent standard of knowledge?
  • Has the intervention contributed to sustainable capacity building of target groups
  • What factors were crucial for the achievement or failure to achieve the project objectives so far (indication of strengths and weaknesses)?
  • What changes have been brought about in performance / behaviour of participating organisations and end beneficiaries?
  • What is the scope or magnitude of the change achieved?  What is the significance/ strategic importance of the achievements?
  • What has not been achieved (failures, disappointments, missed opportunities, challenges) and why?
  • What were the risks identified ? Did these materialise? If so, how did the project deal with them and reduce the impact on the project? Please complete a risk assessment matrix for inclusion in the final report (see Annex 04 for template)
  • Are there any exceptional experiences that should be highlighted, e.g. case studies, stories, best practice.
  • Please include a table of relevant figures showing achievement against the project indicators as stated in the project logframe included as Annex 01.

5.3 Efficiency of Planning and Implementation

Are the objectives achieved in a cost-efficient manner by the intervention? How big is the efficiency or utilisation ratio of the utilised resources? Specific questions include:

  • Is the relationship between input of resources and results achieved appropriate and justifiable? What is the cost-benefit ratio?
  • Which activities were undertaken in order to achieve project results? Were these conducted efficiently (in terms of expertise, time, costs, etc)? Were the activities of the expected quality?
  • Were there any planned activities that did not happen? If so, why?
  • Have the resources been allocated in the most strategic way among partners? Were the results/ outcomes appropriate to the costs incurred? Could the results have been achieved more economically?
  • How successful were the roles that CEC and the partner(s) played in the project management?  How could it be improved?
  • To what extent have individual resources been used economically?
  • Are there any alternatives for achieving the same results with less inputs/funds?

5.4 Impact

Assess the positive and negative, primary and secondary long-term effects produced by a intervention, directly or indirectly, intended or unintended. Specifically it will examine:

  • What has happened as a result of the programme or project?
  • What real difference has the activity made to the beneficiaries?
  • How many people have been affected?
  • Does the development intervention contribute to the achievement of the objectives (tendentially, overall goal)? Please complete an Achievement Rating Scale for inclusion the final report (see Annex 05).
  • What is or are the impact(s)/effects of the intervention compared to the total situation of the those affected:
  • positive and negative, intended and unintended effects
  • technical, economic, social effects
  • Even though there may be no explicit outcomes relating to the environment and gender, please consider any impact on the environment and on women as a result of this project.
  • Has there been any impact on people beyond the target groups (for example, in the wider geographical community or the sector as a whole) through learning shared by project participants or as a result of policy work.
  • To what extent was the intervention exemplary, created structures and/or had a broad effect/impact in terms of leverage?
  • What would the development have been like without the intervention?

5.5 Potential for sustainability and replication

The evaluator will examine the probability of continuation of benefits from the intervention after the project has been completed.

  • Are the positive effects sustainable? How is the sustainability or the continuity of the intervention and its effects to be assessed?
  • To what extent will activities, results and effects be expected to continue after donor intervention has ended?
  • To what extent does the intervention reflect on and take into account factors which, by experience, have a major influence on sustainability like e.g. economic, ecological, social and cultural aspects?
  • How self-supporting in particular are the entities part of or created during the implementation of the project
  • To what extent did the programmes strengthen local ownership and leadership?
  • How committed are participating organisations / partners / participants to continue utilising new skills, knowledge, techniques acquired during the project?
  • Are there indications that elements of the project will be sustainable without further external support? What are these and what are the factors which will determine whether or not they will continue?
  • Is there evidence to suggest that the project activities could be copied, up-scaled or replicated beyond the immediate project area? Why / why not?
  • What is the social and political environment/ acceptance of this intervention
  • Assess and make recommendations on the key strategic options for the future of the project, i.e exit, scale down, replication, scale-up, continuation, major modifications
  • Comment on any existing plans
  • Make recommendations in addition

5.6 Findings and Recommendations

  • What were the main successes and failures of the project, and why?
  • What are the challenges and how can they be addressed?
  • Which changes/ actions are needed for appropriate and successful implementation?

6. Methodology for Evaluation

The evaluation should be conducted in a participatory manner to bring multiple perspectives from key stakeholders in assessing the achievement of the project aim and specific objectives.

A Project Evaluation Group will be established to both inform and support the evaluation process. This will comprise key project staff from CEC, Traidcraft and Bikash Bangladesh as well as other key staff from CEC. The membership and responsibilities of this group are outlined in section 8.2 below.

It is envisaged the Methodology will include:

6.1 Desk Phase

  • Planning meeting with the Project Evaluation Group to develop guiding questions, elaborate and focus methodology and propose a work-plan. 
  • A desk review of existing project documents. (See Annex 02 for a list of the documents to be consulted.)
  • An analysis of existing quantitative and qualitative data which can shed light on some of the indicators of the logical framework.
  • Identification of issues and questions still to be answered and review of methodology.

6.2 Field Phase

Depending on the outcome of the desk phase, this might include:

  • Surveys, direct field visits, interviews, focus group discussions with a small selection of project actors including: small tea grower collectives, project staff (CEC, Traidcraft and Bikash Bangladesh), Tea Boards of India and Bangladesh, bought leaf factory
  • Interacting with individuals/ organisations that are not part of this project but relevant to the work being reviewed.

See Annex 03 for a list of key project informants.

6.3 Synthesis Phase

  • Presentation of Preliminary findings to project partners. This would provide a key opportunity to draw out any lessons learned.
  • The findings of the evaluation will be elaborated into a draft evaluation report, which is structured as per the required format for a final evaluation report (see Annex 06).
  • The draft report will be assessed by the Project Evaluation Group according to quality standards, and feedback will be provided to the consultant.
  • On the basis of comments received, the evaluator will revise and deliver the final evaluation report.
  • Main findings presented at dissemination workshop with participating organisations, partners and key Traidcraft staff.

7. Deliverables

7.1 Desk Phase

  • Methodology design / key questions and timeline for approval.
  • Feedback to the Project Evaluation Group on initial analysis from literature review and data analysis to identify initial findings, key gaps, review sampling criteria etc. 

7.2 Field Phase

  • Written feedback on preliminary findings and conclusions to the Project Evaluation Group.

7.3 Synthesis Phase

  • The evaluator will provide a written Evaluation report. The required format for the evaluation report is attached as Annex 06.  The draft report will be initially submitted to CEC for comment and review (CEC will then consult with the Project Evaluation Group). Any changes necessary will be made by the evaluator to produce the final report. CEC and its partners are interested to preserve the objectivity of the evaluator but reserve the right to ensure the Evaluation report is of the quality expected.   
  • The evaluator will present the final report at a dissemination workshop, where project partners and participating organisations will be present. The findings will be discussed and reviewed.

8. Roles and Responsibilities

8.1 Evaluator’s Responsibility

  • Undertake assignment as outlined in the ToR. 
  • Complete the tasks in ToR in the allocated time.
  • Update project partners on a regular basis concerning progress.

8.2 Project Evaluation Group’s responsibility

The Project Evaluation Group is comprised of:

This group’s main functions are:

  • To ensure that the evaluator has access to and has consulted all relevant information sources and documents.
  • To validate the evaluation framework, questions and methodology.
  • To discuss and comment on reports delivered by the evaluator at each stage of the process.
  • To assist in feedback of the findings, conclusions, and recommendations.

8.3 Provision of logistical and other support

  • For day to day queries and logistical support the evaluator will contact Joy Chakravorty atCEC, who will coordinate any support necessary. He will be responsible for informing stakeholders of any interviews or focus groups, and for providing the contacts of people to be interviewed, dates of visit and itinerary. 
  • For support on the content and methodology of the evaluation, as well as delivery of the report, the evaluator should contact J John at CEC.

9. Time Frame and Reporting

The Final Evaluation will take place in February - March - April for a period covering five weeks (26 working days).

Activity

Deadlines

Commencement of evaluation

March 05, 2016

Completion of desk phase

March 15, 2016

Completion of field phase

 March 30, 2016

Presentation of  initial findings and recommendations

April 01, 2016

Completion of first draft of evaluation report

April 15, 2016

Completion of final evaluation report

 April 25, 2016

Presentation of final report in dissemination workshop

 April 30, 2016

10. Budget

Total Fees and expenses including travel, boarding, lodging and incidental expenses: up to ₹3,25,000.

The evaluator should develop a budget for completing the evaluation as outlined in this ToR by clearly allocating expenses and daily rate.

11. Evaluation Ethics

The Evaluator will take every measure to safeguard the rights and confidentiality of key informants in the collection of data. The Evaluator will respect CEC’s Code of Conduct for Evaluation.

12. Qualification and Experience

  • Minimum of a Master’s Degree with relevant experience in social sciences, or any other relevant discipline;
  • At least 5 years proven understanding and practical experience of working in related projects in the context of sustainable livelihood of small producers and farmers;
  • Minimum 10 years working experience (Proven relevant experience (at least 5 years experience) in designing and conducting development sector projects or similar evaluations;
  • Experience and knowledge of the socio-economic context of Northeast India and South Asia would be an asset.

13. Competencies

  • Demonstrated and excellent written and oral communication skills in English and Hindi;
  • Knowledge of Bengali and Assamese languages, preferably
  • Strong negotiating skills and ability to work independently
  • Cross–cultural management experience and sensitivity;
  • High level planning, organisational and time management skills, including flexibility, attention to detail and the ability to work under pressure to meet changing deadlines;
  • Well developed interpersonal skills, including the ability to liaise effectively at all levels;
  • Knowledge of and experience with monitoring and evaluation (M&E) mechanisms;
  • Demonstrated analytical and written skills and the ability to clearly present findings, drawing practical conclusions and recommendations;
  • Expertise in agricultural development, collectivisation and value chain

14. Submission of Applications

Interested consultants are invited to submit detailed CV and Expression of Interest (EOI) marked “Final Evaluation - EqualiTea” to CEC, 173-A, Khirki Village, Malviya Nagar, New Delhi - 110017 or e-mail application to cec@cec-india.org on or before 28 February 2016.

 


Comments


of 500 characters