Haiti: Final Project Evaluation of HIV Prevention (PrevSIDA) Project, Haiti

Organization: American Red Cross
Country: Haiti
Closing date: 20 Oct 2014

Type of evaluation Final Project Evaluation

(Summative Evaluation) Expected evaluation methodologies Qualitative methodologiesNumber of evaluators One lead evaluator and one national assistant consultant

(Application should be in a team of two)

Expected start/end dates, number of work days Expected start date: 10 November 2014

Expected work days for lead evaluator:29

Expected work days for local assistant: 22

Deadline for receiving applications 20 October 2014 11:59PM EST

1. Description of project to be evaluated

1.1. Background and objectives of project

In partnership with Haitian RedCross (HRC), the American Red Cross (ARC) has joined a Population Services International (PSI)-led consortium aimed at preventing the sexual transmission of HIV in Haiti. The four year, $3,171,236 PrevSIDA project aims to reduce the incidence of HIV among youth at Higher Risk in Haiti and in settings characterized by high risk behavior. The project, which started January 2011, and ends February 28, 2015, will reach 283,005 individuals from high-risk youth and adult populations with HIV prevention interventions.

PrevSIDA follows the USAID-funded youth-focused Together We Can HIV prevention peer education program that began in 2004 in seven sites throughout Haiti. Sites were selected based on assessed need and HRC branch strength. The program later expanded into two sites chosen on the basis of high prevalence (Anse-a-Veau in the Nippes at 3%) and being underserved by other organizations (Port de Paix in the North West). In addition to the nine sites of Cité Soleil, Petit-Goave, Pétion-Ville, Anse-à-Pitres, Cap Haitien, Fort Liberté, Ouanaminthe, Anse-a-Veau, and Port de Paix, three new sites are established in the PrevSIDA project (Tabarre, Carrefour, and Gonaives).

The project activities address most at risk youth and young adult populations and those in settings characterized by high risk behavior. Project activities will reach certain at-risk populations that include young mothers, transport workers, informal sex workers, clients of sex workers, and female market workers. Project activities have also reached camp dwellers to increase awareness around HIV, particularly sexual transmission and the heightened risk through unprotected sex and multiple concurrent partners.

Project goal and objectives:

The overall goal of the PrevSIDA project is to reduce the incidence of HIV among most at risk populations (MARPS) and in settings characterized by high risk behaviors in Haiti. It intends to address the gaps and needs of MARPs through the following objectives:

1. To identify populations and settings with high-risk sexual behavior and to identify the dynamics of the behavior.
2. To improve HIV related knowledge, attitudes, skills and behaviors of MARPs and in settings characterized by high risk behavior through interpersonal communication-based outreach (IPCs) and community wide events (CWEs).
3. To educate and mobilize communities to adopt responsible social norms, attitudes, and values that reduces vulnerability to HIV sexual transmission.
4. To provide leadership and technical support to U.S. government (USG) partners and Haitian institutions involved in evidenced-based behavior change interventions.
5. To strategically link communication and education interventions for sexual prevention of HIV with other health related services and commodities, including targeted condom promotion and availability, as well as livelihoods initiatives for most-at-risk populations and those in settings characterized by high risk behaviors.

The program draws on internal expertise generated through six recent years of HIV prevention experience in Haiti, and has conducted community assessments to identify populations with high-risk sexual behavior and to identify the dynamics of the behavior. With these data, ARC/HRC employed evidence-based approaches that are rigorously monitored and evaluated. The project focuses on HIV prevention among groups deemed to be most at risk for HIV, including youth and adults practicing high risk behaviors. While the targeted groups have distinct characteristics that call for unique interventions, the interventions can share common approaches and tools that can be adapted to reach different target groups or the same target groups in different settings. Community needs assessment in branches have determined the risk factors and social drivers of high risk sex such as multiple partners (sometimes concurrent), inconsistent condom use, transactional sex, trans-generational relationships, and substance abuse which are common among youth in these communities.

Complicating matters is the January 2010 earthquake, one of the worst natural disasters in recent history. Approximately three million Haitians were affected, with more than 222,570 fatalities, over 300,500 injured and a staggering 1.3 million people displaced[1]. This radically impacted how HIV prevention programming moves forward in the three branches that were directly physically hit. For example, specially adapted interventions were required to meet the needs of camp dwellers.

1.2. Scope and reach of project

By June, 2014, the 12 sites were fully operational and reached 296,481 beneficiaries beneficiaries in nine sites of Haiti (Port de Paix, Cap Haitien, Fort Liberte, Ounaminthe, Cite Soleil, Petion-Ville, Anse-a-Pitres, Anse-a-Veau, Petit-Goave, Tabarre, Carrefour and Gonaives)

1.3. Project management

The project is implemented by the Haitian Red Cross (HRC) in collaboration with the American Red Cross (ARC) who is responsible to PSI, the lead consortium member. The PrevSIDA project is supervised onsite by the ARC HIV Technical Advisor based in Port-au-Prince.

1.4. Previous evaluation activities

PSI conducted a baseline mapping exercise to identify appropriate project sites, known as the “PrevSIDA Modified “Priorities for Local AIDS Controls Efforts” (PLACE) in Aug 2012.

In December 2012, PSI reported on their “Youth TRaC” study measuring condom use among high risk young women and men in Haiti. It measured both key sexual behaviors and experiences of the youth with a focus on the use of condoms and number of sexual partners.

Internally, a review of the ARC Health portfolio, including the PrevSIDA project, was performed in March 2013. HIV Delegate who arrived in Feb 2013 conducted M&E trips to four sites and conducted semi-structured interviews with HRC field staff in part to identify successes, challenges and recommendations to inform both the continuation of the project and its expansion into the three new sites.

A midterm evaluation was conducted with the use of an independent consultant in 2013. The report will be available for review

2. Final Evaluation

2.1. Purpose of Final Evaluation

To assess the project’s relevance, achievement, efficiency, effectiveness, impact, sustainability and to provide insights for the design and implementation of similar project in future.

2.2. Objectives of final evaluation

• Assess and describe the results, outcomes, impacts and effectiveness of project – intended and unintended, positive and negative, as well as the major factors that influenced the results
• Draw lessons learned and provide recommendations to better design and implement similar nature of the project in future.
• Investigate the level of coordination with internal and external actors in terms of program planning, resource management and implementation

2.3. Audience of final evaluation

2.4. Themain audience of the final evaluation includes ARC staff (management, project managers, and technical advisors in Haiti and National Headquarters in Washington, DC), relevant HRC staff, including those in the twelve project sites, and PSI program management in Haiti.

2.5. Coverage of the final evaluation

The evaluation is intended to cover all completed activities in all targeted geographic areas on purposive sampling basis

3. Evaluation criteria and questions

CriteriaMain evaluation questions Sub-questions Outcomes/ Impacts 1. Did the project achieve its intended outcomes?

2. How valuable are the outcomes for project beneficiaries? 2.1. Did the project achieve its intended outcomes, such as change in knowledge and practice in HIV Prevention?

2.2. What are the key impacts of the project according to the beneficiaries? How did the project impacted beneficiaries?

2.3. Did the project achieve any important unintended outcomes, either positive or negative?

2.4. What were the main causes that determined or contributed to intended or unintended outcomes?

2.5. How did the integration of several technical components/sectors impact on the project?

Effectiveness

3. How well are project activities being planned and implemented?

3.1. Were activities implemented as planned?

3.2. Was the implementation mechanism and procedures effective to deliver the intended interventions? If not, what were the gaps?

3.3. Were there any quality standards defined, procedures or protocols in place and were they followed in the implementation of activities? Relevance 4. How appropriate is project design? 4.1. Was the design of project interventions the most appropriate way to achieve intended outcomes? Were there other, more appropriate ways in which similar outcomes could have been achieved?

4.2. How appropriate were the overall PrevSida project goal and objectives, given the ARC capacity and Haitian Red Cross context at the moment of project design? Efficiency 5. How efficiently does ARC manage the project? 5.1. Were there any noticeable, verifiable instances of waste or inefficiency in the delivery of project activities in terms of resources and time?

5.2. How efficiently and timely was the project implemented by HRC per plan? Partnership 6. What effect has the project had on the partnership between ARC, HRC, PSI and other consortium members? 6.1. Has the project contributed to strengthening the partnership between HRC and ARC? If so, how and if not, why?

6.2. What were the key partnership learning for both HRC and ARC from this Project?

6.3. What approach would be the best to further promote partnership and strengthen the HRC role in projects design and implementation?

6.4. To what extent is HRC implementing the project in coordination with the other consortium members, and how can that coordination be improved? Coordination and Coverage 7. How well does actual project coverage compare to expectations and identified needs? 7.1. Did the project reach all the intended or targeted beneficiaries?

7.2. How well did the project coordinate with other organizations working on HIV prevention in the same geographic areas? Beneficiary participation and satisfaction 8. How satisfied are beneficiaries with the project? 8.1. In what ways did beneficiaries participate in the design and implementation of the project?

8.2. How satisfied are beneficiaries with project design, implementation and results? Sustainability and Replicability 9. How sustainable and replicable is the project model? 9.1. What exit strategies were incorporated into project design, were they implemented and to what extent are they going to contribute to sustainability?

9.2. How sustainable was any behavior change that was attributable to the project?

9.3. Are there any partners likely to continue the project activities in the project areas? If yes, under what circumstances or conditions?

9.4. Are there any project activities that communities report they can continue to carry out after closeout? Lessons learned 10. What can we learn from this project that would help inform future HIV prevention projects in Haiti? 10.1. What are the lessons learned in terms of project implementation, coordination, and beneficiary satisfaction?

10.2. What are the best practices procedures, materials, protocols or methodologies that can be incorporated in future HIV prevention projects?

10.3. How can donors better support such project in the future?

4. Scope of work and Evaluation design

4.1. Scope of work

The lead evaluator will be responsible for the following:

1. To procure necessary travel documents and visa in the country of origin (visa fee will be reimbursed) and insurance if required (ARC does not pay for insurance)
2. A working laptop
3. Printing of necessary soft copies of the desk review materials while in the country of origin (cost will be reimbursed)
4. To manage the local assistant consultant during evaluation and assign roles and responsibilities.

The lead evaluator will not be responsible for the following:

1. To arrange transportation for the field visit and to and from Hotel and Airport
2. To arrange accommodation while in Haiti

4.2. Methodologies

The Evaluation team will mainly use qualitative and rapid appraisal techniques. The following is a list of methodologies that are considered applicable; the list, however, should not be considered definitive and contractors are free to propose other methodologies.

1. Desk review of key project documents
2. Literature search and review of material on the environment in which the project operates
3. Review of evaluations and annual reports of other projects in the same sector and with the same/similar target population
4. Interviews with key ARC and HRC project staff
5. Interviews of representatives of other project stakeholders
6. Focus groups and key informant interviews of the beneficiary population
7. Physical site inspections

4.3. Discussion of inception report

Prior to conducting the evaluation, the Lead Evaluator will prepare and submit to ARC an inception report detailing the methodologies and work plan of the evaluation. ARC will provide an inception report template for this purpose. The inception report will be discussed with ARC staff and will be subject to approval prior to the start of field activities.

4.4. Reporting relationship

The consultants will report to Senior Accountability, Monitoring & Evaluation and Learning (AMEL) Delegate on all evaluation related issues. During field work, they will closely work with PrevSida team and coordinate and report to PrevSida Project Manager for all day to day issues.

4.5 International standards & Presentation of evidence

Standard evaluation and survey methodologies and good practices utilized in the international humanitarian community should be applied. Such resources should include but are not limited to those promulgated by the Active Learning Network for Accountability and Performance and the Organization for Economic Co-operation and Development. In particular, all findings and conclusions should be based on evidence which is presented in the evaluation report

1. Ethical Guidelines

It is expected that the evaluation will adhere to ethical guidelines as outlined in the American Evaluation Association’s Guiding Principles for Evaluators. A summary of these guidelines is provided below, and a more detailed description can be found at www.eval.org/Publications/GuidingPrinciplesPrintable.asp.

1. Informed Consent: All participants are expected to provide informed consent following standard and pre-agreed upon consent protocols.
2. Systematic Inquiry: Evaluators conduct systematic, data-based inquiries.
3. Competence: Evaluators provide competent performance to stakeholders.
4. Integrity/Honesty: Evaluators display honesty and integrity in their own behavior, and attempt to ensure the honesty and integrity of the entire evaluation process.
5. Respect for People: Evaluators respect the security, dignity and self-worth of respondents, project participants, clients, and other evaluation stakeholders. It is expected that the evaluator will obtain the informed consent of participants to ensure that they can decide in a conscious, deliberate way whether they want to participate.
6. Responsibilities for General and Public Welfare: Evaluators articulate and take into account the diversity of general and public interests and values that may be related to the evaluation.

5.1. Future use of data

All collected data will be the sole property of the American Red Cross. The contractor may not use the data for their own research purposes, nor license the data to be used by others, without the written consent of the American Red Cross.

r6. Expected activities and Deliverables

6.1 Expected activities (this level of effort is more or less fixed)

ActivitiesLead Consultant Local Consultant In/Out Haiti1. Desk review and literature search 3 1 Out 1. Develop and submit inception report for approval, and development of data collection tools 2 1 Out 1. Planning and preparatory work (with key ARC staff) 2 2 In 1. Site visits and field work 12 12 In 1. Preparation and presentation of preliminary findings to ARC/HRC team before departure. 1 1 In 1. Submission of draft report to ARC for comments 8 3 Out 1. Submission of final report to ARC after incorporating comments 1 1 OutTotal expected work days: 29 22

(The above days apply to official work days. During field works, consultants are not paid for weekends; expect the accommodations and per diem)

7. Deliverables

Deliverables1. Inception report 1. Finalized data collection tools 1. Draft report 1. Final report (with properly filed/archived copies of transcripts of all work documents, e.g. field notes)

8. Required qualifications

The following are the desired qualifications of the Lead Evaluator (national or international):

1. Masters degree in public health or relevant field from recognized university
2. Demonstrated experience in leading evaluations of health sector projects/programs (past experience in conducting evaluations of HIV prevention projects preferred)
3. Demonstrated professional experience in post-disaster/humanitarian environments
4. Demonstrated experience in qualitative data collection and analysis
5. Demonstrated experience in leading focus group discussions and conducting interviews of wide range of stakeholders
6. Professional work experience in Haiti preferred
7. Strong analytical thinker and skilled writer in English
8. Fluency in English and French is required, and knowledge of Creole preferred

The following are the desired qualifications of the Local Assistant(Haitian nationals only):

1. Bachelor or Master degree in public health or relevant field
2. Demonstrated past experience in working with the public health sector in the government or with non-government organizations in Haiti
3. Demonstrated experience in conducting rapid rural appraisals using focus group discussions, key informant interviews of beneficiaries, project staff and other stakeholders
4. Past experience in participating in independent external evaluations of humanitarian projects highly desirable.
5. Fluent in English and French, native Haitian Creole speaker required
6. Application and selection details

9.1 Application materials

The proposal should include the following six items. Please note that any proposal which does not contain all six items will be rejected.

1. Summary of experience ( 1 page maximum)
2. Example of one full evaluation report written in English language. Do not include annexes in the report.
3. Detailed CVsof all professional (s) who will work on the evaluation.
4. Professional references:please provide three references from your previous clients.
5. Daily rate: please mention the proposed daily rate for each contractor in USD.

9.2 Summary of experience

This document should be no more than one page and should include the following information:

CriteriaDetails (this column can be deleted for more space) Evaluator 1 Evaluator 2

(if applicable) Experience in leading project/program evaluations Number of evaluations led (with dates, locations and names of organizations); number of evaluations served as team member Experience in qualitative methods Numbers of years of experience;

Tools/methods used in past Experience in Public Health/HIV&AIDS projects/programs Number of years of experience;

Titles of positions held;

Countries worked in;

Organizations worked for Experience in post-disaster / humanitarian context Number of years of experience;

Countries worked in Professional experience in Haiti Number of years of experience;

Organizations worked for Language proficiency Proficiency in English, French and/or Haitian Creole

Note: Please see the whole document attached.

How to apply:

9.3 Application procedures

Interested candidates should send their applications to amrc.evaluation@gmail.com . Please put the following in the subject line: “Application for PrevSIDA Project Final Evaluation. All applications should be submitted in pair of both lead and local assistant consultants. Single applications will not be entertained.

A complete application will be one with all materials listed above in one single file in either Word or pdf format. The title of this file should be the last name of the Lead Evaluator (e.g. if the Lead Evaluator is named Jon Snow, the title of the application document should be “Snow”.).

Application received after the deadline and incomplete applications will not be accepted.

9.4 Deadline for applications

11:59PM EST, 20 October 2014

Note: Please see the whole document attached