Based on the feedback received from the conference call, Social Impact has amended the original RFP as seen on the RFP Amendment document. Uploaded are also responses to the questions received in regard to the RFP for further clarification.
- Request for Proposals (RFP): Amendment I
- Questions & Answers, 1st Round
- Questions & Answers, 2nd Round
Request for proposals (RFP)
|Project:||Evaluation of the MCC Lesotho Compact Metolong Program (MP) and Urban and Peri-urban Water (UPUW) Activity|
|Funder:||Millennium Challenge Corporation (MCC)|
|RFP Release Date:||Thursday, May 31, 2018, 17.00 US Eastern Time|
|Deadline for Questions:||Tuesday, June 5, 2018, 17.00 Maseru time|
|Answers to Questions:||Released Wednesday, June 6, 2018, 17.00 US Eastern Time|
|Deadline for Proposals:||Friday, June 15, 2018, 17.00 ET|
|Contact:||[email protected] ; [email protected]|
|Annexes||Annex A: Budget Template|
|Downloadables||Request For Proposals (PDF)|
Firms interested in bidding on any part of this RFP are asked to express interest in writing (to both email addresses above) by the deadline for questions, June 5, 2018, 17.00 Maseru time. No proposal materials are required with this expression of interest. The expression of interest is requested in order to assist SI for planning purposes.
|Expression of interest||Tuesday, June 5, 2018, 17.00 Maseru time|
To facilitate a clear understanding of the scope of work and provide an opportunity for clarification on answers to questions, SI will host a conference call on June 7th, following the completion of the question and answer period. Firms who express interest in bidding by the date specified above will receive conference call information for the bidders conference call.
|Bidders Conference Call||Thursday, June 7, 2018, 14.00-15.00 Maseru time|
I. Project Background
Social Impact (SI) was contracted by the Millennium Challenge Corporation (MCC) to conduct an evaluation of large-scale water infrastructure projects implemented through the Millennium Challenge Account-Lesotho (MCA-L) as part of the MCC’s Compact with the Government of Lesotho (GoL), between 2008 and 2013.
Social Impact’s evaluation is broadly concerned with the implementation, impact, and sustainability of these projects, which include the Metolong Program and the Urban and Peri-Urban Water (UPUW) Activity. The Metolong Program involved the construction of a new dam, water treatment works, and downstream conveyance system to increase the quantity, quality, and reliability of water supply to the city of Maseru. The UPUW Activity involved a variety of water infrastructure installations, upgrades, and rehabilitations across several urban sites in Lesotho, including: urban and peri-urban Maseru, Semonkong, Mafeteng, Mohale’s Hoek, Quthing, Qacha’s Nek, Leribe, Butha-Buthe, Mokhotlong, and Mapoteng.
The evaluation is meant to generate lessons learned for future water infrastructure projects including future MCC Compacts implementing similar programming. To date, Social Impact has completed one part of the evaluation (the process evaluation), assessing the implementation of the Metolong Program and UPUW Activity. The next phase of the evaluation (the summative evaluation) is focused on impacts of the projects, particularly on households in the urban sites listed above. Data collection for the summative evaluation will be conducted between July and November 2018.
II. Purpose of the Request for Proposals (RFP)
Data collection for the summative evaluation will be conducted between July and November 2018. Social Impact is now looking to subcontract a local data collection firm to carry out quantitative and qualitative data collection for the summative evaluation.
The findings from the qualitative data collection will be used to inform the final parameters of the quantitative data collection. The expected parameters of the quantitative data collection based on the best information available to SI at present are outlined in this RFP (see “Scope of Work”).
Qualitative and quantitative data collection will be carried out as discrete Tasks, with their own timelines, work plans, and deliverables. Qualitative data collection will be completed before the start of any quantitative data collection.
Firms are invited to bid on one or both of these Tasks. Social Impact expects some efficiencies may be gained by having a single Subcontractor conduct both Tasks, and firms are encouraged to think about potential efficiencies and cost savings when developing technical proposals. However, there will be no penalty in assessing proposals from firms bidding on only one Task. If both Tasks are awarded to a single Subcontractor, the quantitative data collection will be structured as a Contract Option to be exercised at SI’s discretion, pending the results of the qualitative data collection and dependent on satisfactory performance of the Subcontractor on the qualitative Task.
III. Scope of Work
The scope of the Subcontract will include qualitative and quantitative data collection in the following urban areas of Lesotho: Maseru urban, Maseru peri-urban – including Mazenod, Roma, Morija, and Teyateyaneng – Semonkong, Mafeteng, Mohale’s Hoek, Quthing, Qacha’s Nek, Mokhtolong, Butha-Buthe, Leribe, and Mapoteng. The parameters of data collection vary by site; further details are provided below. Offerors that bid on both Tasks will be required to prepare separate technical proposals for each.
TASK 1: QUALITATIVE DATA COLLECTION
Qualitative data collection includes two Activities: focus groups (FGDs) and key informant interviews (KIIs) (see Table 1). The objectives of qualitative data collection are to (a) assist the SI evaluation team in finalizing the parameters of quantitative data collection, which will only begin after the qualitative is completed, and (b) to collect perspectives on water supply and topics related to this evaluation, through FGDs and KIIs. All sites will be included in qualitative data collection.
Table 1. Summary of Task 1: Qualitative
|TASK||ACTIVITY||Respondents||Sample Size||Duration per interview|
|Focus Group Discussions||Conducted with local community members||48||120 minutes|
|Key Informant Interviews||Local chief(s) in each site||TBD*||45 minutes|
*Number of chiefs to be interviewed in each urban site will vary; Offerors are asked to provide estimates with justification, for the number of chiefs needed to be interviewed in each urban site, such that the evaluation team can obtain reliable information about internal migration dynamics in each urban site over the last ten years.
Focus Group Discussions
Focus groups will be conducted with different types of households, summarized in Table 2.
Table 2. Focus group sample size by site and type
|UPUW site||#||Focus group types in each site|
|Maseru urban (1 site)||4||Connected, Metolong-supplied: 1
Connected, Not Metolong-supplied: 1
Newly connected: 1
|Maseru peri-urban (4 sites)
Roma, Morija, Mazenod, Teyateyaneng
|8||Newly connected: 1
|Semonkong (1 site)||4||Newly connected: 2
|Other UPUW (8 sites)
Qacha’s Nek, Quthing, Mohale’s Hoek, Mafeteng, Leribe, Butha-Buthe, Mokhotlong, Mapoteng
|32||Newly connected: 2
Residents of rural villages using free water distribution from the Metolong downstream conveyance
|TOTAL||50 FOCUS GROUPS|
In Maseru, focus groups will be organized for connected households (households with a tap on premises, connected to the WASCO network) residing in areas supplied by the Metolong supply, and for connected households in areas not supplied by the Metolong supply. SI has information about which townships in Maseru urban are supplied or not by the Metolong supply and will provide this information to the successful Offeror following award. Households can be sampled randomly for participation from the WASCO customer database for these focus groups. SI will conduct the sampling and provide the Subcontractor with information for sampled households, so that they may be contacted and invited to participate in the focus groups. A back-up sample will be provided to mitigate the potential for households declining to participate.
In all urban sites, focus groups will be conducted with newly connected and unconnected households. Newly connected households are those which have obtained a connection on premises (in the dwelling or on their plot) to the WASCO network after the completion of the MCC water projects. In most cases, the completion date will be 2013. SI will sample households for these focus groups from the WASCO database, and will provide the sample to the successful Subcontractor to contact households and invite them to participate; as above, a back-up sample will be provided in case some households decline to participate. Unconnected households include households that do not have a connection to the WASCO network on premises.
Lastly, a small number of focus groups will also be conducted in areas between the Metolong Dam and Water Treatment Works and the city of Maseru, with residents of rural villages that reside near the downstream conveyance which transfers the supply from Metolong to the city of Maseru. Currently, these areas are provided water free of charge through off-takes from the downstream conveyance, and thus represent a potentially unintended impact of the project, which the SI evaluation team will explore through focus groups. Respondents for these focus groups do not necessarily have to be sampled from the same village; inviting participants from different villages may vary the perspective gained from the focus groups.
All focus group respondents must include those that have resided in the study site for at least ten years (i.e. they should have been residents in this area before any of the water projects had been started). Unconnected households and households from the villages along the downstream conveyance cannot be sampled from the WASCO database, by definition.
Offerors must include in the technical proposal a proposed approach to sampling respondents for these focus groups that ensures broad representation of those households and does not create bias in terms of the perspectives included. Measures specifically taken to recruit participants must be described in the technical proposal. Past experience in conducting this type of recruitment and sampling will be scored favorably.
A draft focus group discussion guide has been prepared for each group, which will be provided to the successful Offeror following award for input and translation. Topics covered in the focus group discussions will include reasons for connecting or not connecting to the network, perceived changes in water supply, reliability, and quality, general water and sanitation practices, perceived economic and health outcomes, and internal migration and local housing market dynamics. Illustrative topics include the following:
Table 3. Focus group topics, by type
|Theme||All connected households||All unconnected households|
|Reasons for connecting
Potential spillover effects
|– Reasons for requesting a connection
– Previous water source situation
– Changes in water service since interventions
– Notable ‘shocks’ (e.g. major interruptions)
– Expectation of benefits versus reality
– Perceived water quality; changes over time
– Changes in water use & consumption patterns; seasonality
– Alternative source use (current and past); reliability, quality, convenience, and price
– Perceived benefits over time
|– Demand for connection, willingness to pay
– Current water source use; seasonality
– Factors influencing source choice; reliability, quality, convenience, and price
– Seasonality of water use practices
– Barriers to connection (now, prior)
– Indirect benefits from increases in access among others or improvements to supply (spillover)
– Perceived changes over time in any of the above
|Other WASH practices||– Household water storage and treatment
– Sanitation & hygiene in household
– Seasonality of these practices
– Time or money cost of these coping behaviors
– Perceived changes in these practices over time
|Other relevant factors||– Internal migration over the last decade
– Housing market and rental prices
– Motivation for living in city of residence, preferred neighborhoods in city of residence
– Constraining factors from moving, if that would be preferred
– Knowledge of the MCC Compact & interventions
– Experience of other interventions and WASH programs
|Perceptions of the impact of the projects||– Perception of time savings, cost savings, diarrheal illness outcomes
– Re-allocation of time or costs saved
– Specific individuals who have benefited (e.g. females, children)
– Unanticipated effects
All focus groups should include 6 to 8 participants, in order to ensure a balance between a sufficient number of participants so that different perspectives can be represented, while ensuring a small enough number that the focus groups may be conducted in a way that all participants are able to participate meaningfully. Targeted respondents for the focus groups include female members of households – either heads of household, or spouses of heads of household, as they are expected to be the most knowledgeable about water affairs in the household, with respect to the topics addressed in the focus groups.
Deliverables for the FGDs will include the full Sesotho and English transcriptions of the FGDs (made using audio recordings of the FGDs), as well as a brief summary of each FGD (SI can provide a suggested template), with interviewer notes to highlight the main themes arising in each FGD. Offerors should detail their proposed approach for providing these transcriptions on a timely basis, to ensure that they can be done as soon after each FGD as possible.
Key Informant Interviews
Key informant interviews will be conducted with local chiefs, who are likely to be knowledgeable about movements in and out of the study areas over time. The purpose of the key informant interviews is primarily to understand dynamics around internal migration and changes in the housing market in specific urban areas over the last few years. Key informant interviews will include topics related to internal migration and local housing market dynamics, as well as general topics related to water supply and water and sanitation programs in the relevant sites.
Social Impact does not have an estimate of the total number of chiefs to interview in each urban site. Offerors are asked to provide estimates with justification, for the number of chiefs needed to be interviewed in each urban site, such that the evaluation team can obtain reliable information about internal migration dynamics in each urban site over the last ten years.
Key informant interviews are expected to last approximately 45 minutes and will be conducted in Sesotho. Key informant interviews should be conducted by personnel recruited by or employed by the Offeror, which are qualified in the conduct of qualitative interviews and possess the experience and professionalism necessary to conduct these interviews. A draft key informant interview guide has been prepared, which will be provided to the successful Offeror following award for input and translation.
Deliverables for the KIIs will include the full Sesotho and English transcriptions of the KIIs (made using audio recordings of the KIIs), as well as a single brief summary of the chief KIIs for each urban site to highlight the main themes arising in each urban site. Offerors should detail their proposed approach for providing these transcriptions on a timely basis, to ensure that they can be done as soon as possible after the KIIs in each site.
TASK 2: QUANTITATIVE DATA COLLECTION
Quantitative data collection involves two types of surveys, referred to as (1) “household surveys” and (2) “customer surveys”. For some sites, a listing activity is expected to be necessary, prior to conducting the household survey. A summary of the Activities included under Task 2 is in Table 4 below.
SI strongly prefers that data collection for Task 2 be carried out using electronic data collection. SI’s preferred electronic data collection platforms are SurveyCTO or Open Data Kit (ODK).
Offerors are asked to highlight any relevant experience with electronic data collection in the past performance section of the technical proposal, and specific experience with the preferred data collection platforms mentioned above, if applicable.
If Offerors do not propose to conduct electronic data collection or cannot demonstrate experience with it, they should explicitly indicate how they will mitigate corresponding risks to data quality. SI will conduct and maintain responsibility over programming electronic data collection tools, while the Subcontractor will be responsible for managing devices and properly managing the electronic tools for all teams involved in carrying out Task 2.
Devices should be furnished by the Offeror; Social Impact will not provide devices for electronic data collection.
Table 4. Summary of Task 2: Quantitative
|TASK||ACTIVITY||Anticipated Sites & Sample||Sample size||Duration|
|Listing Surveys||Sites: Roma, Morija, Teyateyaneng, and Semonkong||n/a: all enumeration areas within a certain distance from WASCO network||15 minutes|
|Household Surveys||Sample will be taken from the listing, and will include newly connected & unconnected households||3,760
Maseru urban: 740
Maseru peri-urban: 22,520 (630 per site)
|Customer Surveys||Sites: Mafeteng, Mohale’s Hoek, Quthing, Qacha’s Nek, Mokhtolong, Butha-Buthe, Leribe, and Mapoteng
Sample: households with connection to public distribution network.
(175 per site)
The above information summarizes the expected parameters of quantitative data collection given information available to SI at present. After Task 1 is complete, SI will determine whether the expected parameters for the quantitative data collection will proceed as planned, or whether adjustments need to be made in terms of which sites are included in the Activities under Task 2. Any modifications made on the basis of Task 1 will be incorporated into the Subcontract (or Subcontract Option if awarded to a single firm) for Task 2.
Social Impact anticipates that it will be necessary to conduct a listing activity in the following urban sites: Roma, Morija, Teyateyaneng (TY), and Semonkong. Listing will take place for enumeration areas that meet the eligibility criteria for inclusion in the study, on the basis of their distance from the WASCO network. SI has the information needed to identify these enumeration areas and will supply the list of enumeration areas to the successful Subcontractor following award.
The purpose of the household listing is to obtain an up to date sampling frame for these areas, from which a sample can be drawn for the household survey that will take place in each site. Offerors are encouraged to propose approaches to the household listing activity that ensure quality while achieving cost or time savings. For example, firms are encouraged to propose approaches that make use of any potential existing listings or households registers that can be used instead of conducting a listing survey from scratch, or that can be used as a basis for updating, to cut down the time needed to conduct the listing.
The household listing should result in a comprehensive list of households for each enumeration area included, that lists the name and contact for each household, along with the number of years that the household has lived in the area, whether or not the household has any children under five years of age, and whether or not that household has a WASCO connection on premises. For households with a connection, the listing survey should also collect the household’s WASCO account and meter number, to be able to link with the WASCO database for sampling purposes.
Firms should describe their approach to conducting household listing activities successfully in the technical proposal, including how it will be conducted in a way to ensure that there is no bias in the resulting sampling frame (e.g. all households in the EA are included with the required information).
In Maseru urban, Maseru peri-urban areas (Roma, Mazenod, Morija, and Teyateyaneng), and Semonkong, Social Impact expects to conduct household surveys with those that were and were not exposed to the water projects. Depending on the site, the definition of “exposed to the water projects” varies, with definitions provided below and corresponding with the definitions used as part of Task 1.
|Site||Households to be surveyed|
|Maseru urban||Households residing in areas supplied by the new Metolong supply (“treatment”), and households residing in areas not supplied by the new Metolong supply (“comparison”)|
|Maseru peri-urban and Semonkong||Households with a new connection installed after the interventions were completed (“treatment”), and comparable households still unconnected after the interventions (“comparison”)|
A selection of general topics covered in the customer survey will include household demographics, assets, water consumption and expenditures, time spent collecting water, other aspects of household water use and coping costs, and water-related illness as well as satisfaction with WASCO service. SI has drafted a version of the household survey that will be revised based on the results of Task 1. The draft tool will be shared with the successful Offeror following award, for input and translation. It is possible that as part of the household survey, SI will also implement low-cost water quality testing. The details of this activity are under development; if included, the Subcontractor will be required to obtain water samples from a sub-sample of households and test them in the field, and for filling in a separate water quality protocol associated with each water quality test. The details of this activity are not planned at this stage and thus Offerors are not expected to respond directly in the technical or cost proposals at this stage for water quality testing. Nonetheless, Offerors are encouraged to highlight any relevant experience conducting water quality testing through past performance reports.
The targeted respondent for household surveys is the same as the qualitative data collection – the senior female member of household, who is expected to be the most knowledgeable about water affairs in the household. Household surveys are expected to last no more than 90 minutes. Pre-testing and piloting activities will be used to test the duration of the survey, with the aim of achieving an average of 75 minutes per household survey, to ensure important topics are addressed while avoiding respondent fatigue. Survey duration will be monitored during fieldwork as part of SI’s quality control measures.
Customer surveys will be implemented in any urban site which is not included in the “household surveys” above. Customer surveys are also household surveys but are referred to as customer survey as the tool will be an abbreviated version of the full household survey, and the sample will include only households that are current customers of WASCO. In other words, in the urban sites included in this Activity, there will be no “comparison” households as part of the design, and thus no comparison households to survey.
In all areas that will implement only the abbreviated customer survey, SI will conduct sampling directly from the WASCO database, and will share the final list of sampled households with the subcontractor. As in the household survey, a selection of general topics covered in the customer survey will include household demographics, assets, water consumption and expenditures, time spent collecting water, other aspects of household water use and coping costs, and water-related illness as well as satisfaction with WASCO service. SI will draft the Customer Survey during the qualitative data collection, to be finalized based on the results of Task 1 after which it will be shared with the successful Offeror, for input and translation.
It is possible that as part of the customer survey, SI will also implement low-cost water quality testing. The details of this activity are under development; if included, the Subcontractor will be required to obtain water samples from a sub-sample of households and test them in the field, and for filling in a separate water quality protocol associated with each water quality test. The details of this activity are not planned at this stage and thus Offerors are not expected to respond directly in the technical or cost proposals at this stage for water quality testing. Nonetheless, Offerors are encouraged to highlight any relevant experience conducting water quality testing through past performance reports.
Preparations for Data Collection
Subcontractors will be required to undertake a number of activities in preparation for data collection. For both Tasks 1 and 2, this will include:
- Comment on data collection tools and protocols: Subcontractor will review and provide feedback, in the form of in-line comments and suggested revisions, on SI’s data collection protocols and instruments, to ensure that they are properly contextualized, and to ensure that SI has properly considered likely eventualities. The Subcontractor will review and provide feedback on SI’s data collection instruments, and suggest revisions as needed for context, flow, or other aspects This will include focus group discussion guides, key informant interview guides, listing survey, household survey, and customer survey. It will also include quality assurance tools (see Quality Assurance section below for more information). Each of the main tools (e.g. tools other than QA tools) will have an associated protocol that describes how the tool should be administered. A water quality tool and protocol may be added later; as noted above, water quality testing activities should not be included in the technical or cost proposals at this stage.
- Develop manuals for field staff: Subcontractor and SI will collaboratively develop comprehensive manuals for field staff. SI must have a chance to review and approve final manuals at least 5 business days before the start of training. In some cases, SI may develop draft manuals, that will then be provided to the Subcontractor for review and further input. The minimum required manuals that must be developed for this activity include: qualitative interviewer manual (with appendices including manuals for each tool), quantitative enumerator manual (with appendices including manuals for each tool), and supervisor manual.
- Data management plan: SI will draft and share a data management plan for each Task with the successful Subcontractor, for input and comment. It will be finalized by SI.
- Translate and Back-Translate instruments: Translations should either be completed by a team of two concurrently, reconciling any differences together afterward, or by two individuals separately with a third person reconciling differences afterward (Offerors should justify their choice of translation method in the technical proposal). Back-translations should be completed by a third party, who was not involved in the translations. SI will review back-translations and ask Subcontractor to make changes to the instrument translations as needed based on the results, in order to finalize instruments. This process must occur before the pre-test begins.
- Obtain local research clearances and permits: Subcontractor will work with SI to lead the submission of applications for local research clearances as needed (e.g. local IRB or research clearance body), as well as obtain any relevant permissions needed in order to enter specific sites in order to collect data (e.g. local government administration letters of introduction).
Pretesting, Training, and Piloting
- Pretesting: Subcontractor will be required to conduct pretesting for all qualitative and quantitative instruments (except for the KII guides for chiefs). Pretesting is focused on the flow, translation, and logic of the instrument. Pre-testing must be done with a sample of respondents/participants that are similar but not part of the sample frame for the data collection. Each type of qualitative FGD tools will be pre-tested with one group; each of the household surveys will be pre-tested by being administered to five households. Pre-testing should be carried out by local data collection firm staff, prior to training. Ideally, pre-testing would be carried out by field coordinators, supervisors, or field managers, so that the leadership of the data collection activity will have had an opportunity to become familiar with the tools prior to training enumerators.
- Training: Training for each component will take place prior to the associated data collection event. Offerors are required to specify the recommended duration and content of field staff training as part of the technical proposal.
- Offerors shall describe in their technical proposal their approach to assessing interviewers’ and enumerators’ readiness to conduct data collection during and after the training, i.e. specifically how they will determine non-performing trainees. It is recommended that more individuals be trained than will be required for each data collection activity, so that top-performers are selected, as well as to maintain a pool of back-up interviewers/enumerators; non-performing trainees will not be selected as part of the main or back-up teams.
- No interviewer/enumerator is to be sent to the field until he/she has demonstrated sufficient understanding of the protocols. Representatives of SI will assist with the training and may test enumerators as needed and may require, at their discretion, replacement of enumerators deemed to be performing inadequately in training or in the field.
- Enumerator performance shall also be monitored closely by the successful bidder during data collection, with independent checks also carried out by SI and SI’s local coordinator. SI requires that subcontractors clearly describe their proposed corrective actions for non-performing interviewers/enumerators during data collection. See additional requirements under “Quality Assurance” below.
- Piloting: Piloting will be done as part of interviewer/enumerator training. Piloting is focused on the entire process of data collection (including practicing and testing the electronic data collection survey) and is meant to be a “real-life” practice of the data collection. In this way, it is different from the pre-test which is specifically focused on the tool. Following piloting, revisions to the tools may be made but it is not expected that major changes to the tool will be needed. For qualitative, piloting will involve 1 focus group of each type. For quantitative surveys, piloting should be done on a sample of at least two households per enumerator. As with pre-testing, piloting must be done with a sample of respondents/ participants that are similar but not part of the sample frame for the data collection.
Subcontractors will be required to implement a series of preventive, routine, and post-hoc quality control measures. Minimum required quality control measures are listed below.
In addition to the requirements for the Subcontractor, SI will conduct independent quality assurance for the duration of this activity. Subcontractors will be required to respond in a timely manner to SI questions regarding data quality control checks. Subcontractors will also be required to report on quality control activities and results in regular reports to SI during data collection, as well as within the final data collection completion report (see “Reporting” section below).
Required Quality Control Measures, to be led by Subcontractor:
- Comprehensive Training: Error prevention is the most important component of quality control. Interviewers/enumerators should be trained thoroughly on all tools relevant for their activity during their respective training. For Task 2, more enumerators than will be fielded must be trained so that the best-prepared enumerators carry out data collection, and to ensure a pool of back-up enumerators to draw upon as needed. (Tasks 1 and 2)
- Translation & Backtranslation: Ensuring accurate translation, so that the intended meaning is communicated, improves data quality. As above, translation and back-translation must be completed by different individuals, and data collection must not begin until these tasks are completed and the tools are finalized. (Tasks 1 and 2)
- Daily team debriefs: Check-ins with the interviewers/enumerators and field management staff (field manager, coordinators, supervisors) to review any challenges faced, allow for questions and clarifications, and provide feedback to the wider group. These are especially important early in the data collection activities to ensure that proper interviewing habits are formed. (Tasks 1 and 2)
- Supervisor checks: Supervisors will check their teams’ forms before they are submitted to the server, to ensure completeness and spot-check for errors. These are especially important early in the data collection activity to ensure that proper interviewing habits are formed. (Tasks 1 and 2)
- Phone/Skype check-in with SI: Upon SI or Subcontractor request, SI and the Subcontractor may hold regular and/or ad hoc check-ins to discuss the result of daily debriefs and supervisor checks, and any other feedback on the early stages of data collection. (Task 2)
- Accompaniment: Subcontractor will ensure that at least 5% of interviews are directly observed by a supervisor or other senior member of the team. All enumerators should be directly observed at least once during the first week of data collection. Observations will be summarized in an accompaniment form developed by Social Impact. (Task 2)
- Co-enumeration: Subcontractor will co-enumerate at least one interview per enumerator during the first two weeks of field work (fill in a duplicate version of the interview form concurrently during direct observation). Thereafter, at least 1% of interviews should be co-enumerated. SI also reserves the right to request co-enumeration for specific enumerators if questions are raised during SI’s regular data quality checks. (Task 2)
- Back-checks: Subcontractor will conduct back-checks on 10% of the total sample, using a back-check tool developed by Social Impact. Back-check surveys should not be made available to enumerators. Back-checks should be conducted by separate teams from the enumerators by phone, and the Subcontractor is expected to work collaboratively with SI to determine the allocation of back-checks throughout data collection (e.g. all enumerators should have 10% of their surveys back-checked within the first two weeks, and thereafter back-checks may be specifically targeted to certain interviews or enumerators). (Task 2)
- Additional back-checks or re-interviews: As requested by SI, based on the Subcontractor’s quality control or SI’s independent quality control, the Subcontractor will conduct additional back-checks or full re-interviews in order to validate any issues raised with individual surveys. (Task 2)
Quality Control Measures conducted by Social Impact:
- Survey programming quality control: SI will program various quality control measures into the electronic survey, for the household survey and customer survey. These may include: speed limits, logic checks, audio audits, and other internal validations. These measures will prevent errors by enumerators and ensure logical branching of the survey. The final set of validations will be established following enumerator training and the pilot activity, incorporating input from the Subcontractor and enumerator teams. As the need arises during data collection, SI may institute or alter internal validations to correct any issues faced in the field. These or any other changes made to the electronic data collection instrument will require all devices used in the field to be updated with the new instrument. SI will notify the Subcontractor of any such changes made, and it is required that following this notification, all devices must be updated before continuing with surveying, to ensure that the most up to date version of the instrument is being used. Survey programming quality control may include random audio audits to check that enumerators are conducted their assigned interviews. Audio clips will be reviewed for quality control and destroyed thereafter. Offerors are asked to note in the technical proposals whether this may cause any unintended consequences for respondent agreement to participate in the interview. (Task 2)
- Routine data checks: For Task 1, Social Impact will review qualitative transcripts, and return questions about the content, transcription, or translation back to the Subcontractor. For Task 2, Social Impact will conduct independent quality checks of the data downloaded directly from the server, at least weekly (more frequently during the first 2-3 weeks of data collection). (Tasks 1 and 2)
- Independent back-checks: SI’s local coordinator will also conduct random back-checks via phone or in-person. Results of SI’s quality checks which generate questions or the need for clarification, back-checking, or re-interviewing, will be communicated with the Subcontractor. The Subcontractor will be required to respond to these questions within 2-3 business days of receiving them. (Task 2)
- Analyze quality control form data: SI will also analyze co-enumeration, accompaniment, and back-check data on an ongoing basis as it is received and will summarize any questions or feedback for the Subcontractor from each check. Subcontractors will be required to respond to these questions within 3-5 business days of receiving them. (Task 2)
Ethics, Respondent Protection, and Data Security
Subcontractors are required to abide by Social Impact’s respondent protection and data security protocols (to be provided upon award). Subcontractors will be given an opportunity to comment on the protocol and provide feedback that allows SI to better contextualize the protocol (without modifying SI’s “required minimums”).
The following is a list of major measures that will be required of Subcontractors, in order that they comply with SI’s respondent protection and data security protocols:
- ensuring several basic protections for devices and other equipment or materials used during data collection (SI to provide to Subcontractor with requirements following award),
- ensuring that all enumerators are thoroughly trained in ethical and professional standards in the field,
- ensuring that data collection staff administer the informed consent properly, reading in full as written and answering respondent questions before beginning interviews or surveys,
- ensuring that all data collection staff sign a non-disclosure agreement and privacy protection promise (to be provided upon award) demonstrating their understanding that respondent information shared with them during an interview is considered confidential and should not be shared,
- ensuring that swift corrective action is taken when any data collection staff are found to be behaving in ways that are not ethical or professional,
- abiding by data security measures agreed upon with Social Impact, including the physical security of any hard-copy materials and the secure storage, transfer, destruction, and access of digital materials.
Personnel and Staffing Plan
Bidders should provide CVs for required team members (key personnel) positions as listed below, which meet the minimum qualifications. In the Personnel section of the technical proposal, bidders should also describe their recruitment strategy for other field staff and should specify the total number of enumerators and supervisors that will conduct the activities under each Task.
Task 1: Qualitative
|Team Leader (1)||Required 5-10 years of relevant experience managing qualitative data collection exercises in Lesotho. Experience in urban WASH and topics described in this RFP is a plus.|
|Qualitative Interviewers (2+)||8+ years of experiencing conducting, analyzing, and reporting on qualitative interviews. Experience conducting focus groups is required. Ideal candidates may be academics, research consultants, evaluators, etc. Academic training in a field relevant to qualitative research required. Experience or exposure to topics related to urban WASH, internal migration, and others described in this RFP a plus.|
A minimum of two qualitative interviewers is required for Task 1, but Offerors are free to propose additional, based on their technical proposals. Focus groups should be conducted by two interviewers simultaneously, so that one can facilitate while the other takes notes. Offerors proposing more than two qualitative interviewers should justify their proposal in terms of any logistical, cost, or other trade-offs considered.
Task 2: Quantitative
|Team Leader (1)||Required 5-10 years of relevant experience managing large-scale quantitative data collection exercises in Lesotho. Experience managing electronic data collection activities strongly preferred. Experience in urban WASH and topics described in this RFP is a plus.|
|Field Manager (1)||5+ years of experiencing coordinating large scale quantitative field data collection. Experience managing electronic data collection activities strongly preferred. Experience in urban WASH and topics described in this RFP is a plus.|
|Field Coordinators (3+)||3+ years of experiencing coordinating large scale quantitative field data collection. Experience managing electronic data collection activities strongly preferred. Experience in urban WASH and topics described in this RFP is a plus.|
Other personnel required for the successful conduct of Task 2 include field supervisors and enumerators, but CVs of those individuals shall not be included with the technical proposals.
Offerors must justify their proposed Supervisor to Enumerator ratio for the quantitative data collection, but this ratio shall not exceed 5 enumerators to each supervisor.
Bidders should describe their approach to ensure that well-qualified enumerators are hired. Enumerators should have relevant interviewing experience in household interviewing, especially using similar instruments to this evaluation.
Recruitment and staffing procedures should be outlined in the technical proposal, along with contingencies for staff replacement, should the need arise, during data collection. Offerors are welcome to submit additional CVs as justified for the proposed teaming structure, with position name and role to ensure no duplication of effort. CVs for such individuals are not required and will not be scored.
For each Task, Subcontractors will be required to submit the following reports. Social Impact will provide report templates as guidance to the Subcontractor following award.
- Inception Report/Work Plan: Includes plan for project management, start-up, translation/pre-testing, training and piloting, sampling and data collection, respondent protection and data security, quality assurance plan, risk reduction plan, reporting, deliverables, and Gantt chart demonstrating timeline for all data collection activities and deliverables.
- Weekly/Bi-weekly Reports: Summarizes progress on sampling, data quality assurance, measurement challenges, and requests for assistance over the past week/two weeks. Weekly reports will be required throughout Task 1. For Task 2, bi-weekly reports will start after enumerator training.
- Pretest Report: Includes summary of pretest process, general observations on the pre-test, and instrument-specific observations on the pre-test including suggested revisions.
- Training & Pilot Report: Includes description of activities conducted, details on recruitment of supervisors and enumerators in training, observed performance of supervisors and enumerators in training, summarized results of piloting, average duration of interviews, edits to instruments, protocols, or manuals, and a final pre-test and pilot dataset.
- Final Report: Summarizes final dates and figures for preparation and execution of data collection. Characterizes final sample and corresponding challenges. Summarizes actual data quality control measures taken compared to target and identifies outstanding challenges. Identifies other remaining measurement challenges. Includes final copies of protocols, manuals, sampling materials, weighting procedures, and the complete dataset.
Bidders should submit a summary of three past performance reports, including contact information for references. SI reserves the right to contact references provided in these past performance reports. Demonstrated experience conducting data collection for WASH sector and collecting data in Lesotho required. Demonstrated experience facilitating focus group discussions required for consideration for Option 1. Demonstrated experience successfully conducting household surveys of at least 3000 households using electronic data collection required for consideration for Option 2. Experience working on a USG-funded evaluation is preferred but not required. This experience should reflect institutional capacity, not just that of individual team members. Of particular importance is relevant work in the management and implementation of data collection, with methodologies and with populations/locations as those in this scope of work.
IV. Deliverables & Payment Schedule
The Subcontractor will submit invoices according to the payments listed below. Weeks correspond to the start of each Task. Task 1 weeks, for example, are relative to contract effective date for the Task 1 contract. Task 2 weeks are relative to the start of Task 2, after Task 1 is completed. Submission dates for each deliverable invoiced and SI approval dates should be specified on the invoice for each payment. Invoices cannot be submitted prior to SI accepting deliverables/milestones in writing. The deliverable and payment schedule for each of the two Tasks is provided below.
TASK 1 DELIVERABLE & PAYMENT SCHEDULE
|Phase||Payment||Deliverables / Milestones||Week
|1: Prep||1||Qualitative Inception report with work plan||1||10|
|1: Prep||2||Feedback to SI on instruments and protocols
(includes FGD guides & KII guides)
Final instruments translated & back-translated
|2: Fieldwork||3||Training Manuals finalized with input from SI
Interviewer Training completed
Pilot & Training Report(s)
|2: Fieldwork||4||All final Transcripts (English and Sesotho) and summaries; all quality assurance reconciled||5-8||30|
|3: Reporting||5||Weekly progress reports||1-9||n/a|
|3: Reporting||5||Data collection completion report||9||30|
TASK 2 DELIVERABLE & PAYMENT SCHEDULE
|Phase||Payment||Deliverables / Milestones||Week
|1: Prep||1||Quantitative inception report with work plan||2||10|
|1: Prep||2||Feedback to SI on instruments and protocols (includes listing survey, household survey, customer survey, and quality assurance tools)
Final instruments translated & back-translated
|2: Fieldwork||3||Training Manuals finalized with input from SI
Enumerator Training completed
Pilot & Training Report(s)
|2: Fieldwork||4||All quality assurance successfully reconciled
Final, raw datasets (if not electronic data collection). If electronic data collection is used, SI will have real-time access to raw datasets through a server. SI strongly prefers electronic.
|3: Reporting||5||Weekly progress reports
Bi-weekly progress reports
|3: Reporting||5||Final data collection completion report||26||30|
V. Scoring Criteria
Social Impact will weigh the following factors to score proposals:
- Technical Proposal: Compliance with requirements of scope of work; understanding of data collection activity requirements; innovative approaches presented if applicable. The technical proposal shall also include a section on operational considerations, demonstrating how the Offeror intends to respond to any challenges posed by the data collection activities over certain seasons or holidays.
- Personnel: Compliance with required qualifications and overall demonstrated experience of the personnel presented.
- Past Performance: Demonstrated, successful experience conducting similar activities in comparable settings, as specified in the Scope of Work. This experience should reflect institutional capacity, not just that of individual team members.
- Cost: Compliance and alignment with technical scope of work; competitiveness; reasonableness.
SI will judge proposals for qualitative and quantitative proposals separately (e.g., a missing or sub-par score on the qualitative/Option 1 proposal will not affect the scoring of the quantitative/Option 2 proposal). Firms bidding on both Tasks should submit only a single budget document, though budgets for each task will be evaluated separately. Awards will be made on a by-Task basis using a best value trade-off approach. If an Offeror has made proposals for multiple Options, SI reserves the right to award the bid for only one of those Options.
Firms based outside of Lesotho are eligible to bid but must demonstrate their level of experience conducting similar work in Lesotho and their ability to rapidly mobilize data collection within Lesotho, including relevant capabilities to carry out high quality data collection in Sesotho.
VI. Submission Instructions
Bidders should follow the instructions below for submission of questions and proposals, for the question period, as well as the technical and cost proposals.
Qualitative and quantitative data collection will be carried out as discrete Tasks, with their own timelines, work plans, and deliverables. Firms are invited to bid on one or both of these Tasks. If both Tasks are awarded to a single Subcontractor, the quantitative data collection will be structured as a Contract Option to be exercised at SI’s discretion, pending the results of the qualitative data collection and dependent on satisfactory performance of the Subcontractor on the qualitative Task.
NOTE: Social Impact reserves the right to award the qualitative and quantitative components to different firms depending on qualifications, or to make a single award with quantitative data collection as an option. Offerors bidding on both tasks are invited to identify opportunities for cost-saving or improved technical quality through economies of scale. Offerors are not required to provide proposals for all Tasks. Offerors will not be evaluated based on their ability to bid on more than one Task (i.e. Offerors will not be penalized for bidding only on one task).
Please use subject line “Evaluation of the MCC Lesotho Compact Metolong Program (MP) and Urban and Peri-urban Water (UPUW) Activity – Questions”. Questions may be submitted on a rolling basis, but answers to questions will be released only once, according to the date on the first page of this RFP. SI will release answers to all questions received, to all Offerors. Questions will not be identified with the Offeror who submitted them.
Please send to both email addresses in the “Contact” field on page 1 by the deadline for questions. Late submission of questions will be considered on a case by case basis by SI.
Bidders must submit technical proposals that consist of the following components: Technical Approach, Technical Prompts, Personnel, and Past Performance. The full technical proposals must not exceed page limitations, as delineated in each section below.
Offerors are asked to pay particular attention to page limits; material that exceeds the page limitations will not be reviewed or scored by SI and SI has the right to disqualify submissions on the basis of non-compliance with page limitations.
Technical proposals must not include any financial information; SI may disqualify bids that include financial information in the technical proposal.
Technical Approach (Page limitations: up to 6 pages per Task)
Offerors must submit a Technical Approach no longer than six (6) pages if submitting for a single Task, or twelve (12) pages if submitting for both Tasks. The technical approach should include the Offeror’s approach to meeting all the requirements in the scope of work for the Task. The technical approach should be supplemented by an answer to the technical prompts listed below. Answers to the technical prompts will not be counted against the page limits above.
Technical Prompts (Page limitations: up to 3 pages per Task)
Offerors are required to submit answers to the following technical prompts, in supplement to the technical approach as described above. Answers should be submitted in no more than 3 pages per task (qualitative, quantitative).
- TASK 1 Qualitative, technical prompts:
- What are the proper procedures to identify which chiefs have informed perspectives related to the study areas? Will there be any challenges, logistical or otherwise, in conducting interviews with chiefs? If so, what are these and how does the Offeror propose to mitigate them?
- Similarly, what is the Offeror’s estimated number of chiefs that will be interviewed as part of the qualitative Task? The answer to the first question in the previous bullet should provide justification for the number provided here.
- Will there be specific challenges conducting data collection in the months outlined because of seasonality (winter, snow)? If so, what are the Offeror’s proposed methods to mitigate these challenges?
- Are the female members of household identified in this RFP indeed the most appropriate respondents for the focus groups (in that they will be the most knowledgeable about the water-related topics outlined in this RFP)? Does the Offeror have any additional or alternative suggestions?
- To understand dynamics of internal migration and changes in the housing market over the last few years (e.g. since at least 2013), are there other key informants that will be very knowledgeable about these topics in the urban areas included in the evaluation, that SI should consider interviewing?
- Note: The Offeror’s answer to this question is not required to be costed into the proposal, as it is not laid out as part of this scope of work. However, SI will consider suggestions provided and if requested, Offerors should be ready to propose a cost proposal for additional qualitative interviews as suggested under the answer to this question.
- What are the local research clearances and permits that will be needed to conduct the household listing and household survey successfully? What are the lead times necessary in order to initiate the required processes/paperwork?
- TASK 2: Quantitative, technical prompts:
- Does the Offeror foresee any specific challenges, logistical or otherwise, to implementing the data collection as specified in the Scope of Work? If so, what are they? What is the Offeror’s proposed approach to mitigating these challenges?
- Who are the community leaders or other individuals that will be the most important to engage as part of the household listing activity, to ensure that listing is done accurately and efficiently?
- What are the local research clearances and permits that will be needed to conduct the household listing and household survey successfully? What are the lead times necessary in order to initiate the required processes/paperwork?
Personnel (Page limitations: up to 2-page narrative per Task; 2 pages per CV)
Offerors must submit a narrative, no longer than two (2) pages summarizing key personnel qualifications and experience along with a description of the approach for recruiting other field staff for the data collection activity. Offerors must also submit CVs for all proposed key personnel. Altogether, CVs should not exceed 2 pages EACH.
The two-page narrative should very clearly explain why the proposed staff are qualified to successfully carry out the requirements of the scope of work for this Subcontract.
It is permissible for the same individual to be proposed for a position under the Qualitative Task and the Quantitative Task. If this is done, there should be a clear justification for the individual’s qualification to perform both, and any cost savings or efficiencies that can be achieved by having a single person in both positions should be clearly stated.
Past Performance (Page limitations: No more than 2 pages per past performance report)
Offerors must submit three (3) past performance reports, not exceeding two (2) pages each. Offerors submitting a proposal for both Tasks are permitted to submit a fourth past performance report if it assists the firm in demonstrating experience in both quantitative and qualitative, but this is not required. Past performance reports must be submitted with the following information:
- Value of the contract
- Scope of work carried out
- Results achieved, challenges overcome
- Reference name and contact information
The cost proposal shall consist of a budget in Excel with traceable formulas and clear explanation of any assumptions made within the budget. Bidders are strongly encouraged (though not required) to use the budget template provided in Annex A to this RFP. Costs should be presented in USD. Regardless of how many Tasks for which an Offeror is submitting a proposal, the financial proposal shall be limited to a single Excel file.
Narrative (optional; not to exceed two pages)
The Offeror may submit an option narrative, not to exceed two additional pages, not counted against the page limit, documenting specific cost-saving measures and the source of the assumptions used in budgeting. For Offerors submitting proposals for both Options, any efficiencies in cost for combining the qualitative and quantitative efforts should be noted on this page.
Submission: Please use subject line “Evaluation of the MCC Lesotho Compact Metolong Program (MP) and Urban and Peri-urban Water (UPUW) Activity – proposal submission”.
Please send to both email addresses in the “Contact” field on page 1 by the deadline for proposals. Late submissions will not be accepted.