SOLICITATION NOTICE
R -- Quantifying Improvements Based on Lab Testing of Questionnaires
- Notice Date
- 5/27/2004
- Notice Type
- Solicitation Notice
- NAICS
- 541990
— All Other Professional, Scientific, and Technical Services
- Contracting Office
- Department of Health and Human Services, Center for Disease Control and Prevention, Acquisition and Assistance Field Branch (Cincinnati), 4676 Columbia Parkway M/S C-4, Cincinnati, OH, 45226
- ZIP Code
- 45226
- Solicitation Number
- Reference-Number-000HCS13200409713
- Response Due
- 6/10/2004
- Archive Date
- 6/25/2004
- Point of Contact
- Jennifer Bayer, Purchasing Agent, Phone (513)533-8586, Fax (513)533-8283, - Dwight Favors, Supervisory Contract Specialist, Phone (513)533-8137, Fax (513)533-8283,
- E-Mail Address
-
jmh8@cdc.gov, dyf3@cdc.gov
- Description
- Negotiation are being conducted on a sole source basis with the University of Massachusetts at Boston, Center for Survey Research for quantifying improvements based on laboratory testing of questionnaires. Background Information: This requirement is requested by the National Center for Health Statistics (NCHS), Office of Research and Methodology, for a research organization to assist in a methodological investigation involving alternative versions of survey questions. Specifically, this study will evaluate whether questionnaire revisions based on cognitive interviewing or other forms of technical appraisal are easier to administer in a standardized manner, pose fewer problems for respondents while answering, or generate response distributions that are more plausible or consistent with expectations than the original versions. The contractor will collaborate with NCHS staff to select appropriate questions to be included in the study; administer a split-ballot telephone survey; behavior-code the results; and collaborate with NCHS staff in analyzing the data; therefore, the contractor must have expertise in all of these areas. Objective: The purpose of this requirement is to procure the professional services of a research organization capable of collecting and interpreting data necessary to explore the benefits of alternative question wordings. Specifically, this will involve selection of appropriate questions and their alternatives, collection of survey data, and performing a highly specific task known as behavior coding. These efforts should be led by a senior researcher with expertise in questionnaire design and survey methods who will oversee this data collection and advise NCHS throughout the project. It is widely accepted that cognitive interviewing is a useful tool for questionnaire design, but there have not been many attempts to demonstrate that cognitive interviewing recommendations result in question wordings that are ?better? (i.e., provide more accurate data or are easier to administer) than previous wordings. Part of the difficulty is practical B often there is no gold standard to determine the accuracy of survey responses (and some gold standards such as medical records are not necessarily better, since they sometimes include errors of their own). One of the few attempts to look into whether cognitive re-designed questions are actually better was conducted by Willis and Schechter (1997). Their study used split-ballot experiments to compare responses from before-and-after question wordings. Although there was no way to externally verify their survey data, responses from the modified question were more plausible and closer to expected findings than the previous versions. Some of the major hypotheses to be considered are:  Some questions are difficult to answer because they actually contain two distinct concepts-- splitting them into two separate questions will change response distributions in expected manners and will reduce administration difficulties.  Presentation of qualifying information is easier to process when it appears after the primary subject matter of the question has been presented. In other words, it may be significantly easier for respondents to answer a question such as, AHave you been to a doctor, not including X, Y, and Z@ than ANot counting X, Y, and Z, have you been to a doctor?@ At present, questionnaire design literature contains no guidance about optimal structuring of complex questions.  Re-wordings that clarify vague concepts will significantly affect response distributions and reduce interviewer-respondent negotiations about meaning; however, expanding question length to explain nuances of meaning that only apply to a small subset of respondents will not affect response distributions, and may increase the complexity of interviewer-respondent interactions. In other words, it may be better to provide such clarifications only when respondents need them. Scope of Work: In order to achieve the above objectives, the following activities will be carried out under this requirement: 1a) Select about 10-15 complex health survey questions. Some of these may be draft questions proposed for surveys such as the NHIS, while others may be questions that have actually been fielded. 1b) Compile (or construct) alternative versions of each of these questions. For example, sometimes laboratory testing or technical reviews have led to alternative versions that split one long question into two smaller ones; restructured the wording within a particular question; or proposed new wordings or response categories. In some cases the alternatives have been incorporated onto fielded surveys, while in other cases the original version was retained. (This could be for the sake of consistency over time, consistency with other surveys, or based on a determination that the evidence in favor of changing the question was not sufficiently compelling.) In either case, there are often two versions of questions available that apparently measure the same thing; however, there may not be clear evidence that one or the other is measurably ?better.? 2) Embed the questions into split-ballot experiments on an RDD telephone survey. With respondent permission, as many of these interviews as possible would be tape-recorded. The overall sample size should be approximately n=450 at the level of funding requested, which should be adequate to detect significant differences across question versions. The actual sample size may vary based on the total number of questions included in the instrument. Data would be collected by telephone due to maximizing the number of interviews conducted at a reasonable cost. It is also more feasible to tape-record and behavior code interviews that are conducted at a central telephone facility. Many of the questions to be tested are used on both face-to-face and telephone surveys and the use of this mode should not diminish the value of findings. 3) Behavior-code those interviews that were successfully tape-recorded. The behavior coding technique was developed by Cannell and colleagues to measure both interviewer and respondent behavior during the process of administering and answering questions. Thus, the method is used to identify questions that:  interviewers have difficulty administering as written  require extensive probing before codeable answers can be obtained  respondents interrupt  need to be repeated 4) Provide tabulations from survey data and behavior coding, copies of datasets and appropriate technical documentation; and collaborate on analysis of the data collected in association with the project officer, including a preliminary report based on findings. Period of Performance: From date of award of a purchase order through December 31, 2004 Deliverables: 1. Alternative versions of questionnaire to be included in the study, to be selected in collaboration with the NCHS project officer. Overview of the research design and general plan of the experiments, including basic procedures to be followed and approximate number of research subjects. The overview should also include suggestions of possible survey items to be included in the experiments. Due approximately 9 weeks following award of contract. 2. Tabulations of the survey data collected in telephone interviews and a report of survey procedures and field results. Due approximately 24 weeks following award of contract. 3. Tabulations of behavior coding data, electronic copies of datasets associated with the project datasets, and a preliminary report of findings. Due approximately 30 weeks after award of contract. Certification Statement: This contract meets applicable policies and procedures in OMB Circular A-120. Contractor shall perform only those services described in the scope of work. All materials relating to this contract are the property of the government and shall be returned to the government at the completion of work. The contractor shall not release any information related to the work performed or the results obtained without the written permission of the project officer. Capabilities Requirements: Through this announcement, alternate sources are being offered the opportunity to demonstrate their capabilities to provide the services specifically identified above. To be considered qualified, sources must submit a capabilities statement which demonstrates in writing that they have the following: At least ten years experience and expertise in: questionnaire design and evaluation for large-scale federal surveys, including technical reviews, cognitive interviewing, and behavior coding experimental designs in social science research, especially split ballot experiments testing alternative versions of similar questions detailed familiarity with the National Health Interview Survey and other federal health surveys This expertise should be demonstrated through a combination of publications, track record of completed methodological projects specifically focusing on questionnaire testing and evaluation, and invited presentations addressing these topics. Ability and experience to carry out all phases of an RDD telephone survey, including sample selection, data collection using a trained and experienced interviewing staff, supervision and monitoring, and production of tabulations and electronic data sets. Ability to tape record telephone interviews At least ten years expertise conducting behavior coding, as described in publications by Cannell, Fowler, and colleagues. Contractors should be able to demonstrate success in applying the method for questionnaire evaluation and achieving high levels of coding reliability in multiple past applications. Experience producing publication-quality reports based on behavior coding for survey questionnaire evaluation is essential. Ability to perform data collection and coding within the timeframe specified in the proposal. Qualified organizations are encouraged to submit a capabilities statement which addresses the requirements and contains material in sufficient detail to allow the Centers for Disease Control and Prevention to determine if the party can perform this requirement. Capabilities are to be received in the contracting office no later than fifteen (15) calendar days from the date of this announcement. Submit written information to: Jennifer Bayer, CDC/PGO/AAFB, 4676 Columbia Parkway, Cincinnati, Ohio 45226, Mail Stop C-4, reference #000HCS13200409713. The intent of this synopsis is to determine whether alternative sources exist. Information received will be used solely for the purpose of determining whether to conduct a competitive procurement. A determination by the Government not to compete this proposed requirement, based upon responses to this notice, is solely within the discretion of the Government. All responsible sources may submit a response which shall be considered by the Agency.
- Place of Performance
- Address: University of Massachusetts at Boston, 100 Morrissey Blvd., Boston MA
- Zip Code: 02125-3393
- Country: USA
- Zip Code: 02125-3393
- Record
- SN00593543-W 20040529/040527211816 (fbodaily.com)
- Source
-
FedBizOpps.gov Link to This Notice
(may not be valid after Archive Date)
| FSG Index | This Issue's Index | Today's FBO Daily Index Page |