SOLICITATION NOTICE
U -- Special Enrollment Examination
- Notice Date
- 9/7/2005
- Notice Type
- Solicitation Notice
- NAICS
- 611710
— Educational Support Services
- Contracting Office
- Department of the Treasury, Internal Revenue Service (IRS), National Office Procurement (OS:A:P), 6009 Oxon Hill Road, Suite 500, Oxon Hill, MD, 20745
- ZIP Code
- 20745
- Solicitation Number
- TIRNO05R00011amendment0001
- Response Due
- 9/28/2005
- Archive Date
- 10/13/2005
- Description
- The purpose of this amendment is to answer questions received from interested offerors. A full copy of this amendment, including the Standard Form (SF) 30 cover sheet can be found at the IRS Procurement website: http://www.irs.gov/opportunities/procurement/. A signed copy of the SF30 acknowledging the amendment is required to be submitted along with all proposals. The due date for proposals is NOT extended as a result of this amendment and all terms and conditions remain unchanged. Question # 1. How many of the 7,618 applicants taking the exam in 2004 took the completed exam (4 parts) and how many only took individual parts? Is there a complete breakdown of who completed what? Answer #1: We can provide estimates for the number signed up to take part of the exam. In 2004, approximately 1,250 signed up to take part of the exam and the remainder took the entire exam. Question # 2. How many challenges and questions should be expected? Answer #2: In 2004 we received 1,456 challenges. Our current policy is to release all the questions to the public for feedback as well as "unofficial answers". After consideration of feedback (i.e. challenges) we arrive at our "official answers" approximately 5 - 6 weeks after the examination. The decision on whether to use this approach or an alternative approach, such as not releasing the questions to the public, should be part of the offeror's solution to the requirement. If the questions are not released to the public we would expect the number of challenges to be reduced significantly. Question # 3. How often has IRS been forced to defend the SEE (Special Enrollment Exam) legally? Answer #3: To the best of our knowledge, never. Question # 4. Does IRS have previous job analysis information that we can use? Answer #4: No. Question # 5. Does IRS have previous test blueprint information that we can use? Answer #5: No. Question # 6. Will IRS provide SMEs (Subject Matter Experts) for item bank development? Answer #6: No. However, to the extent noted in the solicitation, the IRS will provide subject matter experts for the job analysis and pilot testing. Question # 7. How has IRS created cut-scores for the SEE in the past? Answer #7: The passing rate has been set at 60%. Question # 8. How many requests are made for lists of SEE applicants annually? Answer #8: Records are only available for this year to date. From the first of this year, to date there have been 26 paid requests for lists of Enrolled Agents. Question # 9. What metrics does IRS use currently to measure SEE program effectiveness? Answer #9: Comments we receive from candidates and professional organizations help us measure the effectiveness. Question # 10. Is there a prior job analysis and may we have a copy if awarded (the contract)? Answer #10: There is none. Question # 11. The 2004 Special Enrollment Exam Topic List included 30 topics for part 1, 29 topics for Part 2, 15 topics for part 3, and 12 topics for Part 4 (see Appendix). How were those distribution of topics established? Answer #11: We typically have had a revenue agent lead a team of IRS employees in developing questions. This team lead decided on the distribution of topics. Question # 12. Exam validation, including pilot tests of the exam. Provide the IRS with validation report with results of the pilot test. Provide the IRS with a recommendation for the cut-score for final approval. Is the stipulation of a cut score meant to preclude an Adaptive Testing approach? Answer #12: Not necessarily, however detailed explanation would be needed in order to justify the use of Adaptive Testing. Given the breadth of knowledge needed to be an Enrolled Agent, we want to focus on the EAs demonstrating the necessary knowledge rather than on a specific percentage of people passing the exam. Question # 13. Update exam as needed to reflect changes in tax law, exam saturation and scoring. How will "as needed" be determined, and what will be the timeline for the update? Answer #13: We currently administer the examination in September based on the law applicable for the prior year's returns. For instance, an examination administered in 2005 will reflect the law in affect for preparing 2004 tax returns. Question # 14. "8.0 Considerations - b. ....Up to two (2) IRS employees will be available for pilot testing up to 16 hours each." How was the number 2 determined? Answer #14: Two (2) IRS employees will be available for up to 16 hours each for pilot testing because that is the IRS manpower commitment available for this project. Question # 15. There is no restriction on the format(s) of the exam or frequency. The IRS is seeking the best overall approach to the SOO that is economically feasible. Does format unrestricted refer to the T/F, multiple choice or computation components, or the topics tested? Answer #15: Format unrestricted refers to the t/f, multiple choice or computational components. It is up to the vendor to determine topics tested through their job analysis. Question # 16. With regard to Exam development and validation: Are parallel forms required, or will randomized question order suffice? Answer #16: The approach used for this should be part of the offeror's solution offered in their proposal. Question # 17. How many appeal cases occur per year? When a successful appeal of the scoring key has been made, are all other test-takers rescored and re-contacted? Answer #17: There is no appeals process in place. We give ourselves about 5 weeks to consider challenges and come up with final answers. Only after the final answers are determined do we grade the exams. If we accept a challenge, it will impact all test takers, not just the individual who wrote in. Question # 18. How will the contractor be paid for applicants who have taken previous parts under the old fee structure? Answer #18: Accommodation of applicants who have passed part of the exam under the prior format is not a requirement of the solicitation. Question # 19. May we have access to de-identified applicant item-level performance data for prior examinations? Answer #19: We can provide frequency distribution reports to the successful offeror, providing a breakdown of how each question was answered. We can also provide a passing basis summary report showing how many individuals received what scores. Question #20. Paragraph (h) (vi) on page 39 of 48 states that ?a sample exam comparable to that which will be developed in satisfaction of this requirement? is to be submitted with the proposal, in CD format. Two questions with regard to this requirement: a. Please define in some meaningful way what is meant by ?comparable.? One assumes that it does not mean ?comparable? in length to the current examination. Answer # 20 a. "Comparable" means an exam of similar depth and complexity that has been developed for another application. The purpose of this requirement is to demonstrate the quality of exam produced by each offeror. b. Can the Government provide assurances that the sample exam will be kept secure and that it will be destroyed after award? Answer # 20 b. All proposal material is kept secure. One copy of all proposals is retained in the contract file, and all other material is securely destroyed at the conclusion of the acquisition process. Question # 21. Paragraph (4) Price states that ?overall price will include the fees collected for reimbursement of Government expenses plus fees that will be retained by the contractor. Can the Government provide an amount that all bidders can use as the ?fees collected for reimbursement of Government expenses?? Answer # 21. The fee collected for the reimbursement of Government expenses will be determined and provided to the successful offeror for inclusion in their fee collection process. This amount will be the same for all offerors and, therefore, has no impact on the competitive process. The amount has not be definitively determined at this time. Question #22. Attachment A, Phase 2?Exam Development identifies one of the major activities as ?External review of the exam by outside professionals.? Who are these outside professionals? Are they tax experts? Are they testing experts? What organization(s) are involved in the review? Answer #22. Attachment A is an overview of the current program for exam development and administration. It is provided for your background information only and is not intended as a requirement to influence offeror's proposals. Therefore, the answers to this question are not necessarily germane for an offeror's proposal development. Question #23. Will continue the policy of publicly releasing the test items on an annual basis? Answer # 23. While release of the test items was a practice under IRS administration of the exams, it is not an IRS policy and is not a requirement for this solicitation/contract. Question #24. Where are the resumes of key personnel to be included? Answer # 24. While resumes have not been specifically requested, key personnel information should be included in the technical portion of the proposals.
- Record
- SN00889544-W 20050909/050908034418 (fbodaily.com)
- Source
-
FedBizOpps.gov Link to This Notice
(may not be valid after Archive Date)
| FSG Index | This Issue's Index | Today's FBO Daily Index Page |