From NCCAOM Board Chair: Common questions, Clarifications and Transparency

Q1. The NCCAOM announced in mid-May of 2018 that it would be discontinuing year-round testing for the year 2019. What is the rationale and status of that announcement?

Answer: In mid-May, the NCCAOM announced that it would be discontinuing year-round testing during 2019 and moving to linear exams four (4) times during that year in order to increase the number of validated exam questions. After listening to our stakeholder feedback and comments over the past week, the NCCAOM decided to delay the administration of the linear exam until early 2020. This means that all of our exams (offered in English) will be administered in the adaptive (year-round format) for all of 2019. Applicants have one and half years to take the exams in the adaptive, year-round administration format.

Computer adaptive testing (CAT) is a much more secure way of testing competency by a certification exam (i.e., less risk of cheating); however, it proves to be a challenge to keep the required number of validated items (exam questions) in the item bank. Each question on the board exam must go through a minimum of a 1-year process from creation, review by a team of subject matter experts who serve on our Exam Development Committees, as well as our staff exam development specialists and our psychometrician who all check and recheck for reference sourcing, editing, and then psychometric validation. Each question must be psychometrically validated before it can be used to score an exam. When each person takes an exam, a certain number of questions are given, but not scored in order to have enough exposure to be psychometrically validated. For each question, it must be seen and answered by at least 300 test takers.

It is no secret that the number of students taking the exams, annually, has dropped dramatically over the past decade—enough so that between retiring overexposed exam questions and not having enough new ones validated, the NCCAOM’s item bank for each of its board exams is beginning to drop. This is the reason for the temporary cessation of adaptive testing in 2020. If we can obtain more candidates to take the exams, we may not have to offer the exams in linear format for the full year. We have been constrained in recent years with the declining number of ACAOM accredited program graduates taking the NCCAOM certification exams. The Accredited Commission for Acupuncture and Oriental Medicine (ACAOM) accredited school enrollment is currently 1,849 less in 2018 compared to 2012 (Reference: ACAOM Enrollment and Graduation Data, personal communication with Mark McKenzie, June 1, 2018). We must increase the numbers of valid exam items, and linear (form-based) testing is the only way to quickly get those numbers high enough. It is our hope that there are enough test takers during the 2020 linear exam administration cycle so that NCCAOM can return to computer adaptive, year-round testing in 2021 for all examinations. The harsh reality is that, if the number of students taking these exams (directly correlating with the number of students in schools) continue to dwindle at the current rates, we may not be able to return to adaptive testing for a while, if ever. If California adopts the NCCAOM exams, this will add more test takers, and this will help significantly in terms of speeding up the psychometric validation of our new items and we can return to year-round testing in 2021. That is what we believe will facilitate a prompt return to CAT in 2021; however, these circumstances are beyond our control.

Q2. What does the NCCAOM actually do?

Answer: The NCCAOM is a non-profit 501(c)6 organization made up of employees and volunteers including a volunteer Board of Commissioners, which has oversight responsibilities for the organization. The CEO manages the organization. NCCAOM certification or a passing score on the NCCAOM certification examinations are documentation of competency for licensure as an acupuncturist by 46 states plus the District of Columbia which represents 98% of the states that regulate acupuncture. All NCCAOM certification programs are currently accredited by the National Commission for Certification Agencies (NCCA).

For the NCCAOM certification programs in Acupuncture, Chinese Herbology, and Oriental Medicine to remain accredited by the NCCA, the NCCAOM must adhere to strict national standards for examination qualifications and development. All Diplomate level certification exams must meet examination content validity standards set forth by NCCA.

The Mission of the NCCAOM is to assure the safety and well-being of the public and to advance the professional practice of acupuncture and Oriental medicine (AOM) by establishing and promoting national evidence-based standards of competence and credentialing.

Q3. How are minimum competency standards for the protection of the public determined?

Answer: One of NCCAOM’s primary responsibilities to the public and the AOM profession is the creation of fair, and valid, psychometrically sound exams that serves as the standard of what a first-year practitioner should know. The passage of these qualifying exams for licensure and certification and the use of these exams to determine Board certification is important because it is what we (meaning our profession) have determined is competency for the AOM profession. The Job Analysis that the NCCAOM does every 5 years documents, empirically the KSAs (knowledge, skills, and abilities) to be competent to practice as an acupuncturist. The Job Analysis provides the linkage to the content outlines which serves as a blueprint for the development of the NCCAOM examinations, and thus determines what types of questions are asked on the exam. This methodology of determining what is a valid exam is crucial - it is the standard of every medical profession in which board exams are required. It is even more crucial because it is what sets us apart from every other profession that wishes to add acupuncture to their scope of practice. This is why national board certification is so important - it validates why 3-300 hours training is insufficient to train a practitioner.

Q4. What determines NCCAOM policy?

Answer: NCCAOM policies and procedures are determined by a number of factors. First, NCCAOM certification programs are accredited by the Institute for Credentialing Excellence’s accreditation board, National Commission for Certifying Agencies (NCCA). The NCCA create standards for credentialing agencies such as ours. NCCA accredited programs certify individuals in a wide range of professions and occupations including nurses, automotive professionals, respiratory therapists, counselors, emergency technicians, crane operators and more. To date, NCCA has accredited approximately 330 programs from more than 130 organizations. Every five years, the NCCAOM must undergo reaccreditation for EACH active certification program we offer (Acupuncture, Herbology and Oriental Medicine). Our policies and changes to our policies must keep in line with NCCA standards or we can lose credentialing. The last set of updated NCCA accreditation standards occurred in 2016. Maintaining accredited certification programs is important because of acceptance by state licensing boards, the federal government (including the VA, DOD), state governments, and insurance companies. There are some policies where we may ask our stakeholders’ opinions and use that to determine policy. There are many other policies where this is not appropriate because of the requirements of our accrediting body, fiduciary responsibilities (the fiscal health of NCCAOM), or impacts to public safety need to drive our policy decisions. NCCAOM is not a membership organization, it is a credentialing organization. Examples of this include: the elimination of the apprenticeship-only route, elimination of the pre-graduation route, and the revisions to the NCCAOM® Code of Ethics and the Grounds for Professional Discipline. These are examples that our policies are driven by our NCCA accreditation standards.

The NCCAOM does occasionally solicit stakeholder for feedback on policy changes such as the 2013 request for comments on proposed policy changes to add background checks for the application process and to change PDA policies. After receiving feedback from its various stakeholders, the NCCAOM did not change these policies as they were also not contingent on compliance of our accreditation.

Q5. I have heard that “The NCCAOM is sitting on 10 million dollars and is out for a money grab.” - Fact or Fiction?

Answer: I recently heard this rumor at a function where I was presenting. Regardless of the amount we do have in reserve (it is nowhere near 10 million dollars). Please refer to our published NCCAOM Annual Reports where you will be able to see the actual amount in reserves. There is a common misperception the NCCAOM does nothing for the profession and is out for a money grab, and over charging for everything. This is simply not true. If anything, the NCCAOM Board has been shielding applicants and potential licensees from increases in exam and certification costs as well as inflation. Psychometrically valid Certification exams are extremely expensive to create. Each exam item (question) costs anywhere from $1500-2000 to create, then those costs are amortized over a 5-year period. When the NCCAOM item bank was compromised by cheating in 2005, it cost $1 million dollars to redo the item banks; and this rewriting of the questions was by volunteers! The NCCAOM uses the fees generated from certification and exams to continually develop items through an extensive and robust exam development process in which subject-matter experts, testing staff and psychometricians develop and ensure that all items are valid and reliable. In addition, as part of our accreditation, we conduct an extensive job analysis survey every 4-5 year that entails costs and staff and volunteer time. In addition, the NCCAOM uses fees generated from recertification to conduct promote NCCAOM National Board-Certified Acupuncturists™ through a public education campaign (please see press center on website) and advocacy to gain further recognition for acupuncturists within state and federal agencies. We do not have 10 million dollars in the bank, but as a Board, we must ensure that the organization maintains a prudent reserve - currently 6 months of operational costs. This is imperative to any nonprofit organization to be viable. If the world went south, we have enough money to operate for 6 months without any income. Any institution (or business) must have a prudent reserve or risk extinction with unforeseeable outside circumstances.

Q6. Is the NCCAOM creating artificial barriers to keep people from practicing?

Answer: No.; however, NCCAOM’s number one priority and mission is to protect the public. We do so by setting minimum standards of competency for a first-year practitioner as determined by the Job Analysis - which hopefully you as a practitioner participate in by answering the survey questions. Without these standards, we risk others determining what our standards should be.

Q7. Why did the NCCAOM eliminate the pre-graduation route?

Answer: This topic has been an ongoing discussion for the Board for a number of years. There are a few key points leading to the announcement of the elimination of this route in May of 2018. The first is that our NCCA accreditation requires that each of our routes be substantially equivalent in order to be a valid route for testing. The NCCAOM currently has 4 routes for eligibility to test:

Formal education U.S.

Formal Education International

Apprenticeship Only (through 2020),

Combination of Apprenticeship and Formal Education.

The NCCAOM Eligibility Committee has continued to revisit any discrepancies amongst the routes and has identified several - most of which have been addressed, but the elimination of the pre-grad route is the most recent one has been considered on multiple previous occasions with the Board. Currently the option to test prior to completion of one’s education is only offered through the U.S. Formal Education route. It is noteworthy that all international formal education applicants and every other route must complete their education before applying for certification and testing. This is a significant discrepancy between each of the routes and is not standard for any other medical profession in which independent practitioners can test and potentially practice before completing their education. While most states require graduation from schools, some states only require board certification because of the assumption that each practitioner would have finished their education prior to testing. We do not allow the same for any other route. It is a loophole we must close and is not worth the possibility of delaying or risking our reaccreditation by NCCA. We do understand that this can lengthen the process from testing to licensure; however, we are hoping that by delaying the linear exams to our maximum time frame of 2020, this will help to ease some of the challenge