“The conference brings together researchers and practitioners on all aspects of test security, including statistical data forensics, deterrence, proctoring, policies of investigation, and legal implications.” said Cengiz Zopluoglu, associate professor and program director for the Research, Measurement, and Evaluation (RME) Program, who spent more than a year organizing the event. “It had the highest attendance yet.”
Dean Laura Kohn-Wood welcomed the 202 professionals to the three-day conference, including attendees from United States, United Kingdom, Netherlands, Australia, Brazil, Israel, Canada, and the United Arab Emirates. “It is my pleasure to bring together the best and brightest minds and researchers in test security,” she said. “This is certainly an important issue for employment as well as for education.”
Kohn-Wood said she was introduced to test security as a clinical psychology student when she learned how to administer and score the Minnesota Multiphasic Personality Inventory (MMPI). “I learned that you could embed indications of whether or not such a psychological measure was valid and accurate,” she said.
“Today, we have an incredible responsibility to ensure the moral and ethical bases of our testing tools are accurate,” she added. “We need to be able to stand behind a score and know it is a valid measure of knowledge, competence or skill, as high-profile cases have brought this to the attention of the public.
Kohn-Wood also cited the school’s commitment to its Research, Measurement and Evaluation (RME) program. “We understand the importance of ensuring fairness, accuracy and validity in a diverse, multicultural community like South Florida. We know that if you can establish a tool or intervention in this environment, it could work almost anywhere in the country.”
After the dean’s talk, Rachel R. Watkins Schoenig, founder and CEO of Cornerstone Strategies, moderated three fast-paced debates on hot topics in the field. “We like to present both sides of an issue, and listen to the wisdom of the crowd,” she said.
Kimberly Swyggert, director of test materials management at the National Board of Medical Examiners, argued that randomizing options or items to enhance security is a psychometrically sound practice. “Cheaters look for patterns and try to predict what’s coming next,” she said. “It’s the same thing in assessments. If items are always in the same order, it is easier to memorize the answers and for groups to share signals on the day of the test.”
Arguing against randomization, Kim Brunnert, psychometrician at Elsevier, said randomizing items could confuse the test-taker and introduce the possibility of unpredictable errors that could jeopardize the results. “Don’t put test security ahead of validity,” she said. “Randomizing is not a strong enough security measure to risk the validity of the scores.”
Faisel Alam, test security manager at the Law School Admissions Council, and Camille Thompson, executive director at the College Board, debated the second question: “Are miniaturized digital devices a bigger threat to exam security than cell phones today?”
Alam pointed to the use of tiny cameras and recording devices embedded in buttons, eyeglasses and even clothing as a serious threat to steal content from tests. “With cell phones, the damage is limited to a few individuals, but a gang of test takers with tiny cameras can compromise your entire item bank,” he said.
However, Thompson noted that more than 95 percent of U.S. adolescents own or have access to cell phones, which she called “the Swiss army knife of cheaters.” Using a smart phone to find answers doesn’t feel wrong, while using a spy device feels like you’re crossing the line, she added.
In the third debate, Jaime Mulkey from Caveon Test Security argued that certification and license holders should pay a nominal annual fee to help maintain the security of the exam. “Everything today is taxed, so why not test security?” she said. “Having a visible tax lets the public know they are being protected.”
However, Bryan Freiss, director of information security at Pearson VUE, said test security should be embedded through the entire cycle from initial test development through reporting of results. “I don’t pay my credit card company to protect my data, so why should I pay an exam security fee to embed security?” he said. “Test security should be embedded throughout the life cycle of the exam and the pricing should include security.”
The conference had 45 sessions, covering topics such as deterrence strategies, online proctoring, data forensics, cloud-based security, and investigating test security concerns in K-12 schools. “We deeply appreciate the work of our presenters and the support of our sponsors,” said Zopluoglu, who thanked conference sponsors including Ascend Learning, ACT, Alpine Testing Solutions, Caveon, Cisco, Examity, National Council on Measurement in Education, National Council of State Boards of Nursing, National College Testing Association, PearsonVue, ProctorU, Proctorio Pro, Prometric, PSI, Renaissance Learning, and The College Board.
