Innovations+ Connect Program

Below are the Innovations+ Connect sessions in which presenters will partake in a 45-minute discussion on March 29th or 30th. All Innovations+ Connect attendees will receive an email the week before the event with access to the recorded sessions to view and to prepare for discussions with the speakers and other attendees.

Please note all times are EDT. Schedule subject to change.

Wednesday, March 29, 2023

8:00 AM – 8:45 AM | Legal Eagle Update (and Q&A)

This session will include a Legal Eagle Panel with a combined total of 95 years of legal experience that will provide an update on the most pressing legal issues facing testing organizations, following by a Q&A session. The Legal Eagle panel will include four attorneys, each from a different law firm, with extensive experience representing testing organizations and service providers. In the first 25 minutes of this session, each member of the panel will provide a “legal update” on important legal issues in testing, including ADA accommodations, copyright registration and enforcement for secure tests, trade secret law as it pertains to test content, contract terms and enforcement, and invalidation of test scores. In the last 25 minutes, the panel will answer questions from the attendees. If there was ever a question you wanted to ask a lawyer about testing, this is your opportunity!

Speakers: Caroline Mew, Perkins Coie LLP; Jennifer Ancona Semko, Baker & Mckenzie LLP; Allison Mulford, Mulford & Associates, LLC; Marc Weinstein, Marc J. Weinstein PLLC

8:45 AM – 9:00 AM | Break

 

9:00 AM – 9:45 AM | Accommodations in the Age of Hybrid Testing

As people adapt to new ways of working and meeting, there is a strong push to provide workers the flexibility to work and live anywhere. This change will continue to push the shift to greater hybrid assessment opportunities where the test taker can choose whether they test in-person or remotely. The shift to remote testing changes the nature of special accommodations that are requested by test takers and how they might be implemented. Testing programs and exam providers actively seek ways to manage candidate expectations and maintain the fairness and validity of their assessments.

Presenters will share how two organizations partnered to meet the additional challenges of maintaining content security and identification confirmation while providing accommodations support in remote exam administrations. They will take the audience through the challenges and issues and show how technology provided remote support for the most complex accommodations, including readers, scribes, translators, and more while maintaining content security and verifying test-taker identity. The audience will be introduced to innovative features including content blur, screen, and keyboard control, and much more. Leave the session with a solid understanding of what a program should consider when adopting technologies in a hybrid testing environment.

Speakers: Ashley Norris, Meazure Learning; Andy Dua, Uexams; Carl Bowman, CompTIA

9:45 AM – 10:00 AM | Break

 

10:00 AM – 10:45 AM | Exam Misconduct – Building an Intelligence Database to Understand the Threats

To protect an exam program, stakeholders must understand the security threats it faces. This comes from stockpiling knowledge gained by investigating and learning the facts behind individual cases. However, exam security practitioners must also seek to understand the entire security landscape, which means continuing to collect intelligence, even without active cases.

This requires methodical compiling of a database of bad actors and their techniques, employing all the tools available. These include traditional measures such as forensics, collusion analysis, and secret shops, as well as newer techniques like those used by law enforcement and intelligence agencies. The resulting picture will provide good knowledge of the potential threats and the methods bad actors use.

Join Carissa Redfield from PMI, together with Harry Samit and Bryan Friess from Pearson VUE, to learn more about the value of a comprehensive intelligence database - and some tips and techniques for building one.

Speakers: Harry Samit, Pearson VUE; Bryan Friess, Pearson VUE; Carissa Redfield, PMI

10:45 AM – 11:00 AM | Break

 

11:00 AM – 11:45 AM | Threat or Opportunity? The Changing Landscape of Assessment

The assessment industry is facing new threats to the use of assessment as part of entering professions, qualifying for medical reimbursement, etc. but, we are also becoming armed with new ways to broaden our reach and better serve new audiences: microcredentials, assessment-based certificates, technologies enabling more interesting test design & delivery. Are we facing irrelevance or increased ability to serve? The panel will provide an overview of the various challenges to our traditional model and roles, along with recent advances and opportunities to expand our reach.

Takeaways:

• Examples of current threats
• Understanding of emerging technologies for test assembly, analysis, and administration
• Understanding of new exam types, associated standards, and current uses Ideas for broadening scope & reach for testing organizations

Presenters: Donald Balasa, American Association of Medical Assistants; Kellie Early, LSAC; Frank Williams, PSI Services, Credentialing; Criag Mills, Mills Strategy Partners; Christine Mills, Ascend Learning; Bill West, Cornerstone Strategies

Thursday, March 30, 2023

10:00 AM – 10:45 AM | Test Fraud: How Secure is Your Strategy?

“Learn from the mistakes of others. You can never live long enough to make them all yourself.” ~Groucho Marx

In a world where data breaches and identify theft are daily occurrences, come discover what you can do to keep your assessments from becoming headline news. This presentation is a primer on the elements of test fraud, covering the threats and risks of cheating. The panel is comprised of three testing experts who will discuss real-world examples and cover the ins and outs of prevention, deterrence, detection, and what to do when a breach is confirmed.

This session is for those who are new to test security or those who want to explore test security a little further. Attend this to cover the basics of test security, learn from the mistakes and successes of the community, and discover ideas on how to improve the validity of your test program.

Speakers: John Kleeman, Questionmark/Learnosity; Steve Addicott, Caveon; Dave Clements, Accenture

10:45 AM – 11:00 AM | Break

 

11:00 AM – 11:45 AM | The Importance of ‘Diversity of Thought’ in Preventing Bias in Professional Testing

There has been considerable discussion over the last few years about addressing bias in testing by ensuring demographic diversity of program participants. While this is an honorable goal, to truly prevent bias in our programs, we need to go beyond demographic diversity and ensure that our credentialing programs have true diversity of thought.

The reality is that most people within any given group tend to think like other members of the group regardless of their demographic characteristics. Their education and experience define what they believe their profession does and what a professional needs to know. A committee comprised of individuals who all attended the same level of schools or work in the same type or size of companies will tend to have the same experiences and therefore the same thoughts and beliefs. Such groups may have demographic diversity but still lack diversity of thought. If so, the resulting credential will inevitably be biased for members of the “in group” and biased against others who are not in that group.

This presentation looks at how bias affects every step of a credentialing program from test design to test production and scoring and it explores the steps that credentialing bodies can take to prevent bias and ensure that every applicant has the same opportunity to demonstrate the knowledge and skill truly required to do the job.

Speakers: Clarence Chaffee, The Caviart Group, LLC; Morgean Hirt, Association of Talent Development

11:45 AM – 12:00 PM | Break

 

12:00 PM – 12:45 PM | What is the Role of AI in Remote Proctoring?

The COVID-19 pandemic drove an explosion in digital assessment and remote proctoring, as organizations struggled to migrate on short timelines and small budgets. This has also accelerated the conversation, and in some cases backlash, around the use of AI in remote proctoring. Clearly, there is a future for AI in remote proctoring, but what is it?

This debate will bring together experts from four of the leading remote proctoring companies, each with a unique perspective. One is AI only, one is human only, and two are a blend of technology and human. Two are located outside of the US, providing a global perspective. Discussion points include:

• Where does AI excel in protecting test integrity, humans??
• For what problems or situations is AI or human proctoring not a good fit?
• What are best practices and policies to implement AI proctoring?
• How can bias affect AI proctoring, and how do we mitigate it?

At the end, we will welcome questions from the audience for the panelists to discuss, as well as welcoming their viewpoints, especially from a customer perspective.

Speakers: Don Kassner, MonitorEDU; Mike Murphy, ProctorFree; Raul Rivera, Sumadi; Joy Matthews-Lopez, JML Measurement and Testing; Kranthi Bathula, Proctor360

12:45 PM – 1:00 PM | Break

 

1:00 PM – 1:45 PM | Global Considerations for Equity and Technology Based Assessment

Advances in technology have led to innovative approaches to test design and delivery, enabling assessments to be administered worldwide in a range of scenarios. While these technology-driven advances offer a range of benefits, stakeholders have raised potential concerns regarding equity and potential bias. Equity and fairness in testing are longstanding concerns and they are receiving increased attention with technology-based assessments (TBAs), especially in the context of high stakes exams used to make decisions about people. In this session, a panel of experts in test fairness, cultural adaptation, accessibility, and technology-based assessment will discuss key issues and considerations for equity and fairness in TBA. The panel will take a global perspective to consider how these concepts are viewed and addressed in various regions of the world. Considerations in the global environment include translation and adaptation of TBAs across multiple languages and cultures, access to technology, and preparation of test takers for the assessment experience across a wide range of environments that vary with respect to technological resources.

Speakers: Jon Weiner, PSI; Andre Allen, Fifth Theory; Musab Hayatli, cApStAn; Hazel Wheldon, MSI