The Speakers
Speakers included
Cristina Anguiano-Carrasco | ACT, Inc. Iowa City, Iowa
Scott Bartholomew | Purdue University, West Lafayette, IN, USA
Jaehwa Choi | The George Washington University, Washington DC, USA
Greg Chung | UCLA CRESST, CA, USA
Patrick Craven | City and Guilds, UK
Tom Gomersall | BTL, Shipley UK/Malvern, PA, USA
Gareth Hegarty | International Baccalaureate
Graham Hudson | GA Partnership, UK
Benjamin Hunter | Caveon, Midvale, UT, USA
Dee Kanejiya | Cognii, Boston, MA, USA
Elaine Kelsey | Eduworks, Corvallis, OR, USA
Gary Klein | ShadowBox, Washington DC, USA
Pat Leonard | Credly, New York, USA
Edward Metz, Ph.D., Institute of Education Sciences, US Department of Education
Jennifer Murphy | Quantum Improvements Consulting, Florida USA
Andreas Oranje | General Manager at Educational Testing Service (ETS), USA
Graham Orpwood | Professor Emeritus of Education at York University in Toronto, Canada and Director of International Programs for Vretta Inc, Canada and UK
Jacob Pucar | ​Vretta, Canada
Terry Rhodes | Association of American Colleges & Universities, Washington DC, USA
Denis Saunders | Calibrand, Derby, UK
Sae Schatz | Advanced Distributed Learning Initiative, Washington DC, USA
Jeffrey Spranza | Director of Product Management, ITS, USA
Eric Sydell | Shaker, Raleigh, NC, USA
Doug Whatley | BreakAway Games, Hunt Valley, MD, USA
John Winkley | AlphaPlus Consultancy, Manchester, UK
Brian Moon | Sero! Learning Assessments, Fredericksburg, VA, USA
Martyn Roads | Assessment Tomorrow, UK
Jeff Ross | Assessment Tomorrow
click name for more details
Conference Directors
Brian
Moon
​
Chief Technology Officer for Perigean Technologies and Conference Director
​
Mr. Moon is working to enable the assessment of learning using Concept Maps, for efficient classroom and large-scale use. A leading practitioner and researcher in the fields of expertise management and cognitive systems engineering, Mr. Moon applies diagrammatic and naturalistic interviewing techniques to elicit, protect, enable and assess expert knowledge. His practice enables expertise management, design and evaluation of cognitive systems, and the assessment of knowledge. He is widely recognized for his innovative applications of Concept Mapping, which are highlighted in the edited volume, Applied Concept Mapping: Capturing, Analyzing, and Organizing Knowledge. His clients have included numerous Fortune 500 companies, the Advanced Distributed Learning Initiative, Veterans Administration, the Intelligence Community and Department of Defence.
Jeff
Ross
​
Conference Director, Assessment Tomorrow
​
Jeff Ross is an Independent Prince II project manager and consultant with expertise in e-Assessment, e-learning and the use of IT and internet technologies in education, learning and training, including instructional design and qualification development. Previously he was a senior manager with a major awarding body, where he was responsible for change management programmes associated with the introduction of ICT solutions to the delivery of education services and qualifications. His background is in IT, education, publishing, logistics and business operations.
Martyn
Roads
Conference Director, Assessment Tomorrow
​
Martyn is an independent education and assessment consultant. He spent 13 years in Further Education, developing and managing a wide range of vocational programmes before joining BTEC in 1994. In 2001, he was responsible for managing, on behalf of a consortium of UK Awarding Bodies, the successful bid for the development of the Key and Basic Skills national tests in the UK. These were the first tests to be rolled-out in the UK using e-assessment. For the past 10 years, Martyn has been researching and implementing the use of IT in assessment which has included working with a range of organisations on the development of a range of screening and initial assessment tools for major UK employers. Martyn has also undertaken a number of other projects related to the development of vocational e-assessment for a range of awarding organisations including, most recently, supporting various project for Saudi Skills Standards in Saudi Arabia where they are developing a new vocational system which is heavily technology dependent.
Speakers
Cristina
Anguiano-Carrasco, Ph.D
ACT, Inc. Iowa City, Iowa
​
Cristina Anguiano-Carrasco, Ph.D. is currently a Research Scientist at the Center for Social, Emotional, and Academic Learning (SEAL-Team) at ACT, Inc. Iowa City, Iowa. She completed her doctoral dissertation in Research Methods and Psychometrics from the University Rovira I Virgili, Spain in 2013, where she also taught. Thereafter, she served as a Senior Research Associate at the Center for Academic and Workforce Readiness and Success, Educational Testing Service (2014-2017). She has also been a visiting scholar at CITO, in the Netherlands, and the University of Massachusetts, in the United States. Her research focuses on noncognitive skills, related response biases, and improving the measurement and assessment of noncognitive constructs, such as grit-tenacity, cross-cultural competence, teamwork, and self-efficacy. Over the course of her career she has published extensively in peer-review scientific journals, having developed over 20 different noncognitive assessments for different populations (i.e., K-12, higher education, and the workforce). Dr. Anguiano-Carrasco is currently the lead scientist on a range of major ACT research projects, which focus especially on international adaptations and evaluation of social and emotional learning products and services, including ACT Tessera.
Scott R.
Bartholomew, PhD
Purdue University
​
Scott R. Bartholomew, PhD. is an assistant professor of Engineering/Technology Teacher Education at Purdue University. Previously he taught Technology and Engineering classes at the middle school and university level. Dr. Bartholomew’s current work revolves around Adaptive Comparative Judgment (ACJ) assessment techniques, student design portfolios, and Technology & Engineering teacher preparation.
Jaehwa
Choi
The George Washington University​
​
Jaehwa Choi is Associate Professor and Program Director of the Assessment, Testing, and Measurement Program in the Department of Educational Leadership.
He is also the Executive Director of the Computer Adaptive Formative Assessment (CAFA) Laboratory (CAFA Lab, Inc.) and has conducted various Information and Communication Technology (ICT)-based assessment projects in collaboration with Samsung Electronics (Intelligence Solution Team), Korea Institute for Curriculum and Evaluation (KICE; Assessment Advancement Team), Korea Research Institute for Educational Assessment (KRIEA), and others.
Greg
Chung
Associate Director of Technology and Research Innovation, UCLA CRESST
​
Dr. Greg Chung is the Associate Director of Technology and Research Innovation for UCLA CRESST. His current work focuses on the impact of various learning technologies, such as games and intelligent tutoring systems on learning and engagement outcomes, the design of telemetry systems for games, and analytical approaches to support the modeling of learning outcomes from fine-grained data. Greg has led research funded by the U.S. Department of Education, National Science Foundation, Office of Naval Research, Defense Advanced Research Projects Agency, and PBS KIDS. This work spanned the evaluation of learning technologies, development of computer-based games and simulations, assessments of learning processes and outcomes using sensors, web-based measures of problem-solving, concept mapping, and AI-based computational methods.
Patrick
Craven
​
Director – Policy, Quality and Stakeholder Engagement – City & Guilds UK
​
Patrick has over 25 years experience in the field of accreditation, assessment design and educational technology. From the design of educational resources with MESU/NCET and BECTa to running of post-sales technology support departments with Apple Computer, his career has always addressed the fusion of education, skills development and assessment technologies.
For the last 15 years this work has focussed on the specific field of learning and assessment research and development linked to Government Policy and national accreditation programmes. This spanned work with the RSA Examinations Board, OCR Examinations, Cambridge Assessment Group and now he leads on Policy, Quality Assurance, Research/Design and Stakeholder Engagement for City & Guilds. This role considers the growing impact of Policy and new technologies on the learning and assessment industry. It also identifies initiatives to enable City & Guilds to develop and extend the range and quality of learning and assessment products and services they offer. Throughout this period he has remained committed to the notion of ‘fit for purpose’ assessment based on evidence-based research. Patrick currently sits on the Board of the e-Assessment Association, UK eAssessment Advisory Group, Joint Council for Qualifications and is an advisory member of the Federation of Awarding Bodies Board. He sits on the planning committee for the European Association of Test Publishers Conference and is a member of the Society of Leadership Fellows.
Tom
Gomersall
​
BTL, Shipley UK/Malvern, PA, USA
​
Tom joined BTL in 2002 after graduating from the University of Cambridge with an MEng in Electrical and Information Sciences. During this time, he has amassed a significant knowledge and understanding of the assessment industry and built close relationships with global testing organisations such as City & Guilds and Prometric. This has helped him to become the corner-stone of Surpass: The Assessment Platform’s cutting-edge development. As a Technical Architect, Tom works closely with both the Solutions and Development teams to meet the needs and desires of the Surpass Community. Tom pursues the regular addition of new Technology Enhanced Item Types to create more engaging and life-like simulation style questions alongside the more common Multiple Choice and Essay items. Tom is a keen researcher into innovative technologies and is the driving force behind BTL’s adoption of streamlined, AI-based applications of algorithmic computer marking and Item Response Theory (IRT). Recently, Tom has been pivotal in implementing Computer Adaptive Testing (CAT) and Linear On-the-Fly Testing (LOFT) methods directly into Surpass for all members of the Surpass Community. Tom is a classic example of how BTL doesn’t just employ the best in their field, but ensures that team members have a thorough knowledge and understanding of the assessment sector. This helps to ensure that solutions are technically and logically fit for purpose.
Gareth
Hegarty
​
Senior Manager, eTransition, International Baccalaureate
​
Gareth is passionate about developing and delivering fair and high-quality assessment that is relevant to the digital age. Following the successful implementation of a fully digital, media-rich assessment for the IB’s Middle Years Programme (MYP), Gareth is currently focused on introducing on-screen examinations to the IB’s Diploma and Career Related Programmes. Gareth joined the IB after 10 years as a science teacher and in management at schools in the UK, Russia and the Netherlands.
Graham
Hudson
GA Partnership
​
Graham Hudson is a long-standing member of the educational assessment community, having worked in the sector for 35 years. He has been involved in many of the significant developments in general qualifications, including the introduction of GCSEs, Curriculum 2000 and National Curriculum at Key Stage 4. Whilst at QCA he was responsible for the marking and collecting data for the Key Stage tests and after a move to DRS Data Services Limited was part of the team responsible for the first implementation for the live marking of UK examinations using scanned scripts. His interest in eAssessment started whilst at QCA where he led a development change programme involving all the regulatory bodies in the UK looking at the uses of new technologies in assessment, having won several £m of funding from the Treasury. He now runs his own consultancy businesses supporting organisations in the assessment sector in the UK and internationally. His great passion is to harness technology of all kinds to improve the examination experience that candidates face, the reliability of the assessments and marking accuracy.
Benjamin
Hunter
​
Benjamin Hunter, Technology Product Manager & Sales Director, Caveon Technology.
​
A recognized expert in high stakes test security, Benjamin brings a wealth of experience to his position at Caveon. Prior to joining the Caveon team, he began his career in the testing industry with the Pharmacy Technician Certification Board (PTCB) where he developed PTCB’s security program from scratch, and was responsible for the ongoing management and administration of PTCB’s program to ensure exam validity, reliable results, and a secure, consistent testing experience for candidates. His expertise in test development was acquired as the Senior Manager, Test Development and Maintenance at the Conference of State Bank Supervisors (CSBS), where he was responsible for oversight of all aspects of development and maintenance of the SAFE MLO exam program for Mortgage Loan Originators. Benjamin has served previously as an officer on the ATP Security Committee (ATPSC) and was involved in the creation of the 2015 publication of the ATPSC’s “Legal Protection of Assessment Content” document, as well as serving as the lead analyst for the ATPSC 2015 Security Survey, published Q4 2015. He was a speaker for the 2016 ICE Live Webinar series, where he presented on the creation and maintenance of a security plan for testing programs. He regularly presents at industry conferences including the ICE Exchange, ATP’s Innovations in Testing, and the Conference on Test Security. Benjamin is committed to matching clients with cutting-edge technology that will meet their individual program needs and to spearheading a push for new and innovative technology designed to benefit and protect the testing industry in the future.
Dee
Kanejiya
​
Founder and CEO of Cognii
​
Dee Kanejiya is the founder and CEO of Cognii, a leading provider of Artificial Intelligence technology to the education industry. Cognii is truly committed to shifting the assessment practices beyond the multiple-choice questions. Cognii’s award-winning Virtual Learning Assistant provides automatic assessment of open-response (short essay) answers along with personalized tutoring feedback. Dee has over 18 years of experience in technology and business development in the areas of AI, natural language processing, machine learning, and cognitive science. Prior to starting Cognii, he was part of AI companies such as Nuance Communications, and Vlingo Corporation that pioneered the virtual assistant technology for smartphones. Dee studied Master’s and PhD in Electrical Engineering from Indian Institute of Technology Delhi and conducted research at Carnegie Mellon University and Karlsruhe Institute of Technology, Germany. As an innovation leader, he is frequently invited to speak on the topic of AI in Education.
Elaine
Kelsey
​
Senior Software Engineer - Research, Eduworks
Elaine leads a team of developers using natural language processing, artificial intelligence, machine learning and data science to improve processes that require reading and understanding text and interacting through natural language. With a Masters from Georgia Institute of Technolgoy and a graduate of Oregon State, her skills include Natural Language Processing, Computational Linguistics and Intelligent Tutoring Systems
Gary
Klein
​
Cognitive Psychologist and Founder, Shadowbox
Gary Klein, Ph.D., is a cognitive psychologist and the author of Sources of Power: How People Make Decisions, and four other books plus three co-edited volumes. He is known for the cognitive models he described, such as the Recognition-Primed Decision (RPD) model, the Data/Frame model of sensemaking, the Management By Discovery model of planning in complex settings, and the Triple Path Model of Insight. He developed methods including the PreMortem method of risk assessment, techniques for Cognitive Task Analysis, and also pioneered the movement in 1989 — Naturalistic Decision Making. Dr. Klein has founded three companies. He launched his newest company, ShadowBox LLC, in 2014.
Pat
Leonard
resident, Business Development at Credly
Pat Leonard, Vice President, Business Development at Credly, lends a unique perspective that derives from a variety of leadership positions on all sides of the talent development ecosystem. In corporate education, Pat led the US Division of the American Management Association and was President of McGraw-Hill's Lifelong Learning Division. In talent development, she led the largest region of Manpower Group's Right Management division, providing career management and talent development solutions. In higher education, Pat led the Business & Education partnership alliances for Laureate Education's global online universities, was an executive publisher of college text and professional reference materials for Simon & Schuster, and initiated the workforce education division for Pearson. Now with Credly, Pat is leveraging that insight on both the supply and demand side of talent to provide a meaningful tool useful to the issuers of credentials, the earners of those credentials and those who consume or utilize those credentials.
Ed
Metz
Institute of Education Sciences, US Department of Education
Edward Metz is a developmental psychologist and director of the Small Business Innovation Research program (SBIR) at the Institute of Education Sciences in the U.S. Department of Education. Dr. Metz manages competitions and a portfolio of education technology projects, and specializes in working with industry leaders in the development of technology interventions, research to test the implementation of technology in schools, and on business models for the private sector commercialization of emerging forms of technology.
Jennifer
Murphy
​
Chief Executive Officer at Quantum Improvements Consulting
​
I am passionate about using my expertise in research psychology and training to improve the lives of others, especially those who serve our country. As the CEO of Quantum Improvements Consulting, I provide technical expertise in training design, metric development and effectiveness evaluation. I have over 10 years of military selection and training research experience, with an emphasis on leveraging innovative technologies for improving training in a measurably effective way. In addition to a technical background in experimental psychology, my skills include research design and execution, front end analysis, training design, data analysis, research reporting, and training effectiveness evaluation. I started my professional career as a Research Psychologist at ARI in the DC Metro area. There I worked toward developing non-cognitive assessments for Soldier selection and classification, selection measures for vigilance performance, and videogame-based training for night vision goggle use. In 2007, I moved to ARI’s Technology-Based Training unit in Orlando, FL. There, my projects included investigating the role of assessment in virtual and mobile training for Army schoolhouses. Additionally, I led a team of senior government scientists in developing measures for identifying the cognitive and perceptual skills critical to visual IED detection, and then developing and validating video, photo-realistic, and videogame-based training for these skills. Most recently, I served as Director of Defense Solutions at Design Interactive, Inc., where I managed a portfolio of training and performance support efforts incorporating cutting edge technology into training solutions for defense clients. My research has been featured in The New York Times, the Pentagon Channel, Soldier Magazine, and Signal Magazine.
Andreas
Oranje
General Manager at Educational Testing Service (ETS)
​
​Dr. Andreas Oranje is a general manager in Educational Testing Service’s Research division. He received his doctorate in psychology in 2001 and his Masters of Arts cum laude in psychology in 1998, both from the University of Amsterdam. As general manager he oversees research centers focused on the development and validation of generalizable research capabilities. His scope of responsibility includes directing various groups that conduct research on automated scoring, natural language and speech processing, dialogic speech systems and multimodal assessments, cognitive science, and assistive technologies. Before joining Research, Andreas worked on NAEP serving most recently as Project Director for Design, Analysis, and Reporting under the ETS contract of the National Assessment of Educational Progress (NAEP 2013-2017). He is the 2019 program chair for AERA Division D.
Prof. Graham
Orpwood
Professor Emeritus of Education at York University in Toronto, Canada and Director of International Programs for Vretta Inc
​
Graham Orpwood is Professor Emeritus of Education at York University in Toronto, Canada and Director of International Programs for Vretta Inc, an educational technology company dedicated to increasing numeracy. Graham’s background is as a teacher, researcher, curriculum developer and policy advisor with special interests in mathematics, science, and assessment. He was Director of the Science Council of Canada study of science in Canadian schools, Science Coordinator of TIMSS (Third International Mathematics and Science Study) and founding co-Director of the York/Seneca Institute for Mathematics, Science and Technology Education (YSIMSTE). He is co-author of many books, papers and reports, most recently Closing the Numeracy Gap (http://numeracygap.ca/).
Jacob
Pucar
​
Marketing Coordinator, VRETTA INC
Jacob Pucar maintains and manages Vretta’s branding and communication strategies globally. As the Marketing Coordinator at Vretta, he has a strong pulse on market needs, industry innovations and edtech disruptions that lead to the development of Vretta’s award-winning products that are transforming the way students learn mathematics. Jacob works closely with internal product, research, and business teams along with external stakeholders to identify unique ways of creating engaging experiences for the millions of students who Vretta supports internationally. With his shared vision to see a world where everyone enjoys math, Jacob utilizes unique marketing and branding strategies that communicate the value of math education to policy makers, data analysts, and educators at ministries of education and academic institutions across the globe. Jacob’s academic specialization in marketing and communications coupled with his passion for mathematics have helped him manage and strengthen Vretta’s reputation as the leading provider of the highest quality, math educational experience in the world.
Terrel
Rhodes
​
Vice President, Office of Quality, Curriculum, and Assessment and Executive Director of VALUE, AAC&U
Terry Rhodes received his B.A. from Indiana University at Bloomington and his M.A. and Ph.D. in Political Science from the University of North Carolina at Chapel Hill. Before moving into national higher education work, he was a faculty member for twenty-five years. Dr. Rhodes is currently Vice President for the Office of Quality, Curriculum and Assessment at the Association of American Colleges and Universities (AAC&U) where he focuses on the quality of undergraduate education, access, general education, and assessment of student learning. He is also the executive director of the Valid Assessment of Learning in Undergraduate Education (VALUE) project, and director of the annual AAC&U General Education Institute.
Denis
Saunders
​
CEO and Founder, Calibrand the currency for talent®
Denis is developing Calibrand the currency for talent® into a leading e-Assessment solutions organisation operating in the UK and internationally. Providing Calibrand Test, Marker and e-Portfolio systems as well as apps for android and apple the full solution is an online assessment centre. Embracing the 'anytime, anywhere and from any device' ethos of e-Assessment has made Calibrand successful in diverse global markets such as financial services regulation, government compliance, medical competence, awarding bodies, corporate talent management and international certification. In ‘Beyond Multiple-choice’ we will explain how we have adapted our online digital marking tool, Calibrand Marker, for use with scanned scripts specifically for high volume international markets. In these countries here are often up to half a million students sitting down to take the same exam at the same time. There is a demand for an OSM (On Screen Marking) solution to improve data collection and management and reduce fraud. Come and hear how successful this project has been and how we did it.
Sae
Schatz
​
Dr. Sae Schatz, Ph.D., Director, Advanced Distributed Learning (ADL) Initiative
Sae Schatz, Ph.D., Director of the ADL Initiative, is an applied human–systems scientist, with an emphasis on human cognition and learning, instructional technologies, adaptive systems, human performance assessment, and modeling and simulation. Frequently, her work seeks to enhance individuals’ higher-order cognitive skills (i.e., the mental, emotional, and relational skills associated with “cognitive readiness”). Dr. Schatz has applied her efforts to defense training and education for roughly a decade, and she holds a doctorate in Modeling and Simulation from the University of Central Florida.
Eric
Sydell
​
Eric Sydell, VP of Research and Innovation, Shaker
​
Eric Sydell, Ph.D., is one of the founders of Shaker, and currently directs Shaker’s research and product innovation initiatives. Driving growth through ingenuity, Eric has pioneered the transformation and adaptation of the assessment experience from desktop workstations to mobile devices. Over the last 20 years he has developed new ways to leverage technology not only to improve the assessment and selection process, but elevate the candidate experience. Mobilizing resources, fostering collaboration, and challenging assumptions are basic to Eric’s approach to stretching the boundaries of the possibilities for candidate assessment and keeping Shaker on the front line of connecting clients to solutions that best support their business goals. A regular presenter at professional conferences, he has published work on simulation scoring and taught and lectured extensively on assessment innovation, performance, testing procedures, and other human resource issues.
Doug
Whatley
​
Doug Whatley, founder and CEO of BreakAway Games
​
Doug Whatley, founder and CEO of BreakAway Games, has over 30 years of successful game development in the interactive entertainment industry. In the twenty years since he founded BreakAway, Mr. Whatley has positioned the company as a leading developer of serious games, training simulations, and innovative game technology applications for modeling, simulation, and visualization. By applying best practices from the development of real-time strategy games to the creation of simulation tools for experimentation, training, and decision-making analysis, he has attracted high-profile customers looking to apply virtual technology to solve real-world problems. Mr. Whatley was named an Ernst & Young Entrepreneur of the Year for 2004 and one of Business Week's 10 Cutting Edge Designers in 2006. He began his career at America Online where he was instrumental in helping to create the AOL client software. After that, he held senior-level management positions at MicroProse, ABC Sports, and Disney, where he worked on critically acclaimed software titles such as Darklands, Gunship 2000, Sid Meier's Civilization, and ABC's Monday Night Football
John
Winkley
​
John Winkley FCIEA, Director, AlphaPlus
​
John is a director of AlphaPlus Ltd and an experienced leader of projects in education. He is an experienced business manager with a track record of business growth in UK education and technology businesses. He has specialised in providing world-class services for standards, qualifications and assessment agencies in the UK and overseas through the three companies he has helped to found and build over the last 25 years: BTL Group Ltd (www.btl.com), The Virtual College plc (www.virtual-college.co.uk) and his current business, AlphaPlus (www.alphaplusconsultancy.co.uk). He has led design, development and evaluation work for academic, vocational and professional qualification systems, including English and mathematics for adults, UK and international GCSEs and A Levels, and vocational and professional qualifications in several countries. This includes managing the development and delivery of high-stakes examinations and skills-based assessments. His work for these and other clients includes quantitative and qualitative analysis, marking, moderation and standardisation programmes, assessment setting, and various quality assurance roles. John has a BSc and MEng in Electrical Engineering from the University of Bath. He is a Fellow of the Chartered Institute of Educational Assessors. He is also Chairman of The Beckfoot Trust - a Multi Academy Trust of 9 schools in West Yorkshire..
Jeffrey
Spranza
​
Director of Product Management, ITS - Internet Testing Systems
​
Experienced Director of Product Management and Project Management Professional (PMP) familiar with all phases of Systems and Software development including client engagement and relationship development, CONOPS development, requirements life-cycle management, systems development life-cycle management including maintenance and help desk operations. Acting as a liaison between businesspersons and software developers, my team translates business requirements and objectives into product vision, functional requirements and specifications, while balancing the needs of one stakeholder against the needs of all stakeholders.
