Technology-Based Testing: The State of the Art

John Schroeder, CGCC
A new Ocotillo committee was formed this year to investigate technology-based testing. Issues researched and discussed were the following:

What should a technology-based testing system include?
What technology is now available for testing?
What is it going to take to develop a technology-based testing system (or should we)?
What are the implications for teaching and learning?

Discussion revealed there were two different schools of thought driving the interest in technology-based testing at the Maricopa Community Colleges.

Computer-generated tests are needed for areas dealing with hundreds of students in multiple sections that require standardized minimum performance level outcomes. MAT 077 classes are an example, but other courses might have the same potential application.
Computer administered testing is needed to prepare students for the computerized tests that are used (or will be in the near future) by state or federal licensing authorities. Nursing, dental, legal, aviation, and EMT are some of the areas of perceived need.

What should a tech-based testing system include?

After investigating and reviewing commercially available testing systems as well as those currently being used in Maricopa, the following capabilities for a technology-based testing system are recommended:

Support for text, color, graphics, sound, and video
Support for the development of alternative assessment measures
Support for random selection of questions
Support for hard copy of tests
Capabilities for record keeping of student responses for reliability / validity purposes
Support time inputs for timed tests
Support for utilization records
Ease of use for students and faculty
Capability to change, or update questions and feedback easily
Available for multiple computer platforms (Mac, IBM, Unix)
The testing mechanism should not be discipline specific, but usable by all interested disciplines
Support explanatory feedback during review

What is currently being used at MCCD?

There are several applications currently in use within the District:

Nursing faculty at GCC use a Windows-based program for testing their students. The test engine is capable of handling multiple-choice, true/false, matching, and short answer questions. While still early in its use, it is perhaps the most mature application developed within the District.
Nursing faculty at PC are using "RN Challenge" as a review vehicle.
Legal faculty at PC have some similar applications for their students.
Aviation Maintenance Technology at CGCC have text files of 2100 questions for faculty use only.
Some individual faculty have experimented with LXR as a test database engine for written tests, but we did not locate any widespread use or sharing of question databases.>

What is it going to take to develop a technology-based testing system (or should we)?

The committee has concluded that the capability to deliver a test across multiple platforms is one of the most critical issues that should be considered at Maricopa when selecting or developing a technology-based testing system. There are database development tools available that are cross- platform; it would seem that a fast database would be the logical choice. For example, FoxPro2.5 is currently available for DOS and Windows with a Macintosh version due in October; a Unix version is reportedly to follow next year.

What are the implications for teaching and learning?

Concerns about "computer phobia" do not seem to be a major hurdle, provided the software uses a consistent interface. The ability to return to a question and to mark a question for review is seen by some to reduce student concerns. Clearly, anything that makes test item analysis easy has the potential to lead to better teaching and learning.

Review capabilities with explanations on a large selection of test items will allow the student to cover the material in depth before they are evaluated. This will enable the students to insure their understanding of the material and will lead to broader understanding.

Maricopa Center for Learning & Instruction (MCLI)
The Internet Connection at MCLI is Alan Levine --}
Comments to alan.levine@domail.maricopa.edu