Apply to USF Now | Graduate Admissions | Events & Workshops | Giving to the Office of Graduate Studies

Graduate Course Proposal Form Submission Detail - EDF7469

Edit function not enabled for this course.


Current Status: Approved, Permanent Archive - 2005-11-10
Campus:
Submission Type:
Course Change Information (for course changes only):
Comments:


  1. Department and Contact Information

    Tracking Number Date & Time Submitted
    1833 2005-09-28
     
    Department College Budget Account Number
    Educational Measurement and Research ED 171100000
     
    Contact Person Phone Email
    Robert F. Dedrick 45722 dedrick@tempest.coedu.usf.edu

  2. Course Information

    Prefix Number Full Title
    EDF 7469 Introduction to Computer-Based Testing

    Is the course title variable? N
    Is a permit required for registration? N
    Are the credit hours variable? N
    Is this course repeatable?
    If repeatable, how many times? 0

    Credit Hours Section Type Grading Option
    3 C - Class Lecture (Primarily) R - Regular
     
    Abbreviated Title (30 characters maximum)
    Computer-Based Testing
     
    Course Online? Percentage Online
    -

    Prerequisites

    EDF 6432 Foundations of Educational Measurement or equivalent

    Corequisites

    Course Description

    This course should serve as an introduction to the field of computer-based testing. The material covered will be applicable to most operational educational, psychological, credentialing and licensure assessments, for research and measurement.


  3. Justification

    A. Please briefly explain why it is necessary and/or desirable to add this course.

    An analysis of the conference proceedings of national professional organizations (e.g., American Educational Research Association, National Council of Measurement in Education) show a substantial number of measurement/testing sessions devoted specifically

    B. What is the need or demand for this course? (Indicate if this course is part of a required sequence in the major.) What other programs would this course service?

    In addition to students from educational measurement and research, the course may be of interest to doctoral students from other programs in education, as well as doctoral students in psychology, public health, nursing, and business.

    C. Has this course been offered as Selected Topics/Experimental Topics course? If yes, how many times?

    4 times

    D. What qualifications for training and/or experience are necessary to teach this course? (List minimum qualifications for the instructor.)

    Doctoral degree meeting departmental requirements of at least 50% of doctoral coursework in the areas of Measurement, Statistics, and Evaluation. Documented training in computer-based testing.


  4. Other Course Information

    A. Objectives

    This course is intended to provide a broad coverage of the field of computerized assessment. The goal of the course is to inform students about relevant psychometric and computer issues, and to enable them to apply that learning to practical computerized testing applications.

    The successful completion of the course requirements is expected to result in increased ability to (a) intelligently read and evaluate relevant computer-based testing literature, (b) recognize the strengths and limitations of methods used in practical computer-based testing situations, (c) develop computer-based tests that meet professional measurement standards, (d) design research studies requiring the use of computer-based tests, and (e) communicate with peers and other professionals on computer-based testing topics.

    B. Learning Outcomes

    Grades will be based on 3 distinct activities. The first is a set of software evaluations that the student must conduct on different CBT programs. The second is a research proposal in which a CBT is either a major research tool or the focus of the research question. For the third activity, the student must design and develop a CBT, which they will present to the class.

    A description of each of these activities, and the rubrics used for grading them, are provided below.

    Software Evaluations (30%):

    Several computer test programs and demos will be available for students to view and evaluate. Students will also be expected to conduct an online search for information on additional CBT programs. Students will need to conduct a total of 3 software evaluations, describing the computer test software in terms of available measurement features, appropriate use of computer technology, quality of screen design, ease of use, information provided, etc.

    The first evaluation should be 1 page in length; the remaining two should each be 2-3 pages in length. The evaluations should include both descriptive and evaluative components. The first evaluation should be of a computer test program available in the lab. The remaining two evaluations should be of programs of the students choosing, from those in the lab, online selections, and "marketing-demo" versions. (In conjunction with these evaluations, students also need to conduct an online search for CBTs. They will need turn in a printed copy of the home page of a commercial CBT web site, along with a disk containing a demo version of the CBT program.)

    Rubric (for each of the 3 evaluations):

    ___ Required supplementary materials included (0-2)

    ___ Descriptive elements fully addressed (0-5)

    ___ measurement features

    ___ use of technology

    ___ screen design

    ___ ease of use

    ___ help/information provided

    ___ Evaluative elements fully addressed (0-5)

    ___ measurement features

    ___ use of technology

    ___ screen design

    ___ ease of use

    ___ help/information provided

    Research Proposal (35%):

    Students will be expected to develop a research proposal related to computer based testing. The proposal should be approximately 1200-1500 words in length (or, ~3 single-spaced pages, or ~5 double-spaced pages). It should include an introduction, background or literature review, purpose of the current study, methods, and (anticipated) results or conclusions. CBT applications described in the proposals can be for the purpose of conducting research, instrument development, prototype demonstration, or other justified use.

    Students will need to turn in copies of the abstracts of at least 3 relevant articles (used in the literature review), a draft version of the proposal, and a final version of the proposal (in a form suitable for submission). Students will be encouraged (but not required) to submit their proposal for presentation at FERA or other conference.

    Rubric:

    ____ Required supplementary materials included (0-2)

    ____ Rationale for questions included

    ____ Appropriate methods addressed

    ____ Analyses clearly described

    ____ Limitations noted

    ____ Accurate description/discussion of results

    ____ Conclusions consistent with results

    ____ Publishable writing of results section

    Computer Based Test Project/Presentation (35%):

    By the end of the course the students should have designed and developed a computerized exam. The exam should be developed for a particular test purpose and audience, using principles of measurement, CBT development, and screen design. The students may work individually or in small groups (perhaps a group of 2 or 3 students, supplying measurement, computer, and content skills). Several computer test programs will be available in the computer lab, or other programs may be used at the student's discretion. Usability analyses should be conducted on draft versions of the student projects.

    The items will need to be entered in a software program, and a computer test generated. The exam should include introductory and instruction screens, example items, closing screens, and score reporting output. Alternatively, if the student or group has selected a truly ambitious project, an option would be to thoroughly conceptualize the test while entering only some sections into the computer. Some components of the CBT can be illustrated through storyboards instead of generating them within the software program. The final project will be presented in class.

    The supporting materials for this exam should include full test specifications. The purpose of the test should be discussed, perhaps in terms of target audience, construct/domain, and test use (e.g., placement, proficiency, diagnostics). The test delivery model (i.e., CBT) should be specified, perhaps with a rationale for the model chosen. Item and test development can be addressed in terms of item format (i.e., multiple choice), item development and review process, content categories, and test specifications. The most appropriate estimates of reliability and validity for this application can be mentioned. Information about how the test would be administered (i.e., in a classroom setting, on a walk-in basis) could be included. Finally, the method for score reporting, to the examinee or others, could be addressed.

    Rubric:

    ____ Required supplementary materials included (0-2)

    ____ CBT satisfied all requirements (0-15)

    ____ Introductory screens

    ____ Instruction screens

    ____ Example items

    ____ Closing screens

    ____ Score reporting output

    ____ Presentation addressed all points appropriately (0-15)

    ____ Test purpose

    ____ Test delivery model

    ____ Item and test format decisions

    ____ Consideration of reliability and validity

    ____ Test administration plan

    ____ Score reporting plan

    C. Major Topics

    Overview of course

    Overview of CBT

    Administration issues

    Site-based security

    Development issues

    Examinee issues

    Software issues

    Screen design

    Usability testing

    Innovative items

    Research proposals & CBT projects

    Test development

    Classical test theory

    Computer Fixed Tests (CFT)

    Automated Test Assembly (ATA)

    IRT

    Computer adaptive tests (CAT)

    Computer classification tests (CCT)

    Testlets

    Comparison of test delivery methods

    Item pool evaluation

    Professional Guidelines for CBTs

    D. Textbooks

    Required Text:

    Parshall, C. G., Spray, J. A., Kalohn, J. C., & Davey, T. (2002). Practical considerations in computer-based testing. New York: Springer-Verlag.

    Additional Readings:

    Apple, Inc. (1995). Human Interface Design and the Development Process. In Macintosh Human Interface Guidelines. Reading, Ma: Addison-Wesley.

    Association of Test Publishers (2002). Guidelines for Computer-Based Testing.

    Bennett, R. E. (1999). Using new technology to improve assessment. Educational Measurement: Issues and Practice, 18, 5-12.

    CAT mini-tutorial http://www.ericae.net/scripts/cat/startcat.

    E. Course Readings, Online Resources, and Other Purchases

    F. Student Expectations/Requirements and Grading Policy

    G. Assignments, Exams and Tests

    H. Attendance Policy

    I. Policy on Make-up Work

    J. Program This Course Supports


  5. Course Concurrence Information



- if you have questions about any of these fields, please contact chinescobb@grad.usf.edu or joe@grad.usf.edu.