Apply to USF Now | Graduate Admissions | Events & Workshops | Giving to the Office of Graduate Studies

Graduate Course Proposal Form Submission Detail - EDF7439

Edit function not enabled for this course.


Current Status: Approved, Permanent Archive - 2005-11-10
Campus:
Submission Type:
Course Change Information (for course changes only):
Comments:


  1. Department and Contact Information

    Tracking Number Date & Time Submitted
    1834 2005-09-27
     
    Department College Budget Account Number
    Educational Measurement and Research ED 171100000
     
    Contact Person Phone Email
    Robert Dedrick 45722 dedrick@tempest.coedu.usf.edu

  2. Course Information

    Prefix Number Full Title
    EDF 7439 Foundations of Item Response Theory

    Is the course title variable? N
    Is a permit required for registration? N
    Are the credit hours variable? N
    Is this course repeatable?
    If repeatable, how many times? 0

    Credit Hours Section Type Grading Option
    3 C - Class Lecture (Primarily) R - Regular
     
    Abbreviated Title (30 characters maximum)
    Item Response Theory
     
    Course Online? Percentage Online
    -

    Prerequisites

    EDF 6432 Foundations of Educational Measurement or equivalent

    Corequisites

    Course Description

    Basic foundation underlying Item Response Theory (IRT) as well as most common applications in educational and psychological measurement, in terms of the theoretical basis, practical aspects, and specific applications.


  3. Justification

    A. Please briefly explain why it is necessary and/or desirable to add this course.

    An analysis of the premier measurement journals (e.g., Journal of Educational Measurement, Journal of Educational and Behavioral Statistics) and of the conference proceedings of national professional organizations (e.g., American Educational Research Asso

    B. What is the need or demand for this course? (Indicate if this course is part of a required sequence in the major.) What other programs would this course service?

    In addition to students from educational measurement and research the course may be of interest to doctoral students from other programs in education, as well as doctoral students in psychology, public health, nursing, and business.

    C. Has this course been offered as Selected Topics/Experimental Topics course? If yes, how many times?

    4 times

    D. What qualifications for training and/or experience are necessary to teach this course? (List minimum qualifications for the instructor.)

    Doctoral degree meeting departmental requirement of at least 50% of doctoral coursework in the areas of Measurement, Statistics, and Evaluation. Documented training in IRT. Documented experience using IRT in research.


  4. Other Course Information

    A. Objectives

    The course is intended to provide an overview of Item Response Theory (IRT). The goal of the course will be to enable students to understand the fundamentals of IRT and to be able to apply IRT to practical measurement problems. The material covered will be applicable to most educational and psychological applications, for research and practice.

    The successful completion of the course requirements is expected to result in increased ability to (a) intelligently read and evaluate psychometric literature, (b) recognize the strengths and limitations of psychometric analyses in practical testing situations, (c) design research studies requiring the use of IRT, and (d) communicate with peers and other professionals on psychometric topics. More specifically, students will be able to...

    Compare classical test theory and item response theory

    Compare the Rasch model to the 3-parameter logistic IRT model

    Conduct model-data fit analyses in support of model selection

    Use IRT software to calibrate test data

    Evaluate the quality of a calibration run

    Evaluate item and test information functions

    Use IRT for realistic measurement applications such as

    • constructing test forms

    • equating tests

    • conducting DIF analyses

    • preparing CATs

    B. Learning Outcomes

    Grades will be based on brief homework assignments and 4 increasingly extensive projects. Each project will include analyses and a brief write-up, and can be conducted using either Rasch or 3-PL approaches. A brief description of the projects and the rubrics used for grading are provided below.

    Homework Exercises (10%)

    Weekly self-study questions, computational problems, etc.

    Project # 1 – Ability Estimation (10%)

    using the sample Bilog program, EXAMPL01.BLG (and datafile EXAMPL01.DAT):

    • run the program as it is written

    • print out the resultant *.ph1, *.ph2, & *.ph3 files

    • identify where various commands in the *.blg file are confirmed in these files

    revise the command file EXAMPL01.BLG:

    • change the program to use the MLE (instead of EAP) for ability estimation

    • add a command so the program will save a file of examinee Scores

    • print out the resultant *.ph1, *.ph.2, & *.ph3, and *.scr files

    • compare the output from the two models

    turn in the following elements:

    • both sets of *.blg files

    • both sets of *.ph3 output files

    • the revised program’s *.scr file

    • write-up (about 1 page total)

    in your write-up include brief discussions of:

    • the primary data provided in each output file

    • what commands you entered (and where) to make the program revisions

    • what other similar options were available (i.e., for ability estimation & for saved files)

    • what impact resulted from changing the ability estimation method

    Rubric (to be scored on a 0-3 scale):

    ____ Accurate program code

    ____ Correct output

    ____ Accurate discussion of data

    ____ Accurate discussion of commands/program revisions

    ____ Accurate discussion of program options

    ____ Accurate discussion of ability estimation methods

    Project # 2 – Item Parameter Estimation (20%)

    using the datafile (from an 80-item certification test), certific.dat:

    • create the *.blg command file

    • use the 3-PL item response model

    • save the resultant item parameters to a *.par file

    • produce a plot of the test information function (TIF)

    turn in the following elements:

    • *.blg file

    • *.ph2 file

    • *.par file

    • write-up (about 1-2 page total)

    in your write-up include brief discussions of:

    • which ability estimation method you used, and why (or characteristics of that method)

    • what item parameter commands you entered, and their expected effects

    • compare the output of the *.ph2 and *.par files

    • describe the b, a, and c item parameter estimates in terms of:

    o range, mean, fit (using Bilog fit statistics)

    • analyze the set of items:

    o should one or more of these items be removed from the test?

    o would this be a good test form for certification testing?

    Rubric (to be scored on a 0-3 scale):

    ____ Accurate program code

    ____ Correct output

    ____ Accurate discussion of ability estimation method selected

    ____ Accurate discussion of item parameter commands

    ____ Accurate comparison of the 2 output files

    ____ Accurate description of item parameters

    ____ Accurate evaluation of overall set of items

    Project # 3 -- Model-Data Fit (25%)

    using the datafile (from a 20-item mastery test), mastery.dat:

    • create the *.blg command file

    • use the 1-Pl, 2-PL, and 3-PL item response models

    • save the resultant item parameters to *.par files

    • produce plots of the empirical and fitted ICCs for all items

    • examine the fit statistics

    turn in the following elements:

    • all 3 *.blg files (can be cut-and-pasted onto a single page)

    • all 3 *.par files

    • sample fit plots (e.g., best fitting and worst fitting items)

    • write-up (about 2 pages)

    in your write-up include brief discussions of:

    • state the ability estimation method you used, and why

    • state the item parameter commands you entered, and their expected effects

    • propose 1-3 additional analyses of fit that you feel would be most useful, and why

    • analyze the ‘pool’ of items (based on evidence to date) and make a recommendation

    for the ‘best’ item response model:

    o describe effects of the different item response models in terms of model-data fit

    o would this be a good test form (or set of items) for mastery testing?

    o should one or more of these items be removed from the test?

    Rubric (to be scored on a 0-3 scale):

    ____ Correct program code

    ____ Correct output

    ____ Correct plots

    ____ Accurate discussion of ability estimation method selected

    ____ Accurate discussion of item parameter commands and effects

    ____ Accurate description of additional fit analyses

    ____ Accurate evaluation/recommendation of overall set of items

    Project # 4 -- Individual Selection (w/ Instructor) (30%)

    Students will complete an IRT analysis of their own choosing, using real data and relevant questions. The student will identify and/or gather the data, conduct the analyses using appropriate software, evaluate the results, and write a report on the study in APA format, where the results section should be of a quality suitable for publication.

    project possibilities include:

    • Item parameter estimation

    • Information functions

    • Model-data fit

    • Test assembly

    • Test equating

    • DIF

    • CAT

    turn in the following elements:

    • any *.blg files

    • any relevant output files

    • any relevant plots

    • write-up (at least 4-5 pages)

    in your write-up include brief discussions of:

    • rationale for question(s) investigated

    • appropriate level of detail regarding data

    • clearly described analyses

    • program code

    • description/discussion of results

    • limitations of the study

    • conclusions consistent with results

    Rubric (to be scored on a 0-3 scale):

    ____ Correct program code

    ____ Correct output/plots

    ____ Correct plots

    ____ Rationale for questions included

    ____ Appropriate level of detail regarding sample

    ____ Appropriate level of detail regarding variables

    ____ Analyses clearly described

    ____ Model/data fit addressed

    ____ Limitations noted

    ____ Accurate description/discussion of results

    ____ Conclusions consistent with results

    ____ Publishable writing of results section

    C. Major Topics

    Background & overview

    Assumptions

    Models (1-PL, 2-PL, and 3-PL)

    Score scales

    Ability estimation

    Ability & item estimation

    Information functions

    Model-data fit

    Test construction

    Test equating

    DIF

    CAT

    Rasch v. 3-PL

    Rasch overview

    D. Textbooks

    Required Text:

    Hambleton, R. K., & Swaminathan, H.(1985). Item Response Theory – Principles and Applications. Boston: Kluwer-Nijhoff.

    Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of Item Response Theory. Newbury Park: Sage.

    Additional Readings:

    Ban, J., Hanson, B. A., Wang, T., Yi, Q., & Harris, D. (2001). A comparative study of online pretest items – Calibration/scaling methods in CAT. Journal of Educational Measurement, 38, 191-212.

    Ban, J., Hanson, B. A., Yi, Q., & Harris, D. (2002). Data sparseness and online pretest item calibration-scaling

    E. Course Readings, Online Resources, and Other Purchases

    F. Student Expectations/Requirements and Grading Policy

    G. Assignments, Exams and Tests

    H. Attendance Policy

    I. Policy on Make-up Work

    J. Program This Course Supports


  5. Course Concurrence Information



- if you have questions about any of these fields, please contact chinescobb@grad.usf.edu or joe@grad.usf.edu.