Graduate Studies Reports Access
Graduate Course Proposal Form Submission Detail - EDF7439
Tracking Number - 1834
Edit function not enabled for this course.
Current Status:
Approved, Permanent Archive - 2005-11-10
Campus:
Submission Type:
Course Change Information (for course changes only):
Comments:
Detail Information
- Date & Time Submitted: 2005-09-27
- Department: Educational Measurement and Research
- College: ED
- Budget Account Number: 171100000
- Contact Person: Robert Dedrick
- Phone: 45722
- Email: dedrick@tempest.coedu.usf.edu
- Prefix: EDF
- Number: 7439
- Full Title: Foundations of Item Response Theory
- Credit Hours: 3
- Section Type: C -
Class Lecture (Primarily)
- Is the course title variable?: N
- Is a permit required for registration?: N
- Are the credit hours variable?: N
- Is this course repeatable?:
- If repeatable, how many times?: 0
- Abbreviated Title (30 characters maximum): Item Response Theory
- Course Online?: -
- Percentage Online:
- Grading Option:
R - Regular
- Prerequisites: EDF 6432 Foundations of Educational Measurement or equivalent
- Corequisites:
- Course Description: Basic foundation underlying Item Response Theory (IRT) as well as most common applications in educational and psychological measurement, in terms of the theoretical basis, practical aspects, and specific applications.
- Please briefly explain why it is necessary and/or desirable to add this course: An analysis of the premier measurement journals (e.g., Journal of Educational Measurement, Journal of Educational and Behavioral Statistics) and of the conference proceedings of national professional organizations (e.g., American Educational Research Asso
- What is the need or demand for this course? (Indicate if this course is part of a required sequence in the major.) What other programs would this course service? In addition to students from educational measurement and research the course may be of interest to doctoral students from other programs in education, as well as doctoral students in psychology, public health, nursing, and business.
- Has this course been offered as Selected Topics/Experimental Topics course? If yes, how many times? 4 times
- What qualifications for training and/or experience are necessary to teach this course? (List minimum qualifications for the instructor.) Doctoral degree meeting departmental requirement of at least 50% of doctoral coursework in the areas of Measurement, Statistics, and Evaluation. Documented training in IRT. Documented experience using IRT in research.
- Objectives: The course is intended to provide an overview of Item Response Theory (IRT). The goal of the course will be to enable students to understand the fundamentals of IRT and to be able to apply IRT to practical measurement problems. The material covered will be applicable to most educational and psychological applications, for research and practice.
The successful completion of the course requirements is expected to result in increased ability to (a) intelligently read and evaluate psychometric literature, (b) recognize the strengths and limitations of psychometric analyses in practical testing situations, (c) design research studies requiring the use of IRT, and (d) communicate with peers and other professionals on psychometric topics. More specifically, students will be able to...
Compare classical test theory and item response theory
Compare the Rasch model to the 3-parameter logistic IRT model
Conduct model-data fit analyses in support of model selection
Use IRT software to calibrate test data
Evaluate the quality of a calibration run
Evaluate item and test information functions
Use IRT for realistic measurement applications such as
constructing test forms
equating tests
conducting DIF analyses
preparing CATs
- Learning Outcomes: Grades will be based on brief homework assignments and 4 increasingly extensive projects. Each project will include analyses and a brief write-up, and can be conducted using either Rasch or 3-PL approaches. A brief description of the projects and the rubrics used for grading are provided below.
Homework Exercises (10%)
Weekly self-study questions, computational problems, etc.
Project # 1 Ability Estimation (10%)
using the sample Bilog program, EXAMPL01.BLG (and datafile EXAMPL01.DAT):
run the program as it is written
print out the resultant *.ph1, *.ph2, & *.ph3 files
identify where various commands in the *.blg file are confirmed in these files
revise the command file EXAMPL01.BLG:
change the program to use the MLE (instead of EAP) for ability estimation
add a command so the program will save a file of examinee Scores
print out the resultant *.ph1, *.ph.2, & *.ph3, and *.scr files
compare the output from the two models
turn in the following elements:
both sets of *.blg files
both sets of *.ph3 output files
the revised programs *.scr file
write-up (about 1 page total)
in your write-up include brief discussions of:
the primary data provided in each output file
what commands you entered (and where) to make the program revisions
what other similar options were available (i.e., for ability estimation & for saved files)
what impact resulted from changing the ability estimation method
Rubric (to be scored on a 0-3 scale):
____ Accurate program code
____ Correct output
____ Accurate discussion of data
____ Accurate discussion of commands/program revisions
____ Accurate discussion of program options
____ Accurate discussion of ability estimation methods
Project # 2 Item Parameter Estimation (20%)
using the datafile (from an 80-item certification test), certific.dat:
create the *.blg command file
use the 3-PL item response model
save the resultant item parameters to a *.par file
produce a plot of the test information function (TIF)
turn in the following elements:
*.blg file
*.ph2 file
*.par file
write-up (about 1-2 page total)
in your write-up include brief discussions of:
which ability estimation method you used, and why (or characteristics of that method)
what item parameter commands you entered, and their expected effects
compare the output of the *.ph2 and *.par files
describe the b, a, and c item parameter estimates in terms of:
o range, mean, fit (using Bilog fit statistics)
analyze the set of items:
o should one or more of these items be removed from the test?
o would this be a good test form for certification testing?
Rubric (to be scored on a 0-3 scale):
____ Accurate program code
____ Correct output
____ Accurate discussion of ability estimation method selected
____ Accurate discussion of item parameter commands
____ Accurate comparison of the 2 output files
____ Accurate description of item parameters
____ Accurate evaluation of overall set of items
Project # 3 -- Model-Data Fit (25%)
using the datafile (from a 20-item mastery test), mastery.dat:
create the *.blg command file
use the 1-Pl, 2-PL, and 3-PL item response models
save the resultant item parameters to *.par files
produce plots of the empirical and fitted ICCs for all items
examine the fit statistics
turn in the following elements:
all 3 *.blg files (can be cut-and-pasted onto a single page)
all 3 *.par files
sample fit plots (e.g., best fitting and worst fitting items)
write-up (about 2 pages)
in your write-up include brief discussions of:
state the ability estimation method you used, and why
state the item parameter commands you entered, and their expected effects
propose 1-3 additional analyses of fit that you feel would be most useful, and why
analyze the pool of items (based on evidence to date) and make a recommendation
for the best item response model:
o describe effects of the different item response models in terms of model-data fit
o would this be a good test form (or set of items) for mastery testing?
o should one or more of these items be removed from the test?
Rubric (to be scored on a 0-3 scale):
____ Correct program code
____ Correct output
____ Correct plots
____ Accurate discussion of ability estimation method selected
____ Accurate discussion of item parameter commands and effects
____ Accurate description of additional fit analyses
____ Accurate evaluation/recommendation of overall set of items
Project # 4 -- Individual Selection (w/ Instructor) (30%)
Students will complete an IRT analysis of their own choosing, using real data and relevant questions. The student will identify and/or gather the data, conduct the analyses using appropriate software, evaluate the results, and write a report on the study in APA format, where the results section should be of a quality suitable for publication.
project possibilities include:
Item parameter estimation
Information functions
Model-data fit
Test assembly
Test equating
DIF
CAT
turn in the following elements:
any *.blg files
any relevant output files
any relevant plots
write-up (at least 4-5 pages)
in your write-up include brief discussions of:
rationale for question(s) investigated
appropriate level of detail regarding data
clearly described analyses
program code
description/discussion of results
limitations of the study
conclusions consistent with results
Rubric (to be scored on a 0-3 scale):
____ Correct program code
____ Correct output/plots
____ Correct plots
____ Rationale for questions included
____ Appropriate level of detail regarding sample
____ Appropriate level of detail regarding variables
____ Analyses clearly described
____ Model/data fit addressed
____ Limitations noted
____ Accurate description/discussion of results
____ Conclusions consistent with results
____ Publishable writing of results section
- Major Topics: Background & overview
Assumptions
Models (1-PL, 2-PL, and 3-PL)
Score scales
Ability estimation
Ability & item estimation
Information functions
Model-data fit
Test construction
Test equating
DIF
CAT
Rasch v. 3-PL
Rasch overview
- Textbooks: Required Text:
Hambleton, R. K., & Swaminathan, H.(1985). Item Response Theory Principles and Applications. Boston: Kluwer-Nijhoff.
Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of Item Response Theory. Newbury Park: Sage.
Additional Readings:
Ban, J., Hanson, B. A., Wang, T., Yi, Q., & Harris, D. (2001). A comparative study of online pretest items Calibration/scaling methods in CAT. Journal of Educational Measurement, 38, 191-212.
Ban, J., Hanson, B. A., Yi, Q., & Harris, D. (2002). Data sparseness and online pretest item calibration-scaling
- Course Readings, Online Resources, and Other Purchases:
- Student Expectations/Requirements and Grading Policy:
- Assignments, Exams and Tests:
- Attendance Policy:
- Policy on Make-up Work:
- Program This Course Supports:
- Course Concurrence Information:
- if you have questions about any of these fields, please contact chinescobb@grad.usf.edu or joe@grad.usf.edu.