You are here

Assessing Teacher Dispositions
Share

Assessing Teacher Dispositions
Five Standards-Based Steps to Valid Measurement Using the DAATS Model



May 2007 | 296 pages | Corwin
This companion volume to the authors' Assessing Teacher Competency provides schools of education, teacher induction programs, and school districts with the safe, valid, and reliable framework for assessing and measuring teacher dispositions that they have long sought. Teacher affect and disposition have always been more difficult to define and measure than teacher knowledge and skills (competency), but together those three measures define the standards of best practice set by NCATE, INTASC, and NBPTS. The authors' 5-step DAATS model for measuring teacher affect and dispositions is research based, field tested, and presented in a step-by-step sequence with worksheets, training activities, and examples from both pre-service and in-service settings. The DAATS model for assessing teacher dispositions may be used on its own, or it may be used in conjunction with the authors' CAATS model for assessing teacher competency (knowledge and skills) as part of a comprehensive assessment system.
 
List of Tables
 
List of Figures
 
List of Boxes
 
List of Abbreviations and Acronyms
 
Foreword by Richard C. Kunkel
 
Preface
 
Acknowledgments
 
About the Authors
 
1. What Are Dispositions and Why Should We Measure Them?
What This Chapter Is About

 
The Importance of Measuring Dispositions

 
The Challenge

 
What Are Standards-Based Dispositions?

 
Hierarchical Relationships Among Knowledge, Skills, and Dispositions

 
Remembering Bloom

 
Dispositions and Accreditation Requirements: Requirements and Definitions

 
Measuring Dispositions: Sources of Confusion

 
Measuring Dispositions: Morals, Ethics, or Standards Based?

 
Different Construct, Different Assessments, Similar Assessment Design Process

 
Wrap Up

 
Activity 1.1: Questions for Exploration

 
Activity 1.2: What Have You Noticed?

 
Activity 1.3: Assessment Belief Scale

 
Activity 1.4: Cognitive, Affective, and Psychomotor Objectives and Assessments

 
 
2. Methods for Assessing Dispositions
What This Chapter Is About

 
A Conceptual Framework for Measuring Dispositions

 
Measuring Teacher Dispositions: The State of the Art

 
Back to Basics: Bloom and Krathwhol

 
Available Methods for Measuring Dispositions or Affect

 
The Importance of Inference in Measuring Dispositions

 
Wrap Up

 
Activity 2.1: Questions for Exploration

 
Activity 2.2: Bloom and the INTASC Principles

 
Activity 2.3: Field Work

 
Activity 2.4: Review Your Feelings

 
 
3. DAATS Step 1: Assessment Design Inputs
Where We Have Been So Far

 
What This Chapter Is About

 
Why Are Purpose, Use, Propositions, and Content So Important?

 
DAATS Step 1A: Define the Purpose(s) and Use(s) of the System

 
DAATS Step 1B: Define the Propositions or Principles That Guide the System

 
DAATS Step 1C: Define the Conceptual Framework or Content of the System

 
DAATS Step 1D: Review Local Factors That Impact the System

 
Wrap Up

 
Worksheet 3.1: Purpose, Use, Propositions, Content, and Context Checksheet

 
Worksheet 3.2: Purpose, Use, Content, Draft

 
Worksheet 3.3: Propositions

 
Worksheet 3.4: Contextual Analysis

 
 
4. DAATS Step 2: Planning With a Continuing Eye on Valid Assessment Decisions
Where We Have Been So Far

 
What This Chapter Is About

 
DAATS Step 2A: Analyze Standards and Indicators

 
All Those Indicators

 
Why Bother?

 
DAATS Step 2B: Visualize the Teacher Demonstrating the Affective Targets

 
DAATS Step 2C: Select Assessment Methods at Different Levels of Inference

 
DAATS Step 2D: Build an Assessment Framework Correlating Standards and Methods

 
Wrap Up

 
DAATS

 
Worksheet 4.1: Organizing for Alignment (Version 1)

 
Worksheet 4.2: Organizing for Alignment (Version 2)

 
Worksheet 4.3: Visualizing the Dispositional Statements

 
Worksheet 4.4: Selecting Assessment Methods for INTASC Indicators

 
Worksheet 4.5: Assessment Methods for INTASC Indicators: Blueprint

 
Worksheet 4.6: Cost/Benefit and Coverage Analysis of Assessment Methods

 
 
5. DAATS Step 3: Instrument Development
Where We Have Been So Far

 
What This Chapter Is About

 
DAATS Step 3A: Draft items for Each Instrument

 
Thurstone Agreement Scales

 
Questionnaires, Interviews, and Focus Groups

 
Observed Performance

 
Thematic Apperception Tests or Situation Reflection Assessment

 
DAATS Step 3B: Review Items for Applicability to Values, Domain Coverage, Job Relevance

 
Wrap Up

 
Worksheet 5.1: Creating Scales

 
Worksheet 5.2: Creating Questionnaires, Interviews, or K-12 Focus Group Protocols

 
Worksheet 5.3: Creating an Affective Behaviour Checklist

 
Worksheet 5.4: Creating an Affective Behaviour Rating Scale

 
Worksheet 5.5: Creating a Tally Sheet for Affective Observation

 
Worksheet 5.6: Checklist for Reviewing Scale Drafts

 
Worksheet 5.7: Review Sheets for Questionnaires and Interviews

 
Worksheet 5.8: Review Sheets for K-12 Focus Group Protocols

 
Worksheet 5.9: Checklist for Reviewing Observations and Behavioral Checklists

 
Worksheet 5.10: Coverage Check

 
Worksheet 5.11: Rating Form for Stakeholder Review

 
 
6. DAATS Step 4: Decision Making and Data Management
Where We Have Been So Far

 
What This Chapter Is About

 
DAATS Step 4A: Develop Scoring Rubrics

 
Dichotomous Response Scoring Keys

 
Rating Scale Rubrics

 
DAATS Step 4B: Determine How Data Will Be Combined and Used

 
Need for Shared Data

 
Data Storage

 
Data Aggregation

 
Maximizing the Utility of the Data for Decision Making

 
DAATS Step 4C: Develop Implementation Procedures and Materials

 
Preponderance of the Evidence vs. Cut Scores

 
Advising and Due Process

 
Scoring Procedures

 
Implementation

 
Wrap Up

 
Worksheet #6.1: Explanation of Dichotomous Scoring Decisions

 
Worksheet #6.2: Rubric Design

 
Worksheet #6.3: Sample Format for Candidate/Teacher Tracking Form

 
Worksheet #6.4: Format for Data Aggregation

 
Worksheet #6.5: Sample Disposition Event Report

 
Worksheet #6.6: Management Plan

 
 
7. DAATS Step 5: Credible Data
Where We Have Been So Far

 
What This Chapter Is About

 
What Is Psychometric Integrity and Why Do We Have to Worry About It?

 
DAATS Step 5A: Create a Plan to Provide Evidence of Validity, Reliability, Fairness, and Utility

 
Elements of a Plan

 
Element 7.1: Purpose and Use

 
Element 7.2: Construct Measured

 
Element 7.3: Interpretation and Reporting of Scores

 
Element 7.4: Assessment Specifications and Content Map

 
Element 7.5: Assessor/Rater Selection and Training Procedures

 
Element 7.6: Analysis Methodology

 
Element 7.7: External Review Personnel and Methodology

 
Element 7.8: Evidence of Validity, Reliability, and Fairness (VRF)

 
Psychometric Evidence Collected Already

 
Next Steps in Collecting Evidence of Validity, Reliability, and Fairness

 
Future Studies

 
DAATS Step 5B: Implement the Plan Conscientiously

 
Wrap Up

 
Worksheets and Examples

 
Worksheet 7.1: Assessment Specifications

 
Worksheet 7.2: Analysis of Appropriateness of Decisions for Teacher Failures

 
Worksheet 7.3: Analysis of Rehire Data

 
Worksheet 7.4: Program Improvement Record

 
Worksheet 7.5: Expert Rescoring

 
Worksheet 7.6: Fairness Review

 
Worksheet 7.7: Analysis of Remediation Efforts and EO Impact

 
Worksheet 7.8: Psychometric Plan Format

 
Example 1: Logistic Ruler for Content Validity

 
Example 2: Computation of the Lawshe (1975) Content Validity Ratio

 
Example 3: Disparate Impact Analysis

 
Example 4: Computation of Cohen's Kappa (1960) for Inter-rater Reliability

 
Example 5: Two Pearson Correlation Coefficients and Scatterplots: Disposition Scores Correlated with PRAXIS and Portfolio Scores

 
Example 6: Spearman Correlation Coefficient and Scatterplot: Disposition Scores Correlated With Principal Ratings

 
Example 7: Correlation Matrix and Scatterplots Knowledge, Impact, Dispositions, Skills (KIDS)

 
Example 8: T-Test Comparing Dispositions of Mathematics and Science Teachers

 
Example 9: DIF Analysis for Programs

 
 
8. Using Teacher Scores for Continuous Improvement
What This Chapter Is About

 
Reasons Why We Use the Rasch Model

 
The Classical Approach

 
A Quick Overview of Where Rasch Fits Into the Grand Scheme of IRT Models

 
Rasch: The Basics

 
Getting Started

 
Differences That Item Writers Make

 
Guttman Scaling

 
A Sample Rasch Ruler

 
From Pictures to Numbers

 
The Fit Statistic

 
Gain Scores " Real or Imagined'

 
Ratings and Raters

 
Learning More About Rasch

 
Wrap Up

 
Activity #1: Decision-Making Tool for Measurement

 
 
9. Legal Integrity
What This Chapter Is About

 
Why Not Portfolios?

 
Why the Pied Piper?

 
What IF?? A Legal Scenario: Mary Beth JoAnne Sues XYZ University

 
MBJ Helps Us to Understand the Convergency of Psychometrics and Legal Requirements

 
Background Facts

 
Scenario #1

 
Scenarios #2, 3, and 4

 
Psychometric Issues and Legal Challenges in the Real World

 
Legal Issues and Precedents

 
Three Landmark Dispositions Cases in Two Years

 
Tide Changing in NCATE

 
Standards Are the Vanguard!

 
MBJ Revisited

 
End Note

 
 
Resource I. DAATS Steps and Worksheets
 
Resource II. INTASC Disposition Indicators
 
Glossary
 
Index

"The text has been well researched, is standards-based, and includes activities, worksheets, definitions, and rubrics. It addresses a topic that has been a mystery to assessment gurus."

Marilyn K. Troupe, Director, Division of Educator Preparation
Kentucky Education Professional Standards Board

“We have all been through statistics classes and measurement classes, but this text actually helps us apply measurement ideas to the concept of dispositions. It should be extremely helpful to education faculty as well as to their liberal arts and science counterparts who often do not understand the concept of dispositions or why they are important and need to be measured or how one even begins to develop an assessment system that includes such measurements.”

Martha Gage, Director of Teacher Education and Licensure
Kansas State Department of Education

"School leaders will find an important, key tool to assessing teacher dispositions. It's been field-tested, so it isn't just theory. An essential guide."

California Bookwatch, August 2007

"This book is chock full of worksheets, assessment design techniques, staff discussion and training activities, and a variety of examples as to how the above can be used for pre-and in-service settings, with or without a statistical focus. This is not a book written for the average consumer, but rather for teacher educators, assessment designers, and K-12 school administrators and staff developers. From an instructor’s perspective, sections of the text would be excellent as an introduction to quantitative research methods for the social sciences."

Teachers College Record, August 2008

Preview this book

Judy R. Wilkerson

Judy R. Wilkerson is an Associate Professor of Research and Assessment at Florida Gulf Coast University, where she teaches graduate and undergraduate courses in measurement and evaluation. As in this book and all of her research, she focuses her efforts with students on providing a highly pragmatic approach, based in theory, with the goal of instilling a commitment in them to assess K-12 learning. Her Ph.D. is in Measurement and Research from the University of South Florida, where she served for 15 years as Director of Program Review, leading College and University efforts in accreditation. Her career has been dedicated to standards-based... More About Author

William Steve Lang

William Steve Lang is a Professor of Educational Measurement and Research at the University of South Florida St. Petersburg, where he teaches graduate courses in measurement, statistics, and research.   He, too, focuses his teaching on making meaningful and pragmatic uses of the disciplines he teaches.  He earned his Ph.D. from the University of Georgia in 1984.  He has taught as a public school teacher in South Carolina and Georgia and as a college faculty member in South Carolina, Georgia, and Florida.   He has published on a variety of applications in educational testing and works extensively with the Rasch Model of Item Response Theory... More About Author

Purchasing options

Please select a format:

ISBN: 9781412953689
$44.95 
ISBN: 9781412953672
$90.95