Assessing Teacher Dispositions: Five Standards-Based Steps to Valid Measurement Using the DAATS Model

Assessing Teacher Dispositions: Five Standards-Based Steps to Valid Measurement Using the DAATS Model

By: Judy R. Wilkerson (author), William Steve Lang (author)Paperback

1 - 2 weeks availability


This companion volume to the authors' Assessing Teacher Competency provides schools of education, teacher induction programs, and school districts with the safe, valid, and reliable framework for assessing and measuring teacher dispositions that they have long sought. Teacher affect and disposition have always been more difficult to define and measure than teacher knowledge and skills (competency), but together those three measures define the standards of best practice set by NCATE, INTASC, and NBPTS. The authors' 5-step DAATS model for measuring teacher affect and dispositions is research based, field tested, and presented in a step-by-step sequence with worksheets, training activities, and examples from both pre-service and in-service settings. The DAATS model for assessing teacher dispositions may be used on its own, or it may be used in conjunction with the authors' CAATS model for assessing teacher competency (knowledge and skills) as part of a comprehensive assessment system.

Create a review

About Author

Judy R. Wilkerson is an Associate Professor of Research and Assessment at Florida Gulf Coast University, where she teaches graduate and undergraduate courses in measurement and evaluation. As in this book and all of her research, she focuses her efforts with students on providing a highly pragmatic approach, based in theory, with the goal of instilling a commitment in them to assess K-12 learning. Her Ph.D. is in Measurement and Research from the University of South Florida, where she served for 15 years as Director of Program Review, leading College and University efforts in accreditation. Her career has been dedicated to standards-based assessment of programs and teachers, beginning with the creation of an evaluation model for accreditation in 1987, which she implemented in several states. Beginning in 1990 for 15 years, she served as the primary consultant for higher education to the Florida Department of Education, where she drafted the standards for the initial approval of teacher education programs, designed the program approval process, and provided technical assistance to colleges of education in evaluation of teachers and programs statewide. She has consulted nationally on NCATE accreditation and worked with state associations of teacher educators on accreditation related issues. She has also consulted with school districts in Florida on assessment systems for teachers. She was lead designer of the assessment system for the Florida Alternative Certification Program, now used in over 40 of the 68 school districts in the State. William Steve Lang is a Professor of Educational Measurement and Research at the University of South Florida St. Petersburg, where he teaches graduate courses in measurement, statistics, and research. He, too, focuses his teaching on making meaningful and pragmatic uses of the disciplines he teaches. He earned his Ph.D. from the University of Georgia in 1984. He has taught as a public school teacher in South Carolina and Georgia and as a college faculty member in South Carolina, Georgia, and Florida. He has published on a variety of applications in educational testing and works extensively with the Rasch Model of Item Response Theory. He began working extensively with Judy when she joined the faculty of the St. Petersburg Campus in 2001. Since that time, they have collaborated in all aspects of their research and service efforts with the Florida Department of Education, Florida school districts, and teacher education programs nationwide. They are working together to build two teacher assessment scales - one on teacher competencies, the subject of this book, and another on dispositions. Their work in both areas is standards-driven.


List of Tables List of Figures List of Boxes List of Abbreviations and Acronyms Foreword by Richard C. Kunkel Preface Acknowledgments About the Authors 1. What Are Dispositions and Why Should We Measure Them? What This Chapter Is About The Importance of Measuring Dispositions The Challenge What Are Standards-Based Dispositions? Hierarchical Relationships Among Knowledge, Skills, and Dispositions Remembering Bloom Dispositions and Accreditation Requirements: Requirements and Definitions Measuring Dispositions: Sources of Confusion Measuring Dispositions: Morals, Ethics, or Standards Based? Different Construct, Different Assessments, Similar Assessment Design Process Wrap Up Activity 1.1: Questions for Exploration Activity 1.2: What Have You Noticed? Activity 1.3: Assessment Belief Scale Activity 1.4: Cognitive, Affective, and Psychomotor Objectives and Assessments 2. Methods for Assessing Dispositions What This Chapter Is About A Conceptual Framework for Measuring Dispositions Measuring Teacher Dispositions: The State of the Art Back to Basics: Bloom and Krathwhol Available Methods for Measuring Dispositions or Affect The Importance of Inference in Measuring Dispositions Wrap Up Activity 2.1: Questions for Exploration Activity 2.2: Bloom and the INTASC Principles Activity 2.3: Field Work Activity 2.4: Review Your Feelings 3. DAATS Step 1: Assessment Design Inputs Where We Have Been So Far What This Chapter Is About Why Are Purpose, Use, Propositions, and Content So Important? DAATS Step 1A: Define the Purpose(s) and Use(s) of the System DAATS Step 1B: Define the Propositions or Principles That Guide the System DAATS Step 1C: Define the Conceptual Framework or Content of the System DAATS Step 1D: Review Local Factors That Impact the System Wrap Up Worksheet 3.1: Purpose, Use, Propositions, Content, and Context Checksheet Worksheet 3.2: Purpose, Use, Content, Draft Worksheet 3.3: Propositions Worksheet 3.4: Contextual Analysis 4. DAATS Step 2: Planning With a Continuing Eye on Valid Assessment Decisions Where We Have Been So Far What This Chapter Is About DAATS Step 2A: Analyze Standards and Indicators All Those Indicators Why Bother? DAATS Step 2B: Visualize the Teacher Demonstrating the Affective Targets DAATS Step 2C: Select Assessment Methods at Different Levels of Inference DAATS Step 2D: Build an Assessment Framework Correlating Standards and Methods Wrap Up DAATS Worksheet 4.1: Organizing for Alignment (Version 1) Worksheet 4.2: Organizing for Alignment (Version 2) Worksheet 4.3: Visualizing the Dispositional Statements Worksheet 4.4: Selecting Assessment Methods for INTASC Indicators Worksheet 4.5: Assessment Methods for INTASC Indicators: Blueprint Worksheet 4.6: Cost/Benefit and Coverage Analysis of Assessment Methods 5. DAATS Step 3: Instrument Development Where We Have Been So Far What This Chapter Is About DAATS Step 3A: Draft items for Each Instrument Thurstone Agreement Scales Questionnaires, Interviews, and Focus Groups Observed Performance Thematic Apperception Tests or Situation Reflection Assessment DAATS Step 3B: Review Items for Applicability to Values, Domain Coverage, Job Relevance Wrap Up Worksheet 5.1: Creating Scales Worksheet 5.2: Creating Questionnaires, Interviews, or K-12 Focus Group Protocols Worksheet 5.3: Creating an Affective Behaviour Checklist Worksheet 5.4: Creating an Affective Behaviour Rating Scale Worksheet 5.5: Creating a Tally Sheet for Affective Observation Worksheet 5.6: Checklist for Reviewing Scale Drafts Worksheet 5.7: Review Sheets for Questionnaires and Interviews Worksheet 5.8: Review Sheets for K-12 Focus Group Protocols Worksheet 5.9: Checklist for Reviewing Observations and Behavioral Checklists Worksheet 5.10: Coverage Check Worksheet 5.11: Rating Form for Stakeholder Review 6. DAATS Step 4: Decision Making and Data Management Where We Have Been So Far What This Chapter Is About DAATS Step 4A: Develop Scoring Rubrics Dichotomous Response Scoring Keys Rating Scale Rubrics DAATS Step 4B: Determine How Data Will Be Combined and Used Need for Shared Data Data Storage Data Aggregation Maximizing the Utility of the Data for Decision Making DAATS Step 4C: Develop Implementation Procedures and Materials Preponderance of the Evidence vs. Cut Scores Advising and Due Process Scoring Procedures Implementation Wrap Up Worksheet #6.1: Explanation of Dichotomous Scoring Decisions Worksheet #6.2: Rubric Design Worksheet #6.3: Sample Format for Candidate/Teacher Tracking Form Worksheet #6.4: Format for Data Aggregation Worksheet #6.5: Sample Disposition Event Report Worksheet #6.6: Management Plan 7. DAATS Step 5: Credible Data Where We Have Been So Far What This Chapter Is About What Is Psychometric Integrity and Why Do We Have to Worry About It? DAATS Step 5A: Create a Plan to Provide Evidence of Validity, Reliability, Fairness, and Utility Elements of a Plan Element 7.1: Purpose and Use Element 7.2: Construct Measured Element 7.3: Interpretation and Reporting of Scores Element 7.4: Assessment Specifications and Content Map Element 7.5: Assessor/Rater Selection and Training Procedures Element 7.6: Analysis Methodology Element 7.7: External Review Personnel and Methodology Element 7.8: Evidence of Validity, Reliability, and Fairness (VRF) Psychometric Evidence Collected Already Next Steps in Collecting Evidence of Validity, Reliability, and Fairness Future Studies DAATS Step 5B: Implement the Plan Conscientiously Wrap Up Worksheets and Examples Worksheet 7.1: Assessment Specifications Worksheet 7.2: Analysis of Appropriateness of Decisions for Teacher Failures Worksheet 7.3: Analysis of Rehire Data Worksheet 7.4: Program Improvement Record Worksheet 7.5: Expert Rescoring Worksheet 7.6: Fairness Review Worksheet 7.7: Analysis of Remediation Efforts and EO Impact Worksheet 7.8: Psychometric Plan Format Example 1: Logistic Ruler for Content Validity Example 2: Computation of the Lawshe (1975) Content Validity Ratio Example 3: Disparate Impact Analysis Example 4: Computation of Cohen's Kappa (1960) for Inter-rater Reliability Example 5: Two Pearson Correlation Coefficients and Scatterplots: Disposition Scores Correlated with PRAXIS and Portfolio Scores Example 6: Spearman Correlation Coefficient and Scatterplot: Disposition Scores Correlated With Principal Ratings Example 7: Correlation Matrix and Scatterplots Knowledge, Impact, Dispositions, Skills (KIDS) Example 8: T-Test Comparing Dispositions of Mathematics and Science Teachers Example 9: DIF Analysis for Programs 8. Using Teacher Scores for Continuous Improvement What This Chapter Is About Reasons Why We Use the Rasch Model The Classical Approach A Quick Overview of Where Rasch Fits Into the Grand Scheme of IRT Models Rasch: The Basics Getting Started Differences That Item Writers Make Guttman Scaling A Sample Rasch Ruler From Pictures to Numbers The Fit Statistic Gain Scores " Real or Imagined' Ratings and Raters Learning More About Rasch Wrap Up Activity #1: Decision-Making Tool for Measurement 9. Legal Integrity What This Chapter Is About Why Not Portfolios? Why the Pied Piper? What IF?? A Legal Scenario: Mary Beth JoAnne Sues XYZ University MBJ Helps Us to Understand the Convergency of Psychometrics and Legal Requirements Background Facts Scenario #1 Scenarios #2, 3, and 4 Psychometric Issues and Legal Challenges in the Real World Legal Issues and Precedents Three Landmark Dispositions Cases in Two Years Tide Changing in NCATE Standards Are the Vanguard! MBJ Revisited End Note Resource I. DAATS Steps and Worksheets Resource II. INTASC Disposition Indicators Glossary Index

Product Details

  • publication date: 10/07/2007
  • ISBN13: 9781412953689
  • Format: Paperback
  • Number Of Pages: 296
  • ID: 9781412953689
  • weight: 844
  • ISBN10: 1412953685

Delivery Information

  • Saver Delivery: Yes
  • 1st Class Delivery: Yes
  • Courier Delivery: Yes
  • Store Delivery: Yes

Prices are for internet purchases only. Prices and availability in WHSmith Stores may vary significantly