Abstract
School personnel regularly conduct systematic direct observation (SDO) as part of educational evaluation and intervention progress monitoring. Despite the regularity with which SDO is utilized, little research has been devoted to the identification of efficient and effective training procedures. Thirty-two undergraduate students in education participated in the current study and were randomly assigned to one of two methods of feedback as a means of promoting accurate SDO: post hoc feedback and real-time feedback. Participants in each group completed three observations, one pre-training, one training, and one post-training. The dependent variables assessed were changes in interobserver agreement (IOA) with a criterion and overall accuracy of the observation. Results did not reveal significant differences in IOA between the two groups. A significant interaction was found between group assignment and measurement occasion for observation accuracy. Overall, results suggest that real-time feedback did not confer added benefit.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Applied behavior analysis (2nd ed.). Upper Saddle River: Pearson Prentice.
Dart, E. H., Radley, K. C., Briesch, A. M., Furlow, C. M., & Cavell, H. (2016). Comparing the accuracy of group observation methods: Two analyses utilizing simulated data. Behavioral Disorders, 41(3), 148–160.
Dempsey, C. M., Iwata, B. A., Fritz, J. N., & Rolider, N. U. (2012). Observer training revisited: A comparison of in vivo and video instruction. Journal of Applied Behavior Analysis, 25, 827–832.
Egeland, B., & Winer, K. (1974). Teaching children to discriminate letters of the alphabet through errorless discrimination training. Journal of Reading Behavior, 6, 143–150.
Fuchs, L. S. (2004). The past, present, and future of curriculum-based measurement research. School Psychology Review, 33, 188–192.
Gresham, F. M., Cook, C. R., Collins, T., Dart, E., Rasetshwane, K., Truelson, E., & Grant, S. (2010). Developing a change-sensitive brief behavior rating scale as a progress monitoring tool for social behavior: An example using the Social Skills Rating System-Teacher Form. School Psychology Review, 39, 364–379.
Hartmann, D. P., Barrios, B. A., & Wood, D. D. (2004). Principles of behavior observation. In S. N. Haynes & E. M. Hieby (Eds.), Comprehensive handbook of psychological assessment (vol. 3, behavior assessment) (pp. 108–127). New York: Wiley.
Hintze, J. M., Volpe, R. J., & Shapiro, E. S. (2002). Best practices in the systematic direct observation of student behavior. Best Practices in School Psychology IV, 2, 993–1006.
Noell, G. H., Witt, J. C., Gilbertson, D. N., Ranier, D. D., & Freeland, J. T. (1997). Increasing teacher intervention implementation in general education settings through consultation and performance feedback. School Psychology Quarterly, 12, 77–88.
Rapp, J. T., Colby-Dirksen, A. M., Michalski, D. N., Carroll, R. A., & Lindenberg, A. M. (2008). Detecting changes in simulated events using partial-interval recording and momentary time sampling. Behavioral Interventions, 23(4), 237–269.
Ratcliff, N. J., Jones, C. R., Vanden, S. R., Sheen, H., & Hunt, G. H. (2011). Paraprofessionals in early childhood classrooms: An examination of duties and expectations. Early Years: An International Research Journal, 31, 163–179.
Repp, A. C., Nieminen, G. S., Olinger, E., & Brusca, R. (1988). Direct observation: Factors affecting the accuracy of observers. Exceptional Children, 55, 29–36.
Riley-Tillman, T. C., Chafouleas, S. M., Sassu, K. A., Chanese, J. A., & Glazer, A. D. (2008). Examining the agreement of direct behavior ratings and systematic direct observation data for on-task and disruptive behavior. Journal of Positive Behavior Interventions, 10, 136–143.
Riley-Tillman, T. C., Chafouleas, S. M., & Christ, T. J. (2009). Direct behavior rating (DBR): An emerging method for assessing social behavior within a tiered intervention. Assessment for Effective Intervention, 34, 195–200.
Riley-Tillman, T. C., Christ, T. J., Chafouleas, S. M., Boice-Mallach, C. H., & Briesch, A. (2011). The impact of observation duration on the accuracy of data obtained from direct behavior rating (DBR). Journal of Positive Behavior Interventions, 13, 119–128.
Saudargas, R. A., & Zanolli, K. (1990). Momentary time sampling as an estimate of percentage time: A field validation. Journal of Applied Behavior Analysis, 23, 533–537.
Shapiro, E. S., & Heick, P. F. (2004). School psychologist assessment practices in the evaluation of students referred for social/ behavioral/emotional problems. Psychology in the Schools, 41, 551–561.
Simonsen, B., Myers, D., & Briere, D. E. (2011). Comparing a behavioral check-in/check-out intervention to standard practice in an urban middle school setting using an experimental group design. Journal of Positive Behavior Interventions, 13, 31–48.
Solomon, B. G., Klein, S. A., & Politylo, B. C. (2012). The effect of performance feedback on teachers’ treatment integrity: A meta-analysis of the single-case literature. School Psychology Review, 41, 160–175.
Warmington, M., Hitch, G. J., & Gathercole, S. E. (2013). Improving word learning in children using an errorless technique. Journal of Experimental Child Psychology, 114, 456–465.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
All procedures performed in the current study were in accordance with the ethical standards of the institution and the national research committee and with the 1964 Helsinki declaration and its later amendments. Informed consent and assent were obtained for all individual participants included in the study. Prior to conducting the current study, Institutional Review Board approval was obtained from the University of Utah.
Conflict of Interest
The authors declare that they have no conflict of interest.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Radley, K.C., Dart, E.H., Lewis, H.L.J. et al. An Evaluation of Real-Time Feedback for Training Classroom Observers. Contemp School Psychol 26, 241–247 (2022). https://doi.org/10.1007/s40688-020-00304-x
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40688-020-00304-x