Protocol: Comparing advantages and disadvantages of Rating Scales, Behavior Observation Scales and Paired Comparison Scales for behavior assessment of competencies in workers. A systematic literature review

Juan A. Marin-Garcia, Lucia Ramirez Bayarri, Lorena Atares-Huerta

Abstract

Protocolo: comparación de las ventajas e inconvenientes de las “Rating Scales”, “Behavior Observation Scales”, y “ Paired Comparison Scale” para la evaluación basada en comportamientos, de las competencias de los trabajadores de empresas. Una revisión sistemática de la literatura.

 

This is the protocol for a review and there is no abstract. The objectives are as follows: Identify the characteristics of each of the types of scale and how they differ from each other. Estimate the extent to which they are used by organizations in the assessment of skills of their employees. Summarize the advantages and disadvantages of each. Propose which of them would be more appropriate for assessing the competence of innovation in people (performance evaluation, promotion of workers, recruitment, etc.) and the mode of administration (self, peer, expert assessment).

---------------------

Resumen Por tratarse de un protocolo para una revisión sistemática no existe un resumen propiamente dicho. En esta revisión nos planteamos los siguientes objetivos: identificar qué características tiene cada uno de los tipos de escala y cómo se diferencian unas de otras. Es-timar el grado en que son usadas, unas u otras, por las organizaciones en la evaluación de competencias de sus empleados. Resumir las ventajas e inconvenientes que presentan cada una de ellas en general y a la hora de evaluar la competencia transversal de innovación en las personas. Proponer cuál de ellas sería más adecuada, atendiendo a los ob-jetivos específicos de evaluación de la competencia de innovación en las personas (evaluación del desempeño, promoción de trabajadores, selección de personal, etc.) y el modo de administración (self, peer, expert assessment).

Palabras clave: Protocolo; Revisión Sitemática de literatura; Rating Scales; Behavior Observation Scales; Paired Comparison Scales; evaluación comportamiento.


Keywords

Protocol; Systematic literature review; Rating Scales; Behavior Observation Scales; Paired Comparison Scales; behavior assessment;Protocolo; Revisión Sitemática de literatura; evaluación comportamiento

Full Text:

PDF ES

References

Baartman, L. K. J., Bastiaens, T. J., Kirschner, P. A., & van der Vleuten, C. P. M. (2006). The wheel of competency assessment: Presenting quality criteria for competency assessment programs. Studies in Educational Evaluation, 32(2), 153-170. doi:http://dx.doi.org/10.1016/j.stueduc.2006.04.006

Boyatzis, R. E. (2008). Competencies in the 21st century. Journal of management development, 27(1), 5-12. doi:doi:10.1108/02621710810840730

Bretz, R. D., Milkovich, G. T., & Read, W. (1992). THE CURRENT STATE OF PERFORMANCE-APPRAISAL RESEARCH AND PRACTICE - CONCERNS, DIRECTIONS, AND IMPLICATIONS. Journal of Management, 18(2), 321-352. doi:10.1177/014920639201800206

Bryman, A., & Bell, E. (2011). Business Research Methods. USA: Oxford University Press.

Charmaz, K. (2006). Constructing grounded theory. A practical guide through qualitative analysis. London: SAGE.

Dolan, S. L., Valle Cabrera, R., Jackson, S. E., & Schuler, R. S. (2007). La gestión de los recursos humanos. Cómo atraer, retener y desarrollar con éxito el capital humano en tiempos de transformación. Madrid: Mcgraw-Hill.

Dowdy, E., Twyford, J., & Sharkey, J. D. (2013). Methods of Assessing Behavior: Observations and Rating Scales. In D. H. Saklofske, V. L. Schwean, & C. R. Reynolds (Eds.), The Oxford Handbook of Child Psychological Assessment (pp. 623-650): Oxford University Press.

DuVernet, A. M., Dierdorff, E. C., & Wilson, M. A. (2015). Exploring Factors That Influence Work Analysis Data: A Meta-Analysis of Design Choices, Purposes, and Organizational Context. Journal of Applied Psychology. doi:10.1037/a0039084

Friese, S. (2012). Qualitative Data Analysis with ATLAS.ti. London: SAGE Publications Ltd.

Groth-Marnat, G. (2003). Handbook of psichological assessment. Hoboken, New Jersey: John Wiley & Sons, Inc.

Hartig, J. (2008). Psychometric models for the assessment of competencies. In J. Hartig, E. Klieme, & D. Leutner (Eds.), Assessment of competencies in educational contexts (pp. 69-90). Guttingen: Hogrefe & Huber Publishers.

Hatzinger, R., & Dittrich, R. (2012). Prefmod: An R package for modeling preferences based on paired comparisons, rankings, or ratings. Journal of Statistical Software, 48(10). Retrieved from http://www.scopus.com/inward/record.url?eid=2-s2.0-84863314424&partnerID=40&md5=fe41f891ec6e3f17584078fe2d73b5ef

Hayes, A. F., & Krippendorff, K. (2007). Answering the call for a standard reliability measure for coding data. Communication Methods and Measures, 1, 77-89.

Heidemeier, H., & Moser, K. (2009). Self-Other Agreement in Job Performance Ratings: A Meta-Analytic Test of a Process Model. Journal of Applied Psychology, 94(2), 353-370. doi:10.1037/0021-9010.94.2.353

Hoyt, W. T., & Kerns, M. D. (1999). Magnitude and moderators of bias in observer ratings: A meta-analysis. Psychological Methods, 4(4), 403-424. doi:10.1037//1082-989x.4.4.403

Lenburg, C. (1999). The Framework, Concepts and Methods of the Competency Outcomes and Performance Assessment (COPA) Model Online Journal of Issues in Nursing, 4(2).

Lohmann, A., & Prumper, J. (2006). Questionnaire for direct participation in the office (FdP-B) - results concerning its reliability and validity. Zeitschrift fur Arbeits-und Organisationspsychologie, 50(3), 119-134. Retrieved from ://000239034600001

Marin-Garcia, J. A., Aragonés Belgran, P., & Melón, G. (2014). Intra-rater and inter-rater consistency of pair wise comparison in evaluating the innovation competency for university students. Working Papers on Operations Management, 5(2), 24-46. doi:http://dx.doi.org/10.4995/wpom.v5i2.3220

Marin-Garcia, J. A., Garcia-Sabater, J. J., Maheut, J., Valero-Herrero, M., & Andres-Romano, C. (2012). Gestión de recursos humanos para ingenieros de la rama industrial. Harlow: Pearson Education.

Marin-Garcia, J. A., Perez-Peñalver, M. J., & Watts, F. (2013). How to assess innovation competence in services: The case of university students. Direccion y Organizacion(50), 48-62. Retrieved from http://www.revistadyo.com/index.php/dyo/article/viewFile/431/451

Marin-Garcia, J. A., Ramirez Bayarri, L., & Andreu Andres, M. A. (2015). Comparación de los métodos de escalas y frecuencia de comportamiento para valorar la competencia de innovación. El punto de vista de alumnos y profesor en el caso de una asignatura de máster. Paper presented at the Congreso In-Red 2015-Universitat Politècnica de València.

Marin-Garcia, J. A., & Santandreu-Mascarell, C. (2015). What do we know about rubrics used in higher education? Intangible Capital, 11(1), 118-145. doi:http://dx.doi.org/10.3926/ic.

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The, P. G. (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med, 6(7), e1000097. doi:10.1371/journal.pmed.1000097

Moore, D. R., Cheng, M. I., & Dainty, A. R. J. (2002). Competence, competency and competencies: performance assessment in organisations. Work Study, 51(6), 314-319. doi:doi:10.1108/00438020210441876

Rowe, C. (1995). Clarifying the use of competence and competency models in recruitment, assessment and staff development. Industrial and Commercial Training, 27(11), 12-17. doi:doi:10.1108/00197859510100257

Saaty, T. L. (1980). The Analytic Hierarchy Process. New York: McGraw-Hill.

Saaty, T. L. (1996). Decision Making with Dependence and Feedback: The Analytic Network Process. Pittsburgh, PA.: RWS Publication.

Tziner, A., & Kopelman, R. E. (2002). Is there a preferred performance rating format? A non-psychometric perspective. Applied Psychology, 51(3), 479-503. doi:10.1111/1464-0597.00104

Voskuijl, O. F., & Van Sliedregt, T. (2002). Determinants of interrater reliability of job analysis: A meta-analysis. European Journal of Psychological Assessment, 18(1), 52-62. doi:10.1027//1015-5759.18.1.52

Abstract Views

4573
Metrics Loading ...

Metrics powered by PLOS ALM

Refbacks

  • There are currently no refbacks.


 

Cited-By (articles included in Crossref)

This journal is a Crossref Cited-by Linking member. This list shows the references that citing the article automatically, if there are. For more information about the system please visit Crossref site

1. Escala para Evaluar Artículos Científicos en Ciencias Sociales y Humanas- EACSH
Emerson López-López, Sergio Tobón, Luis Gibrán Juárez-Hernández
REICE. Revista Iberoamericana sobre Calidad, Eficacia y Cambio en Educación  vol: 17  issue: 4  first page: 111  year: 2019  
doi: 10.15366/reice2019.17.4.006



This journal is licensed under a Creative Commons Attribution 4.0 International License.

Universitat Politècnica de València

e-ISSN: 1989-9068   https://doi.org/10.4995/wpom