Performance Measurement at Universities: Studying Function and Effect of Student Evaluations of Teaching
DOI:
https://doi.org/10.7146/jookc.v1i1.18908Keywords:
student evaluations of teaching, performance measurement, performance management systems, higher education, knowledge management, knowledge economyAbstract
This paper proposes empirical approaches to testing the reliability, validity, and organizational effectiveness of student evaluations of teaching (SET) as a performance measurement instrument in knowledge management at the institutional level of universities. Departing from Weber’s concept of bureaucracy and critical responses to this concept, we discuss how contemporary SET are used as an instrument of organizational control at Danish universities. A discussion of the current state of performance measurement within the frame of new public management (NPM) and its impact on knowledge creation and legitimation forms the basis for proposing four steps of investigation. The suggested mixed-methods approach comprises the following: first, thematic analysis can serve as a tool to evaluate the legitimacy discourse as initiated by official SET affirmative documents by government, university, and students. Second, constructs for the SET questionnaire can be developed and compared to existing SET questionnaires in terms of reliability and validity. Third, data from SET can be used to corroborate the relationship between the qualitative (comments) and quantitative (scaled questionnaire) sections. Fourth, it can be investigated if SET actually contribute to teaching improvement by examining how the instrument is integrated into systematic ex-ante and ex-post organizational management. It is expected to find discrepancy between the proponents’ intent to evaluate teaching and the way the performance measurement instrument is implemented.
References
Allan, K. D. 2009. Explorations in Classical Sociological Theory: Seeing the Social World. Thousand Oaks, CA: Sage.
Andersen, H. 2002. Universitetsreformen: Topstyring og ensretning. Dansk Sociologi, 13(4): 80-88.
AU. 2013. Course Evaluation. Aarhus: Aarhus University.
Baehr, P. 2001. The “Iron Cage” and the “Shell as Hard as Steel”: Parsons, Weber, and the Stahlhartes Gehäuse Metaphor in the Protestant Ethic and the Spirit of Capitalism. History and Theory, 40(2): 153-169.
Bourdieu, P. 1979. Symbolic Power. Critique of Anthropology, 4(13-14): 77-85.
Bourdieu, P. 1991. Language and Symbolic Power. Cambridge, MA: Harvard University Press.
Bromberg, D. 2009. Performance Measurement: A System with a Purpose or a Purposeless System? Public Performance & Management Review, 33(2): 214-221.
Chan, C. K. Y., Luk, L. Y. Y., & Zeng, M. 2014. Teachers’ perceptions of student evaluations of teaching. Educational Research and Evaluation, 20(4): 275-289.
Choi, B. C. K., & Pak, A. W. P. 2005. A catalog of biases in questionnaires. Preventing Chronic Disease, 2(1).
Choong, K. K. 2013. Understanding the features of performance measurement system: a literature review. Measuring Business Excellence, 17(4): 102-121.
Clark, B. R. 1998. Creating entrepreneurial universities: organizational pathways of transformations. New York: Elsevier.
Clayson, D. E. 2005. Within-Class Variability in Student–Teacher Evaluations: Examples and Problems. Decision Sciences Journal of Innovative Education, 3(1): 109-124.
Crumbley, L. D., & Reichelt, K. J. 2009. Teaching effectiveness, impression management, and dysfunctional behavior. Quality Assurance in Education, 17(4): 377-392.
Dillman, D. A. 2007. Mail and Internet Surveys: The Tailored Design Method. Hoboken, NJ: John Wiley & Sons.
DiMaggio, P. J., & Powell, W. W. 1983. The Iron Cage Revisited: Institutional Isomorphism and Collective Rationality in Organizational Fields. American sociological review, 48(2): 147-160.
Evans, M. 2004. Killing Thinking: Death of the University. London: Bloomsbury Publishing.
Fereday, J., & Muir-Cochrane, E. 2006. Demonstrating rigor using thematic analysis: a hybrid approach of inductive and deductive coding and theme development. International Journal of Qualitative Methods, 5: 1-11.
Galbraith, C., Merrill, G., & Kline, D. 2012. Are Student Evaluations of Teaching Effectiveness Valid for Measuring Student Learning Outcomes in Business Related Classes? A Neural Network and Bayesian Analyses. Research in Higher Education, 53(3): 353-374.
Guest, G., MacQueen, K. M., & Namey, E. M. 2012. Applied thematic analysis. Thousand Oaks, California: Sage.
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. 2006. Multivariate data analysis. Upper Saddle River, NJ: Pearson Prentice Hall.
Hajdin, G., & Pažur, K. 2012. Differentiating between Student Evaluation of Teacher and Teaching Effectiveness. Journal of Information and Organizational Sciences, 36(2).
Hartfiel, G. 1976. Bürokratie, Wörterbuch der Soziologie: 103-104. Stuttgart: Alfred Kröner Verlag.
Hood, C. 1991. A PUBLIC MANAGEMENT FOR ALL SEASONS? Public Administration, 69(1): 3-19.
Kallio, K.-M., & Kallio, T. J. 2012. Management-by-results and performance measurement in universities – implications for work motivation. Studies in Higher Education, 39(4): 574-589.
Kanter, R. 1977. Men and Women of the Corporation. New York: Basic.
Kristensen, J. E., Elstrøm, K., Nielsen, J. V., Pedersen, M., Vind Sørensen, B., & Sørensen, H. 2007. Ideer om et universitet. Aarhus: Aarhus Universitetsforlag.
Kristensen, J. E., Nørreklit, H., & Raffnsøe-Møller, M. 2011. Introduction: University Performance Measurement at Danish Universities. In J. E. Kristensen, H. Nørreklit, & M. Raffnsøe-Møller (Eds.), University Performance Measurement: 7-17. Copenhagen: DJØF Publishing.
Lindahl, M. W., & Unger, M. L. 2010. Cruelty in Student Teaching Evaluations. College Teaching, 58(3): 71-76.
Luft, J., & Shields, M. D. 2003. Mapping management accounting: graphics and guidelines for theory-consistent empirical research. Accounting, Organizations and Society, 28: 169-249.
Lyotard, J.-F. 1984. The Postmodern Condition: A Report on Knowledge. . Manchester: Manchester University Press.
Merton, R. K. 1940. Bureaucratic Structure and Personality. Social Forces, 18: 560-568.
Merton, R. K. 1957. Social Theory and Social Structure. Glencoe, IL: Free Press.
Moed, H. F., Burger, W. J. M., Frankfort, J. G., & van Raan, A. F. J. 1985. The use of bibliometric data for the measurement of university research performance. Research policy, 14(3): 131 - 149.
Moullin, M. 2007. Performance measurement definitions. International Journal of Health Care Quality Assurance, 20(3): 181-183.
Neely, A., Adams, C., & Kennerley, M. 2002. The Performance Prism: The Scorecard for Measuring and Managing Business Success: The Scorecard for Measuring and Managing Stakeholder Relationships (Financial Times Series). Edinburgh Gate: Pearson Education Limited.
Otley, D. 1999. Performance management: a framework for management control systems research. Management Accounting Research, 10(4): 363-382.
Parker, M. 2002. Against Management: Organisation in the Age of Managerialism. Oxford: Polity.
Platt, M. 1993. What student evaluations teach. Perspectives on Political Science, 22(1): 29.
Raffnsøe-Møller, M. 2011. Aims and Formats for performance measurement at Danish Universities. In J. E. Kristensen, H. Nørreklit, & M. Raffnsøe-Møller (Eds.), University Performance Measurement: 49-78. Copenhagen: DJØF Publishing.
Rosanas, J. M., & Velilla, M. 2005. The ethics of management control systems: Developing technical and moral values. Journal of Business Ethics, 57(1): 83-96.
Schubert, T. 2009. Empirical observations on new public management to increase efficiency in public research - boon or bane? ISI - Fraunhofer Institute Systems and Innovation Research. Research Policy, 38: 1225-1234.
Simpson, P. M., & Siguaw, J. A. 2000. Student Evaluations of Teaching: An Exploratory Study of the Faculty Response. Journal of Marketing Education, 22(3): 199-213.
Strang, D., & Macy, M. W. 2001. In Search of Excellence: Fads, Success Stories, and Adaptive Emulation. American Journal of Sociology, 107(1): 147-182.
Stölting, E., & Schimank, U. (Eds.). 2001. Die Krise der Universitäten. Wiesbaden: Westdeutscher Verlag.
Temple, P. 2014. Universities in the Knowledge Economy. Higher education organisation and global change. Abingdon: Routledge.
Van de Walle, S., & Hammerschmid, G. 2011. The Impact of the New Public Management: Challenges for Coordination and Cohesion in European Public Sectors. Halduskultuur - Administrative Culture, 12(2): 190-209.
Ward, S. C. 2011. Neoliberalism and the Global Restructuring of Knowledge and Education. New York: Routledge.
Weber, M. 1905. Die protestantische Ethik und der 'Geist' des Kapitalismus, II. Die Berufsidee des asketischen Protestantismus.
Weber, M. 1968. Economy and Society: an outline of interpretive sociology. New York: Bedminster Press.
Wren, D., & Bedeian, A. 2009. The Evolution of Management Thought. . NJ: Wiley.
Wright, S., & Williams Ørberg, J. 2008. Autonomy and control: Danish university reform in the context of modern governance Learning and Teaching, 1(1): 27-57.
Downloads
Published
How to Cite
Issue
Section
License
§1. Object of the agreement and rights
The author guarantees that she/he has the copyright to the work and that this specific publishing does not offend other persons’, organizations’ or companies’ copyright.
- 1.1. The author gives the Journal of Organizational Knowledge Communication a non-exclusive right to publish the work in the electronic version of the non-commercial journal The Journal of Organizational Knowledge Communication. This journal is an open access journal and will be available for free on the internet and as thus available for all internet users worldwide. The work will be published in English.
- 1.2. The journal is published under a Creative Commons license Attribution Non-commercial No derivatives (cc by-nc-sa) http://creativecommons.org/about/license/. This license allows others to download your work and share it with others as long as they mention you and link back to you, but they can’t change it in any way or use it commercially.
- 1.3. The author is the copyright holder and the author agrees to the above mentioned Creative Commons license.
- 1.4. The Journal of Organizational Knowledge Communication is not entitled to transfer the obtained right in this agreement to a third party.
§2. Publishing on the Internet
The Journal of Organizational Knowledge Communication is under an obligation to publish the work within a reasonable time span and within the first year after the manuscript has been accepted for publication. The Journal of Organizational Knowledge Communication is entitled to use the work or parts of the work for marketing purposes.
§3. Proofreading
The Journal of Organizational Knowledge Communication is edited, peer reviewed and proofread by the editors and the international peer review board in collaboration with the author.
§4. Availability on the Internet
The article will be published on the Internet at www.jookc.com.
