Italian University student evaluation of teaching: does it changes over time?

This paper concerns the measurement of the Student Evaluation of Teaching (SET) byundergraduates. For almost 15 years Italian Universities have been monitoring that aspect,performing on students’ opinions about teaching, e.g. on teacher punctuality, clearnessand willingness, and on suitability of the facility, logistics, and on course scheduling andmanaging. Each University carries out a survey with own procedures and through a questionnaireconsisting of two set of items: a set of items equal for all Italian Universities,and a set of items built by each University. The survey basic aim is to gather informationon teaching skill, educational goals, coordination among teachings, and adequacy of resources.This information is useful to policy makers to improve the SET and the DegreeCourses.The main aim of this paper is to investigate changes over time of the SET. Teacher’squality is obviously a concept that cannot be directly measurable. It is a latent conceptand its measurement needs the specification of convenient measurable dimensions that arestrictly correlated to the concept. The above mentioned questionnaire includes items thatrefer to SET. The items are measured by a Likert-type scale with 4 response categories.To analyse SET and its changes over the time, a Rasch model (Rasch, 1966) is used:it is necessary to transform the data from ordinal scale to interval one and to preserve therank ordering (Doig and Groves, 2006). Rasch model gives items and persons measureson the same continuum: this is very useful to make comparisons among items, amongstudents satisfaction levels, and among items and satisfaction levels, and finally amongdifferent points of time. The partial credit model for rating scales (Masters, 1988; Curtisand Bowman, 2007) appears the useful procedure for the analysis of these items.The dataset regards the questionnaires administrated and collected by the EngineeringFaculty of the University of Pavia during three academic years - from 2004/05 to 2006/07.The 3-years period is taken into consideration because it is the longest period over thequestionnaires structure and items have not been modified. The statistical unit is theteaching. The focus is on 5 teachings chosen among those performed by the same teacherduring the considered period and with many students.Results show: 1) the satisfaction mean level is mainly medium-high, and seems varyingover time, 2) the items and person measures are graphically compared to emphasizeoverall satisfaction vs. single item perceived quality, and more and less appreciated aspectof SET, and their variation over the time.