Staff and students follow up on evaluation

Published Nov 28, 2013

KTH Royal Institute of Technology had hoped for a better outcome from the inspection, undertaken by the Swedish Higher Education Authority (UKÄ), of Sweden’s higher education programmes in technology and engineering. “We were slightly surprised. We believed that more of our programs and courses would have received a more positive appraisal,” states the Vice-Dean of Faculty, Per Berglund.

By the end of the year, a preliminary plan will be in place for putting right the shortcomings indicated.

Fewer courses than expected received the Swedish Higher Education Authorities highest ratings in the recent review. (Photo: Jack Mikrut)

This evaluation is the first of its kind. The former authority (Swedish National Agency for Higher Education) assessed the quality of the education (courses and study programmes) on the basis of self-assessments by the institutions themselves, interviews and visits on site by an assessment panel. This time, however, priority has been given to reviewing and assessing the results of courses and programmes through looking at the students’ degree projects (independent projects).  

In total, the Swedish Higher Education Authority has read over 5,000 degree projects and checked them against the qualitative targets of the Higher Education Ordinance. Of 36 areas of study evaluated at KTH, 3 were considered to be of very high quality, 25 of high quality and 8 of inadequate quality.

“We take these results very seriously. We know that KTH’s programmes possess very high quality through the jobs the students get. The employers are also satisfied with the engineers who have been awarded their degrees at KTH. But now we must become even better at ensuring that we live up to the goals set by the Higher Education Ordinance,” Berglund says.

Per Berglund

He emphasises that it is important to look to the future now. It is also essential that both teaching staff and students participate in this programme development, where the programme co-ordinators play the vital leadership role.

Redesign is called for

Folke Snickars, Professor at KTH and one of six chairmen for the evaluation recently performed by the Higher Education Authority, is concerned that so many of Sweden’s engineering and technology course and programmes lack sufficient quality. He is also surprised that more Master of Science in Engineering courses and programmes did not attain the highest rating, very high quality.

“This sends a very strong signal that Sweden’s universities must really focus on the courses to enhance the quality of the programmes offered. To a large extent, this involves asking those who work with the different degree programmes to go that extra mile. We know that the commitment is there; it is visible in the self-assessments that form part of the evaluation,” he explains.

One trend noticeable both at KTH, and also with the technology and engineering programmes offered elsewhere in Sweden, is that general qualifications i.e. Master’s and Bachelor’s programmes, received more criticisms than the traditional professional qualifications of Master of Science in Engineering and Bachelor of Science in Engineering. The reason for this is that the general qualifications are relatively new here and are yet to achieve their best form, according to Snickars.

In practice, KTH’s Master’s students read the same courses as the Master of Science in Engineering students during the last two years. Nevertheless, the education concerned must fit two different qualitative targets (degree objectives). To judge from the scrutiny undertaken by Sweden’s Higher Education Authority, KTH has not succeeded in designing the courses to meet the requirements of the Master’s Degree (Master of Science) as well as it has for a Master of Science in Engineering degree.   

Education does not fit the qualification

“This is one of the elements that we must look at more closely. We need to ensure that the education at the second cycle level is designed in such a way that the qualitative objectives for both qualifications can be met,” Berglund says.

When seen in terms of Sweden as a whole, there exist similar problems with the Bachelor of Science in Engineering degree programmes, Snickars says. In many cases, the universities and colleges award both a Bachelor’s Degree and Bachelor of Science in Engineering Degree on the basis of one and the same degree project. The fact remains that the degree programmes are seldom adapted – as they should be – to meet the different objectives of these two qualifications.

After the scrutiny performed by the Swedish Higher Education Authority, it is now necessary for KTH to better clarify which parts are necessary to include in the degree project. Per Berglund also believes that there must be a greater consensus at KTH concerning how a degree project shall be designed to achieve the qualitative targets (degree objectives).

Folke Snickars

By the beginning of 2014, the relevant Heads of Department at KTH should have action plans ready for putting right the shortcomings in the courses and programmes rated as being of inadequate quality. The work will continue during the spring, in collaboration with the education committee and academic teacher training experts at the ECE School. In October 2014, KTH will report back to the Swedish Higher Education Authority on this matter.

The evaluation methodology employed by the Higher Education Authority has generated criticism at several of the universities. This is primarily due to the fact that the students’ degree projects have acquired such great weight in the final rating. The appraisal, in fact, is based almost wholly on the evaluation of the degree project quality.

A misleading picture

Snickars understands the criticism that the evaluation offers a misleading picture; he maintains, nevertheless, that the method functions well as an evaluation instrument. It is his view that the Higher Education Authority’s review has both substance and a high degree of credibility.

“A clear advantage is that the students become the focal point. This is due to their being represented on the panels of external experts and the fact that the students’ degree projects are the central element when the programme’s quality is evaluated.”

The evaluators have been divided into six clusters, each of which has some 30 or so experts and assessors tied to them, from both Swedish and foreign higher education institutions.

Snickars likens their work to peer review; that is to say the same method used in the case of applications for research grants or the evaluation of scientific articles. The difference is that this time it has been teachers – experts within their subjects and used to supervising degree projects – who have carried out the review and quality evaluation of the completed degree projects.

Christer Gummeson

How the evaluation was done

The degree projects that were reviewed dated from the years 2010-2012. Between 5 and 25 degree projects (students’ independent projects) have been evaluated for each degree programme. A total of more than 5,000 degree projects are included in this evaluation by the  Swedish Higher Education Authority (UKÄ).

The assessment has been made on the basis of the courses' and programmes’ qualitative targets (degree objectives) as they are described in the Higher Education Ordinance. Depending on which qualification that is reviewed this has covered some 4-6 different targets or objectives. If any of the targets have not been met, then the Higher Education Authority has given that educational programme the rating of inadequate quality.

The next time also that the Authority carries out a national evaluation the degree projects will occupy a central role. On the other hand, it is not certain that they will be given as great prominence as in the recent evaluation. The reason for this is that the European Association for Quality Assurance in Higher Education (ENQA) has levelled criticism at the Swedish Higher Education Authority’s evaluation methods.

The goals contained in the Higher Education Ordinance, against which the qualifications are evaluated, can be divided into three categories: 1) Knowledge and understanding, 2) Proficiency and ability and 3) Values and approach.

KTH results

The three programmes at KTH which were rated as being of very high quality were the Master of Science programme in Industrial Management and the Master’s programmes in Mathematics and Engineering Physics. Amongst the 8 that were rated as being of indaequate quality was a Master of Science in Engineering programme, three Bachelor Science in Engineering programmes, an International Master’s programme and three Master’s courses. The other 25 programmes were rated as being of high quality.