Standards and Guidelines for Quality Assurance in the European Higher Education Area

EXECUTIVE SUMMARY

This report analyses the compliance of the Accreditation Institution with the European Standards and Guidelines for external quality assurance agencies and thus with the membership criteria of the European Association for Quality Assurance in Higher Education (ENQA).

 

The Accreditation Institution was established by law in 2007 for external quality assurance in higher education, namely to accredit study programmes and have the overall responsibility for quality of the higher education system. The Accreditation Council, as the main decision making body of the Accreditation Institution, takes strategic and accreditation decisions and oversees the quality of the agency. It accredits all study programmes although they fall under the auspices of several ministries. Besides the Accreditation Council the Accreditation Institution consists of ACE Denmark, which includes two secretariats that are also defined in the Accreditation Act. The Council Secretariat administratively supports the Council and the Professional Secretariat is carrying out accreditation procedures which include: development of accreditation guidelines, training of experts, receiving applications, preparing site visits, drafting accreditation reports and preparing final reports. In the Danish context ACE Denmark is called “the operator” and carries out accreditation procedures for study programmes that fall under the Ministry of science, technology and innovation.

 

The review of the Accreditation Institution had the sole purpose of examining whether it meets requirement of ENQA membership (“type A review”). The panel received sufficient documentation prior to the visit that provided fruitful additional information. Thus the panel managed to gather evidence for judgement of standards fulfilment.

 

In the very short period of time the Accreditation Institution managed to set up an accreditation system with sufficient human, financial and expert resources. The panel found the system substantially compliant with the European Standards and Guidelines, however, proposes several recommendations in the light of further development of the Accreditation Institution.

 

Page 4 of 39

 

GLOSSARY

 

ENQA

 

The European Association for Quality Assurance in Higher Education

 

ESG

 

Standards and Guidelines for Quality Assurance in the European Higher Education Area

 

HEI

 

Higher Education Institution

 

HEIs

 

Higher Education Institutions

 

Dialogue Forum

 

Dialogue Forum, which was established for more overall and general discussions, comprises representatives from all the universities, students and the business community – a total of 30 representatives. Dialogue Forum meets approx. four times a year with changing themes on the agenda.

 

Page 5 of 39

 

INTRODUCTION

 

This is the report of the review of the Accreditation Institution undertaken in June 2010 for the purpose of determining whether the agency meets the criteria for Full membership of the European Association for Quality Assurance in Higher Education (ENQA). The provisions for membership of ENQA are listed in Annex 1 to this report.

 

BACKGROUND AND OUTLINE OF THE REVIEW PROCESS

 

ENQA’s regulations require all Full member agencies to undergo an external cyclical review, at least once every five years, in order to verify that they fulfil the membership provisions.

 

In November 2004, the General Assembly of ENQA agreed that the third part of the Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG) should be incorporated into the membership provisions of its regulations. Substantial compliance with the ESG thus became the principal criterion for Full membership of ENQA. The ESG were subsequently adopted at the Bergen ministerial meeting of the Bologna Process in 2005.

 

The third part of the ESG covers the cyclical external review of quality assurance and accreditation agencies. In accordance with the principle of subsidiarity, external cyclical reviews for ENQA membership purposes are normally conducted on a national level and initiated by national authorities in an EHEA State, but carried out independently from them. However, external reviews can also be coordinated by ENQA if they cannot be nationally organised, as this is the case for this Accreditation Institution’s external review. In that event, ENQA plays an active role in the organisation of the review, being directly involved as coordinator, whereas, in the case of national reviews, it is only kept informed of progress throughout the whole process.

 

The external review of the Accreditation Institution was conducted in line with the process described in Guidelines for external reviews of quality assurance agencies in the European Higher Education Area and in accordance with the timeline set out in the Terms of Reference. This review of the Accreditation Institution had the sole purpose of examining whether it meets requirement of ENQA membership.

 

The review panel for the external review of the Accreditation Institution was composed of the following members:

 

– Séamus Puirséil, President, Dublin Business School, former Chief Executive

 

HETAC-Ireland, Ireland (Chairman);

 

– Janja Komljenovič, Advisor for Higher Education, Ministry of Higher Education,

 

Science and Technology, Slovenia (Secretary);

 

– Karin Järplid Linde, Assistant Head of Department, National Bologna Expert,

 

Swedish National Agency for Higher Education, Sweden;

 

– Jacques Lanares, Vice Rector, University of Lausanne, Switzerland;

 

Page 6 of 39

 

– Bruno Carapinha, doctoral student in Political science, University of Lisbon,

 

Portugal.

 

The Accreditation Institution produced a self-evaluation report and provided substantial documentation which represented an important portion of the evidence that the panel used to form its conclusions. The panel conducted a sitevisit to validate the self-evaluation and clarify any points at issue. Finally, the review panel produced the present final report on the basis of the self-evaluation report, site-visit and its findings. In doing so it provided an opportunity for the Accreditation Institution to comment on the factual accuracy of the draft report. The review panel confirms that it was given access to all documents and people it wished to consult throughout the review.

 

SELF-EVALUATION REPORT AND THE REVIEW PROCESS OF ACE Denmark

 

Self-evaluation report

 

The Accreditation Institution produced a self-evaluation report (60 pages), which was sent to the review panel prior the site visit. The self-evaluation report was prepared by a project group with participants from the Accreditation Council and ACE Denmark. Other members of the Council and employees at ACE Denmark have been involved in respect of relevant topics.

 

The review panel found the self-evaluation report to be well laid out and informative. However, the panel missed sufficient information in relation to different quality assurance structures in Denmark, namely ACE Denmark, Danish Evaluation Institute (EVA) and the Accreditation Council. The panel was also disappointed that the self-evaluation report was essentially descriptive and lacking in self-analysis.

 

The Accreditation Institution submitted 32 annexes to the self-evaluation report; the most relevant of those translated into English and prepared to translate any further documents upon the panel request. The panel found annexes sufficiently informative and was able to benefit from the additional information that was provided.

 

Site visit

 

The review panel visited the offices of ACE Denmark in Copenhagen on 17 – 18 June 2010. The panel held a telephone briefing with ENQA vice-president and staff member on 14 June 2010 and a preparatory meeting the day before the review to further discuss the self-evaluation report and other available information. The panel also further discussed the initial lines of inquiry and distributed tasks between themselves.

 

During the two-day visit the panel met with different groups of the Accreditation Institution representatives and higher education stakeholders. Programme of the visit is included in the annex 2 of this report. The panel felt convinced that scope

 

Page 7 of 39

 

and level of the programme provided relevant information for the review and provided relevant additional information.

 

The site visit was expeditiously prepared and well organised. ACE Denmark offered strong organisational support prior and during the visit. Its staff was at all times available and provided the panel with all necessary information and access to additional documents. Furthermore, ACE Denmark’s support regarding the logistical organisation, transport, accommodation and meals, was of a very high order.

 

Concluding the report

 

During the visit and after the programme finished the panel members discussed in detail the compliance of the Accreditation Institution with all of the ESG and ENQA membership criteria. They reached high degree of consensus on each criterion separately.

 

After the visit the review secretary prepared the initial draft report in cooperation with the chair and panel members. It was submitted to the Accreditation Institution for comment on its factual accuracy.

 

After receiving comments, further revisions to the report was agreed between the Review Chair and Review Secretary, in consultation with the panel. The final text was then submitted by the review secretary to the ENQA Secretariat for the attention of the Board, and to the Accreditation Institution.

 

CONTEXT OF THE REVIEW

 

Higher education system in Denmark1

 

Danish higher education programmes are organised according to a binary division between research-based and professionally based programmes. The purpose of the research-based programmes is to educate students to the highest international standards within and across the research-based disciplines, whereas the purpose of the professionally oriented programmes is to ensure education closely based on practice and at an international level to meet the need for well qualified professionals in the private and public sectors. The Danish higher education programmes fall under the auspices of different ministries.

 

The research-based programmes are offered by eight universities, and regulated by the Ministry of Science in the Danish University Act (Universitetsloven). There are approximately 1,050 study programmes, educating approximately 121,000 students. ACE Denmark is the operator of accreditation of these research-based higher education programmes in the university sector. Following the University Act of 2003, all universities are organised as self-governing institutions funded by the state. In 2007, a merging of several institutions took place and thus reduced the overall number of universities to 8; for example the Royal Veterinary

 

1

 

Parts of this chapter are taken from:

 

– Eurydice – Eurybase Descriptions of National Education Systems and Policies;

 

– Accreditation Institution’s self evaluation report.

 

Page 8 of 39

 

and Agricultural School is now part of University of Copenhagen and the Danish School of Education merged with University of Aarhus. The aim was to create new and stronger universities with better services as part of the Danish government’s globalization strategy. In addition, 3 major new government research institutes have also been established.

 

The professionally oriented programmes with approximately 83,000 students are predominantly offered by seven University Colleges and ten Academies of Professional Higher Education. Whereas the university programmes are researchbased, these programmes are based on development and close contact with practice. These programmes and institutions are regulated by the Ministry of Education. The programmes fall under the Danish Act on Academy Profession and Professional Bachelor Programmes (Lov om erhvervsakademi- og professionsbacheloruddannelser). As of January 2008, a merging of the university colleges and centres of higher education (CVU) took place and resulted in 8 new regional university colleges (professionshøjskole). Furthermore 10 new academies of professional higher education (consisting of the existing business and technical colleges) have been established. They offer short-cycle education (KVU) and further adult education (VVU). The aim of these new structures is to strengthen the development and cooperation of medium- and short-cycle higher education.

 

A third, and smaller, group of educational programmes fall under the auspices of the Ministry of Culture and encompass educational programmes in the Arts, and provide education for approximately 5,000 students at fifteen institutions. The programmes in the Arts are governed by the Danish Act on Tertiary Artistic Education Institutions under the Auspices of the Ministry of Culture (Lov om videregående kunstneriske uddannelsesinstitutioner under Kulturministeriet) and by the Danish Royal School of Library and Information Science Act (Lov om Danmarks Biblioteksskole). It should be mentioned that specific institutions under the auspices of the Ministry of Culture also offer research-based programmes.

 

Finally, there are the professionally oriented programmes offered at institutions under the auspices of other ministries, e.g. the Ministry of Defence, the Ministry of Economic and Business Affairs and the Ministry of Justice.

 

As already mentioned, most Danish institutions of higher education have been merged to form larger units in recent years. This means both that the institutions have been strengthened with more students and a broader provision of study programmes, but also that the administrative systems are undergoing adaptation and development.

 

The Danish higher education system is organised into four qualification levels, with a number of both ordinary and adult further education degree types at each level. The Danish qualification framework systematically describes the different degree types within the Danish higher education system. Students completing non-university higher education receive a diploma after one to four years. There are a number of medium tertiary (non-university) educations that last three to four years and lead to specialised job-specific qualifications. With the University Reform, the 3+2 structure of 3- year BAs followed by 2-year postgraduate candidate/master programmes has been implemented in all university degrees.

 

Page 9 of 39

 

Historical Overview of the quality assurance system in Denmark

 

In the early 1990’s, the Danish government established Centre for Quality Assurance and Evaluation in Higher Education (Evalueringscenteret). This agency was one of the early quality assurance agencies for higher education and played a significant role in the quality assurance system in Europe. In 1999 Evalueringscenteret was incorporated in a new Danish Evaluation Institute (Danmarks Evalueringsinstitut – EVA). The new agency as part of a wide mandate conducted a number of external programme evaluations of the Danish higher education programmes. These were part of a systematic evaluation of all higher education institutions and their programmes. While the results of the evaluations were not legally binding, and the evaluations also could not have any financial consequences for the institutions the reports were viewed as having considerable influence on the policies and practices of the universities and colleges reviewed by EVA.

 

The Danish Accreditation Institution was established by law in 2007. The establishment came as a natural extension of Denmark’s adoption of the Bologna declaration in 1999, according to which Denmark is under an obligation to incorporate quality assurance in the Danish legislation on higher education. Denmark also undertook to enter into collaboration on quality assurance at the European level.

 

The Danish Government launched a Globalisation Strategy in 2006 which aimed at strengthening Denmark’s position in the increasing international competition. As part of this strategy the government aimed to make Danish higher education programmes world-class and meet international quality standards The programmes would also be required to comply with society’s demand for highly qualified and competitive labour.

 

The new approach set out to reduce the number of universities and higher education institutions and to ensure an independent and focussed strategic approach by these institutions through reform of their governance systems. This was in line with developments, or at least proposals, in other developed countries.

 

Parallel to this approach the government set out to ensure an independent and rigorous approach to the accreditation of study programmes in Danish universities and colleges. Responsibility for accreditation of all higher education study programmes was given to a new statutory body the Accreditation Institution headed by a small nine member board drawn from senior persons among the various partners in higher education.

 

The Accreditation Institution trades under the brand name ACE-DK or AceDenmark2.

 

2 The panel had some difficulty with the nomenclature of the Institution and its secretariats during the early part of the review. To the extent that this report may still reflect this confusion the relevant authorities concerned may consider the extent to which this arises from a fault of the panel or otherwise.

 

Page 10 of 39

 

While all higher education programmes are subject to accreditation under the new system the operation of the reviews and reporting arrangements differ between different types of institutions:

 

• University study programmes are reviewed and reported on by the

 

operating arm of the Accreditation Institution which also (and perhaps

 

confusingly) trades as ACE-Denmark. These programmes are offered by

 

universities under the broad remit and funding of the Ministry Science,

 

Technology and Innovation.

 

• Higher Education programmes in college and other institutions are

 

reviewed and reported on by EVA which continues to operate under the

 

recent legislation with a new mandate. While most of these study

 

programmes come under the remit and funding of the Ministry of

 

Education some programmes are subject to broad oversight of other

 

ministries such as Arts, Culture and Defence.

 

The accreditation decisions are in all cases made by the Accreditation Council while the evaluation procedures leading up to accreditation decisions can be performed by ACE Denmark, EVA or possibly other organisations.

 

While the accreditation system established in 2007 is in large measure sui generis and reflects the particular needs of Denmark at that time the board and executive have succeeded in establishing very good working relations between the various bodies involved.

 

Page 11 of 39

 

COMPLIANCE WITH THE EUROPEAN STANDARDS AND GUIDELINES

 

1. ENQA CRITERION 1

 

1.1 ESG PART 2

 

1.1.1 ESG 2.1 USE OF INTERNAL QUALITY ASSURANCE PROCEDURES

 

STANDARD:

 

External quality assurance procedures should take into account the effectiveness of the internal quality assurance processes described in Part 1 of the European Standards and Guidelines.

 

GUIDELINES:

 

The standards for internal quality assurance contained in Part 1 provide a valuable basis for the external quality assessment process. It is important that the institutions’ own internal policies and procedures are carefully evaluated in the course of external procedures, to determine the extent to which the standards are being met.

 

If higher education institutions are to be able to demonstrate the effectiveness of their own internal quality assurance processes, and if those processes properly assure quality and standards, then external processes might be less intensive than otherwise.

 

According to the Danish Accreditation Act, the Accreditation Institution takes accreditation decisions based on five criteria laid down in the Accreditation order issued by the Minister of Science, Technology and Innovation. Among the five criteria is one which requires the Accreditation Institution review to determine whether study programmes have continuous internal quality assurance in place. In addition to these five criteria there may be additional criteria for a smaller number of study programmes that are under the auspices of the Ministry of Education, Ministry of Culture or other ministries.

 

The review panel gathered additional evidence at the site visit and extra information from the interviews with various stakeholders. The panel understands that during the process of programme accreditation the Accreditation Institution gathers information on internal quality assurance regarding the study programme in question based on the documentation received by the HEI and at the site visit. The Institution also takes into account the information provided by the HEI and the self evaluation report it receives.

 

The panel recognises that the requirements for the documentation of the institution’s quality assurance system have been made more explicit by the reduction of the ten accreditation criteria to the current five criteria. The Accreditation Institution has pointed out to the panel that they recognise the need to establish the effectiveness of the internal quality assurance systems as a key focus of an accreditation and the need to diminish the bureaucratic burden for instance by ensuring that the universities will only have to document standard descriptions once, unless specific circumstances warrant otherwise. However, since the accreditation system in Denmark focuses exclusively on study programmes it sometimes happens that the reviews give insufficient emphasis to an analysis of the university’s internal quality assurance system. The

 

Page 12 of 39

 

Accreditation Institution focuses on elements that are directly connected to the quality assurance of study programmes under revision and less on the institutional quality assurance as a whole. Thus it can happen that some parts of institutional quality assurance are overlooked or that HEIs sometimes provide same information several times.

 

Standard fulfilment: Substantially compliant

 

Recommendations:

 

The Accreditation Institution should develop an approach that takes into account institutional internal quality assurance as a whole that is linked to study programmes.

 

In addition to noting internal quality assurance the Accreditation Institution should put more emphasis on evaluation of its effectiveness as well.

 

1.1.2 ESG 2.2 DEVELOPMENT OF EXTERNAL QUALITY ASSURANCE PROCESSES

 

STANDARD:

 

The aims and objectives of quality assurance processes should be determined before the processes themselves are developed, by all those responsible (including higher education institutions) and should be published with a description of the procedures to be used.

 

GUIDELINES:

 

In order to ensure clarity of purpose and transparency of procedures, external quality assurance methods should be designed and developed through a process involving key stakeholders, including higher education institutions. The procedures that are finally agreed should be published and should contain explicit statements of the aims and objectives of the processes as well as a description of the procedures to be used.

 

As external quality assurance makes demands on the institutions involved, a preliminary impact assessment should be undertaken to ensure that the procedures to be adopted are appropriate and do not interfere more than necessary with the normal work of higher education institutions.

 

Danish Accreditation Act was enacted in 2007 after wide public consultation and debate. Following the Act the Accreditation Order was issued in which 10 criteria for accreditation were defined. The Accreditation Institution developed detailed Guidelines for accreditation in which it further elaborated procedures for accreditation. The criteria and guidelines were published and available prior to accreditation.

 

After the first round of accreditation a thorough revision of accreditation procedures and criteria was made by the Accreditation Institution. In this revision HEIs and other stakeholders were consulted as well as experts that cooperated in the reviews. The Accreditation Institution suggested updates of the criteria to the Minister of Science, Technology and Innovation. In order for criteria to be more comprehensive, the number was reduced from ten to five. The stakeholders the panel interviewed showed satisfaction on this issue. After the reduction was made, the Accreditation Institution updated the Guidelines for

 

Page 13 of 39

 

accreditation. The Institution constantly consults HEIs, other stakeholders and experts in order to develop accreditation procedures to be most fit for purpose. Stakeholders are to a wide extent involved on an ongoing basis in the development and adaptation of the overall accreditation concept. At the same time, the Accreditation Institution is aware at all times that the involvement of stakeholders must not take place at the expense of the independence.

 

Figure 1: Overview of the process for the review of the executive order and the guidelines taken from the self evaluation report

 

At the interviews the panel learned that although stakeholders are included in the process of development of accreditation procedures, their suggestions are not always fully implemented. This is however self evident due to independency of the Institution.

 

Standard fulfilment: Fully compliant

 

Recommendations:

 

We suggest to the Accreditation Institution endeavour to increase involvement of HEIs and students in the ownership of the accreditation guidelines and procedures. This would allow HEIs to better transfer the quality / accreditation requirements into their institutional practice.

 

1.1.3 ESG 2.3 CRITERIA FOR DECISIONS

 

STANDARD:

 

Any formal decisions made as a result of an external quality assurance activity should be based on explicit published criteria that are applied consistently.

 

GUIDELINES:

 

Formal decisions made by quality assurance agencies have a significant impact on the institutions and programmes that are judged. In the interests of equity

 

August 2009

 

The Accreditation Council and ACE Denmark submit a proposal for a new executive order to the Ministry of Science, Technology and Innovation

 

December 2009

 

The Ministry of Science, Technology and Innovation publishes a new executive order

 

1 March 2010

 

The Accreditation Council and ACE Denmark publish guidelines for new and existing study programmes

 

– Joint meetings held with the

 

universities on 17 June, 25

 

June and 18 August

 

– Meeting held with student

 

organisations on 21 August

 

– The Ministry of Science,

 

Technology and Innovation

 

submits the proposal for a new

 

executive order for

 

consultation

 

– The proposal is presented to

 

students and employers at a

 

dialogue forum

 

– Individual meetings held with all

 

universities in December

 

– Joint meeting held with the universities on

 

14 January

 

– Meeting held with employers and students

 

on 25 January

 

– Guidelines submitted for consultation 1-15

 

February

 

Page 14 of 39

 

and reliability, decisions should be based on published criteria and interpreted in a consistent manner. Conclusions should be based on recorded evidence and agencies should have in place ways of moderating conclusions, if necessary.

 

The accreditation criteria, guidelines and procedures are published and publicly available as shown in the documents provided to the panel, on the Accreditation Institution webpage and as discussed at the interviews.

 

Stakeholders at the interviews on the site visit also mentioned that comprehensive understanding and explanation of the criteria by the ACE Denmark’s staff improved over the two years and that criteria are now consistently interpreted and used.

 

In addition the Accreditation Institution puts great emphasis on the consistent usage of criteria and guidelines. This is guaranteed by several measures:

 

– Before each accreditation round ACE Denmark’s staff is divided into teams

 

on the basis of disciplines. Junior and senior staff is mixed as well. The

 

teams then follow the whole accreditation procedure and ensure

 

consistency of methods and criteria for the whole range of programmes

 

that have been assigned to them.

 

– The experts who form review panels take part in the general training

 

organised by ACE Denmark.

 

– In addition prior to the site visit each expert team gathers at the

 

preparation meeting at ACE Denmark to prepare for the site visit. At this

 

meeting participants go through the agenda, questions and lines of

 

inquiry.

 

– At the site visit ACE Denmark’s staff are present but are not full equal

 

members of expert teams.

 

– When final accreditation reports are handed to the Accreditation Council, it

 

again checks and compares the criteria fulfilment through all of the reports

 

in a manner of overall consistency of assessments across universities and

 

educational fields. The analyses comprise all current accreditations as well

 

as experience gained from previous accreditation processes.

 

Standard fulfilment: Fully compliant

 

1.1.4 ESG 2.4 PROCESSES FIT FOR PURPOSE

 

STANDARD:

 

All external quality assurance processes should be designed specifically to ensure their fitness to achieve the aims and objectives set for them.

 

GUIDELINES:

 

Quality assurance agencies within the EHEA undertake different external processes for different purposes and in different ways. It is of the first importance that agencies should operate procedures which are fit for their own defined and published purposes. Experience has shown, however, that there are some widely-used elements of external review processes which not only help to ensure their validity, reliability and usefulness, but also provide a basis for the

 

Page 15 of 39

 

European dimension to quality assurance. Amongst these elements the following are particularly noteworthy:

 

• insistence that the experts undertaking the external quality assurance activity

 

have appropriate skills and are competent to perform their task;

 

• the exercise of care in the selection of experts;

 

• the provision of appropriate briefing or training for experts;

 

• the use of international experts;

 

• participation of students;

 

• ensuring that the review procedures used are sufficient to provide adequate

 

evidence to support the findings and conclusions reached;

 

• the use of the self-evaluation/site visit/draft report/published report/follow-up

 

model of review;

 

• recognition of the importance of institutional improvement and enhancement

 

• policies as a fundamental element in the assurance of quality.

 

Expert teams are always carefully selected by the Accreditation Institution. HEIs are invited to suggest experts to the agency’s data base of experts from which panels are selected. Ultimately the Accreditation Institution finally decides for each expert in order to guarantee objectivity. It can happen however, that a HEI is evaluated by the expert it suggested itself. All experts are trained during workshops and in addition to this training they are invited for initiative meeting at ACE Denmark before they go on the site visit. However, the panel heard on several occasions that the experts were not always fully relevant for the programme under evaluation.

 

Processes are based on a model: detailed guidelines are issued and presented to HEIs, self-evaluation reports and documentation from HEIs, initial workshop for expert teams, external review, initial draft expert report, comments by HEIs, expert report, accreditation decision, published report, and to a varying degree follow-up. A final report provides clear information on fulfilment of each accreditation criteria as well as recommendations for quality improvement. Accreditation criteria are developed for the process and consistently

 

applied. Prior to each accreditation round, ACE Denmark holds a start-up meeting at the universities with the quality managers and representatives of the study programmes which are to be accredited. The meeting is used to present and discuss the process, criteria and measuring points.

 

The Accreditation Institution often uses international experts in the expert teams which are formed by three members. One of those members is always a student. One student is also a full and equal member of the Accreditation Council which has altogether nine members.

 

The guidelines for accreditation are very detailed and the Accreditation Institution gets the evidence for all. It can sometimes happen that the material that is demanded of HEIs duplicates and can be a heavy administrative burden.

 

The panel, however, is concerned about the accreditation of new study programmes. Our concern was endorsed also by various stakeholders at the interviews. The panel realises that due to the legal requirement experts are used for accreditation of existing study programmes. New study programmes are assessed for accreditation by ACE Denmark’s staff who prepare a report for the Accreditation Council. The Accreditation Council decides whether to accredit the

 

Page 16 of 39

 

new programme. Expert panels are used only in exceptional cases. In our view this arrangement does not ensure consistent and expert treatment of all study programmes (new and existing). In addition to this, new study programmes are accredited for the same length as existing, which is 6 years and without the external expert judgement.

 

The panel gave some consideration to the fact that only one academic expert is involved in the accreditation process of existing study programmes. As already mentioned, the expert teams consists of three members: one academic expert, one student and one employer. The panel believes that a peer review model used for subject wise accreditation of study programmes would gain in involving more than one academic expert. This would ensure an accreditation model better fit for its purpose.

 

Another concern is sometimes the involvement of ACE staff (consultants) in the review at content level. The fact that there are two of them on the site visit with each panel is also a source of imbalance in the interactions with programme representatives.

 

Standard fulfilment: Substantially compliant

 

Recommendations:

 

The Accreditation Institution should consider how to include experts also for accreditation of new study programmes.

 

The Accreditation Institution should consider involving more than one academic expert in the accreditation of existing study programmes.

 

1.1.5 ESG 2.5 REPORTING

 

STANDARD:

 

Reports should be published and should be written in a style which is clear and readily accessible to its intended readership. Any decisions, commendations or recommendations contained in reports should be easy for a reader to find.

 

GUIDELINES:

 

In order to ensure maximum benefit from external quality assurance processes, it is important that reports should meet the identified needs of the intended readership.

 

Reports are sometimes intended for different readership groups and this will require careful attention to structure, content, style and tone.

 

In general, reports should be structured to cover description, analysis (including relevant evidence), conclusions, commendations, and recommendations. There should be sufficient preliminary explanation to enable a lay reader to understand the purposes of the review, its form, and the criteria used in making decisions. Key findings, conclusions and recommendations should be easily locatable by readers. Reports should be published in a readily accessible form and there should be opportunities for readers and users of the reports (both within the relevant institution and outside it) to comment on their usefulness.

 

Page 17 of 39

 

All accreditation reports are published and are structured in the same manner. They contain, in chronological order: Introduction; information on the panel and the case processing, timetable etc.; recommendation; summary of assessments; critical legal factors; basic information; competency profile of the study programme; structure of the study programme; assessment; and legality. However, as pointed out by the Accreditation Institution itself, they are not too simple to read by a general reader as the main purpose of the report is to provide information for accreditation decision. Thus the reports do not seem to be used as student information about study programmes.

 

Standard fulfilment: Fully compliant

 

Recommendations:

 

We recommend to the Accreditation Institution that it tries to further develop the accreditation reports in a way that is more comprehensive and useful for a general reader.

 

1.1.6 ESG 2.6 FOLLOW-UP PROCEDURES

 

STANDARD:

 

Quality assurance processes which contain recommendations for action or which require a subsequent action plan, should have a predetermined follow-up procedure which is implemented consistently.

 

GUIDELINES:

 

Quality assurance is not principally about individual external scrutiny events: It should be about continuously trying to do a better job. External quality assurance does not end with the publication of the report and should include a structured follow-up procedure to ensure that recommendations are dealt with appropriately and any required action plans drawn up and implemented. This may involve further meetings with institutional or programme representatives. The objective is to ensure that areas identified for improvement are dealt with speedily and that further enhancement is encouraged.

 

As the Accreditation Institution can make three decisions (positive accreditation, conditional positive accreditation and refusal of accreditation), the follow up procedures are elaborated in case of conditional positive accreditation. Conditional positive accreditation means that a study programme does not meat one or more accreditation criteria and accreditation has a shorter period of ‘validity’, namely 1 or 2 years. During this time the study programme has a possibility to implement changes and improvements in the areas that were found to be ‘problematic’. At the next accreditation only the areas identified before are checked. In this case the Accreditation Council can only make two decisions: positive accreditation or refusal of accreditation.

 

For study programmes with a positive accreditation, follow-up on the part of the Accreditation Council and ACE Denmark is only carried out once the study programme again forms part of the rotation (cycle). These study programmes may – in addition to the decision on the positive accreditation – receive recommendations from the Council in the form of critical factors and comments. Critical factors have no bearing on the formal decision. It is up to the universities

 

Page 18 of 39

 

alone to act on the critical factors. The Accreditation Institution does not carry out actual follow-up on the critical factors, which typically highlight areas on which the Council finds the universities should focus in their further quality development of the study programmes.

 

Standard fulfilment: Fully compliant

 

Recommendations:

 

The panel realises that in the context of such strict accreditation system there is no legal need or opportunity for follow up after the positive decision. However, in order to ensure further development of institutional quality, we suggest the Accreditation Institution to develop follow up procedures to offer support to HEIs even in case of positive accreditation.

 

1.1.7 ESG 2.7 PERIODIC REVIEWS

 

STANDARD:

 

External quality assurance of institutions and/or programmes should be undertaken on a cyclical basis. The length of the cycle and the review procedures to be used should be clearly defined and published in advance.

 

GUIDELINES:

 

Quality assurance is not a static but a dynamic process. It should be continuous and not “once in a lifetime”. It does not end with the first review or with the completion of the formal follow-up procedure. It has to be periodically renewed. Subsequent external reviews should take into account progress that has been made since the previous event. The process to be used in all external reviews should be clearly defined by the external quality assurance agency and its demands on institutions should not be greater than are necessary for the achievement of its objectives.

 

The study programmes are subject to accreditation every 6 years which is clearly defined and published. The Accreditation Institution can decide to evaluate a study programme even before the ‘accreditation expires’. In case of conditional positive accreditation, the programme is subject to accreditation in 1 or 2 years.

 

With the establishment of the Accreditation Institution in 2007, the study programmes are being accredited for the first time. The rotation plan covers the period 2008-14 and comprises the approx. 1,050 university study programmes. The current rotation plan thus runs until 2015, after which time the programmes must be reaccredited. This ensures a cyclical assessment of all university study programmes. The Accreditation Institution is also responsible for accrediting proposals for new study programmes.

 

Standard fulfilment: Fully compliant

 

1.1.8 ESG 2.8 SYSTEM-WIDE ANALYSES

 

STANDARD:

 

Page 19 of 39

 

Quality assurance agencies should produce from time to time summary reports describing and analysing the general findings of their reviews, evaluations, assessments, etc.

 

GUIDELINES:

 

All external quality assurance agencies collect a wealth of information about individual programmes and/or institutions and this provides material for structured analyses across whole higher education systems. Such analyses can provide very useful information about developments, trends, emerging good practice and areas of persistent difficulty or weakness and can become useful tools for policy development and quality enhancement. Agencies should consider including a research and development function within their activities, to help them extract maximum benefit from their work.

 

The Accreditation Institution is a young organisation that successfully set up resources, procedures and methods in a very short period of time. So far it has developed an overall analysis of criteria and methods after the first accreditation round which ended up in reduction of criteria, more comprehensive system and new guidelines for accreditation.

 

At the site visit the panel was informed that as development of the rules and procedures guiding the processes of the work of the Accreditation Institution and on the other hand developing the principles and procedures of the accreditation process itself is now firmly established, the Accreditation Institution has commenced work on system-wide analyses that may serve as a vehicle for quality assurance and quality improvement in the Danish universities.

 

In addition to the information already provided, the panel learned about the following plans of the Accreditation Institution:

 

In the course of 2010, the Accreditation Institution has planned an analysis of “New tendencies in the educational landscape”. The analysis will take as a point of departure the information gathered through the accreditation of new university programme in the period 2008-2010. Apart from discussing the trends and tendencies emanating from the analysis, it will also provide a systematic presentation, based on commonality of characteristics, of the programmes applied for and accredited during the period. Another interesting aspect that may emerge from such an analysis is the possibility to identify the strengths and weaknesses, amongst the new programmes, resulting from the application of the various criteria across the different academic sub-fields. The results will be published and discussed with the stakeholders – amongst other in the Dialogue Forum.

 

In 2011, the Accreditation Institution will select a limited number of clusters of educations that will be undergoing a closer analysis with a focus on mechanisms to assure the continuous relevance of the programmes taught based on an analysis involving amongst other the educational programmes’ on-going contacts with graduated students, prospective employers, and other labour market representatives. The programmes accredited during the first years of activity of the Accreditation Institution will be a key source in the development of the study.

 

Page 20 of 39

 

Further, the Accreditation Institution plans some analyses of a more limited scope: In 2010 the Institution will amongst other publish material targeting students to demonstrate the relevance of programme accreditation to students.

 

In addition, the Institution will be contributing to broader – albeit not necessarily system-wide – analyses in a Nordic and international context, such as the NOQA projects, where the Institution has contributed to the 2009 study on the accreditation/evaluation of joint master programmes and played an active role in the 2010 study of the role of the expert-panels’ visit in the accreditation process. The Institution intends to increase this international activity following the future membership of organisations such as ENQA, ECA and EQAR.

 

Standard fulfilment: Partially compliant

 

1.2 ESG 3.1 USE OF EXTERNAL QUALITY ASSURANCE PROCEDURES FOR

 

HIGHER EDUCATION

 

Standard

 

The external quality assurance agencies should take into account the presence and effectiveness of the external quality assurance procedures described in Part 2 of the European Standard and Guidelines.

 

Guidelines

 

The standards for external quality assurance contained in Part 2 provide a valuable basis for the external quality assessment process. The standards reflect best practices and experiences gained through the development of external quality assurance in Europe since the early 1990s. It is therefore important that these standards are integrated into the processes applied by external quality assurance agencies towards the higher education institutions. The standards for external quality assurance should together with the standards for external quality assurance agencies constitute the basis for professional and credible external quality assurance of higher education institutions.

 

The above sections on the relation between ESG 3.1 and ESG 2.1 – 2.8 include a number of recommendations and reflections, and some findings of substantial rather than full compliance regarding the standards of section 2.

 

The overall conclusion of the review panel is that the Accreditation Institution complies substantially with ESG 3.1.

 

1.3 ESG 3.3 ACTIVITIES

 

STANDARD:

 

Agencies should undertake external quality assurance activities (at institutional or programme level) on a regular basis.

 

GUIDELINES:

 

These may involve evaluation, review, audit, assessment, accreditation or other similar activities and should be part of the core functions of the agency.

 

Page 21 of 39

 

The Accreditation Institution was established with the purpose of external quality assurance. Accreditation is its main activity and is performed on a regular basis.

 

The universities have been informed that the Accreditation Institution is authorised to make the following decisions regarding accreditation of the existing study programmes:

 

– Positive accreditation if the Council assesses that the study programme in

 

general meets the accreditation criteria.

 

– Conditional positive accreditation if the Council assesses that the study

 

programme does not fully meet the conditions for a positive accreditation, but it

 

is assessed that it will be able to address the issues in the short term. The

 

Accreditation Council draws up a schedule for when the issues must be

 

addressed.

 

– Refusal of accreditation if the Council assesses that the study programme in

 

general does not meet the accreditation criteria.

 

As concerns applications for new study programmes, the Accreditation Council can either hand down a positive accreditation or a refusal of accreditation.

 

The accreditations of the first existing study programmes took place in 2008 when 37 programmes were accredited. In the subsequent years, the Accreditation Institution increased the number of study programmes, and plans are currently to accredit approx. 150 existing programmes every year from 2010 on onwards. The plan is organised so that accreditations are initiated of approx.

 

75 study programmes every six months.

 

Standard fulfilment: Fully compliant

 

2. ENQA CRITERION 2: ESG 3.2 OFFICIAL STATUS

 

STANDARD:

 

Agencies should be formally recognised by competent public authorities in the European Higher Education Area as agencies with responsibilities for external quality assurance and should have an established legal basis. They should comply with any requirements of the legislative jurisdictions within which they operate.

 

The Accreditation Institution is formally recognised and set up by the Accreditation Act in 2007 as a professionally independent body within the State administration. The Institution is subject to the State rules and regulations, including the Danish Public Administration Act, the Danish Access to Public Administration Files Act (Offentlighedsloven) and the Ministry of Finance’s Budget Guidelines (Finansministeriets Budgetvejledning). The Accreditation Institution is not subject to the power of instruction of the Minister in relation to accreditation issues. The panel, however, would like to further elaborate and explain different structures within the Accreditation Institution and relations between them.

 

The Accreditation Institution consists of the Accreditation Council as the main decision making body and ACE Denmark, which includes two secretariats that are

 

Page 22 of 39

 

also defined in the Accreditation Act. The Council Secretariat administratively supports the Council. As the panel learned at the site visit this mainly includes help with setting up meetings, document circulation, minutes keeping etc. The task of the Professional Secretariat is carrying out accreditation procedures, including: development of accreditation guidelines, training of experts, receiving applications, preparing site visits, attendance at site visits, drafting accreditation reports, preparing final reports, etc. From an organisational point there is also an Administrative Unit that supports the Accreditation Institution as such: human resources, legal support, etc.

 

The Accreditation Council also makes accreditation decisions for study programmes under the auspices of other ministries than the Ministry of Science, Technology and Innovation and in this case it receives accreditation reports from EVA (Danish Evaluation Institute). EVA is, however, independent from ACE Denmark and from the Accreditation Institution as such.

 

Standard fulfilment: Fully compliant

 

3. ENQA CRITERION 3: ESG 3.4 RESOURCES

 

STANDARD:

 

Agencies should have adequate and proportional resources, both human and financial, to enable them to organise and run their external quality assurance process(es) in an effective and efficient manner, with appropriate provision for the development of their processes and procedures.

 

The Accreditation Institution has adequate resources. The number of staff grew gradually from 21 FTE in 2008 to 29,5 FTE in 2009 and currently 37 employees (34 FTE), which was estimated to be sufficient for the tasks and workload of the Accreditation Institution (approximately 150 existing study programmes and 40 new study programmes annually).

 

The financial resources are also adequate to support the structure and work of the Institution. In 2008 the annual budget was 17.3 million DKK (approximately 2,3 million EUR) and in 2009 18 million DKK (approximately 2,4 million EUR) with supplementary funding in 2009 of 5,4 million DKK (approximately 725.000 EUR).

 

Staff gained in their competence since the establishment of the Institution. Since the staff was young and mainly new to the field, the Institution offered substantial training and support. It was mentioned in the interviews that sometimes the interpretation of criteria by ACE staff was not comprehensive and consistent in the past but that their competence for support of accreditation procedures and criteria interpretation and usage grew.

 

Standard fulfilment: Fully compliant

 

Recommendations:

 

The panel recommends that further training be provided for staff and that they be offered special support in writing accreditation reports.

 

Page 23 of 39

 

4. ENQA CRITERION 4: ESG 3.5 MISSION STATEMENT

 

STANDARD:

 

Agencies should have clear and explicit goals and objectives for their work, contained in a publicly available statement.

 

GUIDELINES:

 

These statements should describe the goals and objectives of agencies’ quality assurance processes, the division of labour with relevant stakeholders in higher education, especially the higher education institutions, and the cultural and historical context of their work. The statements should make clear that the external quality assurance process is a major activity of the agency and that there exists a systematic approach to achieving its goals and objectives. There should also be documentation to demonstrate how the statements are translated into a clear policy and management plan.

 

The Accreditation Institution’s mission, vision and values are publicly available and read as follows:

 

Mission:

 

“As an independent operator, ACE Denmark contributes to ensuring the quality and relevance of higher education in Denmark. Our accreditation work is based on national and international quality assurance standards. ACE Denmark generates knowledge on the quality of higher education through a dialogue with the educational institutions and other relevant interested parties.”

 

Vision:

 

“The Accreditation Council and ACE Denmark strive to be a leading quality assurance institution in Denmark and abroad that documents and generates new, useful knowledge on the quality and relevance of higher education.”

 

Values:

 

“The Accreditation Council and ACE Denmark have four core values which run through all of our accreditation work: The core values are: Independence, transparency, professional capability and openness.

 

In addition, the performance contract, director’s contract, annual review and annual report set out goals and results for the organisation. The goals from the performance and director’s contracts have also been implemented in a management plan.

 

Standard fulfilment: Fully compliant

 

5. ENQA CRITERION 5: ESG 3.6 INDEPENDENCE

 

STANDARD:

 

Agencies should be independent to the extent both that they have autonomous responsibility for their operations and that the conclusions and recommendations made in their reports cannot be influenced by third parties such as higher education institutions, ministries or other stakeholders.

 

Page 24 of 39

 

GUIDELINES:

 

An agency will need to demonstrate its independence through measures, such as:

 

• its operational independence from higher education institutions and

 

governments

 

• is guaranteed in official documentation (e.g. instruments of governance or

 

legislative acts);

 

• the definition and operation of its procedures and methods, the nomination and

 

appointment of external experts and the determination of the outcomes of its

 

quality assurance processes are undertaken autonomously and independently

 

from governments, higher education institutions, and organs of political

 

influence;

 

• while relevant stakeholders in higher education, particularly students/learners,

 

are consulted in the course of quality assurance processes, the final outcomes of

 

the quality assurance processes remain the responsibility of the agency.

 

According to the Accreditation Act, the Accreditation Institution is independent organisation within the public administration. The panel found that the Institution meets the criterion Independence in relation to higher education institutions, ministries and other stakeholders.

 

The Accreditation Council has a Chairman, a Vice-chairman and seven other members. The Minister for Science appoints the Chairman after consulting with the Minister for Education and the Minister for Culture. In addition, the Minister for Science appoints eight members, of whom three have been nominated by the Minister for Education and one by the Minister for Culture. The Minister for Science appoints three members. The last member is a student who is appointed by the Minister for Science following a nomination from the student representatives on the executive committees of the institutions of higher education.

 

Since it is the Minister who appoints members of the Council it might seem to influence the independence of the Accreditation Institution. However, at the site visit the panel learned about the national (legal) context, the procedure that leads to the appointment and the general understanding of the independence by all stakeholders. The choice of the Council members follows wide public consultation in which all stakeholders have opportunity to suggest members. The panel inquired in detail what was the actual process of nomination of members, which happened once until now. We learned that there was generally wide consensus among all relevant parties which members are to be nominated. There were some members that caused some concern among some stakeholders, however, at the interviews all stakeholders confirmed and even assured that the Council is completely independent and is not influenced by either of the ministries, nor by HEIs or other parties. The Accreditation Institution also independently decides upon the processes it uses as well as expert appointment.

 

The Accreditation Institution worked even further on the perceived independence. Although the Accreditation Institution is completely independent in evaluating study programmes according to the criteria and preparing final accreditation reports, it holds discussions with the ministry regarding legal matters that exclude expert, content discussion. Initially reports were not published until after discussions with the ministry on specific legal matters as set

 

Page 25 of 39

 

out in the ministerial order on criteria on the relevance of study programmes. For the avoidance of any possible doubt on the independence of the procedures the Accreditation Institute is in the process of changing this practice so that the reports are issued after the accreditation process itself is finished and prior any discussion on legal matters with the ministry.

 

The panel feels it necessary to elaborate on the appeals procedure and negotiation for the budget in the light of independence as well as both processes are connected to the Ministry of Science.

 

When the draft accreditation report is prepared it is sent to the HEI for comments (the Accreditation Institution calls this ‘a hearing process’) in which HEI has a right to comment on the findings. This is then reviewed by ACE Denmark’s staff and experts involved in the reviews and decided which comments are to be included in the final accreditation reports. Based on the final report the Accreditation Council makes a final decision. HEI can then appeal to the Ministry for the procedural errors and not content decision. This is again compliant to the national legal framework and is not disputed by any stakeholder. That kind of appeals procedure does not seem to be understood to hamper independence by any party.

 

Regarding the budget negotiation the Accreditation Institution has to negotiate annually with the ministry on the basis of its work plan. This is, however, a negotiation about the number of accreditations that will be processed in the following year including the resources needed for the work plan and not about which particular accreditations will be done. Thus the panel estimates that this procedure does not hinder the operational or other type of independence.

 

The panel believes that there is one aspect in which the Accreditation Institution could strengthen its independence, which is regarding the accreditation criteria. As already mentioned before in the report, the general five criteria are laid down in the Accreditation Order by the Minister of science. There is also an Annex to this Order in which the five criteria are further developed. The Institution independently prepares detailed guidelines for accreditation based on these established criteria in which elements to be evaluated for accreditation are elaborated. The panel sees this as an important point for improvement of the independence. From the panel’s point of view the Institution should own the responsibility to elaborate on the criteria itself or at least the elaboration of the general five criteria. However, looking at the criterion “Independence”, the panel does not believe that these circumstances fit in to neither the standard nor the guidelines for the criterion “Independence”. Therefore, the panel concluded this standard to be fully compliant.

 

Standard fulfilment: Fully compliant

 

Recommendations:

 

The Accreditation Institution should identify the whole set of accreditation criteria itself without ministerial interference and freely elaborate them. This would also provide the opportunity to create consistent criteria for all study programmes which run under the auspices of different ministries.

 

Page 26 of 39

 

6. ENQA CRITERION 6: ESG 3.7 EXTERNAL QUALITY ASSURANCE

 

CRITERIA AND PROCESSES USED BY THE AGENCIES

 

STANDARD:

 

The processes, criteria and procedures used by agencies should be pre-defined and publicly available. These processes will normally be expected to include: • a self-assessment or equivalent procedure by the subject of the quality assurance process;

 

• an external assessment by a group of experts, including, as appropriate, (a)

 

student member(s), and site visits as decided by the agency;

 

• publication of a report, including any decisions, recommendations or other

 

formal outcomes;

 

• a follow-up procedure to review actions taken by the subject of the quality

 

assurance process in the light of any recommendations contained in the report.

 

GUIDELINES:

 

Agencies may develop and use other processes and procedures for particular purposes. Agencies should pay careful attention to their declared principles at all times, and ensure both that their requirements and processes are managed professionally and that their conclusions and decisions are reached in a consistent manner, even though the decisions are formed by groups of different people. Agencies that make formal quality assurance decisions or conclusions which have formal consequences should have an appeals procedure. The nature and form of the appeals procedure should be determined in the light of the constitution of each agency.

 

Processes, criteria and procedures used by the Accreditation Institution are predefined and publicly available, cf. the analysis of ESG 2.3 (cf. section 1.1.3) of this report.

 

The evaluation processes involves the self-assessment phase and an external assessment by a group of experts including a student member (cf. section 1.1.4, ESG 2.4). ACE Denmark prepares an accreditation report on every assessment process undertaken based on the initial report and comments received by HEIs. The reports are then published on the Institution website.

 

Follow-up procedures are in place in case of conditional positive accreditation. Comments and conclusions of the review panel on this subject can be found in the analysis of ESG 2.6 (cf. section 1.1.6) of this report.

 

The Accreditation Institution has a procedure in place for HEIs to comment on the results of the assessment processes that lead to the final accreditation report and formal decision. After a decision has been taken there is also an appeals procedure in place (cf. analysis of ESG 3.6 in this report).

 

We found certain unevenness in the views expressed by representatives of universities with whom we discussed experience of accreditation site visits. Some people expressed a high level of satisfaction with the expert teams and their accompanying staff from ACE Denmark. However others found some unevenness in the competence of expert panels. There was a lack of unanimity regarding the influence which may be exercised by ACE Denmark’s staff who accompany the

 

Page 27 of 39

 

panels. The panel recommends the Accreditation Institution to work further on the consistency of site visits and evaluation procedures.

 

Another point, which was also already mentioned before, is the inconsistency of procedures between the accreditation of new study programmes in which the experts are not involved and the accreditation of existing study programmes in which experts are involved (cf. analysis ESG 2.4 in this report).

 

Appeals procedure is also not set up as literally suggested in the guidelines for the standard 3.7 as the agencies that make formal quality assurance decisions or conclusions which have formal consequences should have an appeals procedure. This was already commented previously under the analysis of the ESG 3.6 in this report.

 

Standard fulfilment: Substantially compliant

 

Recommendations:

 

The Accreditation Institution should consider how to include experts also for accreditation of new study programmes.

 

The Accreditation Institution should further work on consistency of site visits in terms of expert teams and ACE Denmark’s staff role in the teams.

 

7. ENQA CRITERION 7: ESG 3.8 ACCOUNTABILITY PROCEDURES

 

STANDARD:

 

Agencies should have in place procedures for their own accountability. GUIDELINES:

 

These procedures are expected to include the following:

 

1. A published policy for the assurance of the quality of the agency itself, made

 

available on its website;

 

2. Documentation which demonstrates that:

 

• the agency’s processes and results reflect its mission and goals of quality

 

assurance;

 

• the agency has in place, and enforces, a no-conflict-of-interest mechanism in

 

the work of its external experts;

 

• the agency has reliable mechanisms that ensure the quality of any activities

 

and material produced by subcontractors, if some or all of the elements in its

 

quality assurance procedure are subcontracted to other parties;

 

• the agency has in place internal quality assurance procedures which include an

 

internal feedback mechanism (i.e. means to collect feedback from its own staff

 

and council/board); an internal reflection mechanism (i.e. means to react to

 

internal and external recommendations for improvement); and an external

 

feedback mechanism (i.e. means to collect feedback from experts and reviewed

 

institutions for future development) in order to inform and underpin its own

 

development and improvement.

 

3. A mandatory cyclical external review of the agency’s activities at least once

 

every five years.

 

Page 28 of 39

 

The Accreditation Institution has a policy for the assurance of the quality of the agency itself. ACE Denmark’s quality assurance policy sets out that the quality assurance work is anchored in the management and comprises the accreditation processes in connection with both new and existing study programmes. In addition, the quality assurance policy stipulates that the quality assurance system must proactively take the core values of the organisation into account and develop the processes in the accreditation work on the basis of systematic cyclical measurements. The Accreditation Institution also has several procedures that function as internal reflection mechanisms in place. One of them is constant internal feedback from staff and focus on process improvement. The other is feedback from experts that take part in expert teams. The Council has thus organised a quality assurance cycle in order to systematically safeguard key elements in the operators’ processes and methods. The quality assurance cycle is based on a transparent and predictable review of selected parts of the ACE Denmark’s processes and methods.

 

Concerning the external feedback mechanism, an evaluation of the universities’ experience with the accreditation process was carried out already after the Accreditation Institution’s first accreditation round. Moreover, an evaluation of the accreditation of new study programmes which took place in spring/summer 2008 has also been carried out. Survey was made of views of stakeholders or relevant actors regarding the satisfaction of the accreditation process. In addition to this conferences with higher education stakeholders are organised to discuss procedures and necessary changes as well as a so-called Dialogue Forum is organised. The Accreditation Institution has also held meetings with representatives of the universities as well as other stakeholders to discuss procedures and necessary changes.

 

At the site visit it was mentioned by several stakeholders that they do have the opportunity to comment on accreditation procedures and some recommendations are taken into account by the Accreditation Institution, but not all. Some of them feel that they are “listened to, but not heard”. The panel understands this concern, although the independence of the Institution has to be safeguarded.

 

It is, however, strangely put in the self evaluation report that the Accreditation Council quality assures ACE Denmark although both bodies are a part of the Accreditation Institution. The panel learned at the site visit that this refers to the general responsibility for the quality of the Institution and that there are sufficient accountability procedures in place.

 

Standard fulfilment: Fully compliant

 

Recommendations:

 

The Accreditation Institution could improve consideration of feedback by stakeholders.

 

8. ENQA CRITERION 8: Miscellaneous

 

i. The agency pays careful attention to its declared principles at all times, and

 

ensures both that its requirements and processes are managed professionally

 

Page 29 of 39

 

and that its judgments and decisions are reached in a consistent manner, even if the judgments are formed by different groups;

 

ii. If the agency makes formal quality assurance decisions, or conclusions which have formal consequences, it should have an appeals procedure. The nature and form of the appeals procedure should be determined in the light of the constitution of the agency;

 

iii. The agency is willing to contribute actively to the aims of ENQA.

 

Regarding the Accreditation Institution attention to its declared principles and the appeals procedures this report already provides sufficient evidence and interpretation.

 

The panel also inquired about the Accreditation Institution’s willingness to contribute to the aims of ENQA. It was clearly stated that the Institution special focus is the international dimension and contribution to the quality assurance development in this area. It is prepared to disseminate information, experiences and good practices in the field of quality assurance (QA) in higher education to other European QA agencies, public authorities and higher education institutions.

 

Standard fulfilment: Substantially compliant

 

Page 30 of 39

 

ADDITIONAL RECOMMENDATION

 

The panel estimates that great work has been done in relatively short time. The Accreditation Institution managed to set up the rules, procedures and support for very demanding accreditation system. However, the panel also felt in order to fully help the Institution to identify further points of development an external evaluation a year or two later or a type B of evaluation seemed to be more appropriate. In the type B evaluation, the review has a number of purposes, one of which is to fulfil the external review requirement of ENQA membership / EQAR listing. Thus the other purpose could be the overall and comprehensive assessment of the new external quality assurance system in Denmark as such.

 

The panel believes that the Accreditation Institution meets most of the standards to a high level. However, in the light of future development of quality of the Institution, we would like to make additional suggestions.

 

The panel found the approach of detailed programme accreditation was legitimate and operated on a fair basis. However it did appear to be quite burdensome for the whole quality assurance system and HEIs. Understanding the political consensus regarding the decision for this approach, the panel recommends thinking about replacing programme accreditation with institutional accreditation in the future or more flexible combination of both. The appropriate time might be after the Accreditation Institution finishes with the first cycle of accreditations. The picture of quality of study programmes will be clearer at that point as well as involvement and competence of all stakeholders. For the time being we suggest to the Institution to think about clustering the study programmes on national level in the process of accreditation. This would allow the Accreditation Institution to have a more proactive approach in the accreditation process, which in turn would improve and facilitate recruitment of experts, consistency in assessments, benchmarking possibilities for HEIs etc.

 

The articulation of External Accreditation and Internal Quality Assurance is not optimal. On the one hand the need to document the effectiveness of the Internal Quality Assurance system creates a significant increase of the bureaucratic burden and on the other hand the effectiveness of these internal quality processes is not checked in a consistent manner.

 

The current framework in which at least three different ministries set at least three different kinds of criteria for accreditation seems to hamper the consistency and general development of quality of study programmes. On the other hand the Accreditation Institution has the overall responsibility for the quality of study programmes as well as the most knowledge about the procedures and standards. The panel thus suggests for the framework to change in a manner to allow the Institution to have more impact on the different approaches used by different ministries. This will allow better consistency and standards achievement for all study programmes.

 

The panel realises the suggestions made impact the whole system and competencies of several ministries, thus we feel the possible changes have to be based on (rational and) real evidence in a sense of process development. During this process careful consideration should be given to the criteria so that they are not an obstacle to innovation.

 

Page 31 of 39

 

In the following paragraphs the panel would like to point out the main recommendations of varying magnitude although some have been already mentioned before.

 

Regarding the experts that are involved in the external evaluations we suggest to include more than one academic expert from the (science) field. The current situation is that the team comprises of three members: an academic expert, a student and an employer. In order for the process to be better fit for purpose the panel suggests to include more than one academic expert. The panel also recommends reducing the role of ACE Denmark’s staff in the expert teams. The panel learned that the accreditation officers as employees of ACE Denmark are not formally equal members of the expert teams. However, there are two officers present and three external members. The number ratio may be perceived as not proper by some stakeholders. The other problem that was detected and is not always present is the active role of the staff at the site visits of HEIs as it can happen that the officer takes over the questioning and even interrupt the expert while asking questions. Using a peer review accreditation model the panel believes that it is important for t he experts to have a responsibility in the process of choosing questions to be asked and the actual questioning at the site visit.

 

For proper consistency and academic as well as expert consideration of all study programmes the panel suggests including academic experts in the process of accreditation of new study programmes. The panel already described the situation and presented the reasons for this suggestion in previous chapters of the report. The panel suggests also considering the evaluation of PhD study programmes as the system in place ensures quality for the whole higher education sector. Thus we believe they should be a part of systematic evaluation.

 

A particular element in the system which could be easily improved is the nomination of a student to the Accreditation Council. Currently a great number of students can suggest members (every student that is a board member at any Danish HEI) which can lead to the case that there is a high number of suggested students out of which the minister chooses one. This can cause additional confusion and lower the transparency procedure. Thus we suggest for the ministry to cooperate with the representative organisation in the process of suggesting the student members.

 

Page 32 of 39

 

CONCLUSION

 

The review panel has in the Accreditation Institution’s self-evaluation report, the many supplied documents and in the context of the site visit found much to commend in the Accreditation Institution’s organisation and operations, especially due to its recent establishment which results in substantial compliance with the European Standards and Guidelines. Much has been done to set up a demanding accreditation processes and the Accreditation Institution gained valuable expertise. All of the stakeholders the panel met pointed out generally positive developments and showed satisfaction with the Accreditation Institution’s efforts to candidate for ENQA membership.

 

There are normally points in the system for further development and improvement which will contribute to higher quality of the system. The panel thus posed several recommendations for further development of the system which are elaborated in the report. The panel recommends the future changes to be evidence based and consulted between all stakeholders.

 

Page 33 of 39

 

Annex 1: Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG)

 

Part 1

 

1.1 – Policy and procedures for quality assurance

 

Institutions should have a policy and associated procedures for the assurance of the quality and standards of their programmes and awards. They should also commit themselves explicitly to the development of a culture which recognises the importance of quality, and quality assurance, in their work. To achieve this, institutions should develop and implement a strategy for the continuous enhancement of quality. The strategy, policy and procedures should have a formal status and be publicly available. They should also include a role for students and other stakeholders.

 

1.2 – Approval, monitoring and periodic review of programmes and awards

 

Institutions should have formal mechanisms for the approval, periodic review and monitoring of their programmes and awards.

 

1.3 – Assessment of students

 

Students should be assessed using published criteria, regulations and procedures which are applied consistently.

 

1.4 – Quality assurance of teaching staff

 

Institutions should have ways of satisfying themselves that staff involved with the teaching of students are qualified and competent to do so. They should be available to those undertaking external reviews, and commented upon in reports.

 

1.5 – Learning resources and student support

 

Institutions should ensure that the resources available for the support of student learning are adequate and appropriate for each programme offered.

 

1.6 – Information systems

 

Institutions should ensure that they collect, analyse and use relevant information for the effective management of their programmes of study and other activities.

 

1.7 – Public information

 

Institutions should regularly publish up to date, impartial and objective information, both quantitative and qualitative, about the programmes and awards they are offering.

 

Part 2

 

2.1 – Use of internal quality assurance procedures

 

External quality assurance procedures should take into account the effectiveness of the internal quality assurance processes described in Part 1 of the European Standards and Guidelines.

 

2.2 – Development of external quality assurance processes

 

The aims and objective of quality assurance processes should be determined before the processes themselves are developed, by all those responsible

 

Page 34 of 39

 

(including higher education institutions) and should be published with a description of the procedures to be used.

 

2.3 – Criteria for decisions

 

Any formal decisions made as a result of an external quality assurance activity should be based on explicit published criteria that are applied consistently.

 

2.4 – Process fit for purpose

 

All external quality assurance processes should be designed specifically to ensure their fitness to achieve the aims and objectives set for them.

 

2.5 – Reporting

 

Reports should be written in a style, which is clear and readily accessible to its intended readership. Any decisions, commendations or recommendations contained in reports should be easy for a reader to find.

 

2.6 – Follow-up procedures

 

Quality Assurance Processes which contain recommendations for action or which require a subsequent action plan, should have predetermined follow-up procedure, which is implemented consistently.

 

2.7 – Periodic reviews

 

External quality assurance of institutions and/or programmes should be undertaken on a cyclical basis. The length of the cycle and the review procedures to be used should be clearly defined and published in advanced.

 

2.8 – System-wide analysis

 

Quality Assurance should produce form time to time summary reports describing and analysing the general findings of their reviews, evaluations, assessments, etc.

 

Part 3

 

3.1 – Use of external quality assurance procedures for higher education The external quality assurance agencies should take into account the presence and effectiveness of the external quality assurance procedures described in Part 2 of the European Standard and Guidelines.

 

3.2 – Official status

 

Agencies should be formally recognised by competent public authorities in the European Higher Education Area as agencies with responsibilities for external quality assurance and should have an established legal basis. They should comply with any requirements of the legislative jurisdiction within they operate.

 

3.3 – Activities

 

Agencies should undertake external quality assurance activities (at institutional or programme level) on a regular basis.

 

3.4 – Resources

 

Agencies should have adequate and proportional resources both human and financial, to enable them to organise and run their external quality assurance

 

Page 35 of 39

 

process(es) in an effective manner with appropriate provision for the development of their processes and procedures.

 

3.5 – Mission Statement

 

Agencies should have clear and explicit goals and objectives for their work, set down in a publicly available statement.

 

3.6 – Independence

 

Agencies should be independent to the extent both that they have autonomous responsibility for their operations and that the conclusions and recommendations made in their reports cannot be influenced by third parties such as higher education institutions, ministries or other stakeholders.

 

3.7 – External quality assurance criteria and processes used by agencies The process, criteria and procedures used by agencies should be pre-defined and publicly available. These processes will normally be expected to include:

 

– a self-assessment or equivalent procedure by the subject of the quality

 

assurance processes;

 

– an external assessment by group of experts, including, as appropriate, (a)

 

student member(s) and site visit as decided by the agency;

 

– publication of a report, including any decisions, recommendations or other

 

formal outcomes;

 

– a follow-up procedure to review actions taken by the subject of the quality

 

assurance process in the light of any recommendations contained in the report.

 

3.8 – Accountability

 

Agencies should have in place procedures for their own accountability.

 

Page 36 of 39

 

Annex 2 – Programme of the site visit

 

Panel activity	Theme
16thJune
The Panel arrives.The Panel has its own meeting.
17thJune
9.00 – 9.15	Team gathering
9.15 a.m. –9.45 a.m.	Meeting with the Council Chairmanship and ACE Denmark’s management:Chairman Mr. Søren Barlebo RasmussenVice Chairman Mr. Christian Thune Acting Director Mr. Sami Stephan BoutaibaHead of the Professional Secretariat Mr. Steffen Westergård Andersen	WelcomePresentation of the organisation, roles, and division of competencies. Presentation of the programme
9.55 a.m. – 10.40 a.m.	Meeting with representatives from the Accreditation Council (Chairman, Vice Chairman and two other representatives)	Questions from the Panel.
10.50 a.m. –11.20 a.m.	Meeting with ACE Denmark’s management:Acting Director Mr. Sami Stephan BoutaibaHead of the Professional Secretariat Mr. Steffen Westergård Andersen	Questions from the Panel
11.30 a.m. – 12.15 p.m.	Meeting with representatives from Ministry of Science, Innovation, and Technology (VTU):Director Mr. Jens Peter Jacobsen Head of Division Ms. Mette Ring RossingSpecial Advisor Ms. Lene Britt Boisen	Questions from the Panel

 

Page 37 of 39

 

Panel activity	Theme
12.15 p.m. – 1.30 p.m.	Lunch – Panel meeting
1.30 p.m. – 2.15 p.m.	Meeting with employees from ACE Denmark:Senior Advisor Mr. Lars Pedersen Senior Advisor Ms. Vibeke Fahlén Special Advisor Ms. Dorthe StadsgaardSpecial Advisor Mr. David Metz Accreditation Consultant Ms. Christina Haulrich Klausen Accreditation Consultant Mr. Jonas Bech Hansen	Questions from the Panel
2.25 p.m. – 3.00 p.m.	Meeting with representatives from the Danish Evaluation Institute (EVA):Head of Division Mr. Tue VinterJørgensen	Questions from the Panel about the Accreditation Council
7.30 p.m.	The Panel has its own dinner
18thJune
8.30 a.m. – 9.00 a.m.	Evaluation in the team.	A liaison person who can access documents or make necessary arrangements. Also access to PC, printer and photocopier.
9.00 a.m. –9.45 a.m.	Meeting with representatives from DSF. (The National Union of Students in Denmark:Ms. Rie Kjær Poulsen Ms. Kristine Bak Nielsen Mr. Andreas Kvist BacherVice President Ms. Lena Skotte	Questions for the students
9.55 a.m. –10.40 a.m.	Meeting with president of Universities Denmark. (The Danish universities guild):President Mr. Jens Oddershede	Questions from the Panel
10.50 a.m.	Meeting with representatives from	Experiencing the accreditation

 

Page 38 of 39

 

Panel activity	Theme
-12.20 a.m.	programmes that have undergone accreditation:Mr. Henrik Vejre, University of CopenhagenMs. Dorthe Kronborg; Copenhagen Business SchoolMs. Birte Hornemann, University of AalborgMs. Eva Sidelmann Karring, University of Aarhus	process. Questions from the Panel
12.20 p.m. – 1.30 p.m.	Lunch – Panel meeting
1.30 p.m. –2.30 p.m.	Meeting with representatives from ACE Denmark’s contacts at universities – Quality Assurance ManagersMs. Hanne Harmsen, University of CopenhagenMs. Geeske de Witte-Westergard, University of RoskildeMr. Morten Vestergaard-Lund, University of Southern Denmark Ms. Marianne Lucht, University of AalborgMs. Iben Westergaard Rasmussen, University of Aarhus	ACE Denmark’s processes, procedures, criteria, transparency, cooperation. Questions from the Panel
2.40 p.m.– 3.10 p.m.	Meeting with Chairman of the Dialogue Forum :Chairman Mr. Mogens Hørder	Discussion of the Dialogue Forum content and experiences. Questions from the Panel
3.10 p.m. –4.30 p.m.	Panel meeting and coffee.
4.30 p.m. – 5.00 p.m.	Meeting with the Council Chairmanship:Chairman Mr. Søren Barlebo RasmussenVice Chairman Mr. Christian Thune	Additional, final questions from the Panel for the Council Chairmanship
5.10 p.m.– 5.40 p.m.	Meeting with ACE Denmark’s management:Acting Director Mr. Sami Stephan Boutaiba	Additional, final questions from the Panel for ACE Denmark’s management.

 

Page 39 of 39

 

Panel activity	Theme
Head of the Professional Secretariat Mr. Steffen Westergård Andersen