Report on an Internal Evaluation of the Accreditation Commission for the Year 2007

Last Updated on Saturday, 30 November 2013 17:37
Written by Dana Mosazná

Internal Evaluation Report of the Accreditation Commission, 2007


 

Report on an by

Milan Sojka, Pavel Höschl, Jiří Sobota

 

November 2007

This report was discussed at a meeting of the Accreditation Commission 
on 20 and 21 November 2007

 

Introduction

The Accreditation Commission of the Czech Republic (hereafter referred to as “AC”) is an institution systematically concerned with the evaluation of the quality of institutions of higher education and their degree programs. Through its activities, the AC creates conditions for quality assurance and improvement in tertiary institutions’ primary activities. Its tools in these areas are institutional evaluations, evaluations of accredited activities and accreditation processes. While evaluation is understood as the primary way in which to improve the quality of Czech institutions of higher education, the process of accreditation primarily fulfils a disciplinary function because it is based on minimum standards and requirements and attempts to separate the quality from the low quality. The AC does not recommend accreditation for poor quality degree programs and non-accredited programs cannot be opened, students may not be admitted to them, teaching may not be conducted, exams carried out or degrees granted and these programs may not be financed using public (government) funds.

The AC is concerned with the quality of tertiary education and evaluates in a comprehensive manner the educational, scholarly, research, artistic and other academic activities of institutions of higher education. In addition to these responsibilities, which are set forth in Law No. 111/1998 Coll., On Institutions of Higher Education as Amended (The Higher Education Act) and in the Statute of the AC as approved by the Government of the Czech Republic in its Resolution No. 744 of 28 July 2004, the AC carries out additional activities in accordance with its obligations arising out of the AC’s membership in ENQA (European Network of Quality Assurance) and the Czech Republic’s inclusion in the European Higher Education Area. In 2005, cooperation between ENQA, EUA, EURASHE and ESIB led to the preparation of the document Standards and Guidelines for Quality Assurance in the European Higher Education Area, which was approved at meeting of government ministers responsible for higher education held in Bergen, Norway that year. These standards and guidelines and conditions along with ENQA membership obligations require the creation of an internal system of quality evaluation for those institutions responsible for quality assurance in higher education and the introduction of external evaluations on either a national or international level.

It is completely proper and legitimate to ask those institutions responsible for quality assurance in higher education to demonstrate through a quality assurance evaluation that their activities are of sufficient quality, that within their framework they cultivate a culture of quality, which in accordance with the law they demand of evaluated institutions of higher education. It is in this context that the AC must strive for systematic improvements in the quality of its activities.

The joint project of the AC of the Czech Republic and the AC of Slovakia to create a system of internal evaluation and prepare for external evaluations came about as the result of long-term good relations and cooperation, common goals and similar problematic areas and of course the advantages of the linguistic relatedness of Czech and Slovak. The goals of this project include cooperation in the creation of a system of internal evaluation for both AC’s and the creation of the necessary foundations for external evaluation as well as further cooperation in the development of national systems of quality assurance in higher education. These activities should eventually lead to the mutual recognition of both accreditations and evaluations.

During the first stage of the project, a proposal for an internal system of evaluation for both AC’s was developed based on the requirements set out in the Standards and Guidelines for Quality Assurance in the European Higher Education Area. In the project’s second stage, a proposal for the realization of external evaluation in cooperation between the Czech Republic and the AC of Slovakia was developed. In this context, all of the documents and other materials created by the AC must demonstrate that:

The system of internal evaluation should be transparent and credible and should result in increasing the qualities of the AC’s activities; at the same it should to an adequate degree meet the Standards and Guidelines for Quality Assurance in the European Higher Education Area so as to meet the requirements for membership in ENQA. Furthermore, this system should create good conditions for the external evaluation of the AC.

First and foremost, this system should serve as a starting point improving the quality of activities carried out by the AC. In this context, it is critical that the report on the internal evaluation be sufficiently analytical and openly address the strengths and weaknesses of the AC’s activities and take the form of recommendations for the resolution of existing problems. The internal evaluation board set up by the AC has used a SWOT analysis to good ends in meeting these goals.

A three-member evaluation board worked on the preparation of this Report on the internal evaluation of the AC. This board’s areas of competence include developing and writing the draft evaluation report, its presentation for discussion at a meeting of the AC and preparing the final Report on the internal evaluation for publication. The board for the internal evaluation was elected as follows: Milan Sojka, Jiří Sobota and Pavel Höschl. The basis for the structure of the report was developed in the first stage of the joint project of the AC of the Czech Republic and the AC of Slovakia. Beginning with 2007, an internal evaluation of the AC will be carried out annually. A part of this internal evaluation is a feedback mechanism that takes the form of a questionnaire, whose results were used in the writing of the report and form an appendix to it.

In its activities, the AC makes every possible effort to apply the Standards and Guidelines for Quality Assurance in the European Higher Education Area and examples of good practice by foreign agencies involved in quality assurance in higher education. At the same time, the AC is able to take advantage of is many years of experience with accreditations and evaluations of quality in Czech higher education and its long-term efforts to improve its activities. There do remain areas where the practices of the AC must be improved because internationally recognized standards are not fully adhered to. This is particularly the case with the active participation of students in the activities of the AC (students are not directly represented in the AC and are only represented in some of the standing Working Groups). The majority of the working groups are made up entirely or predominately of academics and the viewpoints of employers is lacking here. Nor are these viewpoints sufficiently represented in the AC itself.

 

Evaluation of the AC’s Structures

The overall makeup of the AC in terms of the representation of scholarly (educational) disciplines is generally satisfactory. This was the overwhelming opinion of the members of the AC when questioned on this issue. There are concerns about the effectiveness of the AC in light of attempts to increase the number of members on the AC. Some members of the standing Working Groups (hereafter referred to as “WG”) proposed a number of improvements (creating a separate WG for the environmental sciences, including experts from specific fields within a number of the standing WG’s).

From the point of view of the international comparison of approaches to evaluation and criteria, it is very significant that the AC has members from abroad. At the present time, however, foreign members are only from Germany, which somewhat limits international comparability. It is necessary to add representatives from Slovakia, Scandinavia and/or the United Kingdom, or even from Mediterranean countries (a limiting factor here is however language). Academic staff from institutions and the Academyof Sciences of the Czech Republic dominate the AC and its working groups. No students are represented in the AC and the number of experts representing the business community and employers is very low.

In particular it is necessary to increase the number of students and outside experts in the standing and special WG’s. Cooperation with the Student Chamber of the Council of Higher Education Institutions of the Czech Republic is gradually resulting in an increasing number of students in the WG’s. It is also necessary to increase the number of experts working in the field in order to better reflect the view of employers, which is of particular significance in technical and economic disciplines. It is worth noting that many members of the AC and its standing WG’s consider the current makeup of these bodies to be effective; many members of the standing WG’s indicated in their questionnaires opposition to the inclusion of students and some even to representatives from business.

 

Evaluation of Structure at the Level of the AC Secretariat

The AC Secretariat is understaffed and lacks sufficient financial and material support. Given the increasing demand on the development of evaluation materials, growing demands on foreign language skills in the context of cooperation with international bodies (ENQA, CEEN, ECA, etc.) and the growing role of the AC’s international activities, its is necessary to increase financial support and staffing levels for the Secretariat as well as modernizing its technical support. The majority of the AC’s members evaluated the work of the Secretariat very positively in terms of ensuring evaluation and accreditation.

 

Evaluation of the Effectiveness of the AC’s Activities

In its activities the AC makes every effort to apply internationally recognized standards and procedures, to achieve the greatest degree of transparency in these procedures and to adhere to the explicitly formulated evaluation criteria. In accordance with the legally established accreditation criteria, all degree programs and their individual scholarly disciplines and fields for naming docents and professors are all subject to fixed-period accreditation and periodic extensions of that accreditation. This results in extensive demands on resources and time for both the AC and the institutions of higher education.

It is not possible to move from a system of accrediting degree programs to the accreditation of institutions (which is the practice for most member agencies in the European Consortium for Accreditation) when functioning internal systems of quality assurance can be found in only a small number of public and private institutions of higher education. It would however be appropriate to move to the accreditation of institutions for those institutions of higher education that can conclusively demonstrate internal systems of quality assurance that function well. (A necessary precondition for such a shift would have to be the realization of significant legislative changes together with the functioning internal quality assurance systems.)

Since 2002, when the process of accrediting all degree programs included the in Higher Education Act of 1998 had been completed, the AC has been placing a significant emphasis on evaluating institutions of higher education and their constituent faculties and on evaluating accredited activities. Evaluations are gradually becoming the basic tool for increasing quality and creating a culture of quality assurance at Czech tertiary institutions. It would be appropriate from the motivational point of view to introduce the concept of “Center of Excellence” to the evaluation results.

The activities of the AC have thus far been focused primarily on issuing rulings on applications for accreditation and on evaluating institutions of higher education. As a result of the large demands flowing from the agenda relating to granting new accreditations and extending the validity existing accreditations, very little time remains for strategic concerns and discussions of conceptual materials.

Applications for new accreditations and extending the validity of existing accreditations are often submitted without the proper structure or necessary information for evaluating the application. The requirement of submitting each application in triplicate leads to increased costs and increases the difficulty of the entire process. The issuance of newly planned official guidelines on applications for accrediting degree programs should result in a simplification of this process and increased effectiveness. For example, applications will be submitted in a single copy rather than triplicate. Applications should primarily be submitted to the WG’s for evaluation in an electronic format with clearly defined requirements for specific information. Many members of the AC and the standing WG’s expressed support for this change in the questionnaire.

 

Evaluation of Communication

The major issue in this area is the AC’s communication with the general public. It is necessary to find appropriate avenues of communication that will best inform the public about the activities of the AC and its WG’s. One such method is to make better use of the AC’s web pages. A related key issue is to improve the quality of these web pages’ English-language version. Communications with the Department of Higher Education at the Ministry of Education, Youth and Sports are on a very high level and there has been a significant improvement in communication with the Czech Rectors Conference and the Council of Higher Education Institutions of the Czech Republic. There are occasional problems in communications with some tertiary institutions, the causes of which rest on a lack of information (uncertainties about the AC’s mission, and the criteria and procedures it uses in evaluating institutions of higher education on the one hand, and delays and inflexibility in the updating of the AC’s web pages and confusing information on the other).

The questionnaires also revealed certain problems in communication between the AC and the standing WG’s.

 

Evaluation of Process Transparency

In all of their activities, the AC and its standing and special WG’s attempt to maintain the highest levels transparency in both the criteria and procedures it uses. Criteria are made public on the AC’s web pages. These are applied by the WG’s with consideration being given for the situation in specific fields – primarily setting levels of requirements that will correspond with the nature of the discipline. These criteria are held to with only minor exceptions in the activities of both the AC and the standing WG’s. The major issue appears to be that at the level of the standing WG’s, these criteria are not always interpreted in the same way. The AC is addressing this issue and is attempting to ensure that the established criteria are not observed in a merely formal manner. It is also necessary to deal more deeply with the issue of interdisciplinary comparison and consistency in the evaluation of differing fields (the sciences, engineering, the social sciences and humanities and the fine arts).

Legally-mandated deadlines are being observed by the AC. In some cases, the arguments for rejecting an application for accreditation are not sufficiently convincing, which occasionally leads to misunderstandings and incorrect interpretations on the part of the applicants.

 

AC Independence

The functional and procedural independence of the AC is guaranteed by the Higher Education Act of 1998 and the statute of the AC as approved by the Government of the Czech Republic. This independence is strictly maintained in all the activities of the AC and its standing and special WG’s. The moral integrity of the members of the AC and members of the WG’s has thus far meant being able to resist pressures from lobby groups.

A majority of the members of the AC and the standing WG’s consider the adoption of an explicit “Code of Ethics” to be unnecessary. Adoption of such a document, however, could have a positive impact on the public and might lead to the reduction of lobby group pressure. Some respondents consider such a code as appropriate for new members of the standing WG’s, which membership changes more frequently.

 

AC Control Mechanisms

The control mechanisms set out by the Higher Education Act, the Board of Directors and the AC statue work very well; the right of appeal has been established.  These mechanisms have not yet been institutionally implemented to a sufficient degree. It is desirable to include within them formal procedures to be used when rules and regulations or the criteria of the accreditation process have been violated. This should be a separate area of responsibility within the AC Secretariat.

 

Evaluation of International Cooperation

The AC is a member of ENQA, CEEN and ENQAHE. Cooperation with these associations is important for the exchange of experience and good practices. Cooperation with ENQA is of critical importance in improving the professionalism of the quality assurance system (seminars, annual meetings, publications, participating in projects, etc.).

Cooperation with the AC of Slovakia is successfully developing and one of its results is the joint project for the internal and external evaluation of the AC’s and cooperation with selected members of CEEN.

 

Evaluation of Evaluations and Accreditations

In its activities, the AC is placing ever increasing emphasis on the evaluation of institutions of higher education and their accredited activities; this is in spite of the fact that the number of applications for new accreditation and extending the validity of accreditation means that the members of the AC and most of the working WG’s are overburdened. Evaluations and accreditations almost always meet expected quality standards and correspond to accepted criteria. In spite of the fact that starting point for the process of accreditation is the application of minimal standards, this process is fundamentally based on evaluation.

Because of the demands place by legislation on the accreditation process, the AC has not yet been able to create sufficient space for the discussion of conceptual issues. In the future, the AC should systematically focus on an analysis of the impact of the Bologna Process on the quality of Bachelor’s and Master’s degree programs, the possibilities of making the processes of evaluation and accreditation more effective, increasing motivation to improve quality and searching for a closer link between evaluations and the financing of public institutions of higher education.

 

Existing Problems and Recommendations

  1. The AC should attempt to make its activities more effective and create space for discussion of conceptual issues.
    • The starting point for a more effective system in the future should be a shift from accrediting degree programs to the accreditation of institutions (tertiary institutions and/or their constituent faculties). A necessary precondition is creating functional internal systems of assessing quality assurance at tertiary institutions.
    • Applications for accreditation should be submitted and processed primarily in electronic form
    • Further discussions about the criteria used in particular for Master’s and doctoral degree programs and disciplines for naming Docents and Professors.
  2. It is necessary to improve communication between the AC and the standing WG’s. In this context, it is necessary to make better use of the AC’s web pages and transmitting information via FTP.
  3. It is necessary to focus on improving the level of information that tertiary institutions and the general public have about the AC’s activities.
  4. The AC should discuss questions relating to the inclusion of students and outside experts in the activities of the standing and special WG’s.
  5. It is necessary to request that the Ministry of Education, Youth and Sports increase the budget of the AC in view of its new responsibilities arising from the participation of the Czech Republic in the Bologna Process and the need to better provide for the AC Secretariat (in terms of both personnel and material).

 

Internal Evaluation of the Accreditation Commission of the Czech Republic

Přílohy:
Download this file (CZ_vnitrni_hodnoceni_AK_2007.pdf)CZ - Vnitrni hodnoceni AK 2007[ ]156 kB
Download this file (EN_Internal_ Evaluation_Report_AC_2007.pdf)EN - Internal Evaluation Report AC 2007[ ]100 kB