Definition of Learning Outcomes by University Teachers in the Context of the National Qualifications Framework of the Slovak Republic

In order to increase the quality of university education we consider it important to deal with the current trends of the quality system of education, which is a very demanding process conditioned by many factors. One of the tasks of Slovak universities within the process of a complex accreditation in 2014 was to develop learning outcomes in information sheets of study subjects in accordance with the National Qualifications Framework of the Slovak Republic (hereinafter NQF SR), which follows the European Qualifications Framework for Lifelong Learning (hereinafter EQF). There is a minimum of knowledge in the conditions of Slovakia to be used for developing and quality processing of results of education on the level of the study subjects (via information sheets)1. The aim of the study is to assess the quality of processing of the learning outcomes in the information sheets of the study subjects in relation to the formulated knowledge, skills and competences in the NQF SR. Through the content analysis we want to find out whether there is compatibility between the formulated learning outcomes on the level of the information sheets of the study subjects and capabilities formulated in NQF SR.


Introduction
In order to increase the quality of university education we consider it important to deal with the current trends of the quality system of education, which is a very demanding process conditioned by many factors. One of the tasks of Slovak universities within the process of a complex accreditation in 2014 was to develop learning outcomes in information sheets of study subjects in accordance with the National Qualifications Framework of the Slovak Republic (hereinafter NQF SR), which follows the European Qualifications Framework for Lifelong Learning (hereinafter EQF). There is a minimum of knowledge in the conditions of Slovakia to be used for developing and quality processing of results of education on the level of the study subjects (via information sheets) 1 . The aim of the study is to assess the quality of processing of the learning outcomes in the information sheets of the study subjects in relation to the formulated knowledge, skills and competences in the NQF SR. Through the content analysis we want to find out whether there is compatibility between the formulated learning outcomes on the level of the information sheets of the study subjects and capabilities formulated in NQF SR.

Theoretical starting points
Current trends in the system of higher education are focused on achieving the European dimension of university pregradual training. The whole process of changes begins with the Sorbonne Declaration on Harmonisation of the European Higher Education System (1998) 2 . M. Vetráková states that achieving a comparable level of education in regional, national and European areas result in a need to adopt generally applicable standards of higher education 3 . Those were adopted by the Joint Quality Initiative commission of experts (the Joint Quality Initiative, 2001) in Dublin (2002) 4 . The characteristics of successful graduates of different levels of education were specified in the following aspects (also called Dublin Descriptors): knowledge and understanding achieved, application of knowledge and understanding, ability to formulate conclusions (judgment), the ability to communicate knowledge and understanding, ability of further education. Based on R. Matlovič "the Dublin Descriptors became one of the theoretical starting points of implementation of a new paradigm in the area of higher education" 5 . The one is characterized by the approach of focusing on the student and the results of higher education.
Learning outcomes are connected to descriptors of different levels of national qualifications frameworks and the European Qualifications Framework 6 . They are formulated in the information sheets of subjects (hereinafter ISS), a sample of which can be found in the Decree of the Ministry of Education on Credited System of Study 7 . The information sheet demonstrates generally binding document available to both teachers and learners and represents the official curricular document, which should reflect an educational standard. Průcha, Turek, Bajtoš agree on the idea that educational standards are given standards, which formulate the acquired degree or level of educational standards or their results 8 . It is a set of requirements for students, which should be met in particular years. We agree with M. Verešová and M. Čerešník stating that the ISS is missing a definition of a performance standard, which is a necessity on the lower levels of education 9 .
Learning outcomes determine what a learner knows, understands and is able to do after completing a study process 10 . This concerns a broad range of learning outcomes in the plane of knowledge (theoretical and application), skills and metaskills, competences and metacompetences. In the process of implementation of competencies into learning outcomes it is necessary to find a compatibility between international and national binding documents. In a particular reflection of competencies into the learning outcomes on the level of the information sheets of the study subjects in Slovakia, the most frequent method used is the revised Bloom's Taxonomy 11 . From the methodological point of view it is possible to use the knowledge of several conceptions focused on implementation of learning outcomes into higher education presented by Vetráková 12 : -KSC typology (knowledge, skills, competences), -tuning methodology, -Biggs' theory in relation to learning outcomes -SOLO taxonomy.

The objective of the empirical study
In 2014 Slovak high schools and universities had to undergo the difficult process of comprehensive accreditation. In order to improve the quality of education based on the application of the principle of focusing on the student we persumed the developing of the educational standards in particular ISS, which are in accordance with the results of education according to the EQF 13 and followed by NQF SR 14 . NQF SR has defined learning outcomes in the plane of knowledge, skills and competences, which should be compatible with the learning outcomes set in each ISS. Based on the above we have set the following scientific problem: What is the level of formulation of learning outcomes in the ISS in accordance with the NQF SR?
The basic bjective of the research is to analyze and evaluate the level of processing the learning outcomes in the ISS focused on the preparation of pedagogical-psychological competences of students of teacher courses. Our goal was to find out: -the overall level of processing the learning outcomes in the ISS in the context of the NQF SR, -the adequacy of formulation of knowledge (theoretical and application) in accordance with the NQF SR, -the adequacy of formulation of the skills and metaskills in accordance with the NQF SR, -the adequacy of formulation of competences and matacompetences in accordance with the NQF SR.

Methodology
The basic method of research was a content analysis of intentionally selected ISS of the teacher study programmes of the Faculty of Humanities and Natural Sciences, Faculty of Arts and Faculty of Sport of Prešov University in Prešov. The criterion of targeted selection were the subjects focused principally on the pedagogical-psychological competences of the teacher (N = 118). Structure of the ISS is presented in Table 1. Working on the method of content analysis we followed the knowledge of F.N. Kerlinger, Š. Švec, M. Černotová, P. Gavora, R. Seebauer 15 . The stages of the ISS content analysis of teachers study programs are: -an intentional choice of the ISS of social-scientific, pedagogical-psychological core and the subjects didactics, -an initial contents analysis of five ISSs for the purpose of selection of analytical categories -the exact definition of categories, helping examples, encoding rules 16 , -a content analysis of the ISS in relation to the capabilities of the individual dimensions of the designed professional standard of a novice teacher; along with contents analysis checked by an independent researcher, -a repeated content analysis of the ISS, -a random selection of five ISSs analyzed and checking the accuracy of the analysis by another independent expert.
The basic unit of analysis was paragraph. Overall, there were 126 categories analyzed and divided into 4 parts with the exact qualification procedure. Each field had a partial score using the arbitrary process 17 . In the context of the study we offer the partial results of one part of the research -level of formulation of learning outcomes in accordance with the NQR SR, which consists of 6 analytical categories. Their result is the adequacy of the formulation of the theoretical and application knowledge, skills and metaskills in relation to the content and adequacy of the formulation of competences and metacompetences in relation to learning, and thus cognitive process. The content analysis was performed from April 2016 to June 2016. The type of research is descriptive and the study uses the basic statistical characteristics.

The research results
In this section we present the partial results of research focusing on the level of formulation of learning outcomes in the basic curricular document (the information sheet of the subject -the ISS) in accordance with the NQF SR.

Assessment of the level of formulation of learning outcomes in accordance with the NQF
The analysis of applications of learning outcomes into the ISS focused on pedagogicalpsychological competences of the student points to the results presented in Table 2. The results of the content analysis (Table 2) point to the serious shortcomings of university teachers in the formulation of learning outcomes in accordance with the NQF SR. Level A weas achieved by learning outcomes in 7 ISSs (5.93%); level B was achieved by learning outcomes in accordance with NQF SR in 6 ISSs (5.08%), level C was achieved by learning outcomes in 4 ISSs (3.39%); level D -8 ISSs (6.78%); level E -18 ISSs (15.25%). The lowest score (0 points) was achieved in formulated learning outcomes of 8 ISSs, especially subjects of the state exam, where the performance standard was absent. We believe that learning outcomes in the ISS of the state exam reflect the competences which the student obtained during the study. Also the ISS of state examinations should contain the necessary particulars so that students know what they are expected, what kind of knowledge, skills, competencies graduates should have after finishing the study.
The assessment of the level of learning outcomes in the ISS consisted of six analytical categories: the results of education in accordance with the NQF SR on the level of theoretical knowledge, application knowledge, skills, metaskills, competences in relation to the content of the subject and the metacompetences in relation to educational process, where the subject is taught. In the following section we present the partial results of the research. NQF SR has cross-cutting knowledge of the specified branch and focuses on application usage on the level which corresponds with the current state of cognition 18 . It is essential to formulate such results of learning in each ISS so that graduates from a certain educational level may obtain broad knowledge and understanding in a specialised field, including the knowledge of practical context and relations to the related branches. The results of the content analysis of the ISS in relation to the formulation of learning outcomes in the theoretical and application plane are presented in Table 3 and Table 4.  Table 3 shows the formulation of the learning outcomes in the ISS aimed at theoretical knowledge. We find out that 91 ISSs (77.12) have formulated the learning outcomes focused on the knowledge and 27 ISSs (22,88%) have not. Table 4. shows the formulation of the learning results in the ISS in the plane of application knowledge. We have identified 86 ISSs (72.88%) with the learning outcomes formulated and focused on the application knowledge and 32 ISSs (27.12%) without learning outcomes formulated. One of the parts of the content analysis of the ISS was to evaluate the adequacy of the formulation of learning outcomes focused on knowledge. The result is presented in Table 5.  Table 5 illustrates that overall there was adequately formulated knowledge in 58 ISSs (49.15%), partially correctly formulated results were identified in 28 ISSs (23.73%) and incorrectly formulated learning outcomes focused on knowledge were identified in 32 ISSs (27.12%). We can state that the formulation of learning outcomes in the plane of the cognitive processes of the revised Bloom's Taxonomy 19 (in the ISS is predominantly focused on the lower cognitive processes (approximately 72.64%). The finding is illustrated in Graph 1.
19 D.R. Kratwohl, Revision… the formulation of learning outcomes focused on knowledge. The result is presented in Table   5. The skills formulated in the NQF SR require a graduate of bachelor degree to obtain information in active way and to use it for solving practical tasks in the given field, to be able to solve practical tasks within the branch using the usual research and development processes, along with a critical assessment of their appropriateness and adequacy 20 . The results of the education in the analysed ISS focused on this area is shown in Table 6. In the framework of the content analysis of the ISS, in 26 ISSs (22.03%) we hardly spotted adequately formulated results of the training focused on skills in accordance with the NQF SR, partial correctness of the formulation of skills was identified in 72 ISSs (61.02%) and 20 ISSs (16,95%) did not have sufficiently formulated learning outcomes for this component.
Beside knowledge and skills there are also competences considered a part of the formulation of learning outcomes. Within them and besides capabilities related more-less to the content of the study field, the competences connected with the metakognition, or the so-called matacompetences are also highlighted. Their base consists of metaskills, so the skills associated with learning (cognitive) process in the plane of its planning, implementation, and monitoring and evaluation. More specifically, it is e.g. the skill to reconsider ways of collecting information, to plan own learning (cognitive) process,to critically assess the procedures of the various solutions, tasks,etc. Adequacy of formulation of learning outcomes in the ISS aimed at metaskills are illustrated in Table 7. Content analysis of the ISS focused on the learning outcomes in relation to metaskills showed that in 11 ISSs (9.32%) learning outcomes for the analyzed component were for-mulated adequately; in 14 ISSs (11.87%) partially correctly and in 93 ISSs (78,81%) there was inadequate formulation of metaskills or the absence of their formulation.
Another part of the content analysis of the ISS was to evaluate the formulation of learning outcomes with a focus on competences in relation to the content of the study subject. A ompetence is understood as a comprehensive characteristic, which includes the knowledge, skills, but also values, attitudes, etc. In the context of our study we differentiated and analyzed the competences related to the content of the subject, which includes the results of education formulated and competences connected with the metacognition, the so-called metacompetences that are connected with the cognitive and learning process. Such a distinction is only relative, in the interest of a deeper analysis we, however, considered it necessary. The adequacy of the formulation of learning outcomes focused on competencies related to the content of the study subject are expresses in Table 8. We identified the adequacy of the formulation of learning outcomes focused on competences in 32 ISSs (27.12 %), partially correct formulation of the learning outcomes in 72 ISSs (61.01 %) and inadequacy of formulation of learning outcomes with a focus on competences in relation to the content of the study subject in 14 ISSs (11.87%). NQF SR also formulates the competencies that significantly express capability of the graduate to solve specialist tasks and coordinate sub-activities, to take responsibility for the results of the team, can identify and assess the ethical, social and other contet of the problems being solved, can obtain fresh knowledge and actively expand their own knowledge, can present the results of their own work or team work, is able to defend them, argue, etc. 21 . In the study we presented them as metacompetences. Evaluation of the adequacy of the ISS learning outcomes aimed at metacompetences is shown in Table 9. Content analysis ISS focused on learning outcomes in relation to metacompetences showed that 11 ISSs (9.32%) contained adequately formulated learning outcomes towards analyzed components; in 15 ISSs (11.87%) the learning outcomes were identified as partially correct and 92 ISSs (77.97%) contained inadequate formulation of learning outcomes aimed at metacompetence or there was absence of their formulation.

Discussion and conclusions
Learning outcomes according to EQF also indicated in the ECTS Users Guide are an overview of what the student is supposed to know, understand and demonstrate after completing the learning process 22 . Vetráková terms them verifiable statements of what students are supposed to know and are able to perform after completing their study 23 . The author notes that the learning outcomes reflect a link between teaching, learning and evaluation which is expressed by evaluation criteria.
The link should should also exist between the learning outcomes and capabilitie of novice teachers, which are formulated in the draft of professional standards of novicebe teachers. Such an application of learning outcomes specifies the objectives of study subjects, study programmes and practice requirements. The internal quality system of higher education institution in the conditions of Slovakia ( §87a of the Higher Education Act) is based on standards and guidelines for quality assurance in accordance with ESG descriptors 24 . The reported trends should be taken into account when formulating educational standards in the ISS. During the implementation of learning outcomes on the level of study subjects (via ISS) the most frequent methodology used was determining the results according to Bloom's Taxonomy and its revised version 25 .
The results of content analysis ISS aimed at educational-psychological capabilities of teacher study programmes of the Faculty of Humanities and Natural Sciences, the Faculty of Arts and the Faculty of Sport of University of Prešov highlight the need to redefine the content of performance standards (as well as content standard). The overall level of formulating the learning outcomes in the ISS (N = 118) of study subjects (compulsory, compulsory obligatory and elective study subjects of social-scientific and pedagogicalpsychological cores) within the context of NQF SR achieved an average 3.35 (reflecting qualification level Fx). The results of content analysis point to serious shortcomings of university teachers to formulate learning outcomes in accordance with the NQF SR (up to 63.57% ISSs reached Fx level). As part of the partial analysis we came to the following findings: -adequatly formulated learning outcomes focused on knowledge in about 49% ISSs, -the dominance of formulating the learning outcomes for lower cognitive processes in 72.64% ISSs, -adequately formulated learning outcomes focused on skills in about 26 ISSs (22.03%), -adequately formulated learning outcomes fosused on metaskills in 11 ISSs (9.32%), -adequately formulated learning outcomes focused on competences in relation to the subject content in 32 ISSs (27.12%), -adequately formulated learning outcomes focused on metacompetences in 11 ISSs (9.32%).
Based on the results of the search we would like to state that university teachers are not adequately prepared to transfer the knowledge, skills and competences into learning outcomes in accordance with the NQF SR. The overall level of evaluation of the ISS was influenced by the adequacy of knowledge, skills and competences, where we can spot a great amount of shortcomings. Guidelines of the European Parliament 26 emphasize the importance of capability of development, especially in terms of teamwork, analyses of specific situations, etc. In the ISS analyzed the mentioned dimension is quite underestimated. Although the inspirational approach how to formulate learning outcomes and their implementation into the designing study subjects can be found in the publication of M. Verešová and M. Čerešník, while during the planning of learning outcomes they use the Biggs and Tang's model 27 . NQF SR assumes that a bachelor's degree graduate will have more extensive capabilities. While the standards for the lower levels of education are given by the ministry, university standards are absent, and this is the problem that M. Verešová and M. Čerešník warn about 28 . If we want to reach the compatibility between theory and practice on the international level, it is necessary to ensure that university teachers (creators of the curriculum) respect international and national guidelines providing such a quality (Dublin descriptors, EQF, NQF, National System of Occupations, ECTS users manual, standards and regulations providing quality according to ESG, etc.). Demonstrating the competencies on the level of ISS it is necessary to respect and assure that the study subjects have clearly defined performance standards in relation to the skills necessary for novice teachers. On the other hand, in the application of learning outcomes we suggest using the knowledge of several conceptions recommended by Vetráková -typology of KSC (knowledge, skills, competences) methodology for improving performance (Tuning Methodology), SOLO taxonomy, revised Bloom´s taxonomy, etc. 29 Based on the above information we can suggests paying propper attention to the development of standards, which significantly contribute to the European dimension of undergraduates training.
The results of content analysis of the ISS in relation to knowledge, skills and competences formulated in NQF SR also point to broader context. 1. The field of recommendations focuses on the capability of university teachers to process educational standards in the ISS of study subjects. We note that during the preparation of accreditation files (comprehensive accreditation in 2014) the academics were poorly informed about how to proceed with processing the learning outcomes (content and performance standards). It should be obvious that the university teacher in the context of trends in the quality of education should adopt basic knowledge on the European and national scale. We can see a great lack of particular procedures related to the complex accreditation of higher education institutions in Slovakia. Particular steps and impact of comprehensive accreditation on the quality of education have not been systematically considered. First of all it was necessary to train specialists who would be able to prepare university teachers, e.g. in how to transfer learning outcomes according to the EQF and subsequently NQF SR 30 , so that the teacher and student are clear about what the learner knows, understands and is able to perform at the end of training. The definition of the learning outcomes, knowledge, skills, attitudes declare the need to focus more attention on training of university teachers in terms of capability to process educational standard in the ISS on the level of quality corresponding with the international trend. 2. The field of recommendations is focused on the ability of university teachers to link undergraduate training of student teachers in accordance with the requirements given by NQF. The content ISS analysis showed incompatibility between the results formulated in the ISS and the knowledge, skills and competencies that are expected from graduates of the bachelor´s degree according to NQF. We recommend paying proper 29 M. Vetráková, Systém kvality… 30 Národný kvalifikačný… attention to the discussion on the application of learning outcomes in accordance with requirenets given by the Bologna Process of Europeanisation. 3. The field of recommendations leads to the need to strengthen the study subject of University Pedagogy. Since one of the essential topics should also be the implementation of specific objectives into a specific lesson of the approbation subject, we expected better learning outcomes formulated in the ILP. Insufficient ability of teachers to formulate educational outcomes reflects the absence of university pedagogy. Its necessity is emphasized by several authors, e.g. Kosová; Knapík; Černotová and Ištvan; Ferencová and Šuťáková; Krajčová and Ferencová; Rovňanová and others 31 . The study subject of Basics of University Pedagogy was introduced at the Prešov University for doctoral students in the academic year 2015/2016, based on the needs of improving pedagogical activities, which is part of their job description. According to legislative documents scientific-pedagogical staff do not have the proper pedagogical qualifications so that they could perform pedagogical work. Šuťáková and Ferencová state that these employees are aware of their shortcomings in managing, organizing and performing the educational process 32 . On that basis, we recommend teachers take the course in university pedagogy as it is one of the ways how to improve the quality of education at university.
The article is a part of the VEGA project no.