This publication is a Science for Policy report by the Joint Research Centre, the European Commission’s in-house science service. It aims to provide evidence-based scientific support to the European policy-making process. The scientific output expressed does not imply a policy position of the European Commission. Neither the European Commission nor any person acting on behalf of the Commission is responsible for the use which might be made of this publication.

JRC Science Hub

https://ec.europa.eu/jrc

Abstract

How are Higher Education Institutions Dealing with Openness? A Survey of Practices, Beliefs and Strategies in Five European Countries

Open Education is on the agenda of half of the surveyed Higher Education Institutions (HEIs) in France, Germany, Poland, Spain and the United Kingdom. For the other half of HEIs, Open Education does not seem to be an issue, at least at the time of the data collection of the survey (spring 2015). This report presents results of a representative a survey of Higher Education institutions in five European countries (France, Germany, Poland, Spain and the United Kingdom) to enquire about their Open Education (OE) practices, beliefs and strategies (e.g MOOCs). It aims to provide evidence for the further development of OE to support the supports the Opening Up Communication (European Commission, 2013) and the renewed priority on Open Education, enabled by digital technologies, of ET2020

JRC Mission

As the Commission’s in-house science service, the Joint Research Centre’s mission is to provide EU policies with independent, evidence-based scientific and technical support throughout the whole policy cycle.

Working in close cooperation with policy Directorates-General, the JRC addresses key societal challenges while stimulating innovation through developing new methods, tools and standards, and sharing its know-how with the Member States, the scientific community and international partners.

Serving society Stimulating innovation Supporting legislation

Europe Direct is a service to help you find answers to your questions about the European Union

Free phone number (*): 00 800 6 7 8 9 10 11

(*) Certain mobile telephone operators do not allow access to 00 800 numbers or these calls may be billed.

A great deal of additional information on the European Union is available on the Internet.

It can be accessed through the Europa server http://europa.eu

JRC99959

EUR 27750 EN

PDF

ISBN 978-92-79-55330-1

ISSN 1831-9424

doi:10.2791/709253

LF-NA-27750-EN-N

EPUB

ISBN 978-92-79-63915-9

ISSN 1831-9424

doi:10.2791/720120

LF-NA-27-750-EN-E

© European Union, 2016

Reproduction is authorised provided the source is acknowledged.

How to cite: Castaño Muñoz, J., Punie, Y., Inamorato dos Santos, A., Mitic, M. & Morais, R. (2016): How are Higher Education Institutions Dealing with Openness? A Survey of Practices, Beliefs and Strategies in Five European Countries. Institute for Prospective Technological Studies. JRC Science for Policy Report, EUR 27750 EN; doi:10.2791/709253

All images © European Union 2016, except: Cover image, f/sco – Fotolia.com

How are Higher Education Institutions Dealing with Openness?

A Survey of Practices, Beliefs, and Strategies in Five European Countries

Foreword

“Open and innovative education and training, including by fully embracing the digital era” has been confirmed as one of the six new priorities of the strategic framework for European cooperation in Education and Training (ET2020), adopted at the November 2015 Education Council. Issues related to opening up education were first addressed in the September 2013 Communication on “Opening up Education: Innovative teaching and learning for all through new technologies and Open Educational Resources”.

JRC IPTS is conducting a project on behalf of DG EAC to provide evidence to support policies related to Open Education and, at the same time, provide guidance to Higher Education institutions to open up their educational practices: OpenEdu. Besides in-house research, OpenEdu is running 5 studies in collaboration with external partners:

• Moocknowledge: a survey on MOOC learners (ongoing);

• OpenCred: desk research and case studies on recognition of non-formal learning via MOOCs (final report to be published early 2016);

• OpenSurvey: a representative survey of Higher Education institutions in 5 European countries to enquire about their openness strategies (this report);

• OpenCases: case studies on openness in Higher Education (final report to be published early 2016);

• BMOpen: case studies on business models for Open Education (ongoing).

This report presents the results of the OpenSurvey study, a survey of Open Education practices, beliefs and strategies in Higher Education institutions in five European countries (France, Germany, Poland, Spain and the United Kingdom). This survey was carried out by the ICT for Learning and Skills team of JRC IPTS in collaboration with the Academic Cooperation Association (ACA).

The survey presents two novelties as compared with previous research in the field. First, in order to provide reliable data, Opensurvey was designed to avoid positive selection bias towards institutions more prone to offer Open Education. Secondly, Opensurvey allows accounting for country differences regarding the state of Open Education.

Together with the other OpenEdu studies, OpenSurvey aims to provide essential data to build an evidence-based picture of Open Education in European Member States, and also to show what it would take to push the field forward.

Yves Punie

Project Leader, ICT for Learning and Skills

Acknowledgements

First, we would like to thank to Academic Cooperation Association (ACA) team involved in this study, to those persons from the Association that appears as authors but also to other key collaborators in this study: Bernd Wätcher and Irina Ferencz. Thanks for the valuable work during all the study process. Also to Marek Fuchs from TU Darmstadt, who carried out a highly valuable statistical work.

Secondly, we want thank to all the managers of the Higher Education Institutions in France, Germany, Spain, Poland and the UK who generously responded the questionnaire and provided the data on which this report is based.

Thirdly, we would like to thank DG EAC for supporting this study and in particular to Konstantin D. A. Scheller, Julie Anderson, Deirdre Hodson, Irina Sombre-Nizovtseva and Geir Ottestad for their valuable feedback during the process.

Several people were involved in the test of the questionnaire so thanks also go to all members of the Advisory Group for their valuable input.

Finally, we would also like to thank our colleagues from the JRC-IPTS who provided comments and suggestions to the text, in particular: Panagiotis Kampylis and Riina Vuorikari, but also other members of the team for its valuable input. Thanks also go to Patricia Farrer for proof-reading and editing the final version of this report.

Executive summary

Policy context

The European Commission considers the modernisation of Education and Training systems as a key means of achieving sustainable and inclusive growth in Europe¹. The uptake of ICT-based open and flexible education is expected to contribute to this modernisation by facilitating more efficient and effective ways of acquiring the competences that are needed in the 21st century economy and society (European Commission, 2012). This idea was laid out in the Communication “Opening up Education: Innovative teaching and learning for all through new technologies and Open Educational Resources” (European Commission, 2013) where the adoption of Open Education by formal and non-formal education institutions was identified as an important objective. More recently, “open and innovative education and training, including by fully embracing the digital era” has been confirmed as one of the six new priorities of the strategic framework for European cooperation in Education and Training (ET2020), adopted at the November 2015 Education Council.

As the integration of Open Education into Higher Education systems is a policy objective, it is essential to have evidence on how European Higher Education Institutions (HEIs) are dealing with Open Education (OE) practices and strategies. This report aims to contribute to the evidence-base by exploring the supply side of Open Education in five European Higher Education systems (France, Germany, Poland, Spain and the UK). The findings are based on a representative survey conducted in the period February-June 2015 that was responded by 178 Higher Education Institutions in these five EU countries. The survey aimed to find out if, how and to what extent HEIs in the five countries engage in Open Education, and if not, why not. Consequently, the report directly supports the Opening up Education Communication and the renewed priority given to Open Education, enabled by digital technologies, by the ET2020 strategy.

Open Education is understood in this study as a mode of realising education using digital technologies to provide alternative and less restrictive access routes to formal and non-formal education. This perspective is broad to enable a comprehensive view, thus encompassing for instance Open Educational Resources (OER), Massive Open Online Courses (MOOCs), recognition of open learning, etc.

Key conclusions

In general we find that Open Education is on the agenda of half of the surveyed HEIs in the five countries. This is significant. On the other hand, during the survey period Open Education seems not to have been an issue for the other half of the surveyed HEIs.

In order to promote more widespread and deeper use of OE, it is important to design policies that directly address the difficulties institutions face when dealing with OE. On the basis of the data, steps should be taken to:

• integrate OE into HEIs’ overall strategies.

• increase awareness and understanding of OE.

• change existing practices and mechanisms to facilitate the implementation of OE.

As an overall remark, we identified significant differences between the five countries in terms of engaging with Open Education: this too could hold policy lessons.

Main findings

The survey explored views and practices of HEIs on a broad and wide range of OE elements. Overall, some degree of commitment to and engagement in the different forms of Open Education covered by the survey was present in most of the countries surveyed by the study. The main findings of the survey are presented below:

• Use of Information and Communication Technologies (ICT) in face-to-face educational settings is common: Blended learning is much more widely adopted by HEIs than fully online courses or study programmes. Blended learning is perceived by university managers as the most effective way of delivering education.

• The offer of Massive Open Online Courses (MOOCs) is growing but still not widespread: One fifth of the surveyed HEIs stated that they offer at least one MOOC. In addition, about a quarter of the HEIs that are not offering MOOCs at the moment intend to do so in the future. However, this situation varies among countries, ranging from France, where both, the current MOOCs offer and the intentions to offer MOOCs in the future are high, to others like Germany where both current and planned MOOC offers are low. Poland falls between the two extremes: however, though it currently offers low numbers of MOOCs, it has considerable growth potential reflected in the high percentage of HEIs intending to offer MOOCs in the future.

• Recognition of MOOC learning is rare: In all 5 countries studied, HEIs usually lack recognition mechanisms; even in cases where MOOC certificates are based on reliable ways of assessment and linked to a specific number of ECTS. This indicates there is little awareness and/or trust in providing recognition of learning through MOOCs.

• Open Educational Resources (OER) are widely used: More than 50% of HEIs support the use of OER within their institution. In contrast, only just over one third of HEIs support the development (and offer) of OER. Most of those HEIs that use OERs do so to supplement classical face-to-face instruction and do not substitute core learning materials for OER.

• Collaboration occurs within national borders: Cross-border collaboration among institutions is less frequent than local or national collaboration. Cross-border collaboration in MOOCs is even less frequent than in other areas. In national collaboration, countries differ. At national level, French HEIs collaborate the most and Polish ones the least. Cross-border collaboration, however, is rare among HEIs of all five countries.

• HEIs have different motivations for engaging in Open Education: Promotion and visibility of the institution as well as reaching more students are the strongest drivers for HEIs to engage with Open Education. Enhancing the quality of education is also an important motivation. Institutions are less convinced about the financial benefits as a major driver for engaging in Open Education. HEIs offering OER give more importance to institutional strategies which emphasise “free access to education”. Institutions which offer MOOCs see this social aim as less important.

• Skills and recognition are the most important barriers: The main reasons for HEIs not to engage in OE practices are that academic staff is not skilled to use Open Education and also the difficulties associated with formal recognition of Open Education. Pedagogical issues are reported as less important challenges.

• Lecturers get support to engage in Open Education but rarely in terms of career development: Lecturers involved in OE receive mainly technical support, and in half of the cases also some training. Less common are support mechanism related to time allocation for the development of OE, and its recognition for career development.

Related and future JRC work

This survey was carried out under the umbrella of the OpenEdu project where a set of studies about recognition mechanism practices, Open Education case studies, and motivations and profiles of MOOC learners have been also developed. All these studies can stand alone, but together they compose a body of research for the creation of a framework on Open Education strategies for Higher Education institutions².

1.    Introduction

The European Commission considers the modernisation of Education and Training systems as a key means of achieving sustainable and inclusive growth in Europe (see Europe 2020 strategy³). ICT-based open and flexible education is expected to contribute to this modernisation by facilitating more efficient and effective ways of acquiring relevant competences for the 21st century economy and society (Rethinking Education Communication, 2012). The initiative “Opening up Education: Innovative teaching and learning for all through new technologies and Open Educational Resources” (European Commission, 2013) developed this idea in depth. Opening up Education took a systemic approach and identified Open Education (OE) as a potential solution to some of the challenges detected in EU educational systems. In addition, it put forward some important topics to be considered when planning policies for modernisation where Open Education principles can play a significant role, such as promoting digital skills and competences, removing barriers to education access, encouraging the sharing of knowledge, best practices and educational materials across EU borders. Open Education can play this role both for formal and non-formal education institutions. Recently, the inclusion of “open and innovative education and training, including by fully embracing the digital era” as one of the six priorities of the strategic framework for European cooperation in Education and Training (ET2020) has reinforced the policy relevance of OE (European Commission, 2015).

As the integration of Open Education into the Higher Education systems is a policy objective, it is essential to have an overview of current institutional engagement with Open Education across the EU. This is the only way in which potential policy measures for promoting Open Education integration can be assured to be relevant and functional. This report contributes to this overview by exploring the supply side of Open Education in five European Higher Education systems (France, Germany, Poland, Spain and the UK). The findings presented in this report are based on a survey, conducted in the period February-June 2015, that was responded by 178 Higher Education institutions (HEIs) in these five EU countries. The survey’s objective was to show if, how and to what extent, Higher Education institutions in the five countries engage in Open Education - and if not, why not. Consequently, the report is a contribution to the Opening Up Communication (European Commission, 2013) and the renewed priority on Open Education, enabled by digital technologies, in the ET2020 strategy.

A comprehensive working definition of Open Education was used for this survey: “Open Education is understood as a mode of delivering education, usually via information and communication technologies (ICTs) or blended learning, which offers alternative ways of building competences and skills, and enables less restrictive access routes to formal and non-formal education, as well as to opportunities for lifelong learning (with or without formal recognition of learning achievements)”. This broad and comprehensive definition was proposed in order to capture as many different aspects and understandings of the Open Education concept as possible and therefore it encompasses the elements that are usually identified by literature, experts and practitioners as components of OE at HE contexts: the use of ICT in education to overcome place and time constraints and opening up the range of pedagogies, the use and development of Open Educational Resources (OER), the offer of Massive Open Online Courses (MOOCs), the collaboration among institutions, the offer of recognition possibilities for individual’s open learning, the use of open software in educational institutions and the engagement in Open Science activities. Many institutions were just starting to engage in the field, and a broad definition allowed the survey to capture insights from any OE initiative run by the targeted institutions, however rudimentary.

This definition was chosen as ‘openness’ in education has proved to be a multidimensional concept, and though it is open to interpretation, it has a common underlying philosophy on which the items of this survey are based. Empirically, it is supported by the fact that many of the dimensions of Open Education covered by the survey are interrelated. For instance, institutions that offer OER are much more likely to offer MOOCs, and the institutions offering Open Education are also more inclined to have some open science initiatives and to use free and open source software.

The survey sought to complement existing information about best practices and leading institutions in Open Education by providing evidence about the general state of Open Education in five large Higher Education systems. The report presents some trends in OE common to all the five systems and some differences between them. So far, not many surveys of this kind have been done, particularly ones that involve statistical significance. Even though the net sample is not large (178 institutions), statistical soundness was ensured in order to have a realistic picture of OE provision in the surveyed HE systems and be able to obtain valid data on the most prominent differences among them. The survey took appropriate measures to avoid selection bias (especially over-representation of Open Education early-adopter institutions) and to provide data that realistically described how the bulk of Higher Education institutions in these countries deal with openness. At the same time, it was ensured that country differences, when large enough, were visible.

This report presents the main results of the survey and is structured as follows:

• Section 2 discusses the methods used in the data collection and analysis.

• Section 3 presents the main findings by different topics: ICT-based learning; perception of Open Education provision; OER; MOOCs; Recognition; Collaboration; Open Education strategy and organisation; and Open Science and Free and Open Source Software. In this section, a summary is presented at the beginning of each topic. The summaries are recommended reading for those readers who want a quick summary of the results.

• Finally, in Section 4, evidence-based recommendations for promoting the use of OE are proposed.

2.    Methods

This report is based on a representative survey of HEIs in five EU countries (France, Germany, Poland, Spain and the UK). These countries were selected on the basis of their size (sufficient number of HE institutions) and geographical variety. In a first phase, the questionnaire for the survey was designed, revised by experts and piloted in five HEIs, one per country (see final questionnaire in Annex 1). In a second phase, the sampling framework was built up mainly from official national sources (Ministries of Education or Rectors’ Conferences). The sampling frame consisted of 1,264 Higher Education institutions from the five selected countries (see Table 1 for the number of institutions per country).

Given the relative large number of Higher Education institutions in Germany, Poland and France, proportionally stratified samples for these countries were drawn. In the remaining two countries the whole frame was used as a gross sample. The stratified samples were drawn according to: type of institution according to national classifications; region according to national classifications; whether or not the institution had offered massive open online courses (MOOC) in the past 3 years (according to MOOC scoreboard information). In each of the three countries about 200 HEIs were selected. After deleting several institutions because no contact data was available or because they were used in the piloting phase of the questionnaire, a gross sample of 889 institutions for the five countries was confirmed.

The respondents to the questionnaire had the following or similar profiles, depending on the university: vice-rectors for ICT-related activities, vice-rectors for academic affairs/teaching and learning, and in cases where no other information was available, rectors or rectors’ offices were directly contacted. However, sometimes this central contact passed the questionnaire to a more appropriate person, who had a better overview of the state of Open Education in the institution.

After extensive data collection between February and May 2015 and a quality-check of the items, the study had valid responses from a total of 178 Higher Education institutions. This overall response rate of 20% ranges from 11.2% in France to 28.8% in Poland (See Table 1).

Initially, the raw results of the data collection experienced selection bias, meaning that the net sample differed from the frame with respect to the stratification variables (type of institutions, region and MOOCs offering). In Poland, France and Germany, the proportion of universities (=category of the variable “type of institution”) was not properly represented in the net sample. Also, of the responding institutions, those that offered MOOCs were over-represented. In addition serious distortion by region was observed. In order to solve this problem, and be able to draw a precise map of Open Education in the five selected countries, the respondents’ characteristics (see Table 1 for the net sample) were adjusted to the distribution of the stratification variables in the sampling frame via two different sets of weighting factors. One was applied to the bycountry comparisons (calculated using a ranking technique) and the other to the overall sample analysis (which adjusted the sample to the size of the Higher Education systems via post-stratification weights). More details about the weighting procedure can be found in Annex 2.

All in all, the application of weighting factors reduced the size of the sample. Using the Kish approximation, the effective net sample size (neff) was estimated to be 117.75 for the analysis using all countries together (Design effect=1.51), and 146 “by country” for analysis and comparisons (Design effect=1.22). This loss in precision should be taken into account when producing overall estimates. Given the loss in effective sample size, variance estimates were increased which in turn inflated confidence intervals. Nevertheless, the weighted net sample reflected the Higher Education systems in the five participating countries better than the un-weighted sample and largely avoided the problem of bias and over-representation of pro-Open Education institutions.

Table 1: Response rates and confidence intervals (for five countries together)

Sampling Frame

Contacted

Responses

Response Rate

Neff⁴

CI neff(p=q=0.5)

Poland

306

191 (stratified sample)

55

28.8%

50

+-13.9%

France

294

196 (stratified sample)

22

11.2%

19

+-22.5%

Germany

361

198 (stratified sample)

25

12.6%

17

+-23.8%

Spain

157

157

35

22.3%

27

+-18.5%

UK

147

147

41

27.9%

38

+-15.9%

TOTAL

1264

889

178

20%

117.5⁵

+-9%

Overall, the size of the effective net sample is adequate to make inferences about Open Education in the five analysed countries together (Confidence Interval= +-9% in the worst case scenario: p=q=0.5)⁶. However, the lower precision means we must be cautious when making inferences by country. Nevertheless, the estimates resulting from the “by country” analysis allow us to make country comparisons and to include some statistically significant differences among them when they are large enough.

In this report, an alpha level of .05 (confidence level of 95%) has been used in all the inferential statistical analyses. However, the results that failed to reach significance at the 95% level, have been considered as marginally significant so long as the alpha level was not higher than .10 (confidence level of 90%). The description of the results includes, whenever possible, the exact p-value. In terms of the statistical analyses performed, beyond the simply descriptive, the chi-square test of independence was used to test the association of different categorical variables. When the chi-square tests of independence revealed a statistical association between two variables, Cramer’s V (V) is also reported to measure the strength of association between two categorical variables. Cramer’s V can reach a maximum of 1, so it can be interpreted in the same way as a correlation coefficient. Whenever the chi-square test of independence was significant, the adjusted standardized residuals (called only adjusted residuals in SPSS) were analysed to identify which cells are responsible for the statistically significant effect. Other analyses performed for the purpose of the current report also include t-test (comparison of two independent groups) and one-way analysis of variance (comparison of more than two independent groups). These analyses were used when dealing with variables that were measured on a 7-point scale and can thus be treated as continuous variables.

3.  Institutional engagement with Open Education in five EU countries

This section presents the main results the survey. It includes the descriptive findings for the five countries together, plus the most important differences by country. The section is structured in eight topics:

1. ICT-based learning;

2. Perception of Open Education provision;

3. Open Educational Resources;

4. Massive Open Online Courses;

5. Recognition;

6. Collaboration;

7. Open Education strategies and organisation;

8. Open science and free and open source software.

3.1  ICT-based learning

ICT-based learning in a nutshell

• ICTs enable the opening-up of education through for instance, widening access and diversifying the range of pedagogies.

• Use of ICT in HEIs can be divided into two groups according to the degree of adoption:

○ Classical uses (>50%) (online materials, online discussions, video-recording of lectures and social networks) which are widespread.

○ Innovative uses (<50%) (online simulations and laboratories, mobile learning, serious games and learning analytics) which are much less frequent in HEIS.

• Blended learning (72%) is more frequently provided by HEIs than fully online courses (34%) and online study programmes (29%).

• Blended learning is widely used and is perceived by university managers as the most effective way of delivering education.

• Considering three options (face-to-face, blended or fully online), university managers believe that fully online learning is the least suitable form of education for promoting active learning, teacher-student communication, personalized learning, pedagogical innovation, and collaboration between students.

• 100% online learning is however considered as a better option than face-to-face learning for the acquisition of ICT skills and for more efficient use of lecturers’ and students’ time.

While not all kinds of ICT use in education equal Open Education, it is clear that ICTs can contribute to opening-up different aspects of education such as access (overcoming place and time constraints) and the range of available pedagogies (e.g. flipped classrooms, enhanced collaboration through technologies, personalisation via analytics...). For these reasons, we asked respondents about the use of ICT at the different levels of study and forms of provision. The pattern at Bachelor and Master’s level is roughly similar: institutions make more “classical” use of ICT (materials available online, online discussions, video recording of lectures and social networks), but more innovative use (online simulations and laboratories, mobile learning, serious games and learning analytics) is less frequent. Although there have been programmes fully taught online for some time, they are infrequent. At PhD level, blended learning is considerably less common and fully online-delivered programmes are rare exceptions (5.6%).

Figure 1: Use of ICT-based learning at Bachelor, Master or doctoral level

Note: Number of valid responses after weighting: 118

University managers were also asked about their perception of the effectiveness of 100% online learning, blended learning and 100% face-to-face learning for 10 different dimensions (see Figure 2). For all 10 dimensions, blended learning is considered to be the most effective type of education. 100% online learning is seen, in the case of 5 dimensions, as the least suitable form of provision. However, in the categories “acquisition of ICT skills”, “efficient use of lecturer’s time” and “efficient use of the students’ time” 100% online education comes second, albeit with comparatively low percentages of 21% or less. Also, 100% online learning is seen to be almost as effective as blended learning when it comes to lowering the cost of education for students (by 43.5% of the institutions for 100% online learning as compared to 44.4% for blended learning) and similarly in the case of lowering the cost of education provision for the institution (37.1% compared to 43.0% respectively).

Figure 2: Perception of the effectiveness of online, blended and face-to-face learning

Note: Number of valid responses after weighting: 118

3.2  Perception of Open Education provision

Perception of Open Education provision in a nutshell

• Close to 40% of institutions stated they provide some form of Open Education.

• There are variations by HE systems:

○ UK institutions stand out, as 60% of them report to provide OE.

○ In Germany this figure is at 22%.

○ 42-43% of Spanish, French and Polish institutions declared to provide OE.

• In about 1/3 of the HEIs providing OE, it is done within only a few of their faculties (less than 10% of all faculties). Around 29% provide it in several faculties (between 10-50%) and 25% declared they provide it in more than 50% of their faculties.

Since HEIs do not share a common view of what Open Education is, the survey asked them whether they provided Open Education in any of its different forms, according to their own general perception. Close to 40% of the participating HEIs stated they did provide Open Education, over 60% said they did not. For country comparisons, the chi-square test of independence was significant⁷, indicating that institutions in the United Kingdom provided Open Education more often than institutions in Germany. Indeed, the UK is the only country among the five where the institutions (63.0%) are more likely to offer Open Education than not. Germany, on the other hand, had the highest share of institutions that answered “no” to this question – 78.2%. In France, Germany and Spain about 42-43% of institutions said they offered Open Education.

Figure 3: Perception of Open Education provision

Note: Number of valid responses after weighting: 117 (for overall) and 144 (for country comparison)

The survey also explored how many faculties (units, departments and similar) at the institutions provide Open Education. In about one third of institutions, the offer is provided by only a few of such entities. Slightly fewer institutions report provision in several faculties (29.25) or most of them (25.1%).

Figure 4: Provision of Open Education at the level of faculties (or other similar units)

Note: Number of valid responses: 46. Only institutions that replied to ‘Yes’ to perception of OE provision are included

3.3 Open Educational Resources (OER) and other digital materials

OER and other digital materials in a nutshell

• Digital materials and OER are adopted by institutions

○ 51% of HEIs promote the use of OER.

○ 58% of HEIs promote the use of other digital materials in their educational practices.

○ The promotion of development of own resources is lower (35% for the development of OER and 49% for other digital materials).

• HE systems vary:

○ The proportion of institutions which promote the use of OER in France is higher than in the remaining countries.

○ Polish institutions are less likely to promote the use of OER.

• When analysing the institutions which declared that they promote the development of OER (n=38), the survey showed:

○ OER is predominantly used as complementary materials and not as the main teaching material (82% of HEIs recommend their own OER as complementary materials for face-to-face courses but only 51% of HEIs stated they are used by lecturers in these courses).

○ Specific quality assurance approaches for OER are rare (11%). However, 41% of HEIs use already existing QA mechanisms.

• Development and use of OER is correlated with higher importance being placed on offering free education in the overall institutional strategy. Institutions offering OER placed more importance on institutional strategies for free access to education than those offering MOOCs.

According to OECD definition (2012), Open Educational Resources (OER) are “digital learning resources offered online freely (without cost) and openly (without licensing barriers) to teachers, educators, students, and independent learners in order to be used, shared, combined, adapted, and expanded in teaching, learning and research” ⁸ (Hylen et al. 2012).

Institutions in the sample were asked whether they encouraged their lecturers to use or develop OER or other digital educational materials. The majority of respondents said their institution had a supportive attitude to the use of OER (51.4%) and other digital materials (58.6%). Fewer institutions, however, supported the development of OER (35.2%) and other digital educational material not licensed as OER (49.2%). To test differences across countries, a chi-square test of independence was performed. For the “use of OER”, the chi-square test of independence was marginally significant⁹, indicating that in Poland’s institutions the promotion of the use of OER was lower than in the remaining countries. In addition, the proportion of institutions in France promoting the use of OER was higher than in the remaining countries. However, for the “development and offer of OER”, “use of other digital educational materials (not licensed OER)”, and “development and offer of other digital educational materials (not licensed OER)” no significant differences among countries were revealed¹⁰.

Figure 5: Promotion of use and development of OER and other digital materials

Note: Number of valid responses after weighting: from 108 to 114 depending on the variable (for overall) and from 135 to 140 (for country comparison)

Do institutions which do not promote the use of OER yet intend to do so in the future? Only about 10-11% of the institutions said they did. This represents around one fifth of the institutions which do not support it now. A high share of respondents said they did not know. It is worth pointing out that the growth potential of OER is a bit lower than the potential growth of MOOCs mentioned in Section 3.4 (10% vs 19%).

Figure 6: Plans to promote the use of OER in the future

Note: Number of valid responses after weighting: from 108 to 114 depending on the variable

Respondents who said they promoted the development and offer of OER were asked if their institution uses its own OER and if so, in which types of courses (a number of different ones was suggested). Although the number of responses was relatively low, the answers indicate that institutions used their own OER predominantly as complementary materials for blended learning courses and for face-to-face instruction (86.6% and 82.2% respectively) and not in the “heartland” of the education provision.

Figure 7: Institutional use of own OER

Note: Number of valid responses: 34. Only respondents who promote development and offer of OER

Quality assurance (QA) approaches specific to OER were rare (just over 10%). Standard QA methodologies seemed to prevail. Somewhat surprisingly, the share of those stating there was no QA at all or who did not know was relatively high (27.1% and 21.1%).

Figure 8: Quality assurance mechanisms for own OER

Note: Number of valid responses: 38. Only respondents who promote development and offer of OER

3.4  Massive Open Online Courses (MOOCs) offer

MOOC offer in a nutshell

• MOOCs are on the agenda of almost half (41%) of the HE institutions. 22% of HEIs declared they are already offering MOOCs and 19% are planning to do so. This suggests there is an important growth potential in the short term.

• Currently, the use of MOOCs is less widespread than that of OER, but intentions to offer MOOCs in the future are higher than intentions to offer OER.

• HE systems vary:

○ In France, Spain and the UK a similar and relatively high proportion of institutions offer MOOCs (around 35%).

○ These figures fall to 10% and 8% respectively in Germany and Poland.

○ Institutions in France and Poland stand out regarding their plans to offer MOOCs in the future (26% and 24% respectively).

• When analysing the institutions which offer MOOCs (n=25), the survey showed that:

○ In many cases, MOOC offer is part of the institution’s official educational strategy (57%).

○ MOOCs usually have different quality assurance mechanisms to face-to-face courses.

○ The majority of MOOCs are not linked to recognition instruments or qualification frameworks, and when they are, ECTS is used most often (26%).

Massive Open Online Courses are online courses without access barriers (such us price or entrance requirements) and thought to provide education to a large number of learners. They appeared in 2008, but they only became widely used during 2012-2013. The study explored to what extent HEIs had adopted them: one fifth of all respondent institutions said they offered MOOCs and about another fifth said that though they were not offering MOOCs at the moment, they intended to do so in the near future. These figures are lower than those from other recent EU surveys. EADTU stated that 72% of institutions were offering or planning to offer MOOCs in 2014 (Jansen & Schuwer, 2015), and EUA situated this figure at 58% in 2013 (Gaebel et al. 2014). Our survey avoided over-representation of OE leading institutions (Cf. Section 2 on methods).

This general situation varied by country, ranging from France, where a large number of Higher Education institutions offered MOOCs (36%) and a large number of those that did not currently offer them intended to do so in the future (26.2%), to others like Germany where both rates were low (10.1% and 13% respectively). Poland fell between the two extremes. Although its offer was significantly lower than that of the remaining countries¹¹, its growth potential was significantly higher statistically than that of UK¹².

Figure 9: Offer of MOOCs

Note: Number of valid responses after weighting: 117(for overall) and 144 (for country comparison)

The relationship between offering MOOCs and the promotion of Open Educational Resources was examined via a chi-square test of independence and found to be significant in the case of the promotion of use of OER¹³, and also in the case of the promotion of development of OER¹⁴. Institutions that did not promote the use of OER were very likely not to offer MOOCs, which was indeed the case for about 92% of institutions that not promote OER. For HEIs promoting the use of OER, MOOCs were offered in about 35% of cases. In the case of institutions promoting the development and offer of OER, about 60% did not offer MOOCs and 40% did. This difference was much larger for HEIs that did not promote the development and offer of OER. In these cases, MOOCs were provided by only 10% of institutions.

Institutions which said they offered MOOCs were asked for more detail about their offer. There were only 25 of these institutions, a good reason for not over-interpreting the findings. First, they were asked whether offering MOOCs was part of the official educational strategy of their institution. The figures show that, in many cases (57.5%), offering MOOCs was considered an integral part of the institution’s official educational strategy.

Figure 10: Offering MOOCs as part of the institution’s official educational strategy

Note: Number of valid responses: 25. Only respondents who offer MOOCs.

Secondly, these institutions were asked about the relationships between the MOOCs offered by them and a range of recognition instruments and qualification frameworks. Nearly two thirds of the respondents perceived no links of this sort at all. About a quarter reported a link between their institutions’ MOOCs and the ECTS system and only 9% reported a link to national qualification frameworks.

Figure 11: Connection of MOOCs to recognition instruments and qualification frameworks

Note: Number of valid responses: 25. Only respondents who offer MOOCs.

Finally, the 25 HEIs offering MOOCs were asked about the quality assurance mechanisms they used for these courses. Most of them used different quality assurance mechanisms to those they used for face-to-face programmes.

Figure 12: MOOCs quality assurance mechanisms compared with those for face-to-face education

Note: Number of valid responses: 25. Only respondents who offer MOOCs.

3.5  Recognition

Recognition in a nutshell

• The majority of HEIs do not have mechanisms for recognising MOOC certificates in ECTS, even when certificates were based on reliable assessment.

○ Only 30% of institutions are likely to recognise MOOC certificates which had been allocated a specific number of credits, even when the certificate is based on an on-site exam.

○ This figure drops to only 18% when the certificate is based on a proctored online exam.

• The majority of MOOCs are not linked to recognition instruments or qualification frameworks, and when they are, ECTS is used most often (26% see point 3.4).

• Formal recognition of Open Education was mentioned as an important barrier for engagement in Open Education (see section 3.7).

In order to investigate the mechanisms for the recognition of prior learning, the survey looked at a range of certification tools for MOOCs and other forms of Open Education. Respondents were asked whether certificates would be recognised in credit points via prior learning recognition procedures in their institution, and if so, which certificates. Despite a relatively high share of those who said “I don’t know” to each specific recognition procedure, the results show that few institutions had recognition mechanisms for MOOCs, whether certificates were based on an on-site or online exam, or even when they had a specific number of credits allocated (29.9%). This similarly applied to studying with OER or independent studies (17.3% and 16% respectively). Even fewer declared that they would recognise a certificate without an identity control (3.1%).

Figure 13: Recognition of Open Education in credits points via prior learning recognition procedures

Note: Number of valid responses after weighting: from 109 to 113 depending on the variable

Although the figures differed among countries, there were almost no statistically significant differences. The only exception to this was found in the recognition of knowledge achieved via independent self-study¹⁵.UK institutions were more likely than institutions of the remaining countries to recognise self-study knowledge in credit points. Spanish institutions were the least likely to do so.

Figure 14: Recognition of Open Education in credits points via prior learning recognition procedures by country

Note: Number of valid responses after weighting: from 137 to 141 depending on the variable

3.6 Collaboration

Collaboration in a nutshell

• Cross-border collaboration among institutions is limited compared to local or national collaboration. In all cases (collaboration in digital learning strategies, online courses, teacher training, OER and MOOCS), national collaboration is higher than cross-border collaboration. Cross-border collaboration tends to be around 20% (except in the case of MOOCs where it is lower).

• Although the figures came from a subsample, cross-border collaboration in MOOCs seems to be lower than in other areas (13% for development of MOOCs and 4% for recognition of MOOCs).

• As regards national collaboration, differences between countries were detected. In general, France has the highest levels of “within country” collaboration and Poland has the lowest.

• Cross-border collaboration, however, is similar in the five countries studied and no significant differences are observed in the data.

Due to its digital nature, Open Education represents a good opportunity for national and cross-border collaboration, and therefore a specific section of the survey was focused on this topic.

First, the survey investigated collaboration with other Higher Education institutions in the design of digital learning strategies, shared online courses and shared teacher training. Around 50%-55% of institutions said they cooperated with other institutions. They were most likely to collaborate at the national level on teacher training. Around 20% of the institutions collaborated cross-nationally in these three areas.

Figure 15: Collaboration in the design of digital learning strategies, shared online courses, and shared teacher training.

Note: Number of valid responses after weighting: 118

By country, a lower proportion of institutions in Poland said they collaborated with other universities within their own country on the design of digital learning strategies¹⁶ and also on shared online courses¹⁷ as compared with France. This higher level of collaboration between French HEIs might be an impact of the national policy initiative France Université Numeriqué (FUN). It is also worth highlighting that, although there is no statistically significant difference, UK institutions said they had more cross-border collaboration than collaboration with other UK institutions as regards online courses for students (30% vs 19.9%).

Figure 16: Collaboration on the design of digital learning strategies, shared online courses and shared teacher training by country

Note: Number of valid responses after weighting: 146

Over 50% (55.9%) of those institutions currently offering MOOCs said they cooperated at national-level with other institutions on the development of these courses. However this figure is only 12.6% when it refers to cross-border collaboration. The share of those who did not cooperate at all was 39.6%. Recognition of MOOCs was not a strong incentive for institutions to cooperate internationally: 56.8% did not cooperate with other institutions on mutual recognition of MOOCs or other online courses, while two fifths (41.4%) did, but again only at national level. Only 3.9% of institutions offering MOOCs collaborated on the recognition of these courses with institutions from other countries.

Figure 17: Collaboration on MOOCs development and recognition

Note: Number of valid responses: 25. Only respondents who offer MOOCs

As regards OER, over 45% of the institutions which promoted the development of OER declared they collaborated with institutions in their own countries, while 21.4% declared collaborating cross-border. Also it is important to note that 45.7% of these institutions did not collaborate with any other institutions.

Figure 18: Collaboration in the development of OER

Note: Number of valid responses: 38. Only respondents who promote the development of OER

3.7 Open Education strategies and organisation

Open Education strategies and organisation in a nutshell

• In general, elements related to OE are perceived as important for overall institutional strategy. However, elements related to the democratisation of education (access for students not formally registered for a degree and access to education without tuition fees) are perceived as less important.

• Open Education is not formalised in institutional strategies within most HEIs. Around one third (32%) of HEIs have a policy or mission statement that supports Open Education. However, 60% of French institutions take this more formal approach.

• Permission from the hierarchy is usually required to develop MOOCs and OER (In 65% of HEIs central level hierarchy permission is required for developing a MOOC and in 52% for developing OER).

• HEIs indicated that the main challenges for engaging with OE are:

○ That teachers lack skills in OE (92% of respondents agreed with the need for more teacher training and 77% with the fact that lecturers are used to pedagogies that do not include OE).

○ Difficulties associated with the formal recognition of Open Education (78% of agreement), with some significant differences between countries.

• The main reasons given by HEIs for engaging in Open Education are:

○ Enhancing the image and the visibility of the institution (97%).

○ Reaching more learners (97%).

• Regarding the financial benefits of Open Education, the data indicate that the institutions have divided opinions. 76% of the surveyed HEIs considered that Open Education requires more financial resources than anticipated and 56% of them do not see any financial benefits from offering OE.

○ HEIs with experience in offering Open Education are more likely to see potential financial benefits from the engagement in OE: 56% of them see possibilities of using OE as a way of reducing the cost of the education they provided.

○ Moreover, when asked about the financial benefits, almost a quarter (23%) of HEIs with experience in OE declared they have already had financial benefits.

• Institutions offering OE often support lecturers engaging in this type of education. The most common types of support are help from specialist staff (84%), general awareness raising (65%) and training opportunities in OE (52%).

• Relatively little support is provided in the form of job related incentives. Only 27% of HEIs offering OE allow lecturers to develop OE during their working hours and only 15% of HEIs use evaluation mechanisms for lecturers’ career development that take into account their engagement in OE.

Moving from analysis of the offer to analysis of the strategies on Open Education, HEIs were asked about the relative importance they gave to eight “elements” in their overall strategy, from 1=very unimportant to 7=very important. The “elements” mentioned are often perceived as favouring Open Education. Against this background, it was interesting to note that the mean for all questions ranged from 4.55 to 6.06 on a scale of 7. The ‘leader’ amongst the 8 “elements” was the intention to “support collaboration between your institution and the wider society”, with a mean of 6.06. Even the “elements” with the lowest means, the intention to “provide educational opportunities without entry requirements” (4.55) and the intention to “provide free (no tuition fees) access to education (4.66), which were focused on providing services to non-enrolled students, achieved relatively high agreement rates. A similar pattern emerged when looking at the response categories “very important” and “rather important/important (figure 19).

Figure 19: Importance of elements in the universities’ overall strategies

Note: Number of valid responses after weighting: from 110 to 113 depending on the variable. The response categories “important” and “rather important” were aggregated into the category “rather important/important” and the categories “rather unimportant” and “unimportant” were aggregated into the category “unimportant/rather unimportant”. All the remaining categories are identical to the original scale.

The importance of these elements in the HEIs’ overall strategies differed by country (see Figure 20). For example, the importance of providing educational opportunities without entry requirements was, overall, significantly different¹⁸ and was higher in Poland (Mean=4.96) than in the UK (Mean=3.85)¹⁹. However, there were no significant differences among the remaining countries.

The importance of providing free (no tuition fees) access to education was also found to be different among countries²⁰. Post-hoc comparisons revealed that the importance ascribed to this in the UK (Mean= 3.41) was significantly lower than in Germany (Mean= 4.89)²¹, in Poland (Mean= 5.34)²² and, marginally, in Spain (Mean= 4.60)²³. There were no significant differences between the UK and France.

The importance of promoting innovative pedagogies through the use of ICT in the universities’ strategies also depended on the country²⁴. Post-hoc comparisons revealed that it was more important in the UK (Mean= 6.57) than in Poland (Mean= 5.94)²⁵ and Germany (Mean= 5.75)²⁶.

Finally, the last element found to be different across countries was the importance of supporting collaboration with other HEIs²⁷. Post-hoc comparisons revealed that the importance ascribed to this element was lower in the UK (Mean= 5.30) than in Poland (Mean= 6.28)²⁸, and in Spain (Mean= 6.31)²⁹.

Figure 20: Importance of elements in the universities’ overall strategies by country

Note: Note: Number of valid responses after weighting: from 139 to 142 depending on the variable. Numbers represent mean values (from 1 very unimportant to 7 very important)

The relationship between the provision of MOOCs, support to OER and the importance of different strategies for the institution was examined via independent samples t-tests. The results showed that the importance ascribed to institutional strategies only varied as a function of the offer of MOOCs in two cases. Both the ‘production and use of OER’ and the ‘promotion of innovative pedagogies through ICT’ were considered as more important by HEIs offering MOOCs (Mean= 5.44; Mean= 6.40, respectively), than they were in institutions which did not provide MOOCs (Mean= 4.63; Mean=5.90, respectively)³⁰. These data, together with the data presented in Section 3.4, confirmed that there was a relationship between two of the most common forms of Open Education (MOOCs and OER) as part of a common strategy. In addition, MOOCs are perceived as useful tools for innovating pedagogy. Institutions that promoted the use of OER ascribed more importance to provision of free access to education (Mean= 4.92) than the remaining institutions (Mean= 4.14)³¹. This was also true of institutions that promoted the production and use of OER (Mean= 4.98) as compared to institutions that did not do so (Mean= 4.26)³². This data confirm that the discourse around OER is strongly linked to the discourse on the free access to education for all.

In order to further explore the connexion between the Higher Education institutions’ strategies and Open Education, the survey explored the perception of the institutions’ managers of what role their institutions would be playing by 2020. This question was also linked to the unbundling of tasks and aimed to measure how important each of 6 tasks would be in the future (see Figure 21). Looking at the means (from 1=very unimportant to 7=very important), the role most often chosen was that of a research centre and knowledge producer (5.89), followed by the role of a guidance provider for enrolled learner (5.75) and an assessment body validating competences acquired in the own university (5.58). It is interesting to note that the role of servicing students from outside the universities figured much more modestly. This was in the line with the importance of different elements for the overall strategy, where it was shown that strategies that aimed to open up education to outside students were the least valued.

Figure 21: Unbundling the tasks of Higher Education institutions by 2020

Note: Number of valid responses after weighting: from 111 to 116 depending on the variable. The response categories “important” and “rather important” were aggregated into the category “rather important/important” and the categories “rather unimportant” and “unimportant” were aggregated into the category “unimportant/rather unimportant”. All the remaining categories are identical to the original scale.

The survey investigated the degree of formalisation of Open Education initiatives in the institutions. The results showed that the majority of Higher Education institutions in the five countries concerned had no policy or mission statement related to Open Education. Less than one third of the institutions said they had a policy or mission statement that supported Open Education. In Poland, Spain German and UK the great majority of HEIs (from nearly two thirds in Poland to four fifths in the UK) did not have a policy or mission statement on Open Education. In France, however, the chi-square test was significant³³, showing that France had the highest proportion of institutions with a policy or mission statement that supported Open Education (60.1%) and the lowest proportion of institutions with no policy or mission statement on Open Education (39.9%). In addition, France was the only country where more institutions supported Open Education in their policy and mission statements than did not. The existence of institutional policies expressing reservations on Open Education was very rare, with only a few cases in Poland and Spain.

Figure 22: Policy or mission statements on Open Education

Note: Number of valid responses after weighting: 113 (for overall) and 141 (for country comparison)

The relationship between an institution having an official institutional position on OE and the importance of different strategies for that institution was examined. Table 2 displays the descriptive statistics (means and standard deviations) and the results of the comparisons, examined via independent sample t-tests. The results showed that institutions with an official position or mission statement that supported Open Education ascribed more importance to the vast majority of the different institutional strategies under analysis than institutions that did not have any policy or mission statement on OE. The only exceptions were the strategies related to collaboration with other Higher Education institutions and with the wider society. In these cases, the importance ascribed to these strategies did not vary as a function of the existence/non-existence of an institutional policy on OE. This could indicate that these areas are not commonly understood as components of Open Education.

Table 2: Importance of general institutional strategies and the official institutional position towards Open Education

No policy or mission statement is available on Open Education

Policy or mission statement in support of Open Education

Mean

Standard
Deviation

Mean

Standard
Deviation

t-test results

Provide time and place flexibility to learners (when and where they learn)

5.75

1.24

6.35

.97

t(107)= −2.501 p= .014

Provide educational opportunities without entry requirements

4.30

1.50

5.10

1.31

t(104)= −2.593 p= .011

Provide free (no tuition fees) access to education

4.35

1.92

5.14

1.45

t(85.19)= −2.351a p= .021

Production and use of Open Educational Resources

4.37

1.44

5.67

1.28

t(106)= −4.561 p< .001

Promote innovative pedagogies through the use of ICT

5.84

1.09

6.37

.97

t(103)= −2.407 p= .018

To offer recognition for any kind of learning and competences

5.29

1.44

5.87

.95

t(86.73)= −2.456a p= .016

To support the collaboration with other Higher Education Institutions

5.90

1.12

5.89

1.26

t(107)= 0.072 p= .943

To support the collaboration between your institution and society at large

6.01

1.08

6.17

.95

t(104)= −0.766 p= .445

Note: higher mean values indicate a higher level of importance (1= very unimportant; 7= very important). a In these cases, the homogeneity of variances between groups could not be assumed. Therefore, the value of the t-statistic interpreted was the one computed for equal variances not assumed.

We asked those institutions that were engaged in Open Education to grade 5 reasons for doing so from 1(=very unimportant) to 7 (=very important). The mean for the motivation to “enhance the image and the visibility of the institution” was highest (6.29), followed by the intention to “reach more learners” (6.10). “Enhance the quality of the educational offer” and “increase the enrolment in formal education” scored 5.88 and 5.76 respectively. Interestingly, the mean for the category “reduce the cost of the educational provision for the institution” was the lowest at 4.56. As Figure 23 shows, enhancing the image and visibility of the institution was deemed rather/important to very important by most respondents.

Figure 23: Importance of factors for engaging on Open Education

Note: Number of valid responses: 43. Only respondents who provide Open Education. The response categories “important” and “rather important” were aggregated into the category “rather important/important” and the categories “rather unimportant” and “unimportant” were aggregated into the category “unimportant/rather unimportant”. All the remaining categories are identical to the original scale.

When asked if engagement in Open Education had produced any financial benefits for their institution, the majority of the Higher Education institutions said that, so far it had not. Only 23% reported financial benefits. This would seem to be consistent with the philosophy of Open Education, which also entails the notion of no fees. However, the picture could be different in the future. New forms of provision often produce high up-front costs, which could decrease over time. When those institutions which said they had obtained benefits were asked an open question about what type of benefits, most of them mentioned “reach more students”. In addition, other related benefits such as marketing, the small income directly generated by freemium services, external funding, improvements in educational quality and higher retention rates were mentioned.

Figure 24: Financial benefits from engaging in Open Education

Note: Number of valid responses: 43. Only respondents who provide Open Education

The institutions were also asked what they perceived as the main obstacles for Open Education engagement (see Figure 25). They were asked to grade the relative importance (from 1=very unimportant to 7=very important) of different possible barriers. The biggest challenges were seen as the training required for teachers in the use of Open Education (6.08) and the difficulties of obtaining formal recognition of Open Education (5.51). The least important challenges were seen by respondents as the general opposition by the institution to Open Education (“Open Education is not in line with our pedagogical approach” – 3.61) and Open Education resulting in a lowering of the quality of educational provision (3.75).

Figure 25: Importance of barriers for Open Education

Note: Number of valid responses after weighting: from 108 to 115 depending on the variable

The importance given to the barriers varied by HE system. In three cases, there were statistically significant differences. The first of these was related to the barrier “lecturers are used to traditional pedagogies that do not include Open Education”³⁴. Post-hoc comparisons revealed that institutions in France tended to agree more with the statement that lecturers are used to traditional pedagogies that do not include Open Education (Mean= 5.90) than institutions in Germany (Mean= 4.79)³⁵. The second barrier in which significant differences were found was “Open Education is not in line with our pedagogical approach”³⁶. Here, post-hoc comparisons revealed that institutions in the United Kingdom (Mean= 2.84) agreed to a lesser extent with this obstacle than institutions in France (Mean= 4.57)³⁷, and marginally with institutions in Spain (Mean = 3.93)³⁸ and Poland (Mean= 3.65)³⁹. Institutions in Germany (Mean= 2.97) also agreed to a lesser extent with this obstacle than institutions in France (Mean= 4.57)⁴⁰. Finally, the last barriers where significant differences in the importance given by country were detected was “formal recognition of Open Education is still an unresolved issue at the institutional level”⁴¹. Here, post-hoc comparisons showed that institutions in Spain (Mean= 6.09) saw this obstacle as more important than institutions in Germany did (Mean= 4.98)⁴² and marginally than institutions in the United Kingdom (Mean= 5.21)⁴³.

Figure 26: Importance of barriers for Open Education by country

Note: Number of valid responses after weighting: from 138 to 141 depending on the variable. Numbers represent mean values (from 1 very unimportant to 7 very important)

The level of agreement with two barriers to Open Education was found to vary as a function of the offer of MOOCs by an institution. Institutions that did not provide MOOCs agreed more strongly with the notion that OE might affect negatively the quality of the institution’s educational provision than institutions that offered MOOCs (Mean= 4.04 vs. Mean= 2.75)⁴⁴. However, causality cannot be inferred since data did not allow exploring if more experience with MOOCs leads to less doubts about quality of Open Education or if fewer doubts about OE quality is a determinant of MOOCs offer. There was also a marginal relationship between the offer of MOOCs and the level of agreement with the fact that OE was not in line with the institution’s pedagogical approach. Indeed, institutions that offered MOOCs tended to agree slightly more strongly that this was a barrier than institutions that did not provide MOOCs. These results confirm that MOOCs practices are linked to a discourse of pedagogical innovation and quality.

The relationship between the use and the development and offer of Open Education Resources and the level of agreement with different obstacles and barriers to Open Education was also examined. Institutions that promoted the use of OER agreed less strongly with the idea that OE could affect negatively the quality of the university’s educational provision than institutions that did not promote the use of OER (Mean= 3.41 vs. Mean= 4.07)⁴⁵. Similarly, institutions that promoted the development and offer of OER agreed less strongly that OE could negatively affect the quality of the institution’s educational provision than institutions that did not promote the development and offer of OER (Mean= 3.15 vs. Mean= 4.05)⁴⁶.

The survey investigated whether various types of support were given by institutions to lecturers for engaging in Open Education. Only those institutions that said they provided Open Education were asked these questions so the number of respondents ranged around 40. Adding up the percentages of “yes” and “no, but planning to”, support by specialist staff to design courses and resources reached the highest agreement level (94.9%). It was followed by general awareness raising (89.5%) and the provision of training opportunities in Open Education (85.2%). Relatively few institutions agreed that they took or were planning to take engagement in Open Education into consideration in staff evaluation and for career development (43.9%). Furthermore, relatively few institutions agreed that they allowed or were planning to allow lecturers to reserve specific working hours for the development of Open Education (43.3%).

Figure 27: Support for the involvement of lectures in Open Education

Note: Number of valid responses: 40. Only respondents who provide Open Education

The research also looked at the freedom staff had to offer a MOOC or to produce or offer OER. Did they need permission from the ‘hierarchy’ and, if so, from which level of management? In the majority of cases, permission was indeed necessary. However, in about 16% of cases, staff could offer a MOOC without anyone’s consent. It was slightly more common that the production or offer of OER did not need permission (23.3%). In the vast majority of cases where permission was necessary, the institutional, faculty and departmental leadership seemed to play an almost equally important role.

Figure 28: Hierarchical level approval needed for lecturer/s engagement in MOOCs offer or OER production

Note: Number of valid responses after weighting: 118

3.8 Open science and Free and Open Source Software

Open science and Free And Open Source Software in a nutshell

• Open science is on the agenda of about half the institutions.

• The most commonly-supported open science areas are: “shared research infrastructures with other universities” (67%) and “open data in research and publication in open access routes” (54%).

• The least supported areas are: “open peer reviewing” (31%), “use of alternative funding mechanisms” (35%) and “use of alternative metrics for scientific reputation” (39%).

• Universities that support open science tend to do so by allocating time to it (50%) and providing economic support (47%). As in the case of Open Education, they use mechanisms related to career development and promotions less often (32%).

• About half the respondents stated that their university makes use of both FOSS and proprietary software. 18% declared that they almost always use only proprietary software and only 4% declared that they use only FOSS.

The survey explored whether or not institutions were engaged in the area of Open Science, through “yes/no” questions about 8 different dimensions, i.e. “publication in open access routes”, “open data in research”, “open peer reviewing”, “dissemination via social networks and blogs”, “research with non-scientific participants”, “use of alternative funding mechanisms”, “use of alternative metrics for scientific reputation”, “shared research infrastructures with other universities” and “shared research infrastructures with citizens and society”. The highest percentage value was observed for shared research infrastructures with other universities (66.7%), followed by open data in research (54.0%) and publication in open access routes (51.3%). In all the other categories, “no” answers dominated, showing that it was an incipient field with about 1/3 of the institutions supporting open peer reviewing (31.3%), the use of alternative funding mechanisms such as crowdfunding (35.1%) and the use of alternative metrics (38.6%).

Figure 29: Engagement in Open Science

Note: Number of valid responses after weighting: from 94 to 102 depending on the variable

The relationship between institutions’ support to different aspects related to Open Science and the offer of Open Education was examined using a chi-square test of independence. The relationship between the offer of Open Education and publication in Open Access routes was significant (66.5% of support in the case of institutions offering OE vs 41.1% in the case of institutions not offering OE)⁴⁷. There was also a significant relationship between the offer of Open Education and the support to open data in research (69.7% vs 43.42%)⁴⁸, open peer reviewing (42.4% vs 23.67%)⁴⁹ and the use of alternative metrics for scientific reputation (53.4% vs 27.96%)⁵⁰. These results show how openness in educational institutions is a multidimensional concept that, once adopted, goes beyond the educational function of the institution to other functions, such as research.

Furthermore, the research explored what institutions did in practice to support staff in Open Science. It was found that most institutions did not take engagement in Open Science into account for promotion or salary raises, nor did they offer funding for Open Science activities. The institutions that did offer this kind of support were more likely to offer funding for Open Science activities (47.4%) than through promotions and salary increases. However, slightly more than half the institutions (50.1%) did include Open Science activities in job descriptions.

Figure 30: Incentive mechanisms to promote open science

Note: Number of valid responses: from 77 to 78 depending on the variable. Only respondents who support any of the open science topics above.

Finally information about the use of Free and Open Source Software (FOSS) by institutions was collected by the survey. Almost half the respondents stated that their university used FOSS in a mix with proprietary software (46.6%). If we include those institutions which used FOSS only in some faculties, we can conclude that over two thirds of institutions used FOSS at least in part. About 18% of institutions only used proprietary software.

Figure 31: Use of Free and Open Source Software

Note: Number of valid responses after weighting: 112

Finally, the relationship between the use of FOSS and the offer of Open Education was examined via a chi-square test of independence. This relationship was found to be significant⁵¹. The cells responsible for the significant effect pertained to the response category ‘No, the university almost always uses proprietary software’. In institutions that provided Open Education, this situation was much less frequent (2.56%) than it was in institutions that did not offer it (28.3%).

4.    Policy recommendations

Open Education (OE) through the use of digital technologies can widen access to education and improve the relevance and quality of higher education. This is recognised at EU policy level in different policy initiatives. Most recently, one of the six new priorities of the strategic framework for European cooperation in Education and Training (ET2020), adopted at the November 2015 Education Council refers to the promotion of more widespread open and innovative education and training via digital technologies.

OpenSurvey data show that Open Education is on the agenda of half the HEIs in the five European countries surveyed. This means that, while the benefits are recognised by many HEIs, there is still a long way to go to fully mainstream OE in HE systems. The results from this survey help to illuminate a number of issues that need to be tackled to further OE uptake in Europe.

The study identifies nine policy recommendations, structured around three main headings: (1) the incorporation of OE into the overall planning and strategy formulation of HEIs; (2) awareness raising and increasing understanding of OE; (3) changes needed to improve existing practices of HEI.

Each recommendation contains action points for European and national⁵² policy makers and the HEIs themselves.

I. Integrate OE into HEIs overall strategies

1. Bring the different elements of OE together in a holistic approach and strategy: The implementation of OE consists of several different elements (OER, MOOCs, recognition, innovation, quality, training of staff, etc.) and the survey results indicate that these are interrelated.

HEIs should develop a strategy on open education which brings these different elements together and links them to concrete goals of the general institutional strategy.

Policy makers at European and national level could encourage and incentivise HEIs to do so.

2. Make sure that overall quality assurance mechanisms account for OE: The survey shows that quality assurance mechanisms for MOOCs are different from those of onsite courses. Also, OER are rarely quality assured.

European and national policy makers and quality assurance bodies should ensure the inclusion of Open Education quality assurance into overall QA, while considering the similarities and differences between OE and traditional education.

3. Strengthen the links between OE and Open Science: The importance of openness is not limited to the education function of universities. The survey shows a correlation between OE and Open Science elements, pointing to a wider philosophy of openness. A significant number of surveyed HEIs are already committed to sharing research infrastructures and open access publications. More innovative areas such as open peer review, alternative funding and metrics for scientific reputation are emerging.

HEIs and European and national policy makers should address both elements jointly.

II.  Increase awareness and understanding about OE

4. Emphasize the benefits of OE: The level of support institutions give to Open Education is closely related to perceived benefits and usefulness. OE can be a win-win situation for society and individual institutions. In the survey, the following arguments in favour of OE were confirmed by stakeholders: enhancing image and visibility, reaching more students, increasing the number of enrolled students, enhancing the quality of education, and potentially reducing the cost of educational provision for the institution.

Policy makers at both European and national level could provide incentives for HEIs to experiment with OE in order to assess the benefits and risks for their own institutions.

5. Create awareness of different recognition mechanisms for open learning: HEIs tend not to recognize the equivalence of MOOC certificates to ECTS credits, especially if they come from other institutions. This is true even when the assessment uses secure methods of identity validation and the course description includes the number of ECTS points. Recognition issues must be tackled if OE is to develop further.

European and national policy makers should support experimentation, sharing of experiences and cross-institute collaboration to develop awareness, understanding and trust in reliable recognition mechanisms for open learning.

III.  Change existing practices to facilitate the implementation of OE

6. Adapt teacher training and evaluation mechanisms in order to encourage academic teaching staff to engage with OE: The HEIs surveyed reported that academic teaching staff lack the skills and experience to engage in OE.

HEIs should include OE practices and pedagogies in their teacher training and continuous professional development programmes and consider them in staff evaluation mechanisms or career development incentives.

Policy makers at national level should encourage participation in Open Education to be part of teacher evaluation mechanisms.

7. Facilitate more collaboration in OE, both national and cross-border: Not much cooperation is currently taking place among HEIs, particularly across borders. Therefore, Europe could be missing the opportunities that OE offers for internationalising education and promoting virtual mobility. The barriers and benefits of cooperation need to be better understood.

European and national policy makers could support networks of collaboration in Open Education.

8. Foster blended and online learning for efficient and effective education: Overall, blended learning is considered by the HEIs surveyed as the most effective way of implementing pedagogical innovations such as active learning, personalized learning or collaboration between students. Yet, 100% online learning is considered a better option than face-to-face learning for some subjects, such as the acquisition of ICT skills, and can imply a more efficient use of lecturers’ and students’ time.

HEIs should take into account the different purposes that technologies can serve. HEI should support teachers to choose the most suitable use of technology and non-technology based pedagogy for each content item or course activity.

9. Support innovative uses of digital technologies: The survey shows that some uses of ICT in Higher Education, such as online materials, online discussions, video-recording of lectures, or social networks are already quite widespread, but there are a range of innovative uses which are not highly adopted by HEI.

○ HEIs and European and national policy makers should foster more innovative uses of digital technologies, such as online simulations and laboratories, mobile learning, serious games and learning analytics to continue testing and exploiting new options and pedagogical models for a digital-age learning.

References

European Commission (2012): Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the regions. Rethinking Education: Investing in skills for better socioeconomic outcomes. COM(2012) 669 final. Retrieved from: http://ec.europa.eu/education/news/rethinking/com669_en.pdf.

European Commission (2013): Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the regions. Opening up Education: Innovative teaching and learning for all through new technologies and Open Educational Resources. COM (2013) 654 final. Retrieved from: http://bookshop.europa.eu/en/opening-up-education-pbNC0214816/

European Commission (2015): Strategic Framework for European cooperation in Education and Training (ET2020). Retrieved from: http://data.consilium.europa.eu/doc/document/ST-14440-2015-REV-1/en/pdf

Gaebel, M; Kypriyanovam V; Morais, R; Colucci, E. (2014): E-learning in European Higher Education institutions: Results of a mapping survey conducted in October-December 2013. Retrieved from: http://eua.be/Libraries/publication/e-learning_survey

Hylén, J., et al. (2012): “Open Educational Resources: Analysis of Responses to the OECD Country Questionnaire”, OECD Education Working Papers, No. 76, OECD Publishing, Paris. DOI: http://dx.doi.org/10.1787/5k990rjhvtlv-en. ). Retrieved from: http://www.oecd-ilibrary.org/education/open-educational-resources_5k990rjhvtlv-en;jsessionid=1i3vtktbip5d.x-oecd-live-02

Jansen, D. and Schuwer, R (2015): Institutional MOOC strategies in Europe: Status report based on a mapping survey conducted in October - December 2014. Retrieved from: http://www.eadtu.eu/documents/Publications/OEenM/InstitutionalMOOCstrategies_in_Europe.pdf

List of abbreviations and definitions

ACA- Academic Cooperation Association

DG EAC- Directorate General for Education and Culture

EC- European Commission

ECTS-European Credit Transfer System

EADTU- European Association of Distance Teaching Universities

EU- European Union

EUA- The European University Association

HEIs- Higher Education Institutions

ICT- Information and Communication Technologies

JRC- Joint Research Centre

IPTS- Institute for Prospective Technological Studies

MOOCs- Massive Open Online Courses

OE- Open Education

OECD- Organisation for Economic Co-operation and Development

OER- Open Educational Resources

UK- The United Kingdom

List of Figures

Figure 1: Use of ICT-based learning at Bachelor, Master or doctoral level

Figure 2: Perception of the effectiveness of online, blended and face-to-face learning

Figure 3: Perception of Open Education provision

Figure 4: Provision of Open Education at the level of faculties (or other similar units)

Figure 5: Promotion of use and development of OER and other digital materials

Figure 6: Plans to promote the use of OER in the future

Figure 7: Institutional use of own OER

Figure 8: Quality assurance mechanisms for own OER

Figure 9: Offer of MOOCs

Figure 10: Offering MOOCs as part of the institution’s official educational strategy

Figure 11: Connection of MOOCs to recognition instruments and qualification frameworks

Figure 12: MOOCs quality assurance mechanisms compared with those for face-to-face education

Figure 13: Recognition of Open Education in credits points via prior learning recognition procedures

Figure 14: Recognition of Open Education in credits points via prior learning recognition procedures by country

Figure 15: Collaboration in the design of digital learning strategies, shared online courses, and shared teacher training.

Figure 16: Collaboration on the design of digital learning strategies, shared online courses and shared teacher training by country

Figure 17: Collaboration on MOOCs development and recognition

Figure 18: Collaboration in the development of OER

Figure 19: Importance of elements in the universities’ overall strategies

Figure 20: Importance of elements in the universities’ overall strategies by country

Figure 21: Unbundling the tasks of Higher Education institutions by 2020

Figure 22: Policy or mission statements on Open Education

Figure 23: Importance of factors for engaging on Open Education

Figure 24: Financial benefits from engaging in Open Education

Figure 25: Importance of barriers for Open Education

Figure 26: Importance of barriers for Open Education by country

Figure 27: Support for the involvement of lectures in Open Education

Figure 28: Hierarchical level approval needed for lecturer/s engagement in MOOCs offer or OER production

Figure 29: Engagement in Open Science

Figure 30: Incentive mechanisms to promote open science

Figure 31: Use of Free and Open Source Software

List of tables

Table 1: Response rates and confidence intervals (for five countries together)

Table 2: Importance of general institutional strategies and the official institutional position towards Open Education

1.    Annex 1: Questionnaire

OPENSURVEY: Survey on Open Education in European Higher Education Institutions⁵³

Dear colleague,

This online survey is carried out in the framework of the OpenSurvey study and it focuses on the topic of Open Education (OE). The survey is funded by the European Commission’s Institute for Prospective Technological Studies (IPTS) located in Seville and is jointly carried out by IPTS and the Brussels-based Academic Cooperation Association (ACA).

Through this online survey we aim to capture all views on OE including those of Higher Education institutions that do not (yet) engage in OE, irrespective of the reasons. Therefore we encourage particularly institutions that are NOT actively participating in OE to fill in the survey. For respondents from such Higher Education institutions the questionnaire will only consist of a small number of questions.

OE is understood in this survey as a mode of delivering education using Information and Communication Technologies (ICT) which offers alternative ways of learning and access routes to formal and non-formal education and aims to be open to everyone (OpenEdu, 2015).

Although not limited to these, some examples of OE practices are Massive Open Online Courses (MOOCs), Open Educational Resources (OER) and digital certificates or badges.

This survey aims to generate an overview of the current institutional positions and engagement in OE in five countries: France, Germany, Poland, Spain and the UK. The focus of the survey is mainly on teaching and learning and to a lesser extent on other issues such as open access and/or open science.

The final goal is to inform the Directorate-General for Education and Culture (DG EAC) in the design of policy initiatives on Open Education following the 2013 Communication on “Opening up Education: Innovative teaching and learning for all through new Technologies and Open Educational Resources”

Please answer the questions from your point of view and to the best of your knowledge.

The estimated time for filling in the survey is around 30 minutes.

Should you wish to consult the survey before providing your answers, please see the PDF version which is available here. Please note, however, that only the online version can be used to submit your answers.

You may access the online questionnaire multiple times with or without providing answers. No question is mandatory. If you find you cannot answer a particular question or don’t know how to, you can move on to the next question. However, once you have passed the final page, i.e. clicked on the ‘Submit’ button, the questionnaire will be locked.

As a sign of appreciation for your collaboration in filling in the questionnaire you will receive an executive summary (set of presentation slides) of the survey results once the fieldwork will have been completed.

Thank you for the collaboration.

In this questionnaire, Open Education is understood as a mode of delivering education using Information and Communication Technologies (ICT) which offers alternative ways of learning and access routes to formal and non-formal education and aims to be open to everyone. Although not limited to these, some examples of OE practices are Massive Open Online Courses (MOOCs), Open Educational Resources (OER) and digital certificates or badges.

1. Does you institution have any official policy or mission statement related to Open Education?

No, no policy or mission statement is available on Open Education

Yes, policy or mission statement in support of Open Education

Yes, policy or mission statement expressing reservations concerning Open Education

Filter Q2a only displayed if Q1 = Yes

2a. Please describe your institution’s position concerning Open Education.

Filter Q2b only displayed if Q1= No

2b. Please describe why your institution does not (yet) have a policy or mission statement on Open Education.

3. In your view, how important are the following elements in your university’s overall strategy (i.e. the general strategy, not particularly on Open Education)?

Very important

Important

Rather important

Partly important/partly unimportant

Rather unimportant

Unimportant

Very unimportant

Provide time and place flexibility to learners (when and where they learn)

Provide educational opportunities without entry requirements

Provide free (no tuition fees) access to education

Production and use of Open Educational Resources

Promote innovative pedagogies through the use of ICT

To offer recognition for any kind of learning and competences

To support the collaboration with other Higher Education Institutions

To support the collaboration between your institution and society at large

Section: Online-Blended

4. Please indicate which of the following types of study programmes your institution offers (generally, not only in Open Education).

Yes

No

Bachelor programmes

Master programmes

PhD/doctoral programmes

Further/adult education programmes

Language centre programmes

Practical training programmes in cooperation with or for companies

Filter in Q5: only the options marked as ‘Yes’ in Q4 appear as columns in Q5.

5. This question is on the use of Information and Communication Technologies (ICT)-based learning in the various types of programmes offered by your institution. Please indicate whether your institution uses, as far as you know, ICT-based learning in the types of programmes mentioned in the columns.

Bachelor programmes

Master programmes

PhD/doctoral programmes

Further/adult education programmes

Language centre programmes

Practical training programmes in cooperation with or for companies

Study programmes fully taught online

Individual courses fully taught online

Blended learning (mixture of online and face-to face learning within a course)

Materials available online

Video recording of lectures

Online discussion and interactive tools

Social networks

Use of online collaborative tools

Online simulations and online laboratories

Serious games

Mobile learning

Flipped classroom methodologies

Learning analytics

Other (please specify)

6. Please indicate which type of learning – 100% online learning, blended learning, and 100% face-to-face learning – is in your opinion most effective in promoting the following aspects in your institution. (Please indicate one response in each row.)

100% digital/online learning

Blended learning courses (programmes and courses that mix online and face-to face education)

100% face-to-face learning

Personalized learning

Active learning

Efficient use of the students’ time

Efficient use of the lecturers’ time

Pedagogical innovation

Acquisition of ICT skills

Lower cost burden of the education for the students

Lower cost of the education provision for the institution

Collaboration between students

Teacher-student communication

Section: MOOCs

7. Does your institution offer any Massive Open Online Courses (MOOCs)?

Yes

No

Q8.1 is only displayed if Q7= Yes

8.1 Is offering MOOCs part of your institution’s official educational strategy?

Yes

No

Q8.2 is only displayed if Q7= No

8.2 As far as you know, are there any plans for your institution to offer MOOCs in the future?

Yes

No

I don’t know

Q9 is only displayed if Q7= Yes

9. What were in your opinion the main drivers behind the offer of MOOCs by your institution? (Multiple answers possible.)

It was a leadership initiative (top-down approach)

The initiative was driven by the lecturers (bottom-up approach)

It was recommended/mandated by the national/regional government or one of its agencies

It was incentivised by the European Commission’s Lifelong Learning or ERASMUS+ programme funding

Other, please specify ___________

I don’t know*

Q10 is only displayed if Q7= Yes

10. Does your institution participate in any MOOC platforms or initiatives?

Yes

No

Q11 is only displayed if Q10= Yes

11. In which platform/s or inititative/s? (Multiple answers possible)

Coursera

Iversity

FutureLearn

MiriadaX

EDX

Udacity

OpenupEd

European Multiple MOOC Aggregator (EMMA)

Own university platform (Please specify) ____________

Other regional/national platform (Please specify) ____________

Others (Please specify) ____________

Q12 is only displayed if Q7= Yes

12. Does your institution connect its MOOCs to any of the following frameworks? (Multiple answers possible)

Some or all MOOCs are connected to the European Credit Transfer and Accumulation System (ECTS)

Some or all MOOCs are connected to the European Qualifications Framework (EQF)

Some or all MOOCs are connected to the national qualifications framework

Some or all MOOCs are connected to other reference frameworks (Please specify) ___________

No, the MOOCs are not connected to any reference framework

Q13 is only displayed if Q7= Yes

13. Does your institution have any quality assurance mechanism for the MOOCs offered?

No

Yes, it uses the same quality assurance mechanisms as for face-to-face programmes

Yes, it uses different quality assurance mechanisms than those for face-to-face programmes

I don’t know

Q14 is only displayed if Q13= Yes, it uses different quality assurance mechanisms than those for face-to-face programmes

14. How do you think the MOOCs quality assurance mechanisms compare with those for face-to-face education?

The quality assurance mechanisms of MOOCs are more rigorous

The quality assurance mechanisms of MOOCs are equally rigorous

The quality assurance mechanisms of MOOCs are less rigorous

I don’t know

Q15 is only displayed if Q7= Yes

15. Please elaborate on the business model related to MOOCs in your institution, if any (e.g. aims, funding, costs, gains, etc.)

Section: Open Education Resources

According to an OECD definition (2012), Open Educational Resources (OER) are digital learning resources offered online freely (without cost) and openly (without licensing barriers)⁵⁴ to teachers, educators, students, and independent learners in order to be used, shared, combined, adapted, and expanded in teaching, learning and research.

16. Does your institution promote the use or the development of Open Educational Resources (OER) or other digital educational materials by its lecturers?

Yes

No

Promotion of the use of OER

Promotion of the development and offer of OER

Promotion of the use of other digital educational materials (not licensed OER)

Promotion of the development and offer of other digital educational materials (not licensed OER)

Q17 is only displayed if Q16 Promotion of the use of OER= No

17. As far as you know, are there any plans for your institution to promote the use of OER in the future?

Yes

No

I don’t know

Q18 is only displayed if Q16 Promotion of the development and offer of OER= No

18. As far as you know, are there any plans for your institution to promote the development and offer of OER in the future?

Yes

No

I don’t

Q19 is only displayed if
Q16 Promotion of the use of OER= Yes
OR Q16 Promotion of the development and offer of OER= Yes

19. What were in your opinion the main drivers behind your institution’s decision to support OER? (Multiple answers possible)

It was a leadership initiative (top-down approach)

The initiative was driven by the lecturers (bottom-up approach)

The initiative was driven by the students (bottom-up approach)

It was recommended/mandated by the national/regional government or one of its agencies

It was incentivised by the European Commission’s Lifelong Learning or ERASMUS+ programme funding

Other, please specify _________

I don’t know*

Q20 is only displayed if
Q16 Promotion of the development and offer of OER= Yes

20. Does your institution promote uploading its OER to any repository?

Yes

No

Internal repository of your institution

Shared repository (repository owned by your institution together with one or more other institutions)

National external repository (owned by others, e.g. government)

International external repository (owned by others, e.g. international organisation)

Q20.1. is only displayed if Q20 = No (for any of the items)

In the previous question, you indicated the option "No" for the items displayed below (left-hand side).

Q20.1. As far as you know, are there any plans in your institution to promote uploading OER to any of the following types of repositories in the future?

Yes

No

I don’t know

Internal repository of your institution

Shared repository (repositoryowned by your institution together with one or more other institutions)

National external repository (owned by others, e.g. government)

International external repository (owned by others, e.g. international organisation)

Q21 is only displayed if Q16 Promotion of the development and offer of OER= Yes

21. To what extent would you estimate that the own OER (i.e. those developed and offered by your institution) are being used within your institution?

In 50% or more of all programmes

In between 25% and up to 50% of all programmes

In less than 25% of all programmes

In less than 10% of all programmes

In no programmes at all

I cannot estimate

Q21.1 is only displayed if Q16 Promotion of the development and offer of OER= Yes

21.1. As far as you know, in which of the following different types of courses are the own OER (i.e. those developed and offered by your institution) being used within your institution?

Own OER are used by the lecturers in face-to-face courses

Own OER are recommended as complementary materials for face-to-face courses

Own OER are recommended as complementary materials for blended learning courses

Own OER are used in paid (against a fee) online courses

Own OER are used in free of charge (no fee) MOOCs

Q22 is only displayed if Q16 Promotion of the development and offer of OER= Yes

22. Does your institution have any quality assurance mechanisms for the OER offered?

No

Yes, it uses already existing quality assurance mechanisms

Yes, it uses newly developed quality assurance mechanisms specific for OER

I don’t know

Q23 is only displayed if Q22= Yes, it uses newly developed quality assurance mechanisms specific for OER

23. in your view, how do OER quality assurance mechanisms compare with those already existing quality assurance mechanisms?

The quality assurance mechanisms of OER are more rigorous

The quality assurance mechanisms of OER are as rigorous

The quality assurance mechanisms of OER are less rigorous

I don’t know

Q24 is only displayed if Q16 Promotion of the use of OER= Yes
OR Q16= Promotion of the development and offer of OER= Yes

24. Please elaborate on the business model related to OER in your institution, if any (e.g. aims, funding, costs, gains, etc.)

25. Does your institution collaborate as far as you know with other Higher Education Institutions in any of the following topics? (Multiple answers possible)

Yes, with institutions in own country

Yes, cross-border (i.e. with institutions from other countries)

No*

Design of digital learning strategies

Shared Online courses for students

Shared teacher training

Q25.1 is only displayed if Q16= Yes promotion of the development and offer of OER

25.1. Does your institution collaborate with other Higher Education Institutions in the development of OER? (Multiple answers possible)

Yes, with institutions in own country

Yes, cross-border (i.e. with institutions from other countries)

No*

Q25.2. is only displayed if Q7= Yes (offer MOOCs)

25.2. Does your institution collaborate with other Higher Education Institutions in the development of MOOCs? (Multiple answers possible)

Yes, with institutions in own country

Yes, cross-border (i.e. with institutions from other countries)

No*

Q25.3. is only displayed if Q7= Yes (offer MOOCs)

25.3. Does your institution collaborate with other Higher Education Institutions in consortia/agreements for the mutual recognition of MOOCs or other online courses from other partner institutions? (Multiple answers possible)

Yes, with institutions in own country

Yes, cross-border (i.e. with institutions from other countries)

No*

Section: Recognition

Q26. is only displayed if Q7= Yes (offer MOOCs)

26. As far as you know, are any of the following certification mechanisms used by your institution to recognise MOOCs?

Yes

No, but we are planning to

No

Design of digital learning strategies

Non-official completion pass certificates or badges

Digital badges with identity control

Pass certificates with identity control

Official credit point(s) for enrolled students

27. In your institution, could the following cases get recognition in credit points via recognition of prior learning procedures?

Yes, it could be recognised in credit points

No, it could not be recognised in credit points

I don’t know

A certificate based on an on-site exam of a MOOC that has a specific number of credits allocated to it in the course description

A certificate based on an on-site exam of a MOOC that does not have a specific number of credit points allocated to it in the course description.

A certificate based on online proctored exam (identity of the learner supervised using online methods such as cameras or recognition technologies) of a MOOC that has a specific number of credit points allocated to it in the course description.

A certificate based on online proctored exam (identity of the learner supervised using online methods as cameras or recognition technologies) of a MOOC that does not have a specific number of credit points allocated to it in the course description.

A MOOC certificate without any identity control.

A certificate achieved via studying with OER.

Prior knowledge achieved via independent self-study.

Section: Open Education Strategy

Remember that in this questionnaire, Open Education is understood as a mode of delivering education using Information and Communication Technologies (ICT) which offers alternative ways of learning and access routes to formal and non-formal education and aims to be open to everyone. Although not limited to these, some examples of OE practices are Massive Open Online Courses (MOOCs), Open Educational Resources (OER) and digital certificates or badges.

28. Is Open Education (in any of the different forms) provided within your institution?

Yes

No

Q29 is only displayed if Q28= Yes

29. Where is Open Education provided within your institution?

Q29.1. is only displayed if Q28= Yes

29.1 At the level of faculties (or similar units, e.g. departments), Open Education is provided (please estimate)

In most faculties (more than 50%)

In several faculties (between 10 and 50%)

In a few faculties (less than 10%)

In no faculties at all

Q29.2. is only displayed if Q28= Yes

29.2 Open Education is (also) provided by other (than faculties or similar units) entities in my institution, namely: (Multiple answers possible)

Further/adult education centre

Language centre

Information and Communication Technologies/e-Learning centre

Any other centre (Please specify) _____________

In no centres at all*

Q30 is only displayed if Q29.1 = In most faculties (more than 50%)
OR Q29.1 = In several faculties (between 10 and 50%)
OR Q29.1 = In a few faculties (less than 10%)
OR Q29.2 = Further education centre
OR Q29.2 = Language centre
OR Q29.2 = ICT centre OR Q29.2 = Any other centre

30. From your point of view, how important are the following factors for engaging in Open Education?

Very Important

Important

Rather important

Partly important/partly unimportant

Rather unimportant

Unimportant

Very unimportant

Reduce the costs of the educational provision for the institution

Enhance the quality of the educational offer

Reach more learners

Increase enrolment in formal education

Enhance the image and visibility of the institution

Q31 is only displayed if Q29.1 = In most faculties (more than 50%)
OR Q29.1 = In several faculties (between 10 and 50%)
OR Q29.1 = In a few faculties (less than 10%)
OR Q29.2 = Further education centre
OR Q29.2 = Language centre
OR Q29.2 = ICT centre
OR Q29.2 = Any other centre

31. Has the engagement in Open Education produced so far any financial benefits for your institution?

Yes

No

Q32 is only displayed if Q31 = Yes

32. Please detail your answer.

33. Open Education may be confronted with obstacles and barriers. To what extent do you agree or disagree with the following statements concerning Open Education in your institution?

Strongly Agree

Agree

Rather agree

Partly agree/partly disagree

Rather disagree

Disagree

Strongly disagree

Open Education requires more financial resources than anticipated

Open Education requires teacher training before becoming effective

Lecturers are used to traditional pedagogies that don’t include Open Education

Open Education is not in line with our pedagogical approach

There is a risk that Open Education affects negatively the quality of our institution’s educational provision

Formal recognition of Open Education is still an unresolved issue at the institutional level

We do not see financial benefit for our institution to do it

Others (Please specify)

Q34 is only displayed if Q29.1 = In most faculties (more than 50%)
OR Q29.1 = In several faculties (between 10 and 50%)
OR Q29.1 = In a few faculties (less than 10%)
OR Q29.2 = Further education centre
OR Q29.2 = Language centre
OR Q29.2 = ICT centre
OR Q29.2 = Any other centre

34. Does your institution offer any of the following to support the involvement of its own lecturers in Open Education?

Yes

No, but we are planning to

No

General awareness-raising

Training opportunities in Open Education

Support by specialist staff when planning and designing the courses and resources (e.g. ICT experts, learning technologists)

Inclusion of digital skills of the students as part of the curricula in the introductory phase of the study programmes

Publicity of the best practices in Open Education

Evaluation mechanisms for career development of lecturers take into account their engagement in Open Education

Investment of money in Open Education kick-starting initiatives

Allowing the lecturers to reserve specific working hours for the development of Open Education

35. If a lecturer or a group of lecturers from the same faculty (or similar unit) would like to engage in Open Education, would the approval of the following hierarchical levels be required? (Multiple answers possible)

In order to offer a MOOC

In order to produce OER

The central-level leadership of the university

The faculty-level leadership

The department which they are part of

Others (Please specify)

None of the above, individuals are free to offer Open Education

36. How important will the following roles be for your institution by 2020, in your view?

Very important

Important

Rather important

Partly important/partly unimportant

Rather unimportant

Unimportant

Very unimportant

Digital content provider (production of OER and MOOCs)

Guidance provider for non-enrolled learners

Provider of guidance (tutor) for enrolled learners

Assessment and certification body, validating competences acquired in the university

Assessment and certification body, validating competences acquired anywhere

Research centre, producing new knowledge

Section: Research and Open Software

37. Does your institution support any of the following topics related to Open Science?

Yes

No

Publication in Open access routes (green route, gold route)

Open Data in research

Open peer reviewing

Dissemination of research outputs via social networks and blogs

Conducting scholarly research with non-scientific participants (e.g. Citizen science projects)

Use of alternative funding mechanisms (e.g. crowd funding, inducement prizes)

Use of alternative metrics for scientific reputation (e.g. Altmetrics, ResearchGate)

Shared research infrastructures with other universities

Shared research infrastructures with citizens and society

Q38 is only displayed if Q37 = Yes (for any of the items)

38. What type of incentive mechanisms, if any, are used in your institution to promote Open Science?

Yes

No

Mechanisms related to career development and promotion (e.g. salaries taking into account for promotions)

Mechanisms related to economic support (e.g. funding)

Mechanisms related to time allocation (e.g. activities are part of job description)

Others (Please specify) ______________

39. Does your university use Free and Open Source Software (FOSS)?

Yes, extensively across the institution: the university rarely pays proprietary software

Yes, we use it across the institution but mixed with proprietary software

Yes, but only in some faculties

No, but we are planning to do so

No, the university almost always uses proprietary software

Annex 2: Bias corrections

After the data collection phase, the survey faced a problem of bias. First, the responses differed considerably across the five countries (see Table 1). Secondly, the net sample differed from the frame with respect to the stratification variables (type of institutions, region and MOOCs, see Table A1). In particular in Poland, France and Germany the proportion of universities (=category of the variable “type of institution”, see Table A1 below) was not properly represented in the net sample. Also, as expected, among the responding institutions the institutions offering MOOCs during the last three years was overrepresented. In addition, a distortion according to region was observed. In several countries some regions yielded no responding institution at all.

Table A1- Distribution of type of institution, region and MOOCs in the sampling frame, in the gross sample and the un-weighted net sample by country

Type of institution un-weighted sample

Frame %

Gross Sample %

Net Sample n

%

Universities

86.5

86.2

44

80.0

Poland

Vocational institutions

13.5

13.8

11

20.0

Total

100.0

100.0

55

100.0

Grande écoles

71.3

71.2

17

77.3

France

Universities

28.7

28.8

5

22.7

Total

100.0

100.0

22

100.0

University of applied science

56.8

57.1

15

60.0

Germany

Universities for arts and music

14.1

14.1

2

8.0

Universities

29.1

28.8

8

32.0

Total

100.0

100.0

25

100.0

Schools for art. music. dance

50.3

50.3

17

48.6

Spain

Other institutions

4.5

4.5

2

5.7

Universities

452

45.2

16

45.7

Total

100.0

100.0

35

100.0

Other institutions

7.5

7.5

3

7.3

UK

Universities

92.5

92.5

38

92.7

Total

100.0

100.0

41

100.0

Region un-weighted sample

Frame %

Gross Sample %

Net Sample n

%

Dolnośląskie

8.7

8.7

7

12.7

Poland

Kujawsko-pomorskie

4.5

4.6

3

5.5

Łódzkie

6.1

6.1

3

5.5

Lubelskie

5.2

5.1

3

5.5

Lubuskie

1.3

1.5

0

0.0

Małopolskie

8.4

8.7

4

7.3

Mazowieckie

19.7

18.9

11

20.0

Opolskie

1.3

1.0

2

3.6

Podkarpackie

3.9

4.1

4

7.3

Podlaskie

3.2

2.6

1

1.8

Pomorskie

6.1

6.6

3

5.5

Śląskie

11.9

11.7

4

7.3

Świętokrzyskie

4.2

4.1

0

0.0

Warmińsko-mazurskie

2.3

2.6

4

7.3

Wielkopolskie

9.4

9.7

4

7.3

Zachodniopomorskie

3.9

4.1

2

3.6

Total

100.0

100.0

55

100.0

Alsace

3.1

3.7

1

4.5

Aquitaine

4.2

4.2

2

9.1

Auvergne

1.7

1.6

0

0.0

Basse-Normandie

0.7

1.0

0

0.0

Bourgogne

1.0

1.0

1

4.5

Bretagne

5.9

6.3

1

4.5

Centre

1.7

1.0

0

0.0

Champagne-Ardenne

1.0

1.0

1

4.5

Franche-Comté

1.4

1.0

0

0.0

Haute Normandie

0.7

.5

0

0.0

France

Ile de France (Paris)

28.7

28.8

5

22.7

Languedoc-Roussillon

3.1

3.1

1

4.5

Limousin

0.7

1.0

0

0.0

Lorraine

4.2

4.7

1

4.5

Midi-Pyrénées

6.6

6.3

1

4.5

Nord / Pas-de-Calais

7.6

7.9

3

13.6

Normandie

2.4

2.1

2

9.1

Oversea

1.7

1.6

0

0.0

Pays de la Loire

5.9

6.3

1

4.5

Picardie

2.1

2.1

0

0.0

Poitou-Charentes

1.7

1.6

0

0.0

Provence-Alpes-Côte d’Azur

3.1

3.1

0

0.0

Rhône-Alpes

10.7

9.9

2

9.1

Total

100.0

100.0

22

100.0

Bayern

12.2

11.6

3

12.0

Berlin

10.0

10.1

1

4.0

Brandenburg

2.5

2.5

0

0.0

Bremen

1.7

2.0

1

4.0

Baden-Württemberg

17.2

16.7

5

20.0

Hamburg

3.6

3.5

3

12.0

Hessen

7.2

7.1

2

8.0

Mecklenburg-Vorpommern

1.7

1.5

0

0.0

Germany

Nordrhein-Westfalen

17.2

17.2

6

24.0

Niedersachsen

7.2

7.6

1

4.0

Rheinland-Pfalz

4.7

5.1

2

8.0

Saarland

1.4

1.5

0

0.0

Sachsen

6.1

6.1

1

4.0%

Sachsen-Anhalt

2.5

2.5

0

0.0

Schleswig-Holstein

2.8

3.0

0

0.0

Thüringen

2.2

2.0

0

0.0

Total

100.0

100.0

25

100.0%

Andalusia

12.1

12.1

1

2.9

Aragon

3.8

3.8

5

14.3

Asturias

1.9

1.9

2

5.7

Balearic Islands

2.5

2.5

0

0.0

Basque Country

4.5

4.5

1

2.9

Canary Islands

3.8

3.8

1

2.9

Cantabria

0.6

0.6

0

0.0

Castile and Leon

12.1

12.1

4

11.4

Spain

Castile La Mancha

0.6

0.6

0

0.0

Catalonia

14.6

14.6

5

14.3

Extremadura

1.3

1.3

0

0.0

Galicia

6.4

6.4

2

5.7

In multiple regions

0.6

0.6

0

0.0

La Rioja

1.3

1.3

0

0.0

Madrid

14.0

14.0

5

14.3

Murcia

3.8

3.8

1

2.9

Navarre

1.9

1.9

1

2.9

Valencia

14.0

14.0

7

20.0

Total

100.0

100.0

35

100.0

England

80.3

80.3

29

70.7

Northern Ireland

2.7

2.7

1

2.4

UK

Scotland

11.6

11.6

8

19.5

Wales

5.4

5.4

3

7.3

Total

100.0

100.0

41

100.0

MOOCs un-weighted sample

Frame %

Gross Sample %

Net Sample n

%

No

99.4

99.5

54

98.2

Poland

Yes

0.6

0.5

1

1.8

Total

100.0

100.0

55

100.0

No

73.7

73.3

14

63.6

France

Yes

26.3

26.7

8

36.4

Total

100.0

100.0

22

100.0

No

89.5

89.4

21

84.0

Germany

Yes

10.5

10.6

4

16.0

Total

100.0

100.0

25

100.0

No

75.2

75.2

24

68.6

Spain

Yes

24.8

24.8

11

31.4

Total

100.0

100.0

35

100.0

No

70.1

70.1

25

61.0

UK

Yes

29.9

29.9

16

39.0

Total

100.0

100.0

41

100.0

In order to align the net sample to the distribution of the stratification variables in the frame, we developed a set of weighting factors using raking technique. In addition, we weighted the net sample according to the size of the Higher Education systems in the five countries in order to allow overall estimates for the five countries taken together. This was achieved using additional post-stratification weights according to the size of the Higher Education systems. The respective procedures are described in sections below.

Weighting factors 1- used for “by country” analysis and country comparisons (type of institution, region and MOOCs)

When comparing the net sample to the frame, considerable differences with respect to the distribution of the stratification variables were noticed. In order to align the net sample to the frame data, we used a raking technique consisting of four steps: First, we weighted the net sample according to type of institution within each country. Subsequently each country net sample was weighted by region and, in addition by MOOCs within countries. Since weighting steps 2 and 3 distorted the net sample with respect to the type of institution, we weighted the net sample again by type of institution within countries. The weighting according to region afforded a combination of regions since in 4 of the five regions some regions yielded no responding institution at all. The combined weighting factors remained within the typical threshold of 4.0; accordingly, no trimming was necessary.

Table A2- Distribution of weighting factors 1 (by country analysis) according to type of institution, region and MOOCs

Weighting factors 1 (by country analysis)

N

%

Up to 0.7000

19

10.7

0.7001 through 1.3000

86

48.3

1.3001 and larger

73

41.0

Total

178

100.0

Note: Max. weighting factor 3,42; no trimming required.

After using these weighting factors, the overall number of cases remained stable at 178 (=actual number of cases; the effective net sample size, which is the size of an un-weighted simple random sample that yields the same precision as this weighted sample, is lower; see below). However, the distribution of key variables was better aligned according to the distribution in the frame (see Table A3).

Table A3: Distribution of type of institution, region and MOOCs by country. Sample weighted with weighting factor 1 (by country analysis) using type, region and MOOC offer.

“Type of institution” Weighted sample (by country analysis)

Frame %

Gross Sample %

Weighted n

Net Sample %

Universities

86.5

86.2

48

86.5

Poland

Vocational institutions

13.5

13.8

7

13.5

Total

100.0

100.0

55

100.0

Gande ecoles

71.3

71.2

16

71.3

France

Universities

28.7

28.8

6

28.7

Total

100.0

100.0

22

100.0

Germany

University of applied science

56.8

57.1

14

56.8

Universities for arts and music

14.1

14.1

4

14.1

Universities

29.1

28.8

7

29.1

Total

100.0

100.0

25

100.0

Schools for art, music, dance

50.3

50.3

18

50.3

Other institutions

4.5

4.5

2

4.5

Spain

Universities

45.2

45.2

16

45.2

Total

100.0

100.0

35

100.0

Other institutions

7.5

7.5

3

7.5

UK

Universities

92.5

92.5

38

92.5

Total

100.0

100.0

41

100.0

“Region” Weighted sample (by country analysis)

Frame %

Gross Sample %

Weighted Net n

Net Sample %

Dolnośląskie

8.7

8.7

5

8.9

Kujawsko-pomorskie

4.5

4.6

Łódzkie

6.1

6.1

3

6.3

Lubelskie

5.2

5.1

3

5.2

Lubuskie

1.3

1.5

Małopolskie

8.4

8.7

4

6.5

Mazowieckie

19.7

18.9

11

20.2

Opolskie

1.3

1.0

Podkarpackie

3.9

4.1

Poland

Podlaskie

3.2

2.6

Pomorskie

6.1

6.6

3

6.2

Śląskie

11.9

11.7

7

12.2

Świętokrzyskie

4.2

4.1

Warmińsko-mazurskie

2.3

2.6

Wielkopolskie

9.4

9.7

5

9.5

Zachodniopomorskie

3.9

4.1

Combined regions

14

24.9

Total

100.0

100.0

55

100.0

Alsace

3.1

3.7

Aquitaine

4.2

4.2

1

5.0

France

Auvergne

1.7

1.6

Basse-Normandie

0.7

1.0

Bourgogne

1.0

1.0

Bretagne

5.9

6.3

1

6.4

Centre

1.7

1.0

Champagne-Ardenne

1.0

1.0

Franche-Comté

1.4

1.0

Haute Normandie

0.7

.5

Ile de France (Paris)

28.7

28.8

5

24.8

Languedoc-Roussillon

3.1

3.1

Limousin

0.7

1.0

Lorraine

4.2

4.7

1

4.5

Midi-Pyrénées

6.6

6.3

2

7.2

Nord / Pas-de-Calais

7.6

7.9

2

7.2

Normandie

2.4

2.1

Oversea

1.7

1.6

Pays de la Loire

5.9

6.3

1

4.2

Picardie

2.1

2.1

Poitou-Charentes

1.7

1.6

Provence-Alpes-Côte d’Azur

3.1

3.1

Rhône-Alpes

10.7

9.9

3

12.9

Combined regions

6

27.8

Total

100.0

100.0

22

100.0

Bayern

12.2

11.6

3

12.0

Berlin

10.0

10.1

2

9.3

Brandenburg

2.5

2.5

Bremen

1.7

2.0

Baden-Württemberg

17.2

16.7

4

17.1

Hamburg

3.6

3.5

1

3.6

Germany

Hessen

7.2

7.1

2

6.2

Mecklenburg-Vorpommern

1.7

1.5

Nordrhein-Westfalen

17.2

17.2

4

17.4

Niedersachsen

7.2

7.6

2

7.9

Rheinland-Pfalz

4.7

5.1

1

4.8

Saarland

1.4

1.5

Sachsen

6.1

6.1

2

8.0

Sachsen-Anhalt

2.5

2.5

Schleswig-Holstein

2.8

3.0

Thüringen

2.2

2.0

Combined regions

3

13.7

Total

100.0

100.0

25

100.0

Andalusia

12.1

12.1

3

8.9

Aragon

3.8

3.8

2

4.8

Asturias

1.9

1.9

Balearic Islands

2.5

2.5

Basque Country

4.5

4.5

2

5.9

Canary Islands

3.8

3.8

1

4.1

Cantabria

0.6

0.6

Castile and Leon

12.1

12.1

4

11.9

Castile La Mancha

0.6

0.6

Spain

Catalonia

14.6

14.6

5

14.4

Extremadura

1.3

1.3

Galicia

6.4

6.4

2

6.8

In multiple regions

0.6

0.6

La Rioja

1.3

1.3

Madrid

14.0

14.0

5

14.3

Murcia

3.8

3.8

1

4.1

Navarre

1.9

1.9

Valencia

14.0

14.0

5

14.6

Combined regions

4

10.4

Total

100.0

100.0

35

100.1

England

80.3

80.3

32

78.5

Northern Ireland

2.7

2.7

1

3.4

UK

Scotland

11.6

11.6

5

11.8

Wales

5.4

5.4

3

6.4

Total

100.0

100.0

41

100.0

“MOOC offer” Weighted sample (by country analysis)

Frame %

Gross Sample %

Net Sample n

%

No

99.4

99.5

55

99.4

Poland

Yes

0.6

0.5

1

0.6

Total

100.0

100.0

55

100.0

No

73.7

73.3

16

74.8

France

Yes

26.3

26.7

6

25.2

Total

100.0

100.0

22

100.0

No

89.5

89.4

22

89.5

Germany

Yes

10.5

10.6

3

10.5

Total

100.0

100.0

25

100.0

No

75.2

75.2

25

71.4

Spain

Yes

24.8

24.8

10

28.6

Total

100.0

100.0

35

100.0

No

70.1

70.1

29

70.1

UK

Yes

29.9

29.9

12

29.9

Total

100.0

100.0

41

100.0

Note. Differences in the case count due to rounding

The application of weighting factors 1 (by country analysis) reduced the effective sample size. Using the Kish approximation, the effective sample size was estimated for the overall net sample.

After weighting by type of institution, region, MOOCs and then again type of institution within countries, the overall effective net sample size was 146,45 (=size of an un-weighted simple random sample, that yields the same precision as this weighted sample). Consequently, the design effect DEFF was estimated to be 1.22. This is equivalent to a loss of about one fifth of precision due to weighting. When using SPSS for some statistical procedures the confidence intervals have to be adjusted since variance estimates seem smaller than they are when using the weighted sample.

Table A4: number of respondents vs effective net sample size (neff)

n

neff

Poland

55

50

France

22

19

Germany

25

17

Spain

35

27

UK

41

38

Weighting factors 2- used for overall sample analysis (additional weighting by country size)

In order to allow overall estimates based on the net sample of the survey, we developed an additional set of weighting factors that consider the size of the Higher Education systems in the five participating countries. As an indicator for the size of the Higher Education system the number of institutions in each country was used. Based on the weighted net sample using the weighting factors 1 (by country analysis) described above, we used post-stratification procedure in order to align the weighted net sample to the distribution of HEIs in the countries of the frame.

This combined weighting factors exceeded the threshold of 4.0. Overall, nine institutions had weighting factors larger than 4.0; for three institutions the weighting factors exceeded 6.0.Following standard procedures weighting factors larger than 4.0 were trimmed. Trimming of weighting factors applied to Germany only. In order to compensate for the loss in the number of cases in the weighted net sample, we used an additional constant in the weighting factors for Germany. This again led to a situation where some cases (17 institutions) were weighted using a weighting factor larger than 4.0. This, however, could not be avoided given the distribution of weighting factors for Germany. The largest final weighting factor 2 was 4.32. Even though the number of institutions with a weighting factors larger than 4.0 was increased by the trimming, the maximum of the weighting factors could be kept close to 4.0.

Table A5: Distribution of weighting factors 2 (used for overall analysis) including post-stratification by HE systems size.

Weighting factors 2 (overall analysis)

N

%

Up to 0.7000

40

22.6

0.7001 through 1.3000

54

30.3

1.3001 and larger

84

47.1

Total

178

100.0

Note: Max. weighting factor 4,32 after trimming and introduction of constant to maintain number of cases in weighted net sample.

Post-stratification led to a proportional representation of countries in the weighted net sample (see table A6).

Table A6: Distribution of institutions by country in the frame, in the un-weighted net sample and in the weighted net sample (using weighting factors 2 - for overall analysis)

Number of institutions

Frame %

Un-weighted Sample %

Net

Weighted Net Sample %

Poland

24.5

30.9

24.5

France

22.9

12.4

22.9

Germany

28.6

14.0

28.6

Spain

12.4

19.7

12.4

UK

11.6

23.0

11.6

Total

100.0

100.0

100.0

Overall post-stratification by country had no effect of the distribution of type of institution, region and MOOCs. The trimming procedure led only to a very small distortion of the variable type of institution, region and MOOCs for Germany (see table A6). Consequently, we did not apply another round of raking.

Table A6: Distribution of type of institution, region and MOOCs by country (sample weighted with weighting factor 2 – for overall analysis)

“Type of institution” Weighted sample for overall analysis

Frame %

Gross Sample %

Weighted Net n

Sample %

Universities

86.5

86.2

38

86.5

Poland

Vocational institutions

13.5

13.8

6

13.5

Total

100.0

100.0

44

100.0

Gande ecoles

71.3

71.2

29

71.3

France

Universities

28.7

28.8

12

28.7

Total

100.0

100.0

41

100.0

University of applied science

56.8

57.1

27

53.5

Germany

Universities for arts and music

14.1

14.1

8

15.1

Universities

29.1

28.8

16

31.3

Total

100.0

100.0

51

100.0

Schools for art, music, dance

50.3

50.3

11

50.3

Spain

Other institutions

4.5

4.5

1

4.5

Universities

45.2

45.2

10

45.2

Total

100.0

100.0

22

100.0

Other institutions

7.5

7.5

2

7.5

UK

Universities

92.5

92.5

19

92.5

Total

100.0

100.0

21

100.0

“Region” Weighted sample for overall analysis

Frame %

Gross Sample %

Weighted Net n

Sample %

Dolnośląskie

8.7

8.7

4

8.9

Kujawsko-pomorskie

4.5

4.6

Łódzkie

6.1

6.1

3

6.3

Lubelskie

5.2

5.1

2

5.2

Lubuskie

1.3

1.5

Malopolskie

8.4

8.7

3

6.5

Mazowieckie

19.7

18.9

9

20.2

Opolskie

1.3

1.0

Podkarpackie

3.9

4.1

Poland

Podlaskie

3.2

2.6

Pomorskie

6.1

6.6

3

6.2

Śląskie

11.9

11.7

5

12.2

Świçtokrzyskie

4.2

4.1

Warmińsko-mazurskie

2.3

2.6

Wielkopolskie

9.4

9.7

4

9.5

Zachodniopomorskie

3.9

4.1

Combined regions

11

24.9

Total

100.0

100.0

44

100.0

Alsace

3.1

3.7

Aquitaine

4.2

4.2

2

5.0

Auvergne

1.7

1.6

Basse-Normandie

0.7

1.0

Bourgogne

1.0

1.0

France

Bretagne

5.9

6.3

3

6.4

Centre

1.7

1.0

Champagne-Ardenne

1.0

1.0

Franche-Comté

1.4

1.0

Haute Normandie

0.7

.5

Ile de France (Paris)

28.7

28.8

10

24.8

Languedoc-Roussillon

3.1

3.1

Limousin

0.7

1.0

Lorraine

4.2

4.7

2

4.5

Midi-Pyrénées

6.6

6.3

3

7.2

Nord / Pas-de-Calais

7.6

7.9

3

7.2

Normandie

2.4

2.1

Oversea

1.7

1.6

Pays de la Loire

5.9

6.3

2

4.2

Picardie

2.1

2.1

Poitou-Charentes

1.7

1.6

Provence-Alpes-Côte d’Azur

3.1

3.1

Rhône-Alpes

10.7

9.9

5

12.9

Combined regions

11

27.8

Total

100.0

100.0

41

100.0

Bayern

12.2

11.6

7

13.0

Berlin

10.0

10.1

4

8.5

Brandenburg

2.5

2.5

Bremen

1.7

2.0

Baden-Württemberg

17.2

16.7

9

18.5

Hamburg

3.6

3.5

2

3.8

Hessen

7.2

7.1

3

6.7

Mecklenburg-Vorpommern

1.7

1.5

Nordrhein-Westfalen

17.2

17.2

10

18.8

Germany

Niedersachsen

7.2

7.6

4

8.5

Rheinland-Pfalz

4.7

5.1

3

5.2

Saarland

1.4

1.5

Sachsen

6.1

6.1

4

8.5

Sachsen-Anhalt

2.5

2.5

Schleswig-Holstein

2.8

3.0

Thüringen

2.2

2.0

Combined regions

4

8.5

Total

100.0

100.0

51

100.0

Andalusia

12.1

12.1

2

8.9

Spain

Aragon

3.8

3.8

1

4.8

Asturias

1.9

1.9

Balearic Islands

2.5

2.5

Basque Country

4.5

4.5

1

5.9

Canary Islands

3.8

3.8

1

4.1

Cantabria

0.6

0.6

Castile and Leon

12.1

12.1

3

11.9

Castile La Mancha

0.6

0.6

Catalonia

14.6

14.6

3

14.4

Extremadura

1.3

1.3

Galicia

6.4

6.4

1

6.8

In multiple regions

0.6

0.6

La Rioja

1.3

1.3

Madrid

14.0

14.0

3

14.3

Murcia

3.8

3.8

1

4.1

Navarre

1.9

1.9

Valencia

14.0

14.0

3

14.6

Combined regions

2

10.4

Total

100.0

100.0

22

100.0

England

80.3

80.3

16

78.5

Northern Ireland

2.7

2.7

1

3.4

UK

Scotland

11.6

11.6

2

11.8

Wales

5.4

5.4

1

6.4

Total

100.0

100.0

21

100.0

“MOOC offer” Weighted sample For overall analysis

Frame %

Gross Sample %

Net Sample n

%

No

99.4

99.5

43

99.4

Poland

Yes

0.6

0.5

1

.6

Total

100.0

100.0

44

100.0

No

73.7

73.3

30

74.8

France

Yes

26.3

26.7

10

25.2

Total

100.0

100.0

41

100.0

No

89.5

89.4

45

88.7

Germany

Yes

10.5

10.6

6

11.3

Total

100.0

100.0

51

100.0

No

75.2

75.2

16

71.4

Spain

Yes

24.8

24.8

6

28.6

Total

100.0

100.0

22

100.0

No

70.1

70.1

15

70.1

UK

Yes

29.9

29.9

6

29.9

Total

100.0

100.0

21

100.0

Note. Differences in the case count due to rounding

Using again the Kish Approximation, the effective net sample size after post-stratification by country was estimated 117,75. Consequently, the overall design effect of raking according to type of institution, region and MOOCs and post-stratification by country is estimated 1.51.

It means a loss in precision which should be taken into account when producing overall estimates. Given the loss in effective sample size variance estimates are increased which in turn inflates confidence intervals. Nevertheless, the weighted net sample now reflects the Higher Education systems in the five participating countries to a better extend than the un-weighted sample and the sample weighted with weighting factors 1 only.

Notes

  1 Europe 2020 strategy, see http://ec.europa.eu/europe2020/index_en.htm

  2 http://is.jrc.ec.europa.eu/pages/EAP/OpenEdu.html

  3 http://ec.europa.eu/europe2020/index_en.htm

  4 While the overall DEFF is 1.22 for the whole weighted sample (resulting in a weighted overall number of cases of 146.45), in the tables above, country specific DEFF have been used (assuming country specific estimates for one country at a time), i.e. a DEFF for the weighted subsamples for each country. The variance of the weighting factors is somewhat smaller in this case and thus the neff is slightly larger (that is why the neff in the table above adds up to 151, rather than 146.45 - the neff when using the overall DEFF for country estimates for all countries simultaneously). However this is a more realistic approach when computing CI for countries. This has almost no effect on the magnitude of the CI.

  5 The effective net sample size for the total is not the sum of the different countries neff because it is weighted and adjusted to the Higher Education systems size.

  6 CI varies depending on the variance of particular percentages, and it is extremely important in our survey because our percentages usually are far of this 50/50 equilibrium. Therefore, the CI tends to be lower.

  7 X² (4) = 9.959, p = .041, V = .264

  8 Content in parentheses was added to the OECD definition in order to facilitate the understanding of the terms by the respondents.

  9 X²(4) = 8.991, p = .061, V = .256

10 X² (4) = 2.704, p = .609); x²(4) = 7.081, p = .132; and x²(4) = 4.511, p = .341, respectively.

11 X² (4) = 12.673, p = .013, V = .296

12 X² (2) = 8.991, p = .011, V = .384

13 X² (1) = 12.212, p < .001, V = .333

14 X² (1) = 11.558, p = .001, V = .327

15 X² (8) = 16.035, p = .042, V = .24

16 X² (4) = 9.851, p = .043, V = .266

17 X² (3) = 12.034, p = .007, V = .335

18 F(4,143) = 2.821 p = .027, ηp2 = .073

19 p = .011, d = .73

20 F (4,143) = 6.204 p < .001, ηp2 = .148

21 p = .04, d = -.88

22 p < .001, d = -1.21

23 p = .096, d = -.72

24 F (4, 51.34) = 4.642, p = .003

25 p = .016, d = .73

26 p = .011, d = 1.07

27 F (4, 52.998) = 5.436, p = .001

28 p = .002, d = -.94

29 p = .003, d = -.97

30 t (55.86) = 2.947, p = .005; and t (108) = 2.097, p = .038, respectively.

31 t (95.31) = -2.290, p = .024

32 t (89.97) = -2.185, p = .032

33 X² (4) = 10.037, p = .04, V = .272

34 F (4,144) = 2.266, p = .065, ηp2 = .059

35 p = .105, d = .92

36 F (4, 57.602) = 4.481, p = .003

37 p = .006, d = -1.05

38 p = .094, d = -0.67

39 p = .104, d = -0.57

40 p = .042, d = -0.99

41 F (4, 143) = 3.403, p = .011, ηp2 = .087

42 p = .016, d = .79

43 p = .052, d = .70

44 t (112) = -3.440, p = .001

45 t (106) = 2.032, p = .045

46 t (104) = 2.642, p = .009

47 X² (1) = 5.705, p = .017, V = .238

48 X² (1) = 6.004, p = .014, V = .250

49 X² (1) = 4.417, p = .036, V = .216

50 X² (1) = 6.486, p = .011, V = .264

51 X² (3) = 11.539, p = .009, V = .327

52 ‘National’ as used here can also refer to the regional level, depending on the national distribution of competences.

53 All the text in grey (e.g. name of sections) is shown only for informative purposes and is not visible in the online version of the survey. All the response options marked with an asterisk (*) refer to exclusive answers. This means that in multiple-choice questions, if the item marked with * is selected, other items cannot be simultaneously selected.

54 Content in parentheses was added by the research team to OCDE definition in order to facilitate the understanding of the terms.