Design and development of a digital transformation plan for a Spanish university based on an EDR process

Diseño y desarrollo de un plan de transformación digital para una universidad española basada en un proceso EDR

Virginia Viñoles Cosentino*
UJI, Castellón, España
https://orcid.org/0000-0002-9048-5827 | vvinoles@uji.es

Francesc M. Esteve-Mon
UJI, Castellón, España
https://orcid.org/0000-0003-4884-1485 | festeve@uji.es

Jordi Adell-Segura
UJI, Castellón, España
https://orcid.org/0000-0003-4369-6485 | jordi@uji.es

Recibido: 19/03/2025 Aceptado: 11/04/2025

Abstract

The increasing digitisation, coupled with concurrent societal changes at all levels, requires higher education institutions to respond and adapt in a timely manner. While actions and initiatives have been implemented, the rate of these changes necessitates a strategic and evidence-based approach that engages institutions and relevant stakeholders, particularly in regard to the development of digital competence among teaching staff, which is a crucial element. However, there is still a lack of evidence in the academic literature concerning the development of institutional approaches to digital transformation. This study presents a case study of a Spanish university, where the Educational Design Research (EDR) methodology was employed to design a digital transformation proposal with a particular emphasis on the development of digital competence in teaching. A comprehensive description of the process is provided, including the various iterations and validations conducted during the process, based on a sample of approximately 700 participants involved in the different stages. Among the aspects highlighted is the importance of approaching these changes from an institutional, multi-level perspective, involving different actors. To conclude, a set of valuable design principles for higher education institutions are outlined.

Keywords
Higher education; digital transformation; digital competence in teaching; EDR; DigCompEdu.

Resumen

La creciente digitalización, unida a los profundos cambios sociales en todos los niveles, exige que las instituciones de enseñanza superior respondan y se adapten oportunamente. Aunque se han puesto en marcha diversas acciones e iniciativas, el ritmo de estos cambios requiere un enfoque estratégico y basado en evidencias que involucre a las instituciones y a participantes relevantes, especialmente en lo que respecta uno de los temas cruciales, como es el desarrollo de la competencia digital entre el personal docente. Sin embargo, en la literatura académica siguen existiendo lagunas respecto al desarrollo de enfoques institucionales para la digitalización basados en evidencia. Este estudio presenta un estudio de caso de una universidad española, donde se empleó la metodología de investigación basada en el diseño educativo para diseñar una propuesta de transformación digital con especial énfasis en el desarrollo de la competencia digital docente. Se ofrece una descripción exhaustiva del proceso, incluyendo las distintas iteraciones y validaciones llevadas a cabo durante el mismo, a partir de una muestra de aproximadamente 700 participantes implicados en las distintas etapas. Entre los aspectos destacados se encuentra la importancia de abordar estos cambios desde una perspectiva institucional y multinivel, involucrando a diferentes actores. Para concluir, se esboza un conjunto de valiosos principios de diseño para las instituciones de enseñanza superior.

Palabras clave
Enseñanza superior; transformación digital; competencia digital docente; EDR; DigCompEdu.

1. Introduction

Higher education institutions (HEI) around the world are facing the challenge of digital transformation, a situation that has been exacerbated by the COVID-19 pandemic (Garcez et al., 2022). While this event has catalysed the processes of digitisation, it has also revealed the inherent complexity of this process, highlighting the need for in-depth reflection and the development of new competences and strategies at all levels of the institution (Castañeda et al., 2023b). A crucial aspect of this transformation is to strengthen the digital skills of teachers, who are key actors in direct interaction with students (Garcez et al., 2022). In this sense, the development of Digital Competence in Teaching (DCT) emerges as one of the main challenges for HEI (Basilotta et al., 2022; Fernández-Batanero et al., 2020). DCT involves not only the acquisition of knowledge and skills for teachers' personal use of technology, but also its use from a working and pedagogical perspective that promotes digital learning environments that enable students' development (Basilotta et al., 2022; Nascimbeni et al., 2019). Responding to this need, policies, frameworks, and competency models specifically for teaching have been developed over the past decades. One of the most widespread frameworks in the professional field of educators at the European level is the DigCompEdu (Redecker & Punie, 2017). This framework considers that DCT involves the development of teachers in six areas: (1) professional engagement, (2) digital content, (3) teaching and learning, (4) assessment and feedback, (5) student empowerment, and (6) the development of students' digital competence.

However, despite the policies and institutional initiatives implemented so far there are still significant gaps in DCT of academic staff. The evidence indicates that initiatives which have concentrated exclusively on teacher training are not sufficient in themselves (Castañeda et al., 2023b; Seifu, 2020).

The necessity of more comprehensive approaches which consider the institution, its strategy and context, and which also encompass the input of the academics, decision-makers, and support staff, has been identified as a key aspect (Castañeda et al., 2023b; Inamorato et al., 2023).

Authors as Esteve-Mon et al. (2023), have identified a model of key elements for digital transformation processes in higher education, grouped into six areas: (1) Infrastructure: refers to the necessary technological and physical conditions; (2) Policy: aims to promote clear strategies for leadership, coordination, and institutional policies; (3) Training content: involves detecting the training needs of different profiles and delimiting key content; (4) Training strategies: support the implementation of different actions; (5) Stimulation strategies: involve the different members that make up the institution; and (6) Effective communication for creating awareness.

This integration needs to be accompanied by strategic planning and evidence-based decision-making, underpinned by research, to ensure that institutions are adequately prepared to navigate and succeed in these complex transformation processes (García-Peñalvo, 2021). Notwithstanding their importance, we found insufficient evidence of these digital transformation processes systematically collected and documented in the scientific literature (Viñoles-Cosentino et al, 2022b). Notwithstanding their importance, we found insufficient evidence of these digital transformation processes systematically collected and documented in the scientific literature (Viñoles-Cosentino et al., 2022b). While certain aspects, such as the adoption of specific educational technologies or changes in teaching practices, are occasionally reported, more complex institutional dynamics—such as governance structures, strategic planning, cultural shifts, or professional development ecosystems—tend to be underexplored or addressed in a fragmented manner (Basilotta et al., 2022). This gap highlights the need for research that takes a comprehensive institutional perspective, capable of capturing the interplay between organizational, pedagogical, and technological dimensions of transformation (Basilotta et al., 2022; Castañeda et al., 2023b).

In light of the aforementioned context, this research proposes the design, development and evaluation of a digital transformation plan for a Spanish university. It focuses on the training and development of the necessary digital competence of the university community, and specifically of its teaching staff, through a systematic and iterative process, which is described below.

2. Method

2.1. Aim and research questions

The aim of this article is to present the design of an institutional proposal for the development of DCT in Higher Education and to provide a series of evidence-based design principles extracted as a result of the process.

The research questions that have guided this process are the following:

•RQ1. Does the prototype of the institutional proposal for the promotion of DCT in higher education meet the quality criterion of relevance?

•RQ2. Does the prototype proposal meet the quality criteria of consistency and practicality in a local context?

•RQ3. Does the prototype proposal meet the quality criterion of effectiveness in a local context?

•RQ4. Does the institutional proposal meet the quality criteria in an international context?

2.2. Context and participants

This research was conducted between 2020 and 2023 at a Spanish public on-site university operating a single campus. It employs 2,400 staff, of which 1,700 are academic staff, and has around 16,000 students. During the research period, the university implemented a digitalisation plan with the aim of boosting the digital transformation of the university in its areas of initial and lifelong learning, research, knowledge creation, and outreach and social action. One of the central axes is the development of the DCT, which includes activities such as its continuous assessment and the design of training itineraries to promote it (Universitat Jaume I, 2020). From this axis arises the present research that will delve into the strategies to be implemented by the University to develop the DCT.

Due to the characteristics of the open and participatory methodology, different representatives involved in the topic were included throughout the process: (1) university lecturers from different areas of knowledge and different professional categories (n=659); (2) managers, responsible for university policies in the field of training, research and internationalisation (n=8); and (3) Experts in teacher training and technology (EdTech experts), responsible for the digital training plans of the teaching staff (n=23). The number of participants and their typology was chosen deliberately (purposive sampling), following the criteria of intensity sampling and key informants (Plomp & Nieveen, 2009; Tessmer, 1993), and the following section details its distribution.

2.3. Research design

This study followed the educational design research (EDR) methodology, an educational variant of design-based research (DBR) (van den Akker et al., 2006). This research methodology is characterised as a type of systematic study that allows for the design, development, and evaluation of educational interventions with the specific intention of providing an answer or solution to a complex problem (Plomp & Nieveen, 2009). The result is a continuous process of improvement that is based on scientific principles and enables the challenges encountered in daily practice to be linked to research (De Benito & Salinas, 2016). Moreover, upon completion of the research process, this methodology is characterised by the proposal of design principles, which permit the extrapolation of certain elements of this research practice to other contexts.

This type of EDR study are structured in three phases (Plomp & Nieveen, 2009; Romero-Ariza, 2014): (a) a first phase of preliminary research, in which the tasks derived from the needs analysis, problem description and literature review are carried out; (b) a second phase of development and prototyping, in which the various prototypes are developed, revised and improved; (c) and a final evaluation phase, which involves the assessment of the final “product” at a more general level (broader impact), and the deployment of the design principles.

The data collection process typically employs a combination of qualitative and quantitative methods across the three phases. This approach is designed to achieve what is known as multiple validity (Johnson & Christensen, 2017). Four quality criteria are commonly used: (1) relevance, which refers to content validity; (2) consistency, which corresponds to construct validity; (3) practicality or feasibility; and (4) effectiveness or impact. These four criteria are typically associated with a specific phase of the process (Plomp & Nieven, 2009).

The application of the methodology for educational design research, as detailed in this study, is elucidated in the following section and illustrated in Figure 1.

2.3.1. Preliminary research

During the preliminary research stage, the research problem was conceptualised, and the context was analysed. For this purpose, a systematic literature review was conducted of all the most up-to-date scientific literature available on the development of DCT at university level (Viñoles-Cosentino et al., 2022b). In addition, a quantitative and qualitative analysis was conducted to assess the level of DCT among university teaching staff (n=558), based on the DigCompEdu framework (Esteve-Mon et al., 2022). The qualitative analysis was conducted through interviews with various stakeholders, including lecturers (n=9), EdTech experts (n=3), and university managers (n=3) (Viñoles-Cosentino et al., 2021). The results of this phase have been published in previous articles, so that in this paper they are only summarised in a synthetic way, without representing the main focus of the study, but allowing us to understand the global nature of the research, and the way to evaluate the relevance criterion.

2.3.2. Prototyping phase

This was followed by the prototyping phase. Based on the aspects that emerged from this stage, the first prototype was designed. To design the structure of the prototype, the model for the integration of educational technology in Higher Education (Esteve-Mon et al., 2023), described above, was used as a basis, taking three main dimensions to design the prototype: policy, training, and communication. This prototype was first evaluated by the research team, and after minor adjustments it was presented (I1) to three different groups: EdTech experts (n=7), a group of lecturers (n=6) and a group of university managers (n=3).

After the review, the second iteration (I2) was conducted, in this iteration the communication and training aspects were deepened. A series of educational and institutional materials have been developed to introduce the DCT, focusing on key areas such as motivation, communication and alignment with university policies and strategic initiatives. In addition, this content functions as a guide for the categorisation of training in DCT, which allows the teaching staff to choose their development itineraries. This content was validated with EdTech experts (n=5), with lecturers (n=13) and with managers (n=2), through one-to-one evaluation walkthrough sessions. In these two iterations, mostly consistency and practicality criteria were validated. In the third iteration (I3) the evaluation was conducted with several groups of lecturers using try-out sessions to validate the quality criteria, with emphasis on practicality (n=13).

2.3.3. Assessment phase

After refinement of the most relevant aspects that emerged from the last iteration, the final evaluation phase began. First, the quality criterion of local effectiveness was evaluated. For this purpose, a field trial was conducted open to university participants in which lecturers (n=60) participated. The results were evaluated through content analysis and course evaluation results. As a result of this stage, a first version of the design principles was produced. In order to validate and receive feedback from an international perspective, an expert appraisal with international experts in digital education (n=8) was conducted.

2.4. Instruments

Following the principles of the EDR methodology, different data collection instruments were combined in order to evaluate the quality criteria of the prototype (Johnson & Christensen, 2017):

•a) Systematic review (SLR): A systematic review of the current state of the art in DCT at the University was conducted to perform the conceptual delimitation and extract principles for designing the prototypes (Ferreira González et al., 2011). To this end, a search was conducted in the Scopus databases for the years 2015-2020, using the PRISMA method (Urrútia y Bonfill, 2010). A total of twenty articles were analysed in depth (Viñoles-Cosentino et al., 2022b).

•b) DCT Questionnaire (QDCT): A self-perception questionnaire on the level of DCT based on the DigCompEdu framework was administered to teaching staff. A Likert scale test 1-5, with twenty items (Esteve-Mon et al., 2022).

•c) Preliminary interview (PI): Semi-structured in-depth interviews (Corbin & Strauss, 2015) were conducted via video calls with teaching, Edtech experts and management staff at the university (Viñoles-Cosentino et al., 2021).

•d) Focus group (FG): In iteration 1, 90-minute focus group sessions were conducted with lecturers, EdTech experts and managers to validate the consistency of the prototype, i.e. whether the design was logical and coherent (Plomp & Nieveen, 2009), and addressed the policy, training and communication dimensions identified in model for the integration of educational technology in Higher Education (Esteve-Mon et al., 2023).

•e) Walkthrough (WT): In iteration 2, once the prototype was sufficiently detailed the research team conducted a thorough and comprehensive review of the entire prototype with users and stakeholders (one-to-one) collecting data through interviews, lasting 30-60 minutes each, and delving deeper into the training and communication dimensions of the model (Esteve-Mon et al., 2023).

•f) Try-out (TO): In iteration 3, the prototype was evaluated with a small group of users, who tried the prototype in a setting different from their regular usage environment. Data was collected through interviews and observation, reviewing the elements of practicality (Plomp & Nieveen, 2009).

•g) Field trial (FT): In the assessment phase, the stakeholder group used the final prototype in practice in a real environment. The effectiveness of the intervention was evaluated, through testing and evaluation of use.

•h) Expert appraisal (EA): Several expert evaluations were conducted with international EdTech experts to present the prototype and collect their opinions and views. Each of the quality criteria was assessed, with particular emphasis on whether the characteristics of the prototype at the policy, communication and training levels met the criteria of local and widespread effectiveness (Viñoles-Cosentino et al., 2023)

Below is a table summarising the methods used at each stage, the users involved, and the criteria assessed (table 1).

In terms of data analysis, the quantitative results (QDCT) were analysed using the SPSS (v. 25) statistical software. For the qualitative analysis of the interviews and the focus group, the videos were coded according to a deductive approach through concepts or codebooks (Saldaña, 2015), using the qualitative analysis program MAXQDA (v. 2018).

3. Results

From the perspective of the EDR methodology, each phase of the process provides important results that are worth sharing. Therefore, to offer an orderly and understandable account, we will report the results of each phase of the design and evaluation process of the institutional proposal for the promotion of DCT in higher education.

3.1. Preliminary phase

In the first phase of preliminary research, the study problem was defined, and the context of the university was analysed. In order to conduct the conceptual delimitation and the design of the prototypes, the most relevant scientific literature on the development of DCT was researched (Viñoles-Cosentino et al., 2022b). Among the most noteworthy conclusions, an absence of evidence-based strategies was detected, as very few publications were identified that met this requirement. The majority of proposals for enhancing DCT in academic institutions were isolated initiatives, lacking an institutional strategy to facilitate implementation and support for lecturers. Another important aspect that emerged is the difficulty of developing digital competences with merely expository courses, and therefore the need for training with constructivist and collaborative approaches, considering the context in which the lecturer is immersed.

Concurrently, an evaluation of the context was conducted to define the problem and analyse the context, and for that purpose, the level of the DCT of the academic staff was assessed, as seen in figure 2. Overall, lecturers perceive themselves as having a medium-high level of professional engagement, particularly in communication, collaboration, and personal and professional organisation skills. However, the teachers did not perceive their competence in the creation of digital resources and the use of digital technologies for teaching and assessment of learning to be as high. Furthermore, the areas in which the teachers rated their competence as lowest were those related to the digital empowerment of learners.

Subsequently, interviews were conducted with lecturers, EdTech experts and management staff of the university (Viñoles-Cosentino et al., 2021). In line with the results of the questionnaire, the interviews showed that lecturers are interested in and require training to assist them in the development of their DCT. It was expressed that this training should not only encompass instrumental aspects and the utilisation of tools, but also prioritise didactic and pedagogical aspects, as well as the empowerment of students.

With regard to training preferences, the aspects that stood out most were the following: (a) availability of training at different levels adapted to the participants, (b) content relevant to the specific field of knowledge to which they belong, (c) content accessible for self-training on a permanent basis, (e) opportunities for interaction with other lecturers and exchange of good practices, (f) space for suggestions and training needs of the teaching staff.

3.2. Prototyping phase

3.2.1. Design and evaluation of iteration 1 (I1)

Based on the results presented in the previous phase, and taking as reference the model for the integration of educational technology in higher education (Esteve-Mon et al., 2023), the prototype proposal is structured in 3 main dimensions with a set of proposals to promote DCT (table 2):

Policy level: This dimension includes strategies related to the creation and sustainability of institutional plans and policies.

Training level: This level considers those aspects relevant to training strategies that build on current knowledge of higher education training and professional development. In relation to the training contents, the areas that make up the CDD are considered, as well as aspects related to its importance, usability and self-regulation.

Communication Level: This level encompasses those key actions for effective communication and dissemination of the strategy to the whole community throughout the process.

For its evaluation, a screening was conducted by the research team and minor adjustments were made. Then the prototype was developed and the quality criteria -focus on consistency and practicality- were evaluated with the stakeholders. Among the most outstanding results were the following, adding literal phrases from the interviewees to illustrate such aspects.

a) Policy level

At policy level, the aspects that were most valued were related to the integration of the DigCompEdu framework and its alignment with the digital strategy of the institution. In particular, the managers highlighted its usefulness as a guide and reference in the university's digitisation strategy, especially during the critical period of the Covid-19 pandemic. According to the managers "As an institution, it has been good to have a reference framework to respond in the short term - the health crisis - but also to have a global vision in the long term. In fact, other universities and regional administrations have asked us to take our university as a reference and want to do something based on our experience".

b) Training level

With regard to the training level, we have two main approaches, on the one hand at the level of content, where a greater variety of topics and training per level was valued by the teachers. On the other hand, regarding the methodology, the flexibility provided by having open training courses available for access and the possibility of self-training was valued. As the lecturers mentioned, "In my case it is useful for me to be able to review the material and make progress at weekends", "It is very good to know that the materials are open to access at any time", and “It’s important to have different difficulty levels. I have attended some courses that for me were like an unknown language and if they did not show me a picture, I didn't know what they were talking about.”

One of the aspects highlighted by managers was the importance of interest and motivation in learning. “Anyone who has a negative attitude beforehand is going to be blocked and no matter how many hours of training we give them, we will only lower their professional self-efficacy. That's what I've seen happen with lecturers”.

c) Communication level

At the communication level, the need to strengthen a common metalanguage to understand what we mean when we talk about DCT was emphasised. In addition, having spaces for the exchange of experiences, such as forums, is considered valuable for the development of DCT. Here are some quotes from the lecturers: “I think is important to train in the appropriate language when we talk about DCT and what it entails”, "A forum... that generates interaction and communication with people who are working on the same issues as us, that we can contribute and that they can contribute to us", or "It's very interesting to know what others are doing. you compare yourself; I am not doing this; it would be interesting for me".

The aspects considered for the second iteration are the following: (a) work on aspects that help to build the language around the DCT, (b) focus on motivational aspects and professional interest in academics, (c) consider and integrate contextual aspects for the development of the competence.

3.2.2. Design and evaluation of iteration 2 (I2)

According to the previous results, during the I2, emphasis was placed on the levels of training and communication since they were identified as crucial points to strengthen (table 3).

This iteration was evaluated based on a walkthrough with the different agents (lecturers, managers, and experts), validating the criterion of consistency. The results are summarised below, with the addition of literal sentences.

a) Policy level

One of the aspects that were assessed at the Policy level has to do with aligning the university's digitisation strategies with regard to how a framework is applied and how DCT is developed in this particular university, including the contextualisation of training and access to its own resources. According to the EdTech experts, “On the situated approach, the section where the different units of the university are made explicit and what role they have within the development of DCT. We see the situated approach, because we can see which units can help you develop your DCT”. For their part, the lecturers emphasise that “People will understand that this applies to any lecturer nowadays, but that specifically in our university there are already initiatives that you can access... maybe there are people who didn't know about it”, “It refers to our own resources, I think it is very interesting because it puts you in your context, within the university and that is very good, it is not a course that is out there, but that you have it at home”.

b) Training level

The introductory content was rated highly in terms of its capacity to raise awareness of what DCT is, to understand the DigCompEdu framework and its importance at the teaching and institutional level. Its value for professional development was also highlighted, especially in terms of self-diagnosis and identification of areas for improvement. According to the lecturers, “Self-diagnosis is useful, is not enough to put into practice, but it is enough to see what further training is needed”. According to the EdTech experts, “It is important when we want to work with DCT to start with something introductory that orients people who want to undergo training”, “For example, the self-assessment also helps us realize that there were other aspects that perhaps we haven't developed during the pandemic, but that we can continue to work on. So, the proposal is necessary and relevant”.

c) Communication level

The promotion of a common language that guides training and allows managers, EdTech experts and lecturers to be on the same page was highly valued. The importance of establishing a framework and ensuring its dissemination among staff was highlighted. EdTech experts mention, "I find the proposal super relevant, not only because of the need to master and enhance DCT but also because of the frameworks themselves. We take them for granted when we develop training and digitisation plans. And it’s probably the first thing that lecturers and students need to understand because otherwise, they won't know what we're talking about", "We'll help them improve their DCT, but they won't know why we're using these competency lists with numbers and names. This course seems very relevant to me. It seems crucial within all the plans we have in progress". The managers also highlight, "It is relevant and necessary; we have 1500 teachers, and after the pandemic, it became very clear that it's no longer just a need, but an obligation to get up to speed with DCT. Understanding frameworks is essential to explore the possibilities. The proposal is highly relevant for people to familiarize themselves with the framework and the opportunities it provides, understanding the options, what you know, and what you need to know." According to the lecturers, “If all the teachers at the UJI have this joint language and the institution also has it, then we can understand each other and you offer me something and I know what I want and I know how to ask you for it”.

3.2.3. Design and evaluation of iteration 3 (I3)

For this iteration, the introductory course was developed in the university's LMS using the aspects collected in previous iterations. The designed material includes a course that has a duration of approximately 10 hours, structured in three modules: (a) Teaching in the digital world; (b) Digital Competence in Teaching; (c) Itineraries for the development of DCT. It was assessed with a group of EdTech experts and lecturers, who evaluated this aspect of the proposal through a try-out in “non-real” conditions. In this iteration, the criterion of practicality has been validated.

a) Training level: Teaching activities

Among the aspects that have resulted from the practicality of the course, the value of the constructivist methodology and collaborative activities stand out. According to the EdTech experts: “Regarding the constructivist or collaborative approach, you can see in the suggested activities that most of them are activities that you have to share with other colleagues and that will also make you change your perspectives and change your work”.

b) Training level: Teaching content

In terms of content, the framework of competences is highlighted as one of the strong points. Both its usefulness for understanding DCT and its potential as a guide and continuous professional development are highlighted.

According to the lecturers, “Until this course, I did not know the whole framework of competences that has been established as a reference, nor had I considered what aspects could be improved, or where I was doing things well”, "I have found it useful to have a reference framework that is simple to structure and with specific content. Taking it as a reference has helped me to conduct a self-assessment process that has allowed me to know what I need to improve and what strategies I need to follow to achieve this". The EdTech experts point out that “I think it is sufficient considering that it is an introductory course, and it is also very well accompanied by additional materials for people who want to go a little further”.

3.3. Assessment phase

3.3.1. Evaluation in a local context

After completing the iterations and refining the most relevant aspects that emerged, an evaluation was conducted in a real context. Due to the characteristics of the prototype and its different levels, the local evaluation was focused on a field trial open to participants from the university, in which 60 professors participated, conducting the proposed activities, and delivering an action plan for improvement.

a) Policy level

It was emphasised that there is a need for the university to promote the lecturers DCT development, as well as to reinforce actions to raise awareness of the DigCompEdu framework. According to the lecturers, "It is a very useful and necessary framework, each institution should detect its situation and see what resources it lacks to address it and generate those resources to help the university community”, “It is a bit curious that most of us were not aware of the DigCompEdu framework, so perhaps a greater involvement of the university is needed to raise awareness of this framework, to promote its development and to encourage teachers to improve their skills in this area”.

It was also mentioned that “It is very necessary due to the current circumstances of the digital scenarios in which we live, in which students live, and all the possibilities and challenges that it offers us. It is important to implement it, also because there is a lack of training in this area”.

b) Training level

The structure of the DigCompEdu Framework is particularly appreciated both for self-assessment and for reflecting on and facilitating competence development. According to the lecturers, “Until now, learning had been somewhat disorganised, without knowing what the content of each competence was, and without reflecting on possible actions for improvement. Consequently, the experience has been more than satisfactory as it has provided me with a frame of reference on which to build my digital competence as a teacher”, “I have found it most useful to be able to self-assess myself to see where I need to improve”. “The course helps me to put myself in context. It also makes me more aware of the importance of DCT not only in our daily lives, but also in the development of our profession”.

c) Communication level

Aspects related to communication at different levels are highlighted.

The value of disseminating the university's strategy in relation to resources and development opportunities is highlighted, as is the value of facilitating peer-to-peer exchange through both established forums and proposed activities. According to the lecturers, “I have found the reflection forums particularly useful, both in terms of sharing with colleagues and self-reflection, something we rarely do”, “It has helped me to see the whole overview of all the DCT that we need to consider and lots of links to university resources that we can use depending on what we need, it has been very good for me”.

As an example of the concrete impact of peer-to-peer work outside the course: “what had started as a conversation between two lecturers quickly turned into a colloquium in which more colleagues from the department joined in who were just as eager to share their tips and experiences. It has been very useful because I have been able to learn from people with more years of teaching experience. I especially liked the tips on how to control the class with changes in intonation or dynamic activities designed to get the students’ attention. I am also very happy to have been able to share the resources I use. To highlight: we have decided to create a resource bank shared by all lecturers in the department”.

3.3.2. Evaluation in a local context

After the iterations and the evaluation of the prototype in the local context, the final prototype was structured as follows (table 4).

Then, a broader evaluation of the prototype was conducted with a group of experts from an international context. Through an expert appraisal the quality criteria of the prototype were evaluated.

In terms of practicality, they noted the need for context-specific adaptations. However, they agreed that this was covered by the proposed design principles. They considered it appropriate to address the complexity of the process by using strategic actions at macro/micro levels: "Working at different levels helps us to take concrete action, maintaining complexity but moving forward without oversimplification”. They also stressed the importance of linking institutional strategy and teaching practice, and the importance of making strategies stick and not remain on paper: "The link between macro-level strategy and teaching practice is crucial. There are institutional dynamics beyond a static policy that we need to consider to make things happen".

In relation to building a meta-language around DCT, the use of an existing and extended framework (DigCompEdu) as a basis was welcomed: "it is important to work on the construction of the meta-language, but sometimes a lot of time is wasted in trying to define and agree on it, which ends up in fruitless discussions. Having an extended framework that can be adapted makes the work easier”.

4. Discussion

As previously stated, the objective of this article is to present a case study that illustrates the implementation of the EDR methodology for conceptualizing and evaluating an institutional digital transformation proposal, with a particular focus on the development of DCT in higher education. Additionally, it aims to present the design principles that have been extracted as a result of the process.

In relation to RQ1, on the criterion of the relevance of the institutional proposal to promote DCT in higher education, the analysis of the literature revealed an absence of evidence-based strategies, the proposals analysed were in the form of isolated actions with a limited traditional approach (Viñoles et al., 2022b). As other research has indicated, the intricate nature of this subject requires the formulation of institutional proposals that encompass interventions at diverse levels, including the macro, meso, and micro levels, and involve different actors, transcending a constrained focus on lecturer training (Castañeda et al., 2023b; Lyndgaard et al., 2024). In terms of the level of DCT among university's teaching staff, it was confirmed that the proposal is relevant. There is still a gap, particularly in relation to aspects such as the pedagogical use of technology, student empowerment and the development of their digital competence. Although the pandemic has accelerated the integration of technology, it is important to distinguish between mere use of technology and the actual DCT. Previous research shows that the efforts of universities have focused mainly on infrastructure aspects, but there is still a gap in terms of training and incentives for teachers, which translates into a poor DCT (Castañeda et al., 2023a; Mercader & Gairín, 2020; Oliveira et al., 2021).

When answering RQ2 regarding the fulfilment of the criteria of consistency and practicality of the prototype, the following aspects were highlighted. The evaluations indicated that the prototype was designed in a logical and coherent manner, both in terms of content and methodology. Furthermore, it was deemed to be a realistic and applicable solution within the context of the university for which it was designed.

In the context of digital transformation in education, several frameworks have been developed to guide institutions and educators. Among the most influential for the development of the CDD at the European level is the DigCompEdu framework (Redecker & Punie, 2017), which supports the development of educators' digital competence by providing an embeddable structure to institutional digital transformation plans (Caena & Redecker, 2019; Castañeda et al., 2023b). The integration of a framework such as DigCompEdu as a point of reference and for aligning the digital strategy of the institution was highlighted as a particularly important aspect. Frameworks have proven to be useful in implementation process, as they promote a common working logic and understanding, that structures and helps to permeate strategies (Basilotta et al., 2022; Caena & Redecker, 2019; Chugh et al., 2023). In this case, the DigCompEdu framework works in a cross-cutting way, firstly helping to develop and structure the DCT development strategy in a coherent way, and then as a metalanguage to facilitate communication at different levels about what DCT is and what it means for different stakeholders (Castañeda et al., 2023b). In this regard, previous research (Century & Cassata, 2016; Chugh et al., 2023; Spillane et al., 2002) argues that frameworks can also help to link institutional strategy to teaching practice by providing a structure for interpreting messages and experiences.

Similarly, at the level of the training proposal, the importance of working on different topics and levels was highlighted, as well as the flexibility provided by having different materials available for self-study. In particular, the collaborative and constructivist nature of the course was highlighted, as well as the value of the content for understanding DCT. This is in line with previous research that points to the importance of professional collaboration and collective knowledge construction as a key aspect of teachers' professional development (Keller & Hrastinski, 2007; Sánchez-Tarazaga et al., 2022; Spillane et al., 2002; Winberg, 2008).

At the communication level, the need to reinforce a common metalanguage that allows understanding what is being talked about when referring to DCT, was emphasised. Promoting sense making is essential for teachers to effectively understand and integrate the competences (Chugh et al., 2023; Stollman et al., 2020). As mentioned above, the DigCompEdu framework was a key element as it facilitates this process by presenting DCT in a structured way with definitions and levels that are easily identifiable (Inamorato et al., 2023; Spillane et al., 2002). Furthermore, the existence of dissemination spaces where this information can be accessed and shared with colleagues is a fundamental aspect (Esteve-Mon et al., 2023).

In relation to RQ3, on the effectiveness of the proposal in the local context, the participating lecturers were able to complete the training and develop an improvement proposal for their teaching practice. It was emphasised that the course laid the foundations for the development of the DCT by promoting its understanding and scope, and by raising awareness of the DigCompEdu framework and its potential application to teaching practice. Additionally, having a self-assessment tool was an important aspect to promote teachers' reflection. Consistent with previous research, these aspects were useful in deepening their understanding of competence, as well as in promoting their continuing professional development (Inamorato et al., 2023; Winberg, 2008).

At the communication level: The proposal's effectiveness in promoting the exchange of experiences among peers and in building common strategies was emphasized. Previous research highlights the positive impact of creating collaborative spaces where educators can share practice, encouraging the exchange of ideas and knowledge as a motivator for improving teaching practice (Spillane et al., 2002). In this sense collaborative spaces among lecturers are fundamental for the digital transformation of the university. In relation to peer-to-peer collaboration, forums and web platforms were promoted that make real faculty practices visible, favoring peer-to-peer learning and the situated appropriation of frameworks such as DigCompEdu (Viñoles-Cosentino et al., 2022a). In relation to specific institutional collaborative and participatory processes, the UJI Digital Plan integrated representatives from different areas in the strategic design, promoting a shared and sustainable vision (Universitat Jaume I, 2020). Likewise, the EDR methodology applied facilitated collaboration between lecturers, experts and managers, generating contextualized solutions from the continuous dialogue between theory and practice (Plomp & Nieveen, 2009).

Similarly, an understanding of the digital strategy of the university has been essential, not only for comprehension, but also to facilitate active participation and alignment (Stollman et al., 2020). Another important factor was the perception of organisational support and encouragement to work on their competences (Stollman et al., 2020). Finally, the link between the concepts and strategies addressed in the course and their application in the teaching practice was identified as a significant factor in facilitating the enactment of DCT (Chugh et al., 2023).

Lastly, the RQ4 on the validation of the proposal in an international context was positively assessed. It was underlined that the main elements of the proposal cover all areas and are transferable to other contexts. The necessity for context-specific adaptations due to organisational factors (e.g. between centralised and decentralised universities) was mentioned as an aspect to consider. Among the most appreciated elements of the prototype, the multi-level approach and the different axes of the proposal were highlighted, which make it possible to maintain the complexity of the process while working on concrete aspects (Century & Cassata, 2016; Dearing & Kee, 2012; Esteve-Mon et al., 2023).

In the following section, the main elements of the proposal are collected and synthesised in the form of design principles, a fundamental element in EDR research (Plomp & Nieeven, 2009).

5. Conclusion

The systematic process presented in this article offers a proposal intended to be relevant not only to this local context but can also be extrapolated to similar contexts. The main elements extracted from the final prototype are presented in the table 5.

At the institutional level, it is essential that the strategy is aligned and integrated with the university's vision, involving all levels in the process. The training offer must be adapted to the needs of lecturers, ensuring that it is accessible, flexible and in line with their level of development. In addition, training resources should be organised and centralised so that they are easy to find, identify and access. Another important aspect is the promotion of collaborative spaces between lecturers, encouraging the exchange of experiences and peer learning. It is also crucial to build a metalanguage that not only facilitates the incorporation of key concepts at the lecturer level but also enables coherent and effective communication at the institutional level.

Furthermore, from an individual perspective, it is key to motivate and raise awareness among lecturers of the importance of developing DCT, facilitating their self-assessment and enabling them to make informed decisions about their training and continuing professional development.

Finally, it is important to bring institutional policies and actions closer to the reality of teaching practice, adapting, and accompanying changes to facilitate their effective transfer to the classroom.

In the practical implications of this research, a centralised teacher digital competence resource platform, published as TDC Corner¹ has been developed as part of the DigiUGov project involving 8 university institutions from 4 countries. Currently undergoing international validation, this platform aims to provide a scalable and transferable model for teacher digital competence development, with future research focusing on its impact, usability and institutional adoption.

As is the case with all research, this study is not without its limitations. Despite the systematic methodology employed, exemplified by the EDR approach, which lends rigour and validity to the process, the integration of research and management in real scenarios introduces complexities in time management and decision making, such as those introduced by the pandemic caused by the Covid-19. However, it is important to emphasise the intrinsic value that this aspect confers to the study, as it is not a mere theoretical construct or an isolated experience. Rather, it is an integral and tangible application in a higher education institution. Due to the mentioned conditions, it was not always possible to have a large number of participants at all levels and in all phases. Nevertheless, this represents a potential avenue for future research, where the validation and transferability of design principles to broader contexts can be further investigated.

A strategy as the one presented in this study can contribute not only to digital transformation, but also to the development of institutional capacities that favour the integration of future transformations, ensuring that educational institutions are prepared for a present and future of challenges and dizzying changes, as the integration of artificial intelligence in all academic and organisational processes. Finally, we would also like to emphasise that integrate common reference frameworks for digital competence, as is the case of DigCompEdu, not only allows the systematisation of the process, but also facilitates interoperability and transfer at national and regional level, relevant issues in the European educational agenda.

Project funding

The present research has been carried out with the support of Erasmus+ Project (101082408) DigiUGov: Digitalization meets University Governance.

References

Basilotta-Gómez-Pablos, Verónica, Matarranz, Matía, Casado-Aranda, Luis-Alberto & Otto Ana. (2022). Teachers’ digital competencies in higher education: A systematic literature review. International Journal of Educational Technology in Higher Education, 19(1), 8. https://doi.org/10.1186/s41239-021-00312-8

Caena, Francesca, & Redecker, Christine 2019 Aligning teacher competence frameworks to 21st century challenges: The case for the European Digital Competence. Framework for Educators (DigCompEdu) European Journal of Education, 54(3), 114 https://doi.org/10.1111/ejed.12345.

Castañeda Linda Esteve-Mon Francesc Marc & Adell, Jordi (2023a) La universidad digital: Aproximación a un análisis crítico de los planes de transformación digital de las universidades públicas españolas. Profesorado. Revista de Currículum y Formación del Profesorado, 27(1), 175-198. https://doi.org/10.30827/profesorado.v27i1.23870

Castañeda, Linda, Viñoles, Virginia, Concannon, Fiona, Pedersen, Annette, Al-Hmiedat, Paula & Lobato, Natalia. (2023b). The CUTE CANVAS: developing a design tool for planning strategic actions for institutional of digital competencies. Journal of Decision Systems, 33(1), 30–52. https://doi.org/10.1080/12460125.2023.2167274

Century, Jeanne & Cassata, Amy. (2016) Implementation Research: Finding Common Ground on What, How, Why, Where, and Who. Review of Research in Education, 40(1), 169-215 https://doi.org/10.3102/0091732X16665332

Chugh, Ritesh, Turnbull, Darren, Cowling, Michael A., et al. (2023). Implementing educational technology in Higher Education Institutions: A review of technologies, stakeholder perceptions, frameworks and metrics. Education and Information Technologies, 28, 16403-16429. https://doi.org/10.1007/s10639-023-11846-x

Corbin, Juliet & Strauss, Anselm. (2015). Basics of qualitative research: Techniques and procedures for developing grounded theory (4th ed.). SAGE Publications, Inc.

Dearing, James W. & Kee, Kerk F. (2012). Historical roots of dissemination science. In R. Brownson, G. Colditz, & E. Proctor (Eds.), Dissemination and implementation research in health: Translating science to practice, 55–71. Oxford.

de Benito Crosetti, Bárbara & Salinas Ibáñez, José María (2016). La Investigación Basada en Diseño en Tecnología Educativa. RiiTE Revista interuniversitaria de investigación en Tecnología Educativa. https://doi.org/10.6018/riite2016/260631

Esteve-Mon, Francesc Marc, Llopis-Nebot, María Ángeles, Viñoles-Cosentino, Virginia & Adell-Segura, Jordi (2022). Digital Teaching Competence of University Teachers: Levels and Teaching Typologies. iJET International Journal of Emerging Technologies in Learning, 17(13), pp. 200–216. https://doi.org/10.3991/ijet.v17i13.24345.

Esteve-Mon, Francesc Marc, Postigo-Fuentes, Ana Yara & Castañeda, Linda (2023). A strategic approach of the crucial elements for the implementation of digital tools and processes in higher education. Higher Education Quarterly, 77(3), 558–573. https://doi.org/10.1111/hequ.12411.

Fernández-Batanero, José María, Montenegro-Rueda, Marta, Fernández-Cerero, José & García-Martínez, Inmaculada. (2020). Digital competences for teacher professional development. Systematic review. European Journal of Teacher Education, 18(1), 1-19. https://doi.org/10.1080/02619768.2020.1827389

Ferreira González, Ignacio, Urrútia, Gerard & Alonso-Coello, Pablo. (2011). Revisiones sistemáticas y metaanálisis: Bases conceptuales e interpretación. Revista Española de Cardiología, 64(8), 688-96. https://doi.org/10.1016/j.recesp.2011.03.029

Garcez, Ana, Silva, Ricardo & Franco, Mário. (2022). Digital transformation shaping structural pillars for academic entrepreneurship: A framework proposal and research agenda. Education and Information Technologies, 27(2), 1159–1182. https://doi.org/10.1007/s10639-021-10638-5

García-Peñalvo, Francisco José. (2021). Transformación digital en las universidades: Implicaciones de la pandemia de la COVID-19. Education in the Knowledge Society (EKS), 22. https://doi.org/10.14201/eks.25465

Inamorato dos Santos, Andreia, Chinkes, Ernesto, Carvalho, Marco Antonio Garcia, Solórzano, Claudia Marina Vicario & Marroni, Lilian Saldanha. (2023). The digital competence of academics in higher education: Is the glass half empty or half full? International Journal of Educational Technology in Higher Education, 20(9). https://doi.org/10.1186/s41239-022-00376-0.

Johnson, Burke & Christensen, Larry. (2017). Educational research: Quantitative, qualitative, and mixed approaches (6th ed.). SAGE Publications.

Keller, Christina & Hrastinski, Stefan. (2007). Towards digitally literate university teachers. Nordic Journal of Digital Literacy, 4(2), 104-115. https://doi.org/10.18261/ISSN1891-943X-2009-02-04

Lyndgaard, Sibley F., Storey, Rebecca, & Kanfer, Ruth. (2024). Technological support for lifelong learning: The application of a multilevel, person-centric framework. Journal of Vocational Behavior, 153. https://doi.org/10.1016/j.jvb.2024.104027

Mercader, Cristina & Gairín, Joaquín. (2020). University teachers’ perception of barriers to the use of digital technologies: the importance of academic discipline. International Journal of Educational Technology in Higher Education, 17(4), 1-14. https://doi.org/10.1186/s41239-020-0182-x

Nascimbeni, Fabio, Alonso, Joaquin, Sanz, Olga & Burgos, Daniel. (2019). Read, watch, do: Developing digital competence for university educators. En D. Burgos, M. Cimitile, P. Ducange, R. Pecori, P. Picerno, P. Raviolo, & C. M. Stracke (Eds.), Higher education learning methodologies and technologies online (pp. 80-93). Springer. https://doi.org/10.1007/978-3-030-31284-8_7

Oliveira, Gabriella, Grenha Teixeira, Jorge, Torres, Ana & Morais, Carla. (2021). An exploratory study on the emergency remote education experience of higher education students and teachers during the COVID-19 pandemic. British Journal of Educational Technology, 1, 21-40. https://doi.org/10.1111/bjet.13112

Plomp, Tjeerd & Nieveen, Nienke. (2009). An introduction to educational design research. Netherlands Institute for curriculum development (SLO).

Redecker, Christine & Punie Yves. (2017). European framework for the digital competence of educators: DigCompEdu. Publications Office of the European Union. https://doi.org/10.2760/159770

Romero-Ariza, Marta. (2014). Uniendo investigación, política y práctica educativas: DBR, desafíos y oportunidades. Revista Internacional de Investigación en Educación, 7(14), 159-176. https://doi.org/10.11144/Javeriana.M7-14.UIPP

Saldaña, Johnny. (2015). The coding manual for qualitative researchers (3rd ed.). SAGE Publications Ltd.

Sánchez-Tarazaga, Lucía, Ruiz-Bernardo, Paola, Viñoles-Cosentino, Virginia, & Esteve-Mon, Francesc Marc. (2022). University teaching induction programmes: A systematic literature review. Professional Development in Education, 50(2), 279–295. https://doi.org/10.1080/19415257.2022.2147577

Seifu, K. (2020). Determinants of information and communication technology integration in teaching-learning process at Aksum University. Cogent Education, 7(1), 1-23. https://doi.org/10.1080/2331186X.2020.1824577

Spillane, James P., Reiser, Brian J., & Reimer, Todd. (2002). Policy Implementation and Cognition: Reframing and Refocusing Implementation Research. Review of Educational Research, 72(3), 387-431. https://doi.org/10.3102/00346543072003387

Stollman, Saskia, Meirink, Jacobiene, Westenberg, Meirink, & Van Driel, Jan. (2020). Teachers’ learning and sense-making processes in the context of an innovation: a two year follow-up study. Professional Development in Education, 48(5), 718–733. https://doi.org/10.1080/19415257.2020.1744683

Tessmer, Martin. (1993). Planning and conducting formative evaluations: Improving the quality of education and training. Kogan Page.

Universitat Jaume I. (2020). Plan UJI digital. Plan para el impulso digital de la Universitat Jaume I 2020-23. Aprobado en el Consejo de Gobierno núm. 3, de 12 de marzo de 2020.

Urrútia, Gerard & Bonfill, Xavier. (2010). Declaración PRISMA: Una propuesta para mejorar la publicación de revisiones sistemáticas y metaanálisis. Medicina Clínica, 135(11), 507-511. https://doi.org/10.1016/j.medcli.2010.01.015

van den Akker, Jan, Gravemeijer, Koeno, McKenney, Susan, & Nieveen, Nienke. (Eds.). (2006). Educational design research. Routledge.

Viñoles-Cosentino, Virginia, Esteve-Mon, Francesc Marc, Llopis-Nebot, María Ángeles, & Adell-Segura, Jordi. (2021). Validación de una plataforma de evaluación formativa de la competencia digital docente en tiempos de COVID-19. Revista Iberoamericana de Educación a Distancia, 24(2), 87-106. https://doi.org/10.5944/ried.24.2.29102.

Viñoles-Cosentino, Virginia, Esteve-Mon, Francesc Marc, Llopis-Nebot, María Ángeles & Sánchez-Caballé, Anna. (2022a). Diseño de una propuesta formativa para desarrollar la competencia digital docente en el ámbito universitario. Jornadas Internacionales Universitarias de Tecnología Educativa JUTE 2022, Valencia, Universidad de Valencia. https://hdl.handle.net/10550/85264.

Viñoles-Cosentino, Virginia, Sánchez-Caballé, Anna, & Esteve-Mon, Francesc Marc. (2022b). Desarrollo de la competencia digital docente en contextos universitarios. Una revisión sistemática. REICE. Revista Iberoamericana sobre Calidad, Eficacia y Cambio en Educación, 20(2), 11-28. https://doi.org/10.15366/reice2022.20.2.001.

Viñoles-Cosentino, Virginia, Sánchez-Caballé, Anna, Esteve-Mon, Francesc Marc & Llopis-Nebot, María Ángeles. (2023). Institutional approach to the development of Digital Competence in Teaching: An international validation with experts. In Emerging Researchers Conference (ERC), European Conference on Educational Research ECER 2023. University of Glasgow, Scotland. https://eera-ecer.de/ecer-programmes/conference/28/contribution/55917.

Winberg, Christine. (2008). Teaching engineering/engineering teaching: interdisciplinary collaboration and the construction of academic identities. Teaching in Higher Education, 13(3), 353–367. https://doi.org/10.1080/13562510802045394

Biography

Virginia Viñoles-Cosentino: PhD student in Education at the Universitat Jaume I, she develops her research and teaching career in the field of educational technology. She works on projects of digitization of education in Europe and Latin America. She is part of the research staff of the Universitat Jaume I, and participates in the GREAT group (Research Group on Teaching, Learning and Technology). Her research interests focus on: digital transformation in education, development of digital competence, educational policy, higher education.

Francesc M. Esteve Mon: International PhD in Educational Technology from the Universitat Rovira i Virgili. He is a professor in the Department of Pedagogy and delegate of the rector for the Teaching Transformation, Communication and Cabinet Management of the Universitat Jaume I (UJI). He coordinates the research group on Teaching, Learning and Technology (GREAT) and the educational innovation group EDUBOT of the UJI.

Jordi Adell-Segura (Retired): He has been director of the Center for Education and New Technologies (CENT) and Full Professor in the Department of Pedagogy at the Universitat Jaume I, Spain. D. in Philosophy and Educational Sciences, and coordinator of the research group on Teaching, Learning and Technology (GREAT). His teaching and research focused on educational technology. He has participated in different panels and advisory committees in Technology and Education.

¹ TDC corner: https://digiugov.com/tdc-corner.