Publicado: 2020-06-30

Gamificación en educación: Estado del arte

Universidade da Coruña
Universidade da Coruña
gamification education gamification for education technology state of the art gamificación educación gamificación en educación tecnología estado del arte

Resumo

Este artículo es un viaje a través de los avances y ventajas de la gamificación en educación, basado en la investigación y los estudios científicos identificados como los más relevantes. Se centra en recopilar los artículos, textos, libros y conferencias más significativos y en hacer que este estado del arte esté lo más actualizado posible.

1. Introduction

The term 'gamification' did not join the general vocabulary until 2010, and only a year later did it become a reliable trend, derived from the belief in its potential to encourage motivation, behavioral changes, friendly competition and collaboration in different contexts. It has been defined in several ways: "the use of game design elements in contexts that are not games”, “the phenomenon of creating game experience” or “the process of making activities more game-like".

In education, commitment has been identified as a valuable indicator of academic achievement because students committed to their work persist in their academic activities despite the obstacles. Gamification, therefore, is a promising technique that improves the participation of students with a positive impact in the classroom, combined with effective pedagogy.

In other areas, it has been considered a success, so it is pertinent to make a thorough check of its effectiveness to involve students and improve their academic learning outcomes, since there are also some scientific studies on gamification that show that their effect on motivation or participation is lower than the expectations created by advertising.

This study focuses on reviewing the articles published on gamification in education and concludes that, so far, it is not sufficiently 'research sustainable' to position ourselves on the side that gamification serves as motivation and improvement for learning in education. However it seems essential to carry out out more research tests on the subject in the near future, since there are indications that it could be a good practice.

2. State of the art

2.1. Context

A key sector for studies on gamification is education. Motivation is one of the important predictors of students’ academic achievement that influences the effort and time a student devotes to learning. Since games – known to generate motivation and commitment – are remarkably popular, the proposal to incorporate the mechanics and principles of games to motivate student is attractive.

In the field of education, several attempts were made to gamify learning activities in order to improve learning behaviors, encourage student participation or promote active help in the evaluation of peers, as well as to involve students in learning through the use of tutorial or digital tools.

Although gamification is relatively new as a subject of academic study, there are studies that report evidence of academic trap behavior, claiming that students would be able to cheat when they considered tasks irrelevant. Therefore, gamification processes must be carefully designed.

Rayner, a videogame designer who is responsible for creating attractive e-learning software, states that setting goals and objectives, feedback and reward are the key components in developing gamified learning environments.

Goals and objectives should be divided into achievable steps that students can achieve by using their skills, and designers should keep students within their flow channel.

Nicholson presents a theoretical framework centered on the user that recommends:

  • design considering benefits for the user;
  • focus on introducing fun elements instead of punctuation elements;
  • offer the possibility of choosing different ways to achieve objectives; and
  • integrate game mechanics into the non-game settings.

2.2. Educational levels of the studies

As regards education, most of the studies that can be found on gamification are at university level, and some at K-12 level. At university level, there are almost no studies for graduates, and at K-12 level, the proportion is small, but similar between primary, secondary and intermediate studies.

This may be due to the fact that it is easier for university instructors to experiment with the use of gamification in their own courses, as they have better technical support or greater computer skills. On the other hand, studies with different demographic groups are beneficial, since it is not possible to generalize based on results of a study carried out to one demographic group or another.

2.3. Gamified subjects

There is a wide range of academic subjects, including studies with unspecific subjects, where gamified activities do not depend on a topic and focus on the platform, the learning environment, the measures or personalities of the students.

The vast majority of gamification studies relate to Computer Science and Information Technology. This may be due to the same reason, in other words, the fact that there are more university-level studies, and the ease for instructors to experiment in their own courses. On the contrary, gamification experiments aimed at activities related to humanities and social sciences disciplines are limited.

2.4. Activities to which gamification is directed

Formal learning usually involves a mix of instructional activities and support materials – lectures, tutorials, assignments, projects, labs, exercises, class discussions and teamwork. The studies that are conducted the most focus on gamification of the courses as a whole, but also on online courses – that usually require greater motivation – and regular courses with web-based learning support.

2.5. Studies on game elements and combinations thereof

According to Deterding et al. (2011), gamification is the use of game design elements in contexts that are not games. In turn, the game design elements used for the creation of gamification scenarios can be divided into three categories: dynamics, mechanics and components. Dynamics represents the highest conceptual level in a gamified system. It includes restrictions, emotions, narrative, progression and relationships. Mechanics are a set of rules that dictate the result of interactions within the system, while dynamics are the responses of users to the collections of those mechanics. The mechanics of the game refers to the elements that move the action forward. They include challenges, opportunity, competence, cooperation, feedback, acquisition of resources and rewards. The components are at the basic level of the gamification process and cover the specific cases of mechanics and dynamics. They include achievements, avatars, badges, collections, unlocking content, gifts, leaderboards, levels, points, virtual goods, etc. For example, points (components) offer rewards (mechanics) and create a sense of progression (dynamics). However, we note that the gamification terminology is not yet resolved and there are several variations of the expressions introduced above.

Most educational gamification studies are based on the assumption that gamification in education consists mainly of incorporating an appropriate combination of game elements into learning activities. In all the works reviewed, gamification studies include a subset of the following game elements: points, badges, levels, leaderboards and progress bars. However, gamification is more than mapping elements of the game into the existing learning content. It should offer stronger ways to motivate students, rather than simply being a stream of extrinsic motivators. In addition to reward and feedback mechanism, gamified systems should provide safe places where apprentices can gain experience without being judged or punished for failures, using approaches similar to the online learning environments, so students can improve their algorithmic skills by practicing with interactive exercises.

2.6. Types of studies

Although most empirical studies still examine the impact of gamification on the commitment, performance, participation or retention of students, they have been broadening and deepening their focus. A growing number of articles are exploring a range of learning and behavior outcomes that include:

  • knowledge acquisition results;
  • perceptive results;
  • behavior results;
  • commitment results; and
  • motivational results.

The category of motivational results refers to concepts derived from the motivational principles of games, such as explicit objectives, rules, a system of feedback and voluntary participation. The motivation is demonstrated by the choice of an individual to participate in an activity and the intensity of effort or persistence in such activity. Game design has been adopted as an approach to make non-game related activities more enjoyable and motivating. While gamification strives to increase motivation, motivation is not a unitary phenomenon: different people can have different types and amounts of motivation, which can be determined by the activity they perform.

2.7. Types of goals

The two main categories for grouping the studies according to study objectives are ‘student-centered’ and ‘platform-centered’. Most of the works that broaden and differentiate previous research on the effect of gamification on students fall into the first category. This category includes four subcategories that group the studies as described below:

  • Behavioral and cognitive results: focus on the cognitive and behavioral effects caused by gamification.
  • Categories of students: focus on the students.
  • Perception of students: focus on the students' perception of different mechanics and game principles.
  • Measures: focus on the measures used to evaluate results

The second category includes articles that study the effect of incorporating game elements or selected game principles into specific learning platforms or that experiment with conventional game elements assigning them new roles.

3. A review of the most recent articles

3.1. 2016

Paule, as a teacher, has introduced role-play elements in a traditional translation class to increase Roman cultural content in an intermediate Latin course on Vergil’s Aeneid at Earlham College. He concluded that adding some gamified courses in the classic study plans would be beneficial.

Enache & Pop propose an application (for mobile and desktop devices) that uses the principle of gamification in education to create an attractive and fun way of teaching.

Schoenenberger et al. experience and evaluate gamification by applying it to a biology class using digital learning by presenting the BioTourney gaming platform. This study is carried out in grades 10 and 11 of a secondary school (ages 15-17) with a limited number of students (N = 32) with no prior knowledge of the game.

Featherstone reviews the literature on game-based machine learning in the fields of psychology, education and video games. Specifically in the field of education, it has been shown that an optimal learning state can be encouraged and maintained using a variety of reward-based techniques.

Becker & Nicholson analyze the potential and limits of gamification in learning and use in the classroom.

Barreto et al. present the results of a systematic mapping that explored good practices and lessons learned from gamification in education.

Papers Selection Amount
ACM Digital Library 6
IEEE Xplore 52
Google Scholar 69
Scopus 41
Total 168
After applying both criteria 52
Final Selection (after reading and applying both criteria) 20
Table 1.Studies by search library and final selection after criteria enforcement. Source: Johnson (2016).

Johnson provides a comprehensive overview of existing research (see Table 1 ) introducing gamification techniques (game theory or game-based mechanics), which have gained popularity in fields such as marketing, business, military, and education, for use in sports learning, improving performance.

Zarzycka-Piskorz describes the influence of specific elements on players, presenting motivational aspects of participation and investigating elements that could be responsible for increasing motivation and participation in grammar learning.

Sombrio et al. discuss creating a gamified accessible learning object that introduces the principles of design thinking by reviewing the literature focusing on accessibility, gamification, and design thinking.

3.2. 2017

Dichev & Dicheva analyze articles to improve the understanding and provide a realistic picture of the progress of gamification in education.

John et al. propose an approach to gamification based on gamification learning activities rather than learning content with the aim of optimizing the time spent by teachers to gamify a given course.

Shpakova et al. investigate gamification as a means of interaction amongst teachers and students with the intention of discussing the sustainable impact of gamification on knowledge management. These authors further investigate gamification as a means for teachers to interact with each other, supporting flexibility, facilitating transparency, and improving confidence, as well as displaying skills and competencies, and generating requirements for new competencies.

O’Sullivan, in her thesis, presents Teach2Learn – a website that brings together the problem of labeling natural text samples using gamification in education as an incentive. Students tag text samples from a dataset by teaching machine learning algorithms, and in return they learn the algorithm by unlocking researcher-written lessons, incentivizing researchers and students ans thus helping them achieve their goals. The tests showed that 27.7% of users got involved with the content and tagged enough samples to unlock lessons.

Dicheva explores various questions related to learning and behavior change, examiningthe state of the art and gamification challenges in education.

Kanagasingam evaluates gamification as a way of transfering knowledge to students in the educational systems.

Gradvohl discusses the concept of gamification and present results of the application of this concept to operating systems, computer organization, and architecture courses. The results show that student engagement correlates with class performance.

Ortega-Arranz et al. present a review of the use of MOOCs to support learning with the aim of analyzing unexplored implementations in the field, showing that they are still at an early stage.

Arawjo et al. present Reduct, an educational game that incorporates a new understanding approach to teach basic programming concepts (functions, Booleans, equality, conditionals, and mapping functions on sets).

3.3. 2018

Delello et al. show how gamification in education using digital badges has emerged as a means to motivate and reward student learning. They recruited 90 higher education students from four different disciplines. The implementation was successful, but the planning required to promote the student’s awareness of utility proved high.

Chandran et al. explore the use of gamification in mental health as well as its potential in the learning of psychiatric residents combined with digital andragogy. It turns out to be an efficient teaching model for undergraduate and graduate medical education.

Majuri et al. catalog 128 empirical research works in the field of the gamification of education and learning. The flowchart presented in Figure 1 below describes the process implemented in this study.

Figure 1.A flowchart describing the workflow of the study. Source: Majuri et al. (2018).

Dicheva et al. (2018a, 2018b) developed a Course Gamification Platform (OneUp) to overcome the challenges of learning through gamification. They also describe the functionality of the platform, in other words, the support for integrating game design elements into learning activities, creating dynamic problems, and visualizing student performance and process.

Hassan et al. investigate the question “What can we determine about gamification in education at Hogwarts and what implications can it have?” through a method of textual analysis and codified education in the first novel of the Harry Potter saga.

Tan Ai Lin et al. exposed a group of university students from the Malaysian Public University to Kahoot! (game-based learning platform) for one semester. According to the 51 respondents, the approach proved beneficial to motivate and reinforce learning in theoretical and practical aspects.

Glowacki et al. analyze the use of gamification in higher education institutions in Poland and Ukraine Kahoot as one of the gamification technologies to teach English for Specific Purposes (ESP) to students at a Technical University. The results show that in Poland there is considerable experience in the use of computer games in higher education, while in Ukraine these technologies have been hardly implemented.

Yildirim and Cirak-Kurt carried out a meta-analysis study to determine the effect of gamification on student motivation following PRISMA guidelines. They examined 1554 publications and collected 22 experimental studies conducted between 2010 and 2017, estimating 27 standardized effect size values. The effect size value revealed that gamification has a very small negative effect on student motivation (g = 0.098).

Soboleva et al. describe the types of school activities which allow the use of gamification for modeling, for the development of school children's research skills, and for the training of modern tools to solve theoretical and experimental problems. The results can be used to change the techniques and methods of teaching models at school, to improve conventional training programs, university programs of pedagogical departments and faculties, and to develop and improve specific educational programs in school subjects with the aim of improving social integration, vocational guidance and education.

Inocencio reviews the literature on gamification for education and provides a map of the most frequent scales used to measure results, identifying commitment, motivation, self-efficacy and cognitive absorption.

Torrealba and Camacho analyze the perceptions of students regarding methodological strategies that support the communication of learning digital games through Moodle and Kahoot.

Porubän proposes an IT education method, “OpenLab”, to create interactive multimedia applications with natural user interfaces based on continuous challenges and competition among university students in an open laboratory.

Resch et al. discuss a couple of case studies demonstrating the usefulness of Unreal Engine 4 for collaborative research purposes in the field of gamified automation in immersive media for education and research.

Chan et al. carried out an experiment to evaluate the impact of providing two gamification functions in an online learning system on intrinsic motivation.

Rutledge et al. explore how gamification works within the context of medical education using self-determination theory as an explanatory mechanism for greater participation and motivation. They further discuss common obstacles and challenges in implementing gamification.

Harvey reports on teaching and learning within the College of Social Sciences, Arts and Humanities, College of Science and Engineering, and College of Medicine, Biological Sciences and Psychology of the University of Leicester by exploring challenges, practices, and instructors' and instructional designers on teaching through digital games. A workshop was held at the School of Media, Communication and Sociology in May 2016 to understand the emerging challenges and opportunities related to game-based teaching practice, innovations in assessment, available and necessary resources for teaching, and curricula based on games. Among the participants were staff of the University of Leicester who use digital games for learning and assessment, as well as digital game studios and design instructors from across the UK.

Al-Towirgi et al. performed gamification in a data structure course to teach students the main topics of data structure. The effectiveness of gamification in learning assesses student performance and participation. The results show a positive on learning performance and increased student engagement.

Zhu et al. analyze the long-term effects of gamification on reading literacy of primary school students. The study involved 20 students in Hong Kong (14 women and 15 men) and included questionnaires and semi-structured interviews. The results revealed lasting positive impacts on children’s reading literacy.

3.4. 2019

Legaki et al. present a gamified learning platform called F-LauReLxp using three gamification strategies (Horses for Courses, Metrics to Escape and JudgeIt) to help educate statistical forecasting, forecast accuracy and critical judgment. The authors studied the learning performance of 261 students from a university and an MBA course. The results show a positive impact.

Omerovic & Karahasanovic summarize the motivation and objectives of the CyberWiseKids project designed by SINTEFF in 2015 and revised in 2016, with the aim of enabling children and young people to correctly balance online opportunities and their risks.

Dicheva et al. analyze possible motivators and demotivators of the learning activities that will be used to guide the selection of appropriate gamification strategies.

Zhao investigates the effects of a point system on students’ perception of their motivation and performance in class. The analysis of the students’ motivation levels at the end of the semester was carried out using a one-way multivariate analysis of variance (MANOVA).

Vetushinskiy & Zhukov describe a gamification system with the aim of introducing it to 15 schools in Russia in the autumn of 2019 (Moscow, Republic of Tatarstan, Kaluga, Lipetsk, Novgorod and Nizhny Novgorod Regions). It is a system developed by Sberbank Gamification Lab within the framework from the Sberbank Charitable Foundation and consists of a cyclical gamification system based on six elements: avatars, missions, points, levels, achievements and stickers.

Begicheva et al. deal with online learning in higher education using different technologies, including gamification.

Raitskaya and Tikhonova consider the concept of gamification, its dissemination in higher education research and its relevance today.

Alsubhi et al. investigate the most common challenges and problems facing students in an e-learning environment. They used a qualitative approach to identify and confirm challenges (interviews with experts and interview questions derived from previous work).

Denden et al. present a smart gamification project with Moodle (iMoodle), which analyzes learning to provide tools for teachers to control the learning process. The result obtained was 90% accuracy of iMoodle.

Cassel et al. present in a poster the design of empirical studies on the effect of various game element combinations on the motivation and participation of students in Computer Science courses.

Frost et al. contribute to the existing literature on the gamification impact of a Learning Management System (LMS) for an introductory course required in information systems with the aim of increasing a series of desirable results: interest, motivation, learning and satisfaction.

Arjo et al. propose gamification as an approach to improve learning about traditional food among the young population in Minangkabau, using gamification elements in a mobile questionnaire. A hundred and sixty young people between the ages of 17 and 25 were randomly assigned to participate in the study: a survey to measure the level of commitment. The results indicated that gamification provided positive effects by improving user participation.

Balci et al. compared the effectiveness of badges with the leaderboards on student motivation and academic performance. Eighty-eight online course students took part, randomly assigned into four groups: badges only, tables only, badges and tables, and a control (no badges or tables). Academic performance was measured by their grades, and their motivation was measured through surveys. The results did not improve performance compared to the control group; however, the motivation survey revealed positive results.

Godwod analyzes the use of massively multiplayer online (MMO) games for the improvement of the educational experience compared to non-massive games, claiming that the use of game and gamification in education cannot be generalized for a specific group. He concludes that the use of the MMO approach holds the most promise for educational improvement.

Dal Bon et al. present a gamification experience in relation to learning Organic Chemistry through a team competition held within the Agricultural Science and Technology university course at the Università degli Studi di Padova in Italy. The proposal was made on the Moodle platform.

Yildirim and Sen conducted a study to provide a scientific response while exhibiting the effect of gamification on student performance using the meta-analysis method (2010-2016). Gamification adds 7.2% of academic performance according to 45 experimental results with 3.487 students from different countries.

Toda et al. (2019a) provide a set of game item recommendations, based on user preferences, that could be used by teachers and instructors to gamify learning activities by analyzing data from a survey of 733 people (589 men and 164 women) and collecting information on their user preferences regarding the elements of the game. The results obtained four basic elements: objectives, levels, progress and choice. Toda et al. (2019b) advance the state of the art of gamification for education and provide a practical way to help implement this type of system, by applying gamification elements in a learning system through the Design Sprint method. This method is intended to guide designers and developers through the process of developing educational gamification design. Toda et al. (2019c) extracted the game elements from the state of the art and evaluated the resulting taxonomy through 21 gamification and education experts (21 game elements). The general taxonomy was well accepted by experts, who suggested expanding it to include other elements such as narratives and storytelling.

Latham et al. investigated the development of a desktop computer application to replace video, using 700 life science students per year at one of the world’s top 100 universities. The interactive game presents the same information as the video and provides the students’ self-evaluation. The results suggest that the new product is better, but it cannot guarantee that results are statistically significant.

Bovermann et al. compare motivation, relationship, effort, and perceived value in two different gamified learning environments from a distance learning science degree program. In the individualized gamified learning environment, students were rewarded with individual recognition and social status; in the collective gamified environment, students collaborated as a team in a competition and were scored with credit points. A hundred and one students participated in the study, and the data was collected through an online questionnaire. The results showed a significant difference between both groups and further demonstrated that the collective design was better perceived than the individual one.

Bertoncelli Júnior et al. describe the development and application of an educational game, using a learning experience in the Systems and Network Management curricular unit of a Computer Engineering course. The game is of the strategy type and develops a board adapted for mobile devices with Unreal Engine 4. The impact of the game was evaluated through observation and questionnaires.

Figure 2.Scene overview. Source: Bertoncelli Júnior et al. (2019).

Barreal and Jannes describe the development of a gamification tool for the course “Statistics Applied to the Tourism Sector” of the Tourism Degree in order to present a dynamic and attractive training process for the student, but from a cross-sectional perspective that allows its implementation in similar subjects of higher education. The application encourages student interest in future professional experiences.

Chong exploited gamified education for physiotherapy students and investigated other factors such as class design and mechanics that affected the gamified experience. Six case studies were carried out with the theme of neurological physiotherapy, transforming paper cases into multimedia cases created with iSpring suite 8.1. using simulated, real or animated clients. After applying gamification using classification, scoring and prioritization tables, 10 students participated in two focus groups, and 32 students responded to a survey to share their experiences and perceptions. The results show that gamified education is perceived as motivating.

Smiderle et al. incorporated gamification with points, rankings and badges in a programming learning environment with the aim of studying the effects of these elements on the behavior and commitment of extroverted students. The experimental evaluation was carried out for four months with 40 university students from first-year programming courses, where half used the gamified version and the other half the environment without gamification. The authors found evidence that gamification affected the commitment of introverted and extroverted participants differently. Introverts used the gamified version with the most correct solutions, being more committed to earning points and badges, and had a higher-ranking display compared to extroverts.

3.5. 2020

Fernandez Rio et al. explore how gamification can be used in physical education learning and its effects on students and teachers. The study involved 290 students -8aged 6-14 years old) from four Spanish schools and 12 different classes, as well as four teachers from different schools. The intervention program (MarvPE), based on the Marvel universe, lasted 15 weeks (30 sessions of 50 minutes). It followed a mixed methods approach: a questionnaire for older students (n = 161), drawing for younger students (n = 126), discussion groups for teachers (n = 5) and teacher journal (n =2). The data was analyzed in SPSS and evaluated by content analysis and constant comparison, showing a significant increase in student motivation, as well as an increase in teacher workload.

Andreu aims to establish the current state of knowledge on gamification in order to pinpoint future lines of research, identifying representative studies through a systematic search in four databases (EBSCA, SPORTDISCUS, SCOPUS, ISI and GOOGLE SCHOLAR) and excluding data on participant design, study, variables, instruments and results. The sample comprised 22 articles (19 originals and three reviews), and the results show that gamification in higher education motivates students and improves their learning, providing university students with better opportunities to develop commitment skills.

Sprint and Fox explore the use of flipped classooms, lightweight equipment and gamification to enhance CS1 student learning and social experience, expanding past studies by implementing a flipped classroom with gamification based on team study options. The authors designed and implemented a Moodle plugin to reward students with good study habits. To carry out the evaluation, three semesters were considered: without gamification, with gamification, and with gamification linked to incentives. The results showed an early submission of the papers, but unimproved test scores, as well as greater enjoyment and motivation.

Henukh and Guntara investigate the response of students to the use of Kahoot as a gamification tool for learning physics. The research was carried out at SMP Tunas Bangsa, with 16 students from class VII of West Jakarta, and it was evaluated by means of a questionnaire.

Smiderle et al. studied the impact of gamification on student engagement and behavior based on their personality traits. They conducted the experiment for four months with 40 college students from first-year programming courses.

Danka suggests adapting the motivational formative assessment tools used in Massively Multiplayer Online Games (MMORPG) in MOOC courses to enhance their completion rate.

Aries et al. assess the impact of gamified learning on the students’ intentions to become entrepreneurs, as part of the Entrepreneurship or Business Plan courses of BINUS ONLINE Learning. They carried out the study with 400 students during a month using questionnaires. The results were analyzed through multiple linear regression and show that attitude towards behavior (ATB), perceived behavioral (PBS) and subjective norms have a positive impact on the students’ business intentions.

Basal and Kaynak explore the perceptions of English teachers prior to the inclusion of digital badges in an LMS used in classroom courses at university. Seventy-nine future English teachers participated in the study during 14 weeks using a mixed-method design in which data was collected through a questionnaire and open-ended questions. The study shows that participants had positive perceptions on the use of digital credentials as an integral part of the courses.

Glover and Bodzin seek answers to two questions: “What values, attitudes, and beliefs build the non-gambling behaviors of emerging 12th grade health care preprofessionals" and “How do the values, attitudes, and beliefs of the health care preprofessionals of this course align with the important design characteristics of serious games?". The authors conducted semi-structured interviews with students from Health Sciences. The results showed a complex fusion of desire for achievement and affiliation, reinforcing the need for robust student analysis to identify the multifaceted behavioral characteristics of selected students prior to the design and development of serious games.

Bovermann and Bastiaens investigate the relationship between five types of users through six online learning activities used primarily in online distance learning bachelor and master class in educational science using the systematic approach. The study was carried out with 85 students using a questionnaire. It showed an average percentage for all five types.

Anupam et al. present the process of designing a digital game that complements the introductory curricula of quantum mechanics based on three challenges: taking advantage of past student experiences and knowledge of classical mechanics to 'break free' of it; creating an environment that accurately describes the mathematical formulations of quantum mechanics, but in an attractive and playful way; and developing characters that can relate to players without reinforcing gender stereotypes.

4. Conclusion

Currently there are many studies on gamification. Because of this, we have focused on the ones that seemed to us most relevant in recent years, to show that gamification in education should continue to be investigated and the number of people in the experiments increased, since generally they are few and of very specific branches or courses.

From the present review of the state of the art, its conclude that there arent enough arguments to affirm that gamification really works to motivate students and to help them improve their results.

There are many cases of success in studies, such as "Gamification as a tool for enhancing graduate medical education” or “The model for introduction of gamification into e-learning in higher education”. While they may be optimal as characteristics studies and for those students concerned, the samples used are minimal and do not serve to verify the actual overall effectiveness of gamification.

Several studies highlight the fact that gamification is a task that burdens teachers’ work. Therefore, it is relevant to carry out studies that help improve and make this type of work more efficient, for instance in the field of Artificial Intelligence.

Although not enough arguments are considered, gamification is also an interesting and important field of study, and could be very fruitful, in case it is confirmed to improve student outcomes. Therefore, it is essential to invest in research in this field, since education is truly crucial and everything that can support and create improvement is absolutely necessary.

Referências

Alsubhi, M. A., Ashaari, N. S., & Wook, T. S. M. T. (2019, July). The Challenge of Increasing Student Engagement in E Learning Platforms. In Proceedings of the 6th International Conference on Electrical Engineering and Informatics (ICEEI) (pp. 266-271). https://doi.org/10.1109/ICEEI47359.2019.8988908

Al-Towirgi, R. S., Daghestani, L. F., & Ibrahim, L. F. (2018). Increasing Students Engagement in Data Structure Course Using Gamification. International Journal of e-Education, e-Business, e-Management and e-Learning Increasing, 8(4), 193-211. https://doi.org/ 10.17706/ijeeee.2018.8.4.193-211

Andreu, J. M. P. (2020). Una revisión sistemática sobre gamificación, motivación y aprendizaje en universitarios. Teoría de la Educación. Revista Interuniversitaria, 32(1), 73-99. https://doi.org/10.14201/teri.20625

Anupam, A., Gupta, R., Gupta, S., Li, Z., Hong, N., Naeemi, A., & Parvin, N. (2020). Design Challenges for Science Games: The Case of a Quantum Mechanics Game. International Journal of Designs for Learning, 11(1), 1-20. https://doi.org/10.14434/ijdl.v11i1.24264

Arawjo, I., Wang, C. Y., Myers, A. C., Andersen, E., & Guimbretière, F. (2017). Teaching programming with gamified semantics. In Proceedings of the 2017 ACM SIGCHI Conference on Human Factors in Computing Systems (pp. 4911-4923). Association for Computing Machinery. https://doi.org/10.1145/3025453.3025711

Aries, A., Vional, V., Saraswati, L., Wijaya, L., & Ikhsan, R. (2020). Gamification in learning process and its impact on entrepreneurial intention. Management Science Letters, 10(4), 763-768. https://doi.org/10.5267/j.msl.2019.10.021

Arjo, T. R., Yulius, R., & Nasrullah, M. F. A. (2019). Enhancing Learning Engagement on Minangkabau Traditional Food through Gamified Mobile Quiz. Journal of Physics: Conference Series, 1196. https://doi.org/10.1088/1742-6596/1196/1/012027

Astin, A. (1984). Student involvement: A developmental theory for higher education, Journal of college student personnel, 25(4), 297-308.

Attali, Y., & M. Arieli-Attali. (2015). Gamification in assessment: Do points affect test performance? Computers & Education, 83, 57-63. https://doi.org/10.1016/j.compedu.2014.12.012

Bajko, R., Hodson, J., Seaborn, K., Livingstone, P., & Fels, D. (2016). Edugamifying Media Studies: Student Engagement, Enjoyment, and Interest in Two Multimedia and Social Media Undergraduate Classrooms. Information Systems Education Journal, 14(6), 55-72. http://isedj.org/2016-14/ ISSN: 1545-679X

Balci, S., Secaur, J. M., & Morris, B. J. (2019). Badges and Leaderboards: Understanding Their Effectiveness on Academic Performance and Motivation of Online Students. In J. T. Bastiaens (Ed.), Proceedings of EdMedia+ Innovate Learning (pp. 1072-1078). Association for the Advancement of Computing in Education (AACE). https://www.learntechlib.org/primary/p/210264/

Barreal, J., & Jannes, G. (2019). La narrativa como herramienta docente dentro de la gamificación de la estadística en el Grado en Turismo. Digital Education Review, 36, 152-170. https://revistes.ub.edu/index.php/der/article/view/27091/pdf

Barreto, L. de S., Cavaco, I. N., Monteiro, A., Rousy, D., & Silva, C. (2016). Gamification Aspects in Detail: Collectanea of Studies to Renew Traditional Education. Revista Eletrônica Argentina-Brasil de Tecnologias da Informação e da Comunicação, 1(4). http://doi.org/10.5281/zenodo.59479

Başal, A., & Kaynak, N. E. (2020). Perceptions of pre-service English teachers towards the use of digital badges. Innovations in Education and Teaching International, 57(2), 148-162. https://doi.org/10.1080/14703297.2019.1649172

Becker, K., & Nicholson, S. (2016). Gamifying the Classroom: Pros and Cons. ETC Press.

Begicheva, O. L., Beregovskaya, T. A., & Grishaeva, S. A. (2019, April 14-15). Online education in higher education. In A. Nazarov (Ed.), Proceedings of the 1st International Scientific Conference "Modern Management Trends and the Digital Economy: from Regional Development to Global Economic Growth" (698-701). Atlantis Press. https://doi.org/10.2991/mtde-19.2019.142

Bertoncelli Júnior, C., Lopes, R. P., & Júnior, J. A. (2019). Estudo de caso da conceção e aplicação de um jogo educativo. In F. Garrigós Simón, S. Estellés Miguel, I. Lengua Lengua, J. Onofre Montesa, C. M. Dema Pérez, J. V. Oltra Gutiérrez, Y. Narangajavana, & M. J. Verdecho Sáez (Eds.), INNODOCT/18. International Conference on Innovation, Documentation and Education (pp. 1005-1009). Editorial Universitat Politècnica de València.

Blohm, I., & Leimeister, J. M. (2013). Gamification, design of it-based enhancing services for motivational support and behavioural change. Business & Information Systems Engineering, 5(4), 275-278. https://doi.org/10.1007/s12599-013-0273-5

Boticki, I., Wong, L.-H., & Looi, C.-K. (2012) Experiences in implementing and using a technological framework for mobile collaborative learning of mathematics and Chinese. International Journal of Mobile Learning and Organisation, 6(1), 79-93. https://doi.org/10.1504/IJMLO.2012.046884

Bovermann, K., & Bastiaens, T. J. (2020). Towards a motivational design? Connecting gamification user types and online learning activities. Research and Practice in Technology Enhanced Learning, 15(1). https://doi.org/10.1186/s41039-019-0121-4

Bovermann, K., Habla, S., Weidlich, J., & Bastiaens, J. T. (2019). Better Together?–A Case Study Comparison of Individualistic vs. Collectivistic Gamification Design. In J. T. Bastiaens (Ed.), Proceedings of EdMedia+ Innovate Learning (pp. 1079-1088). Association for the Advancement of Computing in Education (AACE). https://www.learntechlib.org/p/210116/

Cassel, L., Dicheva, D., Dichev, C., Guy, B., & Irwin, K. (2019). Student Motivation and Engagement in STEM Courses: Exploring the Potential Impact of Gamification. In Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education (p. 299). Association for Computing Machinery. https://doi.org/10.1145/3304221.3325578

Chan, E., Nah, F. F. H., Liu, Q., & Lu, Z. (2018). Effect of gamification on intrinsic motivation. In F. F .H. Nah, & B. Xiao (Eds.), HCI Business, Government, and Organizations. HCIBGO 2018. Lecture Notes in Computer Science (pp. 445-454). Springer, Cham. https://doi.org/10.1007/978-3-319-91716-0_35

Chandran, S., Prakrithi, S. N., & Kishor, M. (2018). Gamifying education and mental health. Archives of Medicine and Health Sciences, 6(2), 284-289. https://doi.org/10.4103/amhs.amhs_92_18.

Chong, D. Y. K. (2019). Benefits and challenges with gamified multi-media physiotherapy case studies: a mixed method study. Archives of Physiotherapy, 9(1). https://doi.org/10.1186/s40945-019-0059-2

Cohen, A. M. (2011). The gamification of education. The Futurist, 45(5), 16-17.

Connolly, T., Boyle, MacArthur, E., Hainey, T., & Boyle, J. (2012). A systematic literature review of empirical evidence on computer games and serious games. Computers & Education, 59(2), 661-686. https://doi.org/10.1016/j.compedu.2012.03.004

Csikszentmihalyi, M. (1991). Flow: The psychology of optimal experience. Harper Perennial.

Dal Bon, C., Ferranti, C., & Peggion, C. (2019,). Esperienze di gamification all'università con moodle: The Organic Game. In G. Fiorentino & S. Rabellino (Eds.), Atti del MoodleMoot Italia 2019 (pp. 245-250). MediaTouch 2000 & Associazione Italiana Utenti Moodle (AIUM), Università degli Studi di Verona.

Danka, I. (2020). Motivation by gamification: Adapting motivational tools of massively multiplayer online role-playing games (MMORPGs) for peer-to-peer assessment in connectivist massive open online courses (cMOOCs). International Review of Education, 66(1), 75-92. https://doi.org/10.1007/s11159-020-09821-6

Delello, J. A., Hawley, H., McWhorter, R. R., Gipson, C. S., & Deal, B. (2018). Gamifying education: Motivation and the implementation of digital badges for use in higher education. International Journal of Web-Based Learning and Teaching Technologies, 13(4), 17-33. https://doi.org/10.4018/IJWLTT.2018100102

Denden, M., Tlili, A., Essalmi, F., Jemni, M., Chang, M., & Huang, R. (2019). iMoodle: An Intelligent Gamified Moodle to Predict “at-risk” Students Using Learning Analytics Approaches. In A. Tlili, & M. Chang (Eds.), Data Analytics Approaches in Educational Games and Gamification Systems (pp. 113-126). Springer.

Deslis, D., Kosmidis, C. V., & Tenta, E. (2018). Using a Non-educational Mobile Game for Learning in Biology, Geography and Mathematics: Pokémon Go as a Case Study. In M. Tsitouridou, J. A. Diniz, & T. Mikropoulos (Eds.), Technology and Innovation in Learning, Teaching and Education. TECH-EDU 2018. Communications in Computer and Information Science (pp. 388-396). Springer, Cham. https://doi.org/10.1007/978-3-030-20954-4_29

Deterding, S., Dixon, D., Khaled, R., & Nacke, L. (2011). From game design elements to gamefulness: defining gamification. In A. Lugmayr, H. Franssila, C. Safran, & I. Hammouda (Eds.), Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments (pp. 9-15). Association for Computing Machinery. https://doi.org/10.1145/2181037.2181040

Dichev, C., Dicheva, D., Angelova, G. &. Agre, G. (2015). From gamification to gameful desing and gameful experience in learning. Cybernetics and Information Technologies, 14(4), 80-100. https://doi.org/10.1515/cait-2014-0007

Dicheva, D. (2017). Gamification in Education: A Passing Trend or a Genuine Potential?. In B. Rachev & A. Smrikarov (Eds.), Proceedings of the 18th International Conference on Computer Systems and Technologies (p.11). Association for Computing Machinery. https://doi.org/10.1145/3134302.3134305

Dicheva, D., Irwin, K., & Dichev, C. (2018a). OneUp: Supporting Practical and Experimental Gamification of Learning. International Journal of Serious Games, 5(3), 5-21. https://doi.org/10.17083/ijsg.v5i3.236

Dicheva, D., Irwin, K., & Dichev, C. (2018b). Gamifying with OneUp: For Learning, Grades or Fun?. In M. Gentile, M. Allegra, & H. Söbke (Eds.), Games and Learning Alliance. GALA 2018. Lecture Notes in Computer Science International Conference on Games and Learning Alliance (pp. 343-353). Springer, Cham. https://doi.org/10.1007/978-3-030-11548-7_32

Dicheva, D., Irwin, K., & Dichev, C. (2019). The role of activity motivators and demotivators in gamifying learning. International Journal Information Theories and Applications, 26(2), 134-146. http://www.foibg.com/ijita/vol26/ijita26-02-p03.pdf

Dichev, C. & Dicheva, D. (2017). Gamifying education: what is known, what is believed and what remains uncertain: a critical review. International Journal of Educational Technology in Higher Education, 14(9). https://doi.org/10.1186/s41239-017-0042-5

Domínguez, A., Saenz-de Navarrete, J., De-Marcos, L., Fernández-Sanz, L., Pagés, C., & Martinez-Herráiz, J.-J. (2013). Gamifying learning experiences: Practical implications and outcomes. Computers & .Education, 63, 380-392. https://doi.org/10.1016/j.compedu.2012.12.020

Enache, L.-D., & Pop, D.-P. (2016). Gamifying education world of school-a C# application-. Conference Proceedings of »eLearning and Software for Education«, 3(15), 523-528. https://doi/org/10.12753/2066-026X-16-076

Featherstone, M. (2016). Using gamification to enhance self-directed, open learning in higher education. In T. Connolly & L. Boyle (Eds.), Proceedings of the 10th European Conference on Games Based Learning (pp. 824-834). Academic Conferences and Publishing International Ltd. http://shura.shu.ac.uk/12638/

Fernandez-Rio, J., de las Heras, E., González, T., Trillo, V., & Palomares, J. (2020). Gamification and physical education. Viability and preliminary views from students and teachers. Physical Education and Sport Pedagogy, 25(5), 509-524. https://doi.org/10.1080/17408989.2020.1743253

Fitz-Walter, Z., & Tjondronegoro, D. (2011). Exploring the opportunities and challenges of using mobile sensing for gamification. In N. Lane, F. Zhao, & Choudhury, T (Eds.) Proceedings of the UbiComp 11 Workshop on Mobile Sensing: Challenges, Opportunities and Future Directions 2011 (pp. 1-5). Association for Computing Machinery.

Freitas, S. (2006). Learning in immersive worlds, JISC e-Learning and Pedagogy Experts Group Meeting https://pdfs.semanticscholar.org/80e8/5dbfcba4209325fef75646da4ec98df9b848.pdf

Frost, R. D., Matta, V., & MacIvor, E. (2015). Assessing the efficacy of incorporating game dynamics in a learning management system. Journal of Information Systems Education, 26(1), 59-70. https://aisel.aisnet.org/jise/vol26/iss1/6

Gee, J. P. (2003). What Video Games Have to Teach Us about Learning and Literacy. Palgrave Macmillan.

Glover, K., & Bodzin, A. (2020). Learner Analysis to Inform the Design and Development of a Serious Game for Nongaming Female Emerging Health Care Preprofessionals: Qualitative Sample Study. JMIR Serious Games, 8(1). https://doi.org/10.2196/16003

Głowacki, J., Kriukova, Y., & Avshenyuk, N. (2018). Gamification in higher education: experience of Poland and Ukraine. Advanced Education, 10, 105-110. https://doi.org/10.20535/2410-8286.151143

Godwod, P. (2019). Massively Multiplayer Game-based Education. Journal of Modern Science, 40(1), 271-284. https://doi.org/10.13166/jms/108947

Gradvohl, A. L. S. (2017) Boosting the Engagement of Undergraduate Students in the Class Activities through Gamification. In E. Amaral, S. Polydoro, M. A. Garcia de Carvalho, F. Papes, & A. O. Garcia (Eds.), Inovações Curriculares 2017: Desafios do Ensino Superior (pp. 24-28). Biblioteca/Unicamp

Hamari, J., Koivisto, J, & Sarsa, H. (2014). Does gamification work? – A literature review of empirical studies on gamification. In Sprague Jr., R. H. (Ed.), Proceedings of the 47th Annual Hawaii International Conference on System Sciences (pp. 3025-3034). IEEE Computer Society & Conference Publishing Services. https://doi.org/10.1109/HICSS32722.2014

Hamer, J., A. Luxton-Reilly, H. Purchase, & Sheard, J. (2011). Tools for contributing student learning. ACM Inroads, 2(2),. 78-91. https://doi.org/10.1145/1963533.1963553

Hanus, M. D., & Fox, J. (2018)., Assessing the effects of gamification in the classroom: A longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance. Computers & Education, 8, 152-161. https://doi.org/10.1016/j.compedu.2014.08.019

Harvey, A. (2018). Games as Education in the United Kingdom. Journal of Learning and Teaching in Higher Education, 1(1), 83-92. https://journals.le.ac.uk/ojs1/index.php/jlthe/article/view/2539

Hassan, L., Harviainen, T. J., & Hamari, J. (2018). Enter Hogwarts: Lessons on how to gamify education from the wizarding world of Harry Potter. In J. Koivisto & J. Hamari (Eds.), Proceedings of the 2nd International GamiFIN Conference (pp. 30-39). CEUR-WS.

Henukh, A., & Guntara, Y. (2020). Analyzing the response of learners to use kahoot as gamification of learning physics. Gravity: Scientific Journal of Research and Learning Physics, 6(1), 72-76. https://doi.org/10.30870/gravity.v6i1.7108

Ibáñez, M.-B. A., Di-Serio, & C. Delgado-Kloos, (2014). Gamification for engaging computer science students in learning activities: A case study. IEEE Transactions on learning technologies, 7(3), 291-301.

Inocencio, F. (2018). Using Gamification in Education: A Systematic Literature Review. In Proceedings of the 39th International Conference on Information Systems (ICIS): Bridging the Internet of People, Data, and Things (pp. 2098-2114). Association for Information Systems.

John, T., Feldotto, M., Hemsen, P., Klingsieck, K., Kundisch, D., & Langendorf, M. (2017). Towards a Lean Approach to Gamifying Education. In I. Ramos, H. Krcmar, & V. Tuunainen (Eds.), Proceedings of the 25th European Conference on Information Systems (ECIS) (pp. 2970-2979). Association for Information Systems.

Johnson, C. P. (2016). Gamification in Adventure and Wilderness Sports: A literature review of game-based mechanic’s ability to increase attraction, engagement, and retention in outdoor sports. The Sport Journal, 56. https://thesportjournal.org/article/gamification-in-adventure-and-wilderness-sports-a-literature-review-of-game-based-mechanics-ability-to-increase-attraction-engagement-and-retention-in-outdoor-sports/

Julius, K., & Salo, J. (2013). Designing gamification [Master’s Thesis, University of Oulu]. http://jultika.oulu.fi/files/nbnfioulu-201306061526.pdf

Kanagasingam, G. (2017). Increasing the learning motivations using gamification. Journal of Administrative Management, Education and Training, 13, 192-198.

Kapp, K. M. (2012). The gamification of learning and instruction: game-based methods and strategies for training and education. John Wiley & Sons.

Landers, R. N. (2015). Developing a theory of gamified learning linking serious games and gamification of learning. Simulation & Gaming, 45(6), 752-768. https://doi.org/10.1177/1046878114563660

Latham, B., Poyade, M., Finlay, C., Edmond, A., & McVey, M. (2019). New Tools in Education: Development and Learning Effectiveness of a Computer Application for Use in a University Biology Curriculum. Advances in Experimental Medicine and Biology, 1138, 29-46. https://doi.org/10.1007/978-3-030-14227-8_3

Legaki, N. Z., Xi, N., Hamari, J., & Assimakopoulos, V. (2019). Gamification of the future: an experiment on gamifying education of forecasting. In T. X. Bui (Ed.), Proceedings of the 52nd Hawaii International Conference on System Sciences (pp. 1813-1822). https://doi.org/10.24251/HICSS.2019.219

Li, W., Grossman, T., & Fitzmaurice, G. (2012). Gamicad: a gamified tutorial system for first time autocad users. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (pp. 103-112). Association for Computing Machinery. https://doi.org/10.1145/2380116.2380131

Linehan, C., Kirman, B., Lawson, S., & and Chan, G. (2011). Practical, appropriate, empirically-validated guidelines for designing educational games. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1979–1988). Association for Computing Machinery. https://doi.org/10.1145/1978942.1979229

Looi, C.‐K., Seow, P., Zhang, B., So, H.‐J., Chen, W. and Wong, L.‐H. (2010). Leveraging mobile technology for sustainable seamless learning: a research agenda. British Journal of Educational Technology, 41(2), 154-169. https://doi.org/10.1111/j.1467-8535.2008.00912.x

Majuri, J., Koivisto, J., & Hamari, J. (2018). Gamification of education and learning: A review of empirical literature. In J. Koivisto & J. Hamari (Eds.), Proceedings of the 2nd International GamiFIN Conference (pp. 11-19). CEUR-WS

McCoy, L., Lewis, J. H., & Dalton, D. (2016). Gamification and Multimedia for Medical Education: A Landscape Review. The Journal of the American Osteopathic Association, 116, 22–34. https://doi.org/10.7556/jaoa.2016.003

McGonigal, J. (2011). Reality is broken: Why games make us better and how they can change the world. Penguin

McMahon, B., & Portelli, J. P. (2004). Engagement for what? Beyond popular discourses of student engagement. Leadership and Policy in Schools, 3(1), 59-76. https://.doi.org/10.1076/lpos.3.1.59.27841

Mekler, E. D., Brühlmann, F., Opwis, K, & Tuch, A. N. (2013). Disassembling gamification: the effects of points and meaning on user motivation and performance. In P. Baudisch, M. Beaudoin-Lafon, & W. E. Mackay (Eds.), Proceedings of CHI’13 Extended Abstracts on Human Factors in Computing Systems (pp. 1137-1142). Association for Computing Machinery

Moccozet, L. & Tardy, C (2013). Gamification-based assessment of group work. In Proceedings of 2013 International Conference on Interactive Collaborative Learning (ICL) (pp. 171-179) . https://doi.org.pt/10.1109/ICL.2013.6644565

Moncada, S. M., & Moncada, T. P. (2014). Gamification of learning in accounting education. Journal of Higher Education Theory & Practice, 14(3),1-9.

Morschheuser, B. S., Rivera-Pelayo, V., Mazarakis, A, & Zacharias, V. (2014). Interaction and reflection with quantified self and gamification: an experimental study. Journal of Literacy and Technology, 15(2), 136-156. http://www.literacyandtechnology.org/uploads/1/3/6/8/136889/jlt_special.pdf

Nevin, C. R., Westfall, A. O., Rodriguez, J. M., Dempsey, D. M., Cherrington, A., Roy, B., Patel, M., & Willing, J. H. (2014) Gamification as a tool for enhancing graduate medical education, Postgraduate Medical Journal, 90(1070), 685-693.

Nicholson S. (2015) A RECIPE for Meaningful Gamification. In T. Reiners & L. Wood (Eds) Gamification in Education and Business (pp. 1-20). Springer, Cham. https://doi.org/10.1007/978-3-319-10208-5_1

Nicholson, S. (2012) A user-centered theoretical framework for meaningful gamification. In C. Martin, A. Ochsner, & K. Squire (Eds.), Proceedings of GLS 8.0 - Games+ Learning + Society Conference. (pp. 223-230). ETC Press.

Omerovic, A., & Karahasanovic, A. (2019). "CyberWiseKids": Gamifying Education of Kids on Online Opportunities and Security Risks. http://hdl.handle.net/11250/2593380

Ortega-Arranz, A., Munoz-Cristóbal, J. A., Martínez-Monés, A., Bote-Lorenzo, M. L., & Asensio-Pérez, J. I. (2017). How gamification is being implemented in MOOCs? A systematic literature review. In E. Lavoué, H. Drchsler, K. Verbert, J. Broisin, & M. Pérez-Sanagustín (Eds.), Data Driven Approaches in Digital Education. 12th European Conference on Technology Enhanced Learning, EC-TEL 2017, Tallinn, Estonia, September 12-15, 2017, Proceedings (pp. 441-447). Springer.

O'Sullivan, J. J. (2017). Teach2Learn: gamifying education to gather training data for natural language processing [Doctoral dissertation, Massachusetts Institute of Technology]. http://hdl.handle.net/1721.1/117320

Paule, M. T. (2016). Companions of Aeneas: Gamifying Intermediate Latin. Teaching Classical Languages, 6(2), 66-81. https://tcl.camws.org/sites/default/files/TCL%207.1%20Paule.pdf

Porubän, J. (2018). Challenging the Education in the Open Laboratory. In Proceedings of 16th International Conference on Emerging eLearning Technologies and Applications (ICETA) (pp. 439-444). IEEE.

Prensky, M. (2003). Digital game-based learning. Computers in Entertainment, 1(1), 21. https://doi.org/10.1145/950566.950596

Raitskaya, L., & Tikhonova, E. (2019). Gamification as a field landmark in educational research. Journal of Language and Education, 5(3), 4-10. https://doi.org/10.17323/jle.2019.10688

Raymer, R. (2011). Gamification: Using game mechanics to enhance elearning. eLearn magazine, 9. https://doi.org/10.1145/2025356.2031772

Resch, J., Ehrentraut, J., & Barnett-Cowan, M. (2018). Gamified Automation in Immersive Media for Education and Research. arXiv. https://arxiv.org/pdf/1901.00729.pdf

Rutledge, C., Walsh, C. M., Swinger, N., Auerbach, M., Castro, D., Dewan, M., ... & Maa, T. (2018). Gamification in action: theoretical and practical considerations for medical educators. Academic Medicine, 93(7), 1014-1020. https://doi.org/10.1097/ACM.0000000000002183

Schoenenberger, C. A., Korkut, S., Jaeger, J., & Dornberger, R. (2016, October). BioTourney: Gamifying a Biology Class by Applying a Content-Independent Learning Game Framework. In T. Connolly & L. Boyle (Eds.), Proceedings of the 10th European Conference on Games Based Learning (pp. 575-583). Academic Conferences International Limited.

Shpakova, A., Dörfler, V., & MacBryde, J. (2017). Changing the game: a case for gamifying knowledge management. World Journal of Science, Technology and Sustainable Development, 14(2/3), 143-154. https://doi.org/10.1108/WJSTSD-01-2017-0002

Sillaots, M. (2014). Gamification of higher education by the example of course of research methods. In E. Popescu, R. W. H. Lau, K. Pata, H. Leung, & M. Laanpere (Eds.), Advances in Web-Based Learning – ICWL 2014. ICWL 2014. Lecture Notes in Computer Science (pp. 106-115). Springer, Cham. https://doi.org/10.1007/978-3-319-09635-3_11

Simões, J., Redondo, R. D., & Vilas, A. F. (2013). A social gamification framework for a k-6 learning platform. Computers in Human Behavior, 29(2), 345-353. https://doi.org/10.1016/j.chb.2012.06.007

Smiderle, R., Marques, L., Coelho, J. A. P. D. M., Rigo, S. J., & Jaques, P. A. (2019). Studying the Impact of Gamification on Learning and Engagement of Introverted and Extroverted Students. Proceedings of 2019 IEEE 19th International Conference on Advanced Learning Technologies (ICALT), 2161, 71-75. https://doi.org/10.1109/ICALT47348.2019

Smiderle, R., Rigo, S. J., Marques, L. B., de Miranda Coelho, J. A. P., & Jaques, P. A. (2020). The impact of gamification on students’ learning, engagement and behavior based on their personality traits. Smart Learning Environments, 7(1), 1-11. https://doi.org/10.1186/s40561-019-0098-x

Soboleva, E. V., Galimova, E. G., Maydangalieva, Z. A., & Batchayeva, K. K. M. (2018). Didactic value of gamification tools for teaching modeling as a method of learning and cognitive activity at school. Eurasia Journal of Mathematics, Science and Technology Education, 14(6), 2427-2444. https://doi.org/10.29333/ejmste/89843

Sombrio, G., Schimmelpfeng, L. E., Ulbricht, V. R., & Villarouco, V. (2016, July). Gamification in Education Through Design Thinking. In A. Marcus (Ed.), Design, User Experience, and Usability: Novel User Experiences. DUXU 2016. Lecture Notes in Computer Science (pp. 311-321). Springer, Cham. https://doi.org/10.1007/978-3-319-40355-7_30

Sprint, G., & Fox, E. (2020). Improving Student Study Choices in CS1 with Gamification and Flipped Classrooms. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education (pp. 773-779). Association for Computing Machinery. https://doi.org/10.1145/3328778.3366888

Tan Ai Lin, D., Ganapathy, M., & Kaur, M. (2018). Kahoot! It: Gamification in Higher Education. Pertanika Journal of Social Sciences & Humanities, 26(1), 565 - 582.

Toda, A. M., Oliveira, W., Shi, L., Bittencourt, I. I., Isotani, S., & Cristea, A. (2019a). Planning Gamification strategies based on user characteristics and DM: A gender-based case study. In C. F. Lynch, A. Merceron, M. Desmarais, & R. Nkambou (Eds.), Proceedings of the 12th International Conference on Educational Data Mining (pp. 438-443). Educational Data Mining. https://dro.dur.ac.uk/28609/

Toda, A. M., Palomino, P. T., Oliveira, W., Rodrigues, L., Klock, A. C., Gasparini, I., ... & Isotani, S. (2019b). How to Gamify Learning Systems? An Experience Report using the Design Sprint Method and a Taxonomy for Gamification Elements in Education. Journal of Educational Technology & Society, 22(3), 47-60. https://doi.org/10.2307/26896709

Toda, A., Oliveira, W., Klock, A., Palomino, P., Pimenta, M., Bittencourt, I., ... & Cristea, A. (2019c). A taxonomy of game elements for gamification in educational contexts: Proposal and evaluation. IProceedings of 2019 IEEE 19th International Conference on Advanced Learning Technologies (ICALT), 2161, 84-88. https://doi.org/10.1109/ICALT47348.2019

Torrealba, L. G., & Camacho, E. D. (2018). Juegos, conocimientos y aceptación de la tecnología en la era digital. Revista Ibérica de Sistemas e Tecnologias de Informação, E16, 171-181. http://www.risti.xyz/issues/ristie16.pdf

Urh, M., Vukovic, G., Jereb, E., & Pintar, R. (2015). The model for introduction of gamification into e-learning in higher education. Procedia-Social and Behavioral Sciences, 197, 388-397. https://doi.org/10.1016/j.sbspro.2015.07.154

Vetushinskiy, A., & Zhukov, P. (2019). Gamification of School Education in Russia: Case Study. In J. Arnedo-Moreno, C.S. González, & A. Mora (Eds.), Proceedings of the 3rd International Symposium on Gamification and Games for Learning (GamiLearn’19). Universitat Oberta de Catalunya and Universidad de La Laguna. http://ceur-ws.org/Vol-2497/paper14.pdf

Werbach, K., & Hunter, D. (2012). For the Win: How Game Thinking Can Revolutionize Your Business. Wharton Digital Press.

Werbach K. (2014). (Re)Defining Gamification: A Process Approach. In A. Spagnolli, L. Chittaro, & L. Gamberini (Eds.) Persuasive Technology. PERSUASIVE 2014. Lecture Notes in Computer Science (pp. 266-272). Springer, Cham. https://doi.org/10.1007/978-3-319-07127-5_23

Winoto, P., & Tang, T. Y. (2015). From market place to collusion detection: Case studies of gamification in education. In T. Reiners & L. C. Wood (Eds.), Gamification in Education and Business (pp. 277-290). Springer International Publishing.

Yıldırım, İ., & Cirak-Kurt, S. (2018). The Effect of Gamification on Learner Motivation: A Meta-Analysis Study. In J. Vopava, V. Douda, R. Kratochvil, & M. Konecki (Eds.), Proceedings of MAC 2018 (pp. 310-312). MAC Prague Consulting Ltd.

Yıldırım, İ., & Şen, S. (2019). The effects of gamification on students’ academic achievement: a meta-analysis study. Interactive Learning Environments, 1-18. https://doi.org/10.1080/10494820.2019.1636089

Zarzycka-Piskorz, E. (2016). Kahoot it or not? Can games be motivating in learning grammar?. Teaching English with Technology, 16(3), 17-36.https://files.eric.ed.gov/fulltext/EJ1135685.pdf

Zhao, G. Z. (2019). Using a Gamified Points-Based Grading System in Technology Courses for Pre-Service Teachers [Doctoral dissertation, University of Houston]. https://hdl.handle.net/10657/5719

Zhu, J., Li, X., Zhang, Y., Zainuddin, Z., Lee, S. Y. C., & Chu, S. K. W. (2018). Can Gamification Bring Long-term Effects for Elementary Students’ Learning? [Paper presentation]. ASIS&T AP 2019 regional conference, Phnom Penh, Cambodia.