Data Loading...

MOBILE DEVICES Flipbook PDF

MOBILE DEVICES

155 Views
105 Downloads
FLIP PDF 896.96KB

Computers & Education 94 (2016) 252e275

Contents lists available at ScienceDirect

Computers & Education journal homepage: www.elsevier.com/locate/compedu

The effects of integrating mobile devices with teaching and learning on students' learning performance: A meta-analysis and research synthesis Yao-Ting Sung a, Kuo-En Chang b, Tzu-Chien Liu a, * a b

Department of Educational Psychology and Counseling, National Taiwan Normal University, Taiwan, ROC Grad. Institute of Information and Computer Education, National Taiwan Normal University, Taiwan, ROC

a r t i c l e i n f o

a b s t r a c t

Article history: Received 12 August 2015 Received in revised form 17 November 2015 Accepted 19 November 2015 Available online 23 November 2015

Mobile devices such as laptops, personal digital assistants, and mobile phones have become a learning tool with great potential in both classrooms and outdoor learning. Although there have been qualitative analyses of the use of mobile devices in education, systematic quantitative analyses of the effects of mobile-integrated education are lacking. This study performed a meta-analysis and research synthesis of the effects of integrated mobile devices in teaching and learning, in which 110 experimental and quasiexperimental journal articles published during the period 1993e2013 were coded and analyzed. Overall, there was a moderate mean effect size of 0.523 for the application of mobile devices to education. The effect sizes of moderator variables were analyzed and the advantages and disadvantages of mobile learning in different levels of moderator variables were synthesized based on content analyses of individual studies. The results of this study and their implications for both research and practice are discussed. © 2015 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

Keywords: Evaluation methodologies Pedagogical issues Teaching/learning strategies

1. Introduction 1.1. Integrating mobile devices with learning and instruction Mobile computers have gradually been introduced into educational contexts over the past 2 decades. Mobile technology has led to most people to carry their own individual small computers that contain exceptional computing power, such as laptops, personal digital assistants (PDAs), tablet personal computers (PCs), cell phones, and e-book readers. This large amount of computing power and portability, combined with the wireless communication and context sensitivity tools, makes one-to-one computing a learning tool of great potential in both traditional classrooms and outdoor informal learning. With regard to access to computers, large-scale one-to-one computing programs have been implemented in many countries globally (Bebell & O'Dwyer, 2010; Fleischer, 2012; Zucker & Light, 2009), such that elementary- and middle-school students and their teachers have their own mobile devices. In addition, in terms of promoting innovation in education via information technology, not only does mobile computing support traditional lecture-style teaching, but through convenient

* Corresponding author. Department of Educational Psychology and Counseling, National Taiwan Normal University, 162, HePing East Road, Section 1, Taipei, Taiwan, ROC. E-mail addresses: [email protected] (Y.-T. Sung), [email protected] (K.-E. Chang), [email protected] (T.-C. Liu). http://dx.doi.org/10.1016/j.compedu.2015.11.008 0360-1315/© 2015 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/ licenses/by-nc-nd/4.0/).

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

253

information gathering and sharing it can also promote innovative teaching methods such as cooperative learning (Lan, Sung, & Chang, 2007; Roschelle et al., 2010), exploratory learning outside the classroom (Liu, Lin, Tsai, & Paas, 2012), and gamebased learning (Klopfer, Sheldon, Perry, & Chen, 2012). Therefore, mobile technologies have great potential for facilitating more innovative educational methods. Simultaneously, these patterns in educational methods will likely not only help subject content learning, but may also facilitate the development of communication, problem-solving, creativity, and other high-level skills among students (Warschauer, 2007). However, despite the proposed advantages of using mobile computing devices for increasing computer accessibility, diverse teaching styles, and academic performance, currently researchers found mixed results regarding the effects of mobiledevices (e.g., Warschauer, Zheng, Niiya, Cotten, & Farkas, 2014), and very few studies have addressed how best to use mobile devices, and the effectiveness of doing so.

1.2. Review of the research into integrating mobile devices with teaching and learning There are seven studies which reviewed the research into integrating mobile devices with teaching and learning and can be divided into two types according to the devices they focused on: (1) those focused on how laptops are used in schools and (2) those focused on the applications of various types of mobile device in education (see Appendix A). Regarding the review of laptop-based programs, Zucker and Light (2009) believed that school programs integrating laptops into schools have a positive impact on student learning. However, they also believed that laptop use did not achieve the goals of increasing higher-level thinking and transformation of classroom teaching methods. Penuel (2006) reviewed 30 studies that examined the usage of laptops with wireless connectivity in one-to-one computer programs. Those studies found that students most often used the laptops to do homework, take notes, and ﬁnish assignments. General-purpose software such as word processors, web browsers, and presentation software were relatively common. Bebell and O'Dwyer (2010) examined four different empirical studies of laptop programs in schools. They discovered that in most schools participating in one-to-one programs there were signiﬁcant increases in grade-point averages or standardized tests of student achievement, relative to schools that did not provide such programs. In addition, they found that most students used their laptops to write, browse the Internet, make presentations, do homework, or take tests. Furthermore, teachers made more changes to their teaching methods when they had increased opportunities to use laptops. Students participating in one-to-one programs also had a deeper engagement with what they were learning when compared to control groups. Fleischer (2012) conducted a narrative research review of 18 different empirical studies on the usage of laptops. These studies found a large range in the number of hours that students used laptops, from a few days to as little as 1 h per week. The most frequently used computer functions were searches, followed by expression and communication. In most studies it was found that students had a positive attitude toward laptops, and felt that they were more motivated and engaged in their learning, and it was further believed that teachers conducted more student-centered learning activities. Moreover, considerable differences in classroom educational practices arose from the diversity of teachers' beliefs about the usefulness of laptops. Fleischer (2012) also found several challenges regarding the use of laptops in classrooms, such as encouraging teachers to change their previous beliefs and teaching methods (e.g., teacher-centered lectures) in response to their students' greater ﬂexibility and autonomy; how to reconcile the conﬂict between the students' desire for independent study and the need for teachers' guidance; and how to facilitate teachers' competence by designing an appropriate curriculum and teaching models for laptop usage programs. With respect to the research on the use of mobile technology in education, Hwang and Tsai (2011) provided a broad discussion of studies on mobile and ubiquitous learning published in six journals between 2001 and 2010. In their review of 154 articles, they discovered that the use of mobile and ubiquitous learning accelerated markedly during 2008; researchers mostly studied students of higher education, and the ﬁelds most often researched were language arts, engineering, and computer technology. Frohberg, Goth, and Schwabe (2009) categorized 102 mobile-learning projects, and discovered that most mobile-learning activities occurred across different settings, and took place within a physical context and an ofﬁcial environment, such as a classroom or workplace. Regarding the pedagogical roles that mobile devices play in education, most research has used mobile devices primarily as a sort of reinforcement tool to stimulate motivation and strengthen engagement, and secondarily as a content-delivery tool. Few projects have used mobile devices to assist with constructive thinking or reﬂection. Furthermore, most learning activities using mobile devices have been controlled by the teacher, with there being only a handful of learner-centered projects in existence. Concerning the communication functions, very few projects have made any use of cooperative or team communication. Moreover, the vast majority of studies have made use of novice participants; little research has involved experienced participants. When sorted according to educational goals, it was found that the vast majority of research has focused on lower-level knowledge and skills, and ignored higher-level tasks such as analysis and evaluation. Wong and Looi (2011) investigated the inﬂuence of mobile devices on seamless learning. Seamless learning refers to a learning model that students can learn whenever they want to learn in a variety of scenarios and that they can switch from one scenario or one context to another easily and quickly (Chan et al., 2006; Wong & Looi, 2011). Wong and Looi (2011) selected and analyzed a sample of 54 articles on the use of mobile devices to facilitate seamless learning, and found that all 54 articles contained 10 features, including formal and informal learning, personalized and social learning, and learning across multiple durations and locations.

254

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

1.3. Purposes of this study While analyzing the overall effectiveness of using mobile devices in education, the review research described above has two major limitations. First, all of the reviews adopted a qualitative approach, which may be able to describe and summarize how related studies were conducted and the problems encountered during their execution, but this makes it difﬁcult to evaluate the effects actually produced by the mobile devices in general and the speciﬁc moderator variables. Second, much of the previous review research has focused on the usage of laptop computers as the subject of their investigation (e.g., Penuel, 2006), and most of the research participants in those reviewed articles were in primary and secondary schools. However, the many new developments in mobile hardware have meant that diverse age groups now use different devices. Therefore, many different moderators need to be accounted for when attempting to determine whether or not intervening variables have an effect. In the context of this background, the primary goal of this study was to perform a meta-analysis and research synthesis of the research on the usage of mobile devices in education published in the last 2 decades. Speciﬁcally, the purposes of this study were as follows: 1. To provide an overview of the status of the use of mobile devices in educational experimental studies, including who is using them, which domain subjects are being taught, what kinds of mobile device and software are being used, where such programs take place, how the devices are used in teaching, and the duration of the interventions. 2. To quantify the overall effectiveness of integrating mobile technologies into education on student learning achievement. 3. To determine how the moderator variables inﬂuence the effects of mobile devices on learning achievement. 4. To synthesize the advantages and disadvantages of mobile learning in levels of moderator variables based on the content analysis of articles related with moderator variables.

2. Method 2.1. Data sources and search strategy Journal articles published during the period 1993e2013 were searched electronically and manually, and via reference-list checking to retrieve the relevant literature. For electronic searches, the main databases were the Education Resources Information Center (ERIC) and the Social Sciences Citation Index database of the Institute of Science Index (ISI). Two sets of keywords were searched: (1) mobile-device related keywords, including mobile, wireless, ubiquitous, wearable, portable, handheld, cell phone, personal digital assistant, PDA, palmtop, pad, web pad, tablet PC, tablet computer, laptop, e-book, digital pen, pocket dictionary, and classroom response system; and (2) learning-related keywords, including teaching, learning, training, and lectures. The two sets of keywords were combined when searching the electronic databases. Manual searches included the major journals in educational technology and e-learning, such as the Australian Journal of Educational Technology, British Journal of Educational Technology, Computers & Education, Computer Assisted Language Learning, Educational Technology Research and Development, Journal of Computer Assisted Learning, Language Learning & Technology, and ReCall. After collating all of the related literature, another round of searches was conducted using the reference lists found in the literature yielded by the electronic search to ﬁnd any omitted but relevant works.

2.2. Search results 2.2.1. Initial screening The initial search yielded 4121 abstracts published between 1993 and 2013 (1718 in ERIC and 2403 in ISI) that were related to mobile learning. Two authors read each abstract of the article and judged whether or not the article was related to teaching and learning with a mobile device, which resulted in the selection of 925 articles. 2.2.2. Screening for experimental and quasiexperimental research In the second stage, the studies were screened according to the research method. Experimental studies (including the pretest-posttest equivalent-group, posttest-only equivalent-groups, and randomized matched subjects and posttest-only control-group designs) and quasi-experimental studies (including the pretest-posttest nonequivalent-groups and counterbalanced designs; see Ary, Jacobs, & Razavieh, 2002; Best & Kahn, 1998, for a reference) were included. Conceptual analysis or research reviews, case studies and qualitative research, survey research, and pre-experimental studies were all excluded at this stage. At the completion of this stage there remained 182 articles. 2.2.3. Application of inclusion and exclusion criteria for the meta-analysis Studies were eligible for inclusion in the meta-analysis if they conformed with the following three criteria:

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

255

1. The application of mobile devices was the key variable of the study. The experimental group had an intervention that used mobile devices, and was compared with a control group that used traditional learning. If both the experimental and control groups used mobile-device interventions, and only the teaching methods were compared, then the study was excluded (e.g., Hsu, Hwang, Chang, & Chang, 2013; Jeong & Hong, 2013; Li, Chen, & Yang, 2013; Ryu & Parsons, 2012). 2. Sufﬁcient information was presented to calculate effect sizes, such as means, standard deviations, t, F, or c2 values, or the number of people in each group. Articles in which the sample sizes of each group were not cited, lacked any inferential statistical results, or had inferential statistical results but were still inadequate for calculating an effect size according to Lipsey and Wilson (2000) were excluded (e.g., Gleaves, Walker, & Grey, 2007; Langman & Fies, 2010; Purrazzella & Mechling, 2013; Yang et al., 2013). 3. Experimental results were presented with learning achievement as a major dependent variable measured by standardized or researcher-constructed tests. Studies for which the results were related to affective variables (e.g., learning attitude or learning motivation) or interaction between peers but without learning achievement were excluded (e.g., Jian, Sandnes, Law, Huang, & Huang, 2009; Lan et al., 2007; Mouza, 2008; Siozos, Palaigeorgiou, Triantafyllakos, & Despotakis, 2009). Application of these criteria yielded 110 articles that were acceptable for inclusion in the meta-analysis. For a complete list of these references, please see our online supplemental archive.

2.3. Selection and coding of the outcome variables One of the most used framework for representing the research content and dimensions is the activity theory (AT), which uses activity as a unit for analyzing human practices (Bakhurst, 2009). Recently, several researchers have used the AT as a theoretical basis for analyzing mobile learning studies (e.g., Frohberg et al., 2009; Sharples, Taylor, & Vavoula, 2007) or for designing mobile learning scenarios (e.g., Zurita & Nussbaum, 2007). This study used six major components of AT to select moderator variables and analyze mobile learning: (a) Subjects: which involve all the people who may be involved in learning curriculums through mobile devices, such as students of different age levels or teachers of different levels of teaching expertise. (b) Objects (or objectives) of the mobile learning, which focus on the goal such as acquiring cognitive skills or enhancing learning motivation through mobile devices. (c) Tools/instruments in the mobile learning, which may be artifacts (e.g., hardware and software) or learning resources (e.g., tutors). (d) Rules/control for the activity, which are norms or regulations that circumscribe the mobile activities, such as the procedure in teaching scenarios designed for the learning pace or styles designated. (e) Context of the activity, which refers to the physical (e.g., classroom or museum) or social (e.g., ambience of learning in a group) environments for conducting mobile learning. (f) Communication/interaction, which refers to the method of interaction between users and mobile technologies (such as the process teachers' adaption to mobile devices) or the communications styles among learners. 2.3.1. Research name This refers to the ﬁrst author's name, the year of publication, and the article title. 2.3.2. Research participants In this review, for all the reviewed articles, the research participant corresponded to the “subject” of the AT framework, and was coded by their learning stages, including kindergarten, elementary school, middle school, (senior) high school, university, graduate school, teachers, adults, and mixed. 2.3.3. Treatments The treatments of the reviewed articles corresponded to the “tools” component (e.g., the hardware and software), the “rules/control” component (e.g., the teaching methods and domain subjects), and the “context” component (e.g., intervention settings, intervention duration). The description for each of these treatment variables are as follows: 1. Hardware: Different types of mobile hardware, which comprised PDAs, laptops, tablet PCs, cell phones, iPods, MP3 players, e-book readers, pads, digital pens, pocket dictionaries, and classroom response systems (CRSs), or any mixture of thereof. 2. Software: Different types of software, which encompassed general-purpose software and learning-oriented software (Sung & Lesgold, 2007), the former referring to commercial software currently in circulation that was not designed especially for teaching and learning (e.g., word processors or spreadsheets), and the latter having been designed specifically for educational programs or goals. 3. Teaching method: Different teaching methods, including lectures, cooperative learning (students were divided into groups and completed learning tasks collaboratively, e.g., Chang, Lan, Chang, & Sung, 2010; Huang, Liang, Su, & Chen, 2012), inquiry-oriented learning (using problem-, project-, or inquiry-based methods with mobile devices for learning, e.g., Chen, 2010; Lowther, Ross, & Marrison, 2003), self-directed study (teachers/researchers did not designate or implement speciﬁc teaching scenarios for students to follow, students use mobile devices for self-paced learning, e.g., Chen & Li, 2010; Chen,

256

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

Tan, & Lo, 2013), computer-assisted testing/assessment (using mobile devices for formative assessment or quizzes in classroom or outdoors, e.g., Agbatogun, 2012), and mixed methods thereof. 4. Domain subject: Domain subjects were analyzed to establish the relative effectiveness of mobile devices for teaching different subjects, including language arts, social studies, science, mathematics, multidisciplinary (if the mobile devices were used in several subjects, but measurement of the achievement was presented as a whole instead of separately, this was coded as multidisciplinary), speciﬁc abilities (e.g., spatial ability or creativity), health-care programs, education, psychology, and computer and information technology. 5. Implementation setting: Implementation settings were included to establish whether the impact of mobile devices on learning differed according to the environment in which they were used, which included classrooms, outdoors (e.g., zoo or campus gardens), museum, laboratory, workplaces, and unrestricted settings (devices may be used anywhere). 6. Intervention duration: Different periods of time for the intervention, including periods no more than four hours (4 h), between ﬁve and 24 h (>4 and 24 h), between one day and seven days (>1 day and 7 days), between one week and four weeks (>1 week and 4 weeks), between one month and six months (>1 month and 6 months), and more than six months (>6 months).

2.3.4. Dependent variables The dependent variables corresponded to the “Objective” of the AT model, including two categories: the learning achievement dependent variables refer to measurement of cognitive outcomes such as knowledge application, retention, problem solving…etc. The affective variables refer to measurement of motivation, interest, participation…etc.

2.4. Data analysis 2.4.1. Calculating the effect size The following meta-analysis steps recommended by Borenstein, Hedges, Higgins, and Rothstein (2009) were employed in this study: (a) determine the effect sizes of each article, (b) determine the weighted mean effect size across articles, (c) calculate the conﬁdence interval for the average effect size, and (d) determine whether the effect size of any particular group was inﬂuenced by a moderator variable based on a heterogeneity analysis (QB). Two formulae were used to calculate the effect sizes of the studies. Cohen's d formula (Cohen, 1988) was used to determine the effect size for the experimental research with random assignment and without a pretest:

X1 X2 d ¼ rﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ

(1)

ðn1 1Þs21 þðn2 1Þs22 ðn1 þn2 2Þ

where X 1 and X 2 represent the mean scores, n1 and n2 represent the sample sizes, and s21 and s22 represent the variances of the experiment and control groups, respectively. For experimental or quasiexperimental research with pretests, it was proposed that the pretest should be taken into consideration instead of using the posttest in order to mitigate possible selection bias (Furtak, Seidel, Iverson, & Briggs, 2012; Morris, 2008). Hence, the formula developed in Comprehensive Meta Analysis (version 2.0) was used to obtain effect sizes for research with pre- and posttests:

ESPre=Post Test Two Groups ¼

X1

Post

X1

X2 SDPost

Pre

Post

X2

Pre

(2)

where X 1 Pre and X 1 Post represent the mean scores of the experimental group for the pretest and posttest, respectively, and X 2 Pre and X 2 Post represent the mean scores of the control group for the pretest and posttest, respectively. SDPost can be calculated as follows:

SDPost

sﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ ﬃ n2 Post 1 s22 Post þ n1 Post 1 s21 Post ¼ n2 Post þ n1 Post 2

(3)

where n1 Post and n2 Post represent the sample sizes of the experimental and control groups, respectively, for the posttest, while s21 Post and s22 Post represent the variances of the experimental and control groups, respectively. The two types of effect sizes were calibrated using the sample weights to calculate a Hedges' g according to

g¼

1

3 d 4df 1

(4)

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

257

2.4.2. Evaluating publication bias The fail-safe N (i.e., classic fail-safe N) of Rosenthal (1979) was used to estimate how many insigniﬁcant effect sizes (unpublished data) would be necessary to reduce the overall effect size to an insigniﬁcant level. The comparison criterion was 5nþ10, where n is the number of studies included in the meta-analysis. If the fail-safe N is larger than 5nþ10, it means that the estimated effect size of unpublished research is unlikely to inﬂuence the effect size of the meta-analysis. Moreover, the present study also adopted Orwin's fail-safe N (Orwin, 1983) to estimate the number of missing null studies that would be required to bring the mean effect size to a trivial level. 3. Results and discussion 3.1. Descriptive statistics information Table 1 presents the distribution of moderator variables and their corresponding effect sizes (g). In total there were 110 articles, 419 effect sizes, and 18 749 participants. The largest proportion of studies involved the college-student-level learning stage (38.4%); the next largest group was elementary-school students (33.9%). More studies used learning-oriented software (62.7%) than general-purpose software (34.5%). Handheld devices (including PDA, cell phone, iPod, MP3 player, digital pen, pocket dictionary, and classroom response system) were the most widely studied of the hardware (72.7%), followed by laptops (21.8%, including laptop, pad, tablet PC, and e-book reader). The largest proportion of studies were set in the classroom (50.0%), followed by outdoors (15.5%) and unrestricted settings (16.4%). For teaching methods, self-directed study (30.9%) was the most frequently researched, and the most frequently studied intervention duration was >1 month and 6 months (32.7%), followed by > 1 week and 4 weeks (25.5%) and 4 h (20.9%). Finally, language arts were the most often studied domain subject (34.7%), followed by science (22.9%). In addition, among those moderating variables, the evolution of hardware used, implementation setting, and domain subjects may have seen the greatest amount of change during 1993e2013. The trends of those moderating variables during the two decades are shown in Figs. 1e3. Fig. 1 shows the evolution of the use of different mobile devices. Compared with laptop and mixed categories, handheld devices (e.g., cell phone) had been used more since 2009e2013 and showing an obviously rising trend. Moreover, Fig. 2 shows the evolution of the use of different implementation settings. Compared with informal settings (e.g., museums; outdoors) and unrestricted categories, formal settings (e.g., classroom; laboratories) had been set more since 2004e2008 and showing an obviously rising trend. Finally, Fig. 3 shows the evolution of the domain subject. Compared with other domain subjects, language arts had been studied more since 2009e2013 and showing an obviously rising trend. 3.2. Overall effect size for learning achievement The distribution of the effect sizes of the 110 articles is shown in Fig. 4. The forest plot of effect sizes and the 95% conﬁdence interval of the 110 articles are shown in Appendix B. There were two unusually large effect sizes, g ¼ 4.045 (Hsu & Lee, 2011) and g ¼ 3.050 (Wu, Sung, Huang, Yang, & Yang, 2011), which were larger than the average effect size for the entire collection of 110 articles (g ¼ 0.628) more than three standard deviations, and so these were not included in further analyses (Lipsey & Wilson, 2000). Using the procedure of Lipsey and Wilson (2000) with a random-effects model to integrate the effect sizes of the 108 articles, there was an overall moderate mean effect size of 0.523, with a 95% conﬁdence interval of 0.432e0.613. Researchers (e.g., McMillan, Venable, & Varier, 2013; Van der Kleij, Feskens, & Eggen, 2015) have proposed that Hattie's (2009) criterion is appropriate for evaluating the effect sizes in educational contexts. Therefore, we adopt Hattie's (2009) criterion to interpret the effect size of our research, in which an effect size of 0.60 is high, around 0.40 is medium, around 0.20 is low, and 1 month and 6 months duration (g ¼ 0.566, z ¼ 6.870, p < .001), those of >1 week and 4 weeks duration (g ¼ 0.552, z ¼ 5.644, p < .001), and those 1 week had medium effect sizes (g ¼ 0.479, z ¼ 5.175, p < .001). Interestingly, interventions conducted for durations of >6 months had a non-signiﬁcant

264

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

effect size (g ¼ 0.287, z ¼ 1.942, p ¼ .052). The QB was not signiﬁcant (QB ¼ 4.924, p ¼ .295), which suggests that the effect size did not differ signiﬁcantly between these categories. The non-signiﬁcance of the effect size in long-term duration (>6 months) is counterintuitive, but consistent with those of Kulik and Kulik (1991), who found that computer-based instruction had a greater effect when the duration was shorter. Kulik and Kulik (1991) and Cheung and Slavin (2013) proposed three reasons for why short-term treatments have better effects: high novelty value, stronger interventional supports, and different measurement tools for the dependent variables. These explanations are also applicable to the present ﬁndings. In most studies with intervention durations less than 6 months, the use of mobile devices and the applied teaching methods were both novel, so the students were more easily engaged in the activity. Cross-analysis of intervention duration with other moderator variables provides data that supports these arguments. For example, most research that took place over a 6-month period used general-purpose software (66.7%; Table C2 of Appendix C), which did not necessarily match the needs of the learning scenarios in speciﬁc learning topics. Furthermore, around half of the studies (44.4%; Table C2 of Appendix C) with durations of >6 months placed the computers directly in the classroom and did not specify the teaching methods to be used to achieve speciﬁc educational goals. Conversely, 57.1% of the studies lasting for >1 month and 6 months used learning-oriented software for speciﬁc teaching and learning goals, and 94.3% speciﬁed a speciﬁc teaching strategy instead of simply using computers for some unspeciﬁed purpose in the classroom (Table C2 of Appendix C). In terms of the interventional supports, in most short-term studies, researchers could gather all of their resources for one shot, so they chose the most appropriate hardware and software with more diverse functionality, prepared more elaborate learning activities, and made every effort to control confounding factors. However, in studies lasting >6 months, the longer duration made it more difﬁcult to support the use of diverse resources, ﬁnding logistic assistance for technological problems, and maintaining the enthusiasm associated with using new technologies. For example, Shapley, Sheehan, Maloney, and Caranikas-Walker (2010) found that in laptop immersion schools, after four years of implementation, only 6 of 21 schools reached a substantial level of immersion, and the level of student access and use of laptops in classrooms declined during the period of implementation because of insufﬁcient support. Research in the ﬁeld of education mostly advocates that long-term teaching interventions are important for obtaining reliable results (Hsieh et al., 2005; Pressley & Harris, 1994), but in the present study it was found that long-term interventions with mobile devices in classrooms did not necessarily lead to better effects. Such ﬁndings echo comments made by many researchers about the use of laptops in the classroom: If computers are simply given to teachers and students to use for a long time without any positive guidance, it will not necessarily produce satisfactory educational outcomes (Holcomb, 2009; Zucker & Light, 2009), especially for higher levels learning skills such as reasoning and problem solving (Drayton et al., 2010). In order for there to be abundant effects, long-term interventions need logistical support to integrate advanced technologies with innovative and elaborate educational methods. Information technology applications in the classroom must ﬁrst go through adoption and adaptation before they can proceed to innovation. These processes are also likely to take longer than 1 year (Gerard et al., 2011), or even up to 3 years (Bebell & O'Dwyer, 2010). During such a long-term process, if the main support provided to teachers and students is enthusiasm rather than appropriate support such as hardware, software, and instructional designs, computer use in the classroom will ultimately be merely superﬁcial. 3.3.7. Domain subjects The data in Table 2 indicate the effect sizes for different domain subjects. Social studies (g ¼ 0.768, z ¼ 3.682, p < .001) had a high effect size, while professional subjects (g ¼ 0.592, z ¼ 6.808, p < .001), science (g ¼ 0.565, z ¼ 6.397, p < .001), language arts (g ¼ 0.473, z ¼ 6.352, p < .001) and mathematics (g ¼ 0.337, z ¼ 2.628, p ¼ .009) had medium effect sizes. No signiﬁcant effect size was obtained for using mobile devices for domain-general abilities (g ¼ 0.151, z ¼ 0.868, p ¼ .386). The QB did not achieve statistical signiﬁcance (QB ¼ 9.108, p ¼ .105), which shows that the average effect size did not differ signiﬁcantly among these categories.

3.4. Evaluation of publication bias The classic fail-safe N and Orwin's fail-safe N were adopted to demonstrate the publication bias for the 108 selected studies. As suggested by the data in Table 3, the classic fail-safe N test determined that a total of 4144 studies with null results would be needed in order to nullify the effect size. Moreover, the results of Orwin's fail-safe N test (see Table 4) show that the

Table 3 Results of the classic fail-safe N. Z value for observed studies p value for observed studies Alpha Tail Z for alpha Number of observed studies Number of missing studies that would bring the p value to >alpha

22.51 0.00 0.05 2.00 1.96 108.00 4144.00

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

265

Table 4 Results of Orwin's fail-safe N. Hedges' g in observed studies (ﬁxed effect) Criterion for a ‘trivial’ Hedges' g Mean Hedges' g in missing studies Number of missing studies needed to bring Hedge's g to under 0.01

0.33 0.01 0.00 3423.00

number of missing null studies required to bring the existing overall mean effect size to a trivial level (g ¼ 0.01) was 3423. Both tests suggest that publication bias could not explain the signiﬁcant positive effects observed across all studies. 4. Conclusions and implications Analysis of the empirical research on the use of mobile devices as tools in educational interventions that were published in peer-reviewed journals has revealed that the overall effect of using mobile devices in education is better than when using desktop computers or not using mobile devices as an intervention, with a moderate effect size of 0.523. Through the analysis of moderator variables, we found that many different combinations of hardware, software, and intervention durations for mobile devices have been applied to various ages of users, implementation settings, teaching methods, and domain subjects. The effect of such usage was greater for handhelds than for laptops; usage in inquiry-oriented learning was more effective than usage along with lectures, self-directed study, cooperative learning, and game-based learning; informal educational environments were more effective than their formal counterparts, and medium- and short-duration interventions were superior to long-term interventions. These ﬁndings will contribute to a better understanding of where, for whom, and in which way the use of mobile devices in the learning environment will best highlight the effects of particular educational methods, and reveal the limitations of mobile devices in education. Based on the ﬁndings of this study, it is proposed that more elaborate instructional design developments are needed to more thoroughly exploit the educational beneﬁts possible by utilizing mobile devices. We believe that the three implications proposed below will be helpful for facilitating and achieving these goals. 4.1. Leveraging the pedagogical effects of mobile devices through elaborate designs of learning/teaching scenarios Mobile devices have various distinctive features such as individualized interfaces, real-time access to information, context sensitivity, instant communication, and feedback. These features may be able enhance the effects of certain pedagogies, such as self-directed learning, inquiry learning, or formative assessment. However, it is note-worthy that the features of mobile devices are not sufﬁcient conditions for positive learning effects. The minor effects of mobile-device-based cooperative and game-based learning in our study illustrated this fact. Instructional strategies are important for effective learning with information technology (Lan, 2014; Lan, Sung, Cheng, & Chang, 2015; Liu, Lin, & Paas, 2014). Researchers must ﬁnd the “key” to integrating mobile devices with instructional strategies and ingeniously match the unique features of mobile devices to the resolution of speciﬁc pedagogic challenges. Doing so will maximize the impact of those features on learning outcomes. Some examples include using the instant-feedback functions to solve the difﬁculty of efﬁciently executing and managing formative assessment in a class with many students (Penuel, Roschelle, & Shechtman, 2007) and, for cooperative groups, using wireless communication to facilitate between-group scaffoldings and to avoid idling (Lan et al., 2007). As one of the most used strategies in mobile learning/teaching, self-directed study is an example of a method that deserves more attention paid to pairing speciﬁc features to speciﬁc challenges to yield improved results. In addition, most of the studies in our research utilized mobile devices' features of individuality and wireless communication capacity for self-directed learning, such as learning vocabularies through messaging services or using word processors for writing. However, few studies in our research provided their mechanisms for using the instant feedback to facilitate the interaction between mobile devices and users (e.g., Oberg & Daniels, 2013; Ozcelik & Acarturk, 2011), which is an important element of effective self-directed learning with computers. Therefore, more elaborate methods of implementation, such as a monitoring mechanisms for learning EFL vocabularies through the message services of cell phones, an annotation system for reading e-books (e.g., Hwang, Shadiev, & Huang, 2011), speech recognition for providing feedback to students' oral practices (e.g., Tanner & Landon, 2009), etc., should be considered to enhance the interaction between learner and computers and the effects of self-directed learning. 4.2. Enhancing the quality of the experimental design for mobile intervention While it was found in this study that mobile devices can enhance educational effects, the actual impact of mobile learning programs needs to be enhanced by longer intervention durations, closer integration of technology and the curriculum, and further assessment of higher-level skills. The intervention duration will affect the reliability and ecological validity of mobile learning programs. Of all of the included interventions in this study, those with durations of >6 months constituted only about 8.3% of the research, and more than 27.2% took place within 1 week. With short programs, and especially those that last for only hours, it is difﬁcult to prove that any effects are produced by the features of mobile-integrated instruction rather than by the experience of technology

266

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

novelty. Moreover, short-term projects may adapt poorly to regular classroom practices that may last for several months. Another issue related to teaching duration is the closeness of the integration between mobile devices and the curriculum. Most of the short programs included in our study involved only one or two units of teaching materials in the curriculum of a whole semester. Although it is not necessary for a teacher to use mobile devices in every class, different units or topics may involve different instructional designs when such devices are being used, and hence an iterative trial process is likely to be needed to determine the optimal procedure for the best effects. Therefore, an abundance of mobile learning units will help to provide exemplar models for teachers and enhance the possibility of transferring practices to different lessons. Furthermore, in terms of research, it can improve the reliability and ecological validity of mobile programs in education. Based on the above considerations, researchers may consider appropriate intervention durations according to the skills or teaching methods to be developed with mobile devices. For example, for vocabulary-learning, bite-size materials and short-term durations may be appropriate for learners, but for more complex skills or methods such as inquiry or cooperative learning, longer interventions may be needed to warrant the effect of mobile programs. Another effect of mobile usage that could be strengthened is the expansion of measurements of dependent variables. Most of the studies in our research currently still placed the interests on achievement in content knowledge (e.g., Liu, 2009; Wang & Wu, 2011), and methods for measuring higher-level skills were scarce. Mobile devices were expected to encourage innovation in education and increase high-level abilities (Frohberg et al., 2009; Sung, Chang, & Yang, 2015; Zucker & Light, 2009). Yet most of the research collected for this study focused on increasing content learning, and even though the designed educational activities involve explorative, communication, and cooperative skills, the dependent variables had almost no connection with these skills. For example, in the database of our research, only 5 of the 9 experimental/quasi experimental studies explored the interactive behaviors of students during their mobile learning; furthermore, none of the 24 inquiry-oriented learning recorded and investigated process-related skills such as hypothesis-formation and hypothesistesting. Therefore, including dependent variables besides content knowledgedsuch as problem-solving, critical thinking, interactive communication, or creative innovation skillsdin the measurements will make the persuasiveness of the educational effects of mobile devices much more convincing. 4.3. Empowering educational practitioners through the orchestration of mobile devices, software, and pedagogical design Scholars (e.g., Gao, Liu, & Paas, in press; Liu et al., 2012) have gradually reached a consensus that exerting the maximum effect of information technology in the educational ﬁeld requires reconciliation of the connection among the components of technology (hardware and software), educational context and missions (e.g., learning and teaching processes in different settings), and users (teachers and students) in order to overcome many of the limitations present in the ﬁeld. Scholars rez, 2013) came to agree that the (Dillenbourg, Nussbaum, Dimitriadis, & Roschelle, 2013; Dimitriadis, Prieto, & Asensio-Pe efforts of building harmonious relationships among those components to enable compatible, efﬁcient, and effective technology-enhanced teaching and learning environments may be called orchestration. To achieve orchestration in mobileintegrated education requires the pursuit of at least two directions for research and practices. The ﬁrst is strengthening the functions and expanding the applicability and breadth of learning-oriented software. For example, the research analyzed in this study paired many different learning-oriented software programs with educational activities (e.g., reciprocal teaching, inquiry learning, and formative assessment) that have already proven effective. That software may be modiﬁed to provide the functionality of authoring tools that allow teachers to ﬂexibly arrange their own teaching and learning ﬂows in the classroom. The second direction is strengthening professional teacher-development programs for mobile-enhanced instruction. Most review research into the use of mobile devices for education has emphasized that one of the largest obstacles to implementing effective mobile learning programs is insufﬁcient preparation of the teachers (Frohberg et al., 2009; Penuel, 2006). The essence of effective professional development for technology-enhanced inquiry proposed by Gerard et al. (2011) is also applicable to mobile learning programs. Teachers should be encouraged to modify already developed mobile-integrated education programs, and to gradually customize them into their own personalized program rather than simply designing their own program around the use of technology. The latter approach implicitly leads teachers to technology-adapted instruction, which means that the educational practices of the teachers may be restricted by the functions of technology, and may make it difﬁcult for teachers to change their existing beliefs and habits. In contrast, customizing existing research-based mobile learning programs not only transfers researchers' visions and experiences for the use of technology to teachers, but also minimizes the time teachers spend on formulating new ideas and performing trial-and-error iterative procedures (Gerard et al., 2011; Penuel et al., 2007). To facilitate the transition of researchers' vision, experiences, and skills to school teachers, it is also helpful to involve university-level researchers as mentors or collaborators. Diverse functions and types of hardware and software are available for mobile devices, but conversely the complexity is also high, and hence designing and using them can readily impose additional overhead on teachers. The plethora of technological knowledge and resources that are available to researchers for educational technology means that their participation in a program can result in their knowledge and experience greatly assisting the teachers' autonomy in implementation. Another note-worthy fact is that, despite the importance of teachers' professional development during their adoption of and adaptation to mobile-device based teaching (Newhouse, Williams, & Pearson, 2006; Penuel & Yarnall, 2005), the investigations into increasing the education of teachers regarding the use of mobile devices have been extremely limited. Therefore, more in-depth experimental research is needed into how teachers reconcile mobile hardware and software, lesson content, teaching methods, and educational goals.

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

267

Acknowledgment This research was supported by grants from the Ministry of Science and Technology, Taiwan (101-2511-S-003-047-MY3; 102-2911-I-003-301; 102-2511-S-003-001-MY3; 103-2911-I-003-301; 103-2511-S-003 -058 -MY3; 104-2511-S-003 -018 -MY3), and the Aim for the Top University Project of National Taiwan Normal University. Appendix A. Related review of the research into Integrating Mobile Devices with Teaching and Learning.

Study

Devices are focused on

Method

Number of studies Result

Penuel (2006) synthesized ﬁndings from research and evaluation studies that analyzed implementation and effects of one-to-one initiatives from a range of countries. Factors related to successful implementation reported in the research include extensive teacher professional development, access to technical support, and positive teacher attitudes toward student technology use. Penuel (2006) found that outcome studies with rigorous designs are few, but those studies that did measure outcomes consistently reported positive effects on technology use, technology literacy, and writing skills. Frohberg et al. Laptops Narrative 102 (mobile Frohberg et al. (2009) used a mobile learning framework to evaluate and categorize (2009) review learning projects) 102 mobile learning projects, and to brieﬂy introduce exemplary projects for each category. Despite the fact that mobile phones initially started as a communication device, communication and collaboration play a surprisingly small role in Mobile Learning projects. Zucker and Laptops Narrative 31(not provided Zucker and Light (2009) found research in many nations suggests that laptop programs Light (2009) review publication list) will be most successful as part of balanced, comprehensive initiatives that address changes in education goals, curricula, teacher training, and assessment. Laptops Narrative 5 Bebell and O'Dwyer (2010) summarized evidence that participation in the 1:1 Bebell and review computer programs was associated with increased student and teacher technology O'Dwyer use, increased student engagement and interest level, and modest increases in student (2010) achievement. Hwang and Various types of Content 154(not provided Hwang and Tsai (2011) examined the mobile or ubiquitous learning papers published Tsai (2011) mobile device analysis publication list) in the Social Science Citation Index (SSCI) database from 2001 to 2010. Hwang and Tsai (2011) found that the number of articles has signiﬁcantly increased during the past 10 years; moreover, researchers from the different countries have contributed to the related ﬁeld in recent years. Wong and Looi Laptops Narrative 54 Wong and Looi (2011) aimed to further investigate the meaning of seamless learning (2011) review and the potential ways to put it in practice. Through a thorough review of recent academic papers on mobile-assisted seamless learning (MSL), Wong and Looi (2011) identify ten dimensions that characterize MSL. Fleischer Narrative review Narrative 18 Fleischer (2012) reviewed cross-disciplinary accumulated empirical research on one(2012) review to-one computer projects in school settings as published in peer-reviewed journals between 2005 and 2010, particularly the results of teacher- and pupil-oriented studies. The results of Fleischer (2012) show that the research field has not developed substantially since the previously published reviews. One the other hand, Fleischer (2012) discussed the reasons for this lack of development, as well as the need for political, scholarly and epistemological awareness when researching questions of oneto-one computer projects.

Penuel (2006) Laptops

Narrative 30(not provided review publication list)

Appendix B. Forest plot of the effect sizes and 95% CI of the 110 articles.

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

269

Appendix C. Cross analyses of moderator variables.

Table C1 Cross-analysis of teaching methods, domain subjects, and hardware used. Teaching method Not Lectures mentioned

Inquiry-oriented Cooperative learning learning

Domain Language 6 (15.4%) 1 (2.6%) 2 (5.1%) subjects arts Social studies 0 (0.0%) 0 (0.0%) 3 (60.0%) Science 2 (7.4%) 5 (18.5%) 11 (40.7%) Mathematics 2 (16.7%) 0 (0.0%) 2 (16.7%) General 3 (50.0%) 0 (0.0%) 0 (0.0%) Professional 1 (3.7%) 6 (22.2%) 6 (22.2%) subjects Total 14 (12.1%) 12 (10.3%) 24 (20.7%) Hardware Not used mentioned Handhelds Laptops Mixed Total Software used

Not mentioned General purpose Learningoriented

Total

0 (0.0%) 2 6 1 9

0 (0.0%)

Game-based learning

Selfdirected study

5 (12.8%)

0 (0.0%)

20 (51.2%)

0 (0.0%) 1 (3.7%) 2 (16.7%) 0 (0.0%) 2 (7.4%)

1 (20%) 0 (0.0%) 3 (25.0%) 0 (0.0%) 0 (0.0%)

0 (0.0%) 2 (7.4%) 1 (8.3%) 3 (50.0%) 8 (29.6%)

4 (3.4%)

34 (29.3%)

10 (8.6%)

1 (50.0%)

0 (0.0%)

0 (0.0%)

(2.5%) 10 (12.7%) 18 (22.8%) (25.0%) 2 (8.3%) 5 (20.8%) (33.3%) 0 (0.0%) 0 (0.0%) (8.3%) 12 (11.1%) 24 (22.2%)

7 (8.9%) 2 (8.3%) 0 (0.0%) 9 (8.3%)

3 (3.8%) 1 (4.2%) 0 (0.0%) 4 (3.7%)

1 (50.0%) 27 4 2 34

Mixed

Computerassisted testing

Total

4 (10.3%) 1 (2.6%)

39 (100%)

0 (0.0%) 5 (18.5%) 1 (8.3%) 0 (0.0%) 0 (0.0%)

5 27 12 6 27

10 (8.6%)

1 (20%) 1 (3.7%) 1 (8.3%) 0 (0.0%) 4 (14.8%) 8 (6.9%)

0 (0.0%)

0 (0.0%)

(34.2%) (16.7%) (66.7%) (31.5%)

6 (7.6%) 4 (16.7%) 0 (0.0%) 10 (9.3%)

6 (7.6%) 0 (0.0%) 0 (0.0%) 6 (5.5%)

0 (0.0%)

0 (0.0%)

(100%) (100%) (100%) (100%) (100%)

116 (100%) 2 (100%) 79 24 3 108

(100%) (100%) (100%) (100%)

1 (33.3%)

0 (0.0%)

0 (0.0%)

1 (33.3%)

0 (0.0%)

1 (33.3%)

6 (16.2%)

9 (24.3%)

5 (13.5%)

1 (2.7%)

0 (0.0%)

13 (35.1%)

2 (5.4%)

1 (2.7%)

37 (100%)

2 (2.9%)

3 (4.4%)

19 (27.9%)

7 (10.2%)

4 (5.9%)

20 (29.4%)

8 (11.8%) 5 (7.4%)

68 (100%)

9 (8.3%)

4 (3.7%)

34 (31.5%)

9 (8.3%)

12 (11.1%) 24 (22.2%)

10 (9.3%)

6 (5.6%)

3 (100%)

108 (100%)

Table C2 Cross-analysis for intervention durations, hardware used, software used, and teaching methods. Intervention duration Not mentioned

1 week

>1, 4 weeks

>1 month, 6 months

>6 months

Total

Hardware used

Not mentioned Handhelds Laptops Mixed Total

0 (0.0%) 4 (5.1%) 3 (12.5%) 0 (0.0%) 7 (6.5%)

0 (0.0%) 23 (29.1%) 5 (20.8%) 2 (66.7%) 30 (27.8%)

0 (0.0%) 20 (25.3%) 6 (25.0%) 1 (33.3%) 27 (25.0%)

2 (100.0%) 29 (36.7%) 4 (16.7%) 0 (0.0%) 35 (32.4%)

0 (0.0%) 3 (3.8%) 6 (25.0%) 0 (0.0%) 9 (8.3%)

2 79 24 3 108

(100.0%) (100.0%) (100.0%) (100.0%) (100.0%)

Software used

Not mentioned General purpose Learning-oriented Total

1 2 4 7

(33.3%) (5.4%) (5.9%) (6.5%)

0 (0.0%) 7 (18.9%) 23 (33.8%) 30 (27.8%)

2 7 18 27

(66.7%) (18.9%) (26.5%) (25.0%)

0 (0.0%) 15 (40.5%) 20 (29.4%) 35 (32.4%)

0 (0.0%) 6 (16.2%) 3 (4.4%) 9 (8.3%)

3 37 68 108

(100.0%) (100.0%) (100.0%) (100.0%)

Teaching method

Not mentioned Lecture Inquiry-oriented Cooperative learning Game-based learning Self-directed study Mixed Computer-assisted testing Total

1 (11.1%) 1 (8.3%) 3 (12.5%) 0 (0.0%) 0 (0.0%) 1 (2.9%) 0 (0.0%) 1 (16.7%) 7 (6.5%)

1 (11.1%) 5 (41.7%) 9 (37.5%) 2 (22.2%) 2 (50.0%) 9 (26.5%) 0 (0.0%) 2 (33.3%) 30 (27.8%)

1 (11.1%) 0 (0.0%) 5 (20.8%) 4 (44.4%) 2 (50.0%) 13 (38.2%) 1 (10.0%) 1 (16.7%) 27 (25.0%)

2 (22.2%) 5 (41.7%) 7 (29.2%) 2 (22.2%) 0 (0.0%) 11 (32.4%) 6 (60.0%) 2 (33.3%) 35 (32.4%)

4 (44.4%) 1 (8.3%) 0 (0.0%) 1 (11.1%) 0 (0.0%) 0 (0.0%) 3 (30.0%) 0 (0.0%) 9 (8.3%)

9 12 24 9 4 34 10 6 108

(100.0%) (100.0%) (100.0%) (100.0%) (100.0%) (100.0%) (100.0%) (100.0%) (100.0%)

270

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

Table C3 Cross-analysis for implementation settings and software used. Software used

Implementation setting

Total

Not mentioned Formal settings Informal settings Unrestricted

Not mentioned

General purpose

Learning-oriented

Total

0 (0.0%) 2 (3.3%) 0 (0.0%) 1 (4.0%) 3 (2.8%)

1 22 4 10 37

1 36 17 14 68

2 60 21 25 108

(50.0%) (36.7%) (19.0%) (40.0%) (34.3%)

(50.0%) (60.0%) (81.0%) (56.0%) (63.0%)

(100.0%) (100.0%) (100.0%) (100.0%) (100.0%)

References* Ary, D., Jacobs, L. C., & Razavieh, A. (2002). Introduction to research in education (6th ed.). Belmont, CA: Wadsworth. Bakhurst, D. (2009). Reﬂections on activity theory. Educational Review, 61(2), 197e210. http://dx.doi.org/10.1080/00131910902846916. lu, E. B., & Akdemir, O. (2010). A comparison of undergraduate students' English vocabulary learning: using mobile phones and ﬂash cards. The Turkish Bas¸og Online Journal of Educational Technology, 9, 1e7. Retrieved from http://eric.ed.gov/?id¼EJ898010. Bebell, D., & O'Dwyer, L. M. (2010). Educational outcomes and research from 1:1 computing settings. Journal of Technology, Learning, and Assessment, 9, 5e15. Retrieved from http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1606. Becker, H. J. (1991). How computers are used in United States schools: basic data from the 1989 I.E.A. Computers in Education survey. Journal of Educational Computing Research, 7, 385e405. http://dx.doi.org/10.2190/P2UT-R3U3-FK1L-B89L. Becker, H. J. (2001, April). How are teachers using computers in instruction?. Paper presented at the 2001 Annual Meeting of American Educational Research Association, Seattle, WA. Best, J. W., & Kahn, J. V. (1998). Research in education (8th ed.). Boston, MA: Allyn and Bacon. Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. Chichester, UK: Wiley. Byun, H., Lee, J., & Cerreto, F. A. (2014). Relative effects of three questioning strategies in ill structured, small group problem solving. Instructional Science, 42, 229e250. http://dx.doi.org/10.1007/s11251-013-9278-1. Chan, T. W., Roschelle, J., Hsi, S., Kinshuk, Sharples, M., Brown, T., et al. (2006). One-to-one technology-enhanced learning: an opportunity for global research collaboration. Research and Practice in Technology Enhanced Learning, 1(01), 3e29. http://dx.doi.org/10.1142/S1793206806000032. Chen, C. M., & Chen, M. C. (2009). Mobile formative assessment tool based on data mining techniques for supporting web-based learning. Computers & Education, 52, 256e273. http://dx.doi.org/10.1016/j.compedu.2008.08.005. Cheung, A. C. K., & Slavin, R. E. (2013). The effectiveness of educational technology applications for enhancing mathematics achievement in K-12 classrooms: a meta-analysis. Educational Research Review, 9, 88e113. http://dx.doi.org/10.1016/j.edurev.2013.01.001. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum. Dierking, L. D., Falk, J. H., Rennie, L., Anderson, D., & Ellenbogen, K. (2003). Policy statement of the “informal science education” ad hoc committee. Journal of Research in Science Teaching, 40, 108e111. http://dx.doi.org/10.1002/tea.10066. Dillenbourg, P., Nussbaum, M., Dimitriadis, Y., & Roschelle, J. (2013). Design for classroom orchestration. Computers & Education, 69, 485e492. http://dx.doi. org/10.1016/j.compedu.2012.10.026. rez, J. I. (2013). The role of design and enactment patterns in orchestration: helping to integrate technology in Dimitriadis, Y., Prieto, L. P., & Asensio-Pe blended classroom ecosystems. Computers & Education, 69, 496e499. http://dx.doi.org/10.1016/j.compedu.2013.04.004. Drayton, B., Falk, J. K., Stroud, R., Hobbs, K., & Hammerman, J. (2010). After installation: ubiquitous computing and high school science in three experienced, high-technology schools. Journal of Technology, Learning, and Assessment, 9, 4e56. Retrieved from http://ejournals.bc.edu/ojs/index.php/jtla/article/ view/1608. Ertmer, P., & Ottenbreit-Leftwich, A. (2010). Teacher technology change: how knowledge, beliefs, and culture intersect. Journal of Research on Technology in Education, 42, 255e284. http://dx.doi.org/10.1080/15391523.2010.10782551. Fleischer, H. (2012). What is our current understanding of one-to-one computer projects: a systematic narrative research review. Educational Research Review, 7, 107e122. http://dx.doi.org/10.1016/j.edurev.2011.11.004. Frohberg, D., Goth, C., & Schwabe, G. (2009). Mobile learning projects e a critical analysis of the state of the art. Journal of Computer Assisted Learning, 25, 307e331. http://dx.doi.org/10.1111/j.1365-2729.2009.00315.x. Furtak, E. M., Seidel, T., Iverson, H., & Briggs, D. C. (2012). Experimental and quasi-experimental studies of inquiry-based science teaching: a meta-analysis. Review of Educational Research, 82, 300e329. http://dx.doi.org/10.3102/0034654312457206. Gao, Y., Liu, T.C., & Paas, F. (in press). Effects of mode of target task selection on learning about plants in a mobile learning environment: effortful manual selection versus effortless QR-code selection. Journal of Educational Psychology. doi: http://dx.doi.org/10.1037/edu0000080. Gerard, L. F., Varma, K., Corliss, S. B., & Linn, M. C. (2011). Professional development for technology-enhanced inquiry science. Review of Educational Research, 81, 408e448. http://dx.doi.org/10.3102/0034654311415121. Gillies, R. M., & Haynes, M. (2011). Increasing explanatory behavior, problem solving, and reasoning within classes using cooperative group work. Instructional Science, 39, 349e366. http://dx.doi.org/10.1007/s11251-010-9130-9. Gleaves, A., Walker, C., & Grey, J. (2007). Using digital and paper diaries for learning and assessment purposes in higher education: a comparative study of feasibility and reliability. Assessment & Evaluation in Higher Education, 32, 631e643. http://dx.doi.org/10.1080/02602930601117035. Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. London, England: Routledge. Holcomb, L. B. (2009). Results & lessons learned from 1:1 laptop initiatives: a collective review. TechTrends, 53, 49e55. http://dx.doi.org/10.1007/s11528009-0343-1. Hsi, S. (2003). A study of user experiences mediated by nomadic web content in a museum. Journal of Computer Assisted Learning, 19, 308e319. http://dx.doi. org/10.1046/j.0266-4909.2003.jca_023.x. Hsieh, P., Acee, T., Chung, W., Hsieh, Y. P., Kim, H., Thomas, G. D., et al. (2005). Is educational intervention research on the decline? Journal of Educational Psychology, 97, 523e529. http://dx.doi.org/10.1037/0022-0663.97.4.523. Hsu, C. K., Hwang, G. J., Chang, Y. T., & Chang, C. K. (2013). Effects of video caption modes on English listening comprehension and vocabulary acquisition using handheld devices. Educational Technology & Society, 16(1), 403e414. Retrieved from http://www.ifets.info/journals/16_1/35.pdf. Hsu, L., & Lee, S. N. (2011). Learning tourism English on mobile phones: how does it work? Journal of Hospitality, Leisure, Sport and Tourism Education, 10, 85e94. http://dx.doi.org/10.3794/johlste.102.348.

*

Please see the Further readings for the studies included for meta-analysis.

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

271

Huizenga, J., Admiraal, W., Akkerman, S., & ten Dam, G. (2009). Mobile game-based learning in secondary education: engagement, motivation and learning in a mobile city game. Journal of Computer Assisted Learning, 25, 332e344. http://dx.doi.org/10.1111/j.1365-2729.2009.00316.x. Hwang, G. J., Chu, H. C., Lin, Y. S., & Tsai, C. C. (2011). A knowledge acquisition approach to developing Mindtools for organizing and sharing differentiating knowledge in a ubiquitous learning environment. Computers & Education, 57, 1368e1377. http://dx.doi.org/10.1016/j.compedu.2010.12.013. Hwang, G. J., & Tsai, C. C. (2011). Research trends in mobile and ubiquitous learning: a review of publications in selected journals from 2001 to 2010. British Journal of Educational Technology, 42, 65e70. http://dx.doi.org/10.1111/j.1467-8535.2011.01183.x. Hwang, W. Y., Shadiev, R., & Huang, S. M. (2011). A study of a multimedia web annotation system and its effect on the EFL writing and speaking performance of junior high school students. ReCALL, 23(2), 160e180. http://dx.doi.org/10.1017/S0958344011000061. Inan, F. A., & Lowther, D. L. (2010). Laptops in the K-12 classrooms: exploring factors impacting instructional use. Computers & Education, 55, 937e944. http://dx.doi.org/10.1016/j.compedu.2010.04.004. Jabbar, A. I. A., & Felicia, P. (2015). Gameplay engagement and learning in game-based learning: a systematic review. Review of Educational Research, 85(4), 740e779. http://dx.doi.org/10.3102/0034654315577210. Jeong, H. Y., & Hong, B. H. (2013). A practical use of learning system using user preference in ubiquitous computing environment. Multimedia Tools and Applications, 64, 491e504. http://dx.doi.org/10.1007/s11042-012-1026-z. Jian, H. L., Sandnes, F. E., Law, K. M. Y., Huang, Y. P., & Huang, Y. M. (2009). The role of electronic pocket dictionaries as an English learning tool among Chinese students. Journal of Computer Assisted Learning, 25, 503e514. http://dx.doi.org/10.1111/j.1365-2729.2009.00325.x. Ketamo, H. (2003). An adaptive geometry game for handheld devices. Educational Technology & Society, 6(1), 83e95. Retrieved from http://www.ifets.info/ journals/6_1/ketamo.html. Kim, P., Buckner, E., Kim, H., Makanya, T., Taleja, N., & Parikh, V. (2011). A comparative analysis of a game-based mobile learning model in lowsocioeconomic communities of India. International Journal of Educational Development, 32, 329e340. http://dx.doi.org/10.1016/j.ijedudev.2011.05.008. Klopfer, E., Sheldon, J., Perry, J., & Chen, V. H.-H. (2012). Ubiquitous games for learning (UbiqGames): Weatherlings, a worked example. Journal of Computer Assisted Learning, 28, 465e476. http://dx.doi.org/10.1111/j.1365-2729.2011.00456.x. Kondo, M., Ishikawa, Y., Smith, C., Sakamoto, K., Shimomura, H., & Wada, N. (2012). Mobile assisted language learning in university EFL courses in Japan: developing attitudes and skills for self-regulated learning. ReCALL, 24, 169e187. http://dx.doi.org/10.1017/S0958344012000055. Kukulska-Hulme, A., & Shield, L. (2008). An overview of mobile assisted language learning: from content delivery to supported collaboration and interaction. ReCALL, 20, 271e289. http://dx.doi.org/10.1017/S0958344008000335. Kulik, C. L. C., & Kulik, J. A. (1991). Effectiveness of computer-based instruction: an updated analysis. Computers in Human Behavior, 7, 75e94. http://dx.doi. org/10.1016/0747-5632(91)90030-5. Lai, C. Y., & Wu, C. C. (2006). Using handhelds in a jigsaw cooperative learning environment. Journal of Computer Assisted Learning, 22, 284e297. http://dx. doi.org/10.1111/j.1365-2729.2006.00176.x. Lan, Y. J., Sung, Y. T., & Chang, K. E. (2007). A mobile-device-supported peer-assisted learning system for collaborative early EFL reading. Language Learning & Technology, 11, 130e151. Retrieved from http://llt.msu.edu/vol11num3/pdf/lansungchang.pdf. Lan, Y. J., Sung, Y. T., & Chang, K. E. (2009). Let us read together: development and evaluation of a computer-assisted reciprocal early English reading system. Computers & Education, 53, 1188e1198. http://dx.doi.org/10.1016/j.compedu.2009.06.002. Lan, Y. J. (2014). Does second life improve mandarin learning by overseas Chinese students? Language Learning & Technology, 18, 36e56. Retrieved from http://llt.msu.edu/issues/june2014/action2.pdf. Lan, Y. J., Sung, Y. T., Cheng, C. C., & Chang, K. E. (2015). Computer-supported cooperative prewriting for enhancing young EFL learners' writing performance. Language Learning & Technology, 19, 134e155. Retrieved from http://llt.msu.edu/issues/june2015/lansungchengchang.pdf. Langman, J., & Fies, C. (2010). Classroom response system-mediated science learning with English language learners. Language and Education, 24, 81e99. http://dx.doi.org/10.1080/09500780903096553. Li, L. Y., Chen, G. D., & Yang, S. J. (2013). Construction of cognitive maps to improve e-book reading and navigation. Computers & Education, 60, 32e39. http:// dx.doi.org/10.1016/j.compedu.2012.07.010. Lipsey, M. W., & Wilson, D. B. (2000). Practical meta-analysis. Thousand Oaks, CA: Sage. Liu, T. C., Lin, Y. C., & Paas, F. (2014). Effects of prior knowledge on learning from different compositions of representations in a mobile learning environment. Computers & Education, 72, 328e338. http://dx.doi.org/10.1016/j.compedu.2013.10.019. Liu, C. C., Tao, S. Y., & Nee, J. N. (2008). Bridging the gap between students and computers: supporting activity awareness for network collaborative learning with GSM network. Behaviour & Information Technology, 27, 127e137. http://dx.doi.org/10.1080/01449290601054772. Liu, T. C., Lin, Y. C., Tsai, M. J., & Paas, F. (2012). Split-attention and redundancy effects in mobile learning in physical environments. Computers & Education, 58, 172e180. http://dx.doi.org/10.1016/j.compedu.2011.08.007. Liu, T. Y. (2009). A context-aware ubiquitous learning environment for language listening and speaking. Journal of Computer Assisted Learning, 25(6), 515e527. http://dx.doi.org/10.1111/j.1365-2729.2009.00329.x. Liu, T. Y., Tan, T. H., & Chu, Y. L. (2009). Outdoor natural science learning with an RFID-supported immersive ubiquitous learning environment. Educational Technology & Society, 12, 161e175. Retrieved from http://www.ifets.info/journals/12_4/15.pdf. Looi, C. K., Zhang, B., Chen, W., Seow, P., Chia, G., Norris, C., et al. (2011). 1:1 mobile inquiry learning experience for primary science students: a study of learning effectiveness. Journal of Computer Assisted Learning, 27, 269e287. http://dx.doi.org/10.1111/j.1365-2729.2010.00390.x. Lu, M. (2008). Effectiveness of vocabulary learning via mobile phone. Journal of Computer Assisted Learning, 24, 515e525. http://dx.doi.org/10.1111/j.13652729.2008.00289.x. Lu, Z. J. (2012). Learning with mobile technologies, handheld devices, and smart phones: Innovative methods (pp. 1e272). IGI-Global. McMillan, J. H., Venable, J. J., & Varier, D. (2013). Studies of the effect of formative assessment on student achievement: so much more is needed. Practical Assessment, Research & Evaluation, 18(2). Retrieved from http://pareonline.net/pdf/v18n2.pdf. Morris, S. B. (2008). Estimating effect sizes from pretest-posttest-control group designs. Organizational Research Methods, 11, 364e386. http://dx.doi.org/10. 1177/1094428106291059. Mouza, C. (2008). Learning with laptops: implementation and outcomes in an urban, under-privileged school. Journal of Research on Technology in Education, 40, 447e472. http://dx.doi.org/10.1080/15391523.2008.10782516. Newhouse, C. P., Williams, P. J., & Pearson, J. (2006). Supporting mobile education for pre-service teachers. Australasian Journal of Educational Technology, 22, 289e311. Retrieved from http://www.ascilite.org.au/ajet/ajet22/newhouse.html. Oberg, A., & Daniels, P. (2013). Analysis of the effect a student-centred mobile learning instructional method has on language acquisition. Computer Assisted Language Learning, 26(2), 177e196. http://dx.doi.org/10.1080/09588221.2011.649484. Orwin, R. G. (1983). A fail-safe N for effect size in meta-analysis. Journal of Educational and Behavioral Statistics, 8, 157e159. http://dx.doi.org/10.3102/ 10769986008002157. Ozcelik, E., & Acarturk, C. (2011). Reducing the spatial distance between printed and online information sources by means of mobile technology enhances learning: using 2D barcodes. Computers & Education, 57(3), 2077e2085. http://dx.doi.org/10.1016/j.compedu.2011.05.019. Penuel, W. R. (2006). Implementation and effects of 1:1 computing initiatives: a research synthesis. Journal of Research on Technology in Education, 38, 329e348. http://dx.doi.org/10.1080/15391523.2006.10782463. Penuel, W. R., Roschelle, J., & Shechtman, N. (2007). Designing formative assessment software with teachers: an analysis of the co-design process. Research and Practice in Technology Enhanced Learning, 2, 51e74. http://dx.doi.org/10.1142/S1793206807000300. Penuel, W. R., & Yarnall, L. (2005). Designing handheld software to support classroom assessment: analysis of conditions for teacher adoption. Journal of Technology, Learning, and Assessment, 3, 1e46. Retrieved from http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1658.

272

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

Pressley, M., & Harris, K. R. (1994). Increasing the quality of educational intervention research. Educational Psychology Review, 6, 191e208. http://dx.doi.org/ 10.1007/BF02213181. Purrazzella, K., & Mechling, L. C. (2013). Evaluation of manual spelling, observational and incidental learning using computer-based instruction with a tablet PC, large screen projection, and a forward chaining procedure. Education and Training in Autism and Developmental Disabilities, 48, 218e235. Retrieved from http://daddcec.org/Portals/0/CEC/Autism_Disabilities/Research/Publications/Education_Training_Development_Disabilities/Full_Journals/ETADD_ 48_2_218-235.pdf. Riconscente, M. M. (2013). Results from a controlled study of the iPad fractions game motion math. Games and Culture, 8, 186e214. http://dx.doi.org/10.1177/ 1555412013496894. Roschelle, J., & Pea, R. D. (2002). A walk on the WILD side: how wireless handhelds may change computer-supported collaborative learning. International Journal of Cognition and Technology, 1, 145e168. Retrieved from https://hal.archives-ouvertes.fr/ﬁle/index/docid/190615/ﬁlename/A110_Roschelle_Pea_ 02_WILD.pdf. Roschelle, J., Rafanan, K., Bhanot, R., Estrella, G., Penuel, B., Nussbaum, M., et al. (2010). Scaffolding group explanation and feedback with handheld technology: impact on students' mathematics learning. Educational Technology Research and Development, 58, 399e419. http://dx.doi.org/10.1007/ s11423-009-9142-9. Rosenthal, R. (1979). The ﬁle drawer problem and tolerance for null results. Psychological Bulletin, 86, 638e641. http://dx.doi.org/10.1037/0033-2909.86.3. 638. Runyan, J. D., Steenbergh, T. A., Bainbridge, C., Daugherty, D. A., Oke, L., & Fry, B. N. (2013). A smartphone ecological momentary assessment/intervention “App” for collecting real-time data and promoting self-awareness. PLoS ONE, 8, e71325. http://dx.doi.org/10.1371/journal.pone.0071325. Ryu, H., & Parsons, D. (2012). Risky business or sharing the load? Social ﬂow in collaborative mobile learning. Computers & Education, 58, 707e720. http:// dx.doi.org/10.1016/j.compedu.2011.09.019. lu, G., & Cag ıltay, K. (2012). Mobile language learning: contribution of multimedia messages via mobile phones in consolidating voSaran, M., Seferog cabulary. The Asia-Paciﬁc Education Researcher, 21, 181e190. Retrieved from http://www.ejournals.ph/index.php? journal¼TAPER&page¼article&op¼viewArticle&path%5B%5D¼4826. Shapley, K. S., Sheehan, D., Maloney, C., & Caranikas-Walker, F. (2010). Evaluating the implementation ﬁdelity of technology immersion and its relationship with student achievement. Journal of Technology, Learning, and Assessment, 9, 5e68. Retrieved from http://ejournals.bc.edu/ojs/index.php/jtla/article/ view/1609. Sharples, M., Taylor, J., & Vavoula, G. (2007). A theory of learning for the mobile age. In R. Andrews, & C. Haythornthwaite (Eds.), The Sage Handbook of Elearning Research (pp. 221e247). London, UK: Sage Publications. Shih, S. C., Kuo, B. C., & Liu, Y. L. (2012). Adaptively ubiquitous learning in campus Math path. Educational Technology & Society, 15, 298e308. Retrieved from http://www.ifets.info/journals/15_2/25.pdf. Siozos, P., Palaigeorgiou, G., Triantafyllakos, G., & Despotakis, T. (2009). Computer based testing using “digital ink”: participatory design of a tablet PC based assessment application for secondary education. Computers & Education, 52, 811e819. http://dx.doi.org/10.1016/j.compedu.2008.12.006. Slavin, R. E. (1977). Classroom reward structure: an analytical and practical review. Review of Educational Research, 47, 633e650. Retrieved from http://www. jstor.org/stable/pdfplus/1170003.pdf?acceptTC¼true&jpdConﬁrm¼true. Slavin, R. E. (1993). When and why does cooperative learning increase achievement? Theoretical and empirical perspectives. In H. Daniels, & A. Edwards (Eds.), The RoutledgeFalmer reader in psychology of education (pp. 271e293). New York, NY: RoutledgeFalmer. Slavin, R. E. (2012). Instruction based on cooperative learning. In R. E. Mayer, & P. A. Alexander (Eds.), Handbook of research on learning and instruction (pp. 344e360). New York, NY: Routledge. Sung, Y. T., Chang, K. E., Hou, H. T., & Chen, P. F. (2010). Designing an electronic guidebook for learning engagement in a museum of history. Computers in Human Behavior, 26, 74e83. http://dx.doi.org/10.1016/j.chb.2009.08.004. Sung, Y. T., Hou, H. T., Liu, C. K., & Chang, K. E. (2010c). Mobile guide system using problem-solving strategy for museum learning: a sequential learning behavioural pattern analysis. Journal of Computer Assisted Learning, 26, 106e115. http://dx.doi.org/10.1111/j.1365-2729.2010.00345.x. Sung, Y. T., & Lesgold, A. (2007). Software infrastructure for teachers: a missing link in integrating technology with instruction. Teachers College Record, 109, 2541e2575. Retrieved from http://www.tcrecord.org/Content.asp?contentid¼14536. Sung, Y. T.,Chang, K. E., & Yang, J. M. (in press). How effective are mobile devices for language learning? A meta-analysis. Educational Research Review. doi: 10.1016/j.edurev.2015.09.001. Tamim, R. M., Bernard, R. M., Borokhovski, E., Abrami, P. C., & Schmid, R. F. (2011). What forty years of research says about the impact of technology on learning: a second-order meta-analysis and validation study. Review of Educational Research, 81, 4e28. http://dx.doi.org/10.3102/0034654310393361. Tan, T. H., Liu, T. Y., & Chang, C. C. (2007). Development and evaluation of an RFID-based ubiquitous learning environment for outdoor learning. Interactive Learning Environments, 15, 253e269. http://dx.doi.org/10.1080/10494820701281431. Tanner, M. W., & Landon, M. M. (2009). The effects of computer-assisted pronunciation readings on ESL learners' use of pausing, stress, intonation, and overall comprehensibility. Language Learning and Technology, 13, 51e65. Retrieved from: http://llt.msu.edu/vol13num3/tannerlandon.pdf. Van der Kleij, F. M., Feskens, R. C., & Eggen, T. J. (2015). Effects of feedback in a computer-based learning environment on students' learning outcomes: a meta-analysis. Review of Educational Research, 85(4), 1e37. http://dx.doi.org/10.3102/0034654314564881. Wang, S. L., & Wu, C. Y. (2011). Application of context-aware and personalized recommendation to implement an adaptive ubiquitous learning system. Expert Systems with Applications, 38(9), 10831e10838. http://dx.doi.org/10.1016/j.eswa.2011.02.083. Warschauer, M. (2007). A teacher's place in the digital divide. Yearbook of the National Society for the Study of Education, 106, 147e166. http://dx.doi.org/10. 1111/j.1744-7984.2007.00118.x. Warschauer, M., Zheng, B., Niiya, M., Cotten, S., & Farkas, G. (2014). Balancing the one-to-one equation: equity and access in three laptop programs. Equity & Excellence in Education, 47, 46e62. http://dx.doi.org/10.1080/10665684.2014.866871. Weston, M., & Bain, A. (2010). The end of techno-critique: the naked truth about 1:1 laptop initiatives and educational change. Journal of Technology, Learning, and Assessment, 9, 1e26. Retrieved from http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1611. Wong, L. H., & Looi, C. K. (2011). What seams do we remove in mobile assisted seamless learning? A critical review of the literature. Computers & Education, 57, 2364e2381. http://dx.doi.org/10.1016/j.compedu.2011.06.007. Wouters, P., van Nimwegen, C., van Oostendorp, H., & van der Spek, E. D. (2013). A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology, 105(2), 249e265. http://dx.doi.org/10.1037/a0031311. Wu, T. T., Sung, T. W., Huang, Y. M., Yang, C. S., & Yang, J. T. (2011). Ubiquitous English learning system with dynamic personalized guidance of learning portfolio. Educational Technology & Society, 14, 164e180. Retrieved from http://www.ifets.info/journals/14_4/15.pdf. Yang, Y., Zhang, L., Zeng, J., Pang, X., Lai, F., & Rozelle, S. (2013). Computers and the academic performance of elementary school-aged girls in China's poor communities. Computers & Education, 60, 335e346. http://dx.doi.org/10.1016/j.compedu.2012.08.011. Zhang, B. H., Looi, C. K., Seow, P., Chia, G., Wong, L. H., Chen, W., et al. (2010). Deconstructing and reconstructing: transforming primary science learning via a mobilized curriculum. Computers & Education, 55, 1504e1523. http://dx.doi.org/10.1016/j.compedu.2010.06.016. Zhang, J., Sung, Y. T., Hou, H. T., & Chang, K. E. (2014). The development and evaluation of an augmented reality-based armillary sphere for astronomical observation instruction. Computers & Education, 73, 178e188. http://dx.doi.org/10.1016/j.compedu.2014.01.003. Zucker, A. A., & Light, D. (2009). Laptop programs for students. Science, 323, 82e85. http://dx.doi.org/10.1126/science.1167705. Zurita, G., & Nussbaum, M. (2004). Computer supported collaborative learning using wirelessly interconnected handheld computers. Computers & Education, 42, 289e314. http://dx.doi.org/10.1016/j.compedu.2003.08.005. Zurita, G., & Nussbaum, M. (2007). A conceptual framework based on activity theory for mobile CSCL. British Journal of Educational Technology, 38(2), 211e235. http://dx.doi.org/10.1111/j.1467-8535.2006.00580.x.

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

273

Further readings (The studies included for meta-analysis) Ahmed, S., & Parsons, D. (2013). Abductive science inquiry using mobile devices in the classroom. Computers & Education, 63, 62e72. http://dx.doi.org/10. 1016/j.compedu.2012.11.017. Agbatogun, A. O. (2012). Exploring the efﬁcacy of student response system in a sub-Saharan African country: a sociocultural perspective. Journal of Information Technology Education, 11, 249e267. Retrieved from http://eric.ed.gov/?id¼EJ990469. lu, E. B., & Akdemir, O. (2010). A comparison of undergraduate students' English vocabulary learning: using mobile phones and ﬂash cards. The Turkish Bas¸og Online Journal of Educational Technology, 9, 1e7. Retrieved from http://ﬁles.eric.ed.gov/fulltext/EJ898010.pdf. Bebell, D., & Kay, R. (2010). One to one computing: a summary of the quantitative results from the Berkshire wireless learning initiative. The Journal of Technology, Learning, and Assessment, 9, 1e60. Retrieved from http://napoleon.bc.edu/ojs/index.php/jtla/article/viewFile/1607/1462. Billings, E. S., & Mathison, C. (2012). I get to use an iPod in school? Using technology-based advance organizers to support the academic success of English learners. Journal of Science Education and Technology, 21, 494e503. http://dx.doi.org/10.1007/s10956-011-9341-0. Brooks, G., Miles, J. N. V., Torgerson, C. J., & Torgerson, D. J. (2006). Is an intervention using computer software effective in literacy learning? A randomised controlled trial. Educational Studies, 32, 133e143. Retrieved from http://www.jeremymiles.co.uk/mestuff/publications/p41.pdf. Bruce-Low, S. S., Burnet, S., Arber, K., Price, D., Webster, L., & Stopforth, M. (2013). Interactive mobile learning: a pilot study of a new approach for sport science and medical undergraduate students. Advances in Physiology Education, 37, 292e297. http://dx.doi.org/10.1152/advan.00004.2013. Chaiprasurt, C., & Esichaikul, V. (2013). Enhancing motivation in online courses with mobile communication tool support: a comparative study. The International Review of Research in Open and Distance Learning, 14, 377e401. Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1416/ 2610. Chang, K. E., Lan, Y. J., Chang, C. M., & Sung, Y. T. (2010). Mobile-device-supported strategy for Chinese reading comprehension. Innovations in Education and Teaching International, 47, 69e84. http://dx.doi.org/10.1080/14703290903525853. Chao, P. Y., & Chen, G. D. (2009). Augmenting paper-based learning with mobile phones. Interacting With Computers, 21, 173e185. http://dx.doi.org/10.1016/j. intcom.2009.01.001. Chen, C. M., & Chen, M. C. (2009). Mobile formative assessment tool based on data mining techniques for supporting web-based learning. Computers & Education, 52, 256e273. http://dx.doi.org/10.1016/j.compedu.2008.08.005. Chen, C. M., & Li, Y. L. (2010). Personalised context-aware ubiquitous learning system for supporting effective English vocabulary learning. Interactive Learning Environments, 18, 341e364. http://dx.doi.org/10.1080/10494820802602329. Chen, C. M., Tan, C. C., & Lo, B. J. (2013). Facilitating English-language learners' oral reading ﬂuency with digital pen technology. Interactive Learning Environments, 1e23. http://dx.doi.org/10.1080/10494820.2013.817442. Chen, N. S., Teng, D. C. E., Lee, C. H., & Kinshuk. (2011). Augmenting paper-based reading activity with direct access to digital materials and scaffolded questioning. Computers & Education, 57, 1705e1715. http://dx.doi.org/10.1016/j.compedu.2011.03.013. Chen, Y. (2010). Dictionary use and EFL learning. A contrastive study of pocket electronic dictionaries and paper dictionaries. International Journal of Lexicography, 23, 275e306. http://dx.doi.org/10.1093/ijl/ecq013. Chen, Y. S., Kao, T. C., & Sheu, J. P. (2005). Realizing outdoor independent learning with a butterﬂy-watching mobile learning system. Journal of Educational Computing Research, 33, 395e417. Retrieved from http://www.csie.ntpu.edu.tw/~yschen/mypapers/JECR-2005.pdf. Chiu, L. L., & Liu, G. Z. (2013). Effects of printed, pocket electronic, and online dictionaries on high school students English vocabulary retention. The AsiaPaciﬁc Education Researcher, 22, 619e634. http://dx.doi.org/10.1007/s40299-013-0065-1. Chu, H. C., Hwang, G. J., & Tsai, C. C. (2010). A knowledge engineering approach to developing mindtools for context-aware ubiquitous learning. Computers & Education, 54, 289e297. http://dx.doi.org/10.1016/j.compedu.2009.08.023. De Joodea, E. A., Van Heugtena, C. M., Verheya, F. R. J., & Van Boxtela, M. P. J. (2013). Effectiveness of an electronic cognitive aid in patients with acquired brain injury: a multicentre randomised parallel-group study. Neuropsychological Rehabilitation: An International Journal, 23, 133e156. http://dx.doi.org/ 10.1080/09602011.2012.726632. Du, H., Hao, J. X., Kwok, R., & Wagner, C. (2010). Can a lean medium enhance large-group communication? Examining the impact of interactive mobile learning. Journal of the American Society for Information Science and Technology, 61, 2122e2137. http://dx.doi.org/10.1002/asi.21376. Dunleavy, M., & Heinecke, W. F. (2007). The impact of 1:1 laptop use on middle school math and science standardized test scores. Computers in the Schools: Interdisciplinary Journal of Practice, Theory, and Applied Research, 24, 7e22. http://dx.doi.org/10.1300/J025v24n03_02. Eden, S., Shamir, A., & Fershtman, M. (2011). The effect of using laptops on the spelling skills of students with learning disabilities. Educational Media International, 48, 249e259. http://dx.doi.org/10.1080/09523987.2011.632274. Edwards, C. M., Rule, A. C., & Boody, R. M. (2013). Comparison of face-to-face and online mathematics learning of sixth graders. Computers in Mathematics and Science Teaching, 32, 25e47. Retrieved from http://www.editlib.org/noaccess/39231/. Gardner, J., Morrison, H., & Jarman, R. (1993). The impact of high access to computers on learning. Journal of Computer Assisted Learning, 9, 2e16. http://dx. doi.org/10.1111/j.1365-2729.1993.tb00259.x. Gebru, M. T., Phelps, A. J., & Wulfsberg, G. (2012). Effect of clickers versus online homework on students' long-term retention of general chemistry course material. Chemistry Education Research and Practice, 13, 325e329. http://dx.doi.org/10.1039/C2RP20033C. Gulek, J. C., & Demirtas, H. (2005). Learning with technology: the impact of laptop use on student achievement. The Journal of Technology, Learning, and Assessment, 3, 1e38. Retrieved from http://napoleon.bc.edu/ojs/index.php/jtla/article/view/1655/1501. Hassan, I. S., Ismail, M. A., & Mustapha, R. (2010). The effects of integrating mobile and CAD technology in teaching design process for Malaysian polytechnic architecture student in producing creative product. Turkish Online Journal of Educational Technology, 9, 162e172. Retrieved from http://ﬁles.eric.ed.gov/ fulltext/EJ908082.pdf. Hayati, A., Jalilifar, A., & Mashhadi, A. (2013). Using short message service (SMS) to teach English idioms to EFL students. British Journal of Educational Technology, 44, 66e81. http://dx.doi.org/10.1111/j.1467-8535.2011.01260.x. Ho, K., Lauscher, H. N., Broudo, M., Jarvis-Selinger, S., Fraser, J., Hewes, D., et al. (2009). The impact of a personal digital assistant (PDA) case log in a medical student clerkship. Teaching and Learning in Medicine: An International Journal, 21, 318e326. http://dx.doi.org/10.1080/10401330903228554. Hsu, L., & Lee, S. N. (2011). Learning tourism English on mobile phones: how does it work? Journal of Hospitality, Leisure, Sport and Tourism Education, 10, 85e94. http://dx.doi.org/10.3794/johlste.102.348. Huang, Y. M., Liang, T. H., Su, Y. N., & Chen, N. S. (2012). Empowering personalized learning with an interactive e-book learning system for elementary school students. Educational Technology Research and Development, 60, 703e722. http://dx.doi.org/10.1007/s11423-012-9237-6. Huang, Y. M., Lin, Y. T., & Cheng, S. C. (2010). Effectiveness of a mobile plant learning system in a science curriculum in Taiwanese elementary education. Computers & Education, 54, 47e58. http://dx.doi.org/10.1016/j.compedu.2009.07.006. Huizenga, J., Admiraal, W., Akkerman, S., & ten Dam, G. (2009). Mobile game-based learning in secondary education: engagement, motivation and learning in a mobile city game. Journal of Computer Assisted Learning, 25, 332e344. http://dx.doi.org/10.1111/j.1365-2729.2009.00316.x. Hung, P. H., Hwang, G. J., Lin, Y. F., Wu, T. H., & Su, I. H. (2013). Seamless connection between learning and assessment applying progressive learning tasks in mobile ecology inquiry. Educational Technology & Society, 16, 194e205. Retrieved from http://www.ifets.info/journals/16_1/17.pdf. Hwang, G. J., & Chang, H. F. (2011). A formative assessment-based mobile learning approach to improving the learning attitudes and achievements of students. Computers & Education, 56, 1023e1031. http://dx.doi.org/10.1016/j.compedu.2010.12.002. Hwang, G. J., Chu, H. C., Lin, Y. S., & Tsai, C. C. (2011). A knowledge acquisition approach to developing Mindtools for organizing and sharing differentiating knowledge in a ubiquitous learning environment. Computers & Education, 57, 1368e1377. http://dx.doi.org/10.1016/j.compedu.2010.12.013. Hwang, G. J., Kuo, F. R., Yin, P. Y., & Chuang, K. H. (2010). A Heuristic Algorithm for planning personalized learning paths for context-aware ubiquitous learning. Computers & Education, 54, 404e415. http://dx.doi.org/10.1016/j.compedu.2009.08.024.

274

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

Hwang, G. J., Shi, Y. R., & Chu, H. C. (2011). A concept map approach to developing collaborative Mindtools for context-aware ubiquitous learning. British Journal of Educational Technology, 42, 778e789. http://dx.doi.org/10.1111/j.1467-8535.2010.01102.x. Hwang, G. J., Wu, C. H., Tseng, J. C. R., & Huang, I. (2011). Development of a ubiquitous learning platform based on a real-time help-seeking mechanism. British Journal of Educational Technology, 42, 992e1002. http://dx.doi.org/10.1111/j.1467-8535.2010.01123.x. Hwang, G. J., Wu, P. H., Zhuang, Y. Y., & Huang, Y. M. (2013). Effects of the inquiry-based mobile learning model on the cognitive load and learning achievement of students. Interactive Learning Environments, 21, 338e354. http://dx.doi.org/10.1080/10494820.2011.575789. Hwang, W. Y., & Chen, H. S. L. (2013). Users' familiar situational contexts facilitate the practice of EFL in elementary schools with mobile devices. Computer Assisted Language Learning, 26, 101e125. http://dx.doi.org/10.1080/09588221.2011.639783. Hwang, W. Y., Chen, H. S. L., Shadiev, R., Huang, R. Y. M., & Chen, C. Y. (2012). Improving English as a foreign language writing in elementary schools using mobile devices in familiar situational contexts. Computer Assisted Language Learning, 27, 1e20. http://dx.doi.org/10.1080/09588221.2012.733711. Jones, S. J., Crandall, J., Vogler, J. S., & Robinson, D. H. (2013). Classroom response systems facilitate student accountability, readiness, and learning. Journal of Educational Computing Research, 49, 155e171. http://dx.doi.org/10.2190/EC.49.2.b. Kang, H., Lundeberga, M., Wolter, B., delMas, R., & Herreid, C. F. (2012). Gender differences in student performance in large lecture classrooms using personal response systems (‘clickers’) with narrative case studies. Learning, Media and Technology, 37, 53e76. http://dx.doi.org/10.1080/17439884.2011.556123. Karaman, S. (2011). Effects of audience response systems on student achievement and long-term retention. Social Behavior and Personality: An International Journal, 39, 1431e1439. http://dx.doi.org/10.2224/sbp.2011.39.10.1431. Kert, S. B. (2011). The use of SMS support in programming education. Turkish Online Journal of Educational Technology, 10, 268e273. Retrieved from http:// ﬁles.eric.ed.gov/fulltext/EJ932245.pdf. Kert, S. B. (2013). Using J-Query mobile technology to support a pedagogical proﬁciency course. Journal of Educational Computing Research, 48, 431e445. http://dx.doi.org/10.2190/EC.48.4.b. Ketamo, H. (2003). An adaptive geometry game for handheld devices. Educational Technology & Society, 6, 83e95. Retrieved from http://www.ifets.info/ journals/6_1/ketamo.html. Kim, P., Buckner, E., Kim, H., Makany, T., Taleja, N., & Parikh, V. (2012). A comparative analysis of a game-based mobile learning model in low-socioeconomic communities of India. International Journal of Educational Development, 32, 329e340. http://dx.doi.org/10.1016/j.ijedudev.2011.05.008. Kim, P., Hagashi, T., Carillo, L., Gonzales, I., Makany, T., Lee, B., et al. (2011). Socioeconomic strata, mobile technology, and education: a comparative analysis. Educational Technology Research and Development, 59, 465e486. http://dx.doi.org/10.1007/s11423-010-9172-3. Kondo, M., Ishikawa, Y., Smith, C., Sakamoto, K., Shimomura, H., & Wada, N. (2012). Mobile assisted language learning in university EFL courses in Japan: developing attitudes and skills for self-regulated learning. ReCALL, 24, 169e187. http://dx.doi.org/10.1017/S0958344012000055. de La Fuente, M. J. (2012). Learners' attention to input during focus on form listening tasks: the role of mobile technology in the second language classroom. Computer Assisted Language Learning, 27, 261e276. http://dx.doi.org/10.1080/09588221.2012.733710. Lai, C. H., Yang, J. C., Chen, F. C., Ho, C. W., & Chan, T. W. (2007). Affordances of mobile technologies for experiential learning: the interplay of technology and pedagogical practices. Journal of Computer Assisted Learning, 23, 326e337. http://dx.doi.org/10.1111/j.1365-2729.2007.00237.x. Lai, C. Y., & Wu, C. C. (2006). Using handhelds in a Jigsaw cooperative learning environment. Journal of Computer Assisted Learning, 22, 284e297. Retrieved from http://140.115.126.240/mediawiki/images/6/6d/Jigsaw.pdf. Lan, Y. F., Tsai, P. W., Yang, S. H., & Hung, C. L. (2012). Comparing the social knowledge construction behavioral patterns of problem-based online asynchronous discussion in e/m-learning environments. Computers & Education, 59, 1122e1135. http://dx.doi.org/10.1016/j.compedu.2012.05.004. Lan, Y. J., Sung, Y. T., & Chang, K. E. (2009). Let us read together: development and evaluation of a computer-assisted reciprocal early English reading system. Computers & Education, 53, 1188e1198. http://dx.doi.org/10.1016/j.compedu.2009.06.002. Lan, Y. J., Sung, Y. T., Tan, N. C., Lin, C. P., & Chang, K. E. (2010). Mobile-device-supported problem-based computational estimation instruction for elementary school students. Educational Technology & Society, 13, 55e69. Retrieved from http://www.thai-library.org/Resource/0000005765.pdf#page¼60. Lawrence, L. E., & Victorina, W. (2004). Laptops, technology, and algebra 1: a case study of an experiment. Mathematics Teacher, 97, 136e142. Retrieved from http://personal.stevens.edu/~llevine/mt_laptop_study.pdf. Lin, Y. C., Liu, T. C., & Chu, C. C. (2011). Implementing clickers to assist learning in science lectures: the clicker-assisted conceptual change model. Australasian Journal of Educational Technology, 27, 979e996. Retrieved from http://ascilite.org.au/ajet/ajet27/lin.html. Liu, C. C., Tao, S. Y., & Nee, J. N. (2008). Bridging the gap between students and computers: supporting activity awareness for network collaborative learning with GSM network. Behaviour & Information Technology, 27, 127e137. http://dx.doi.org/10.1080/01449290601054772. Liu, T. Y. (2009). A context-aware ubiquitous learning environment for language listening and speaking. Journal of Computer Assisted Learning, 25, 515e527. http://dx.doi.org/10.1111/j.1365-2729.2009.00329.x. Liu, T. Y., & Chu, Y. L. (2010). Using ubiquitous games in an English listening and speaking course: impact on learning outcomes and motivation. Computers & Education, 55, 630e643. http://dx.doi.org/10.1016/j.compedu.2010.02.023. Liu, T. Y., Tan, T. H., & Chu, Y. L. (2009). Outdoor natural science learning with an RFID-supported immersive ubiquitous learning environment. Educational Technology & Society, 12, 161e175. Retrieved from http://www.ifets.info/journals/12_4/15.pdf. Looi, C. K., Zhang, B., Chen, W., Seow, P., Chia, G., Norris, C., et al. (2011). 1:1 mobile inquiry learning experience for primary science students: a study of learning effectiveness. Journal of Computer Assisted Learning, 27, 269e287. http://dx.doi.org/10.1111/j.1365-2729.2010.00390.x. Lowther, D. L., Ross, S. M., & Marrison, G. M. (2003). When each one has one: the inﬂuences on teaching strategies and student achievement of using laptops in the classroom. Educational Technology Research and Development, 51, 23e44. http://dx.doi.org/10.1007/BF02504551. Lu, M. (2008). Effectiveness of vocabulary learning via mobile phone. Journal of Computer Assisted Learning, 24, 515e525. http://dx.doi.org/10.1111/j.13652729.2008.00289.x. nez, L., Martínez, J. J., Gutie rrez, J. A., et al. (2010). An experiment for improving students performance in de-Marcos, L., Hilera, J. R., Barchino, R., Jime secondary and tertiary education by means of m-learning auto-assessment. Computers & Education, 55, 1069e1079. http://dx.doi.org/10.1016/j.compedu.2010.05.003. Martin-Dorta, N., Saorin, J. L., & Contero, M. (2011). Web-based spatial training using handheld touch screen devices. Educational Technology & Society, 14, 163e177. Retrieved from http://www.ifets.info/journals/14_3/14.pdf. Martin, F., & Ertzberger, J. (2013). Here and now mobile learning: an experimental study on the use of mobile technology. Computers & Education, 68, 76e85. http://dx.doi.org/10.1016/j.compedu.2013.04.021. Mayer, R. E., Stull, A., DeLeeuw, K., Almeroth, K., Bimber, B., Chun, D., et al. (2009). Clickers in college classrooms: fostering learning with questioning methods in large lecture classes. Contemporary Educational Psychology, 34, 51e57. http://dx.doi.org/10.1016/j.cedpsych.2008.04.002. McNaughton, D., Hughes, C., & Clark, K. (1997). The effect of ﬁve proofreading conditions on the spelling performance of college students with learning disabilities. Journal of Learning Disabilities, 30, 643e651. http://dx.doi.org/10.1177/002221949703000608. Morris, J. P., Mahajan, N., Pasek, K. H., Golinkoff, R. M., & Collins, M. F. (2013). Once upon a time: parentechild dialogue and storybook reading in the electronic era. Mind, Brain, and Education, 7, 200e211. http://dx.doi.org/10.1111/mbe.12028. neza, J. J., & Osuna-Pe rezb, M. C. (2013). Development and evaluation of a 3D mobile application for learning manual therapy in the Noguera, J. M., Jime physiotherapy laboratory. Computers & Education, 69, 96e108. http://dx.doi.org/10.1016/j.compedu.2013.07.007. Oberg, A., & Daniels, P. (2013). Analysis of the effect a student-centred mobile learning instructional method has on language acquisition. Computer Assisted Language Learning, 26, 177e196. http://dx.doi.org/10.1080/09588221.2011.649484. lez, J. R. H., Herra iz, J. J. M., Mesa, J. A. G. D., et al. (2011). Using M-learning on nursing courses to Ortega, L. D. M., Plata, R. B., Rodríguez, M. L. J., Gonza improve learning. Computers, Informatics, Nursing, 29, TC98eTC104. http://dx.doi.org/10.1097/NCN.0b013e3182285d2c. Ozcelik, E., & Acarturk, C. (2011). Reducing the spatial distance between printed and online information sources by means of mobile technology enhances learning: using 2D barcodes. Computers & Education, 57, 2077e2085. http://dx.doi.org/10.1016/j.compedu.2011.05.019.

Y.-T. Sung et al. / Computers & Education 94 (2016) 252e275

275

Powell, C. B., & Mason, D. S. (2013). Effectiveness of podcasts delivered on mobile devices as a support for student learning during general chemistry laboratories. Journal of Science Education and Technology, 22, 148e170. http://dx.doi.org/10.1007/s10956-012-9383-y. Radosevich, D. J., Salomon, R., Radosevich, D. M., & Kahn, P. (2008). Using student response systems to increase motivation, learning, and knowledge retention. Innovate, 5, 7. Retrieved from http://eric.ed.gov/?id¼EJ840506. Rau, P. L. P., Gao, Q., & Wu, L. M. (2008). Using mobile communication technology in high school education: motivation, pressure, and learning performance. Computers & Education, 50, 1e22. http://dx.doi.org/10.1016/j.compedu.2006.03.008. Riconscente, M. M. (2013 July). Results from a controlled study of the iPad fractions game motion math. Games and Culture, 8, 186e214. http://dx.doi.org/10. 1177/1555412013496894. Rockinson-Szapkiw, A. J., Courduff, J., Carter, K., & Bennett, D. (2013). Electronic versus traditional print textbooks: a comparison study on the inﬂuence of university students' learning. Computers & Education, 63, 259e266. http://dx.doi.org/10.1016/j.compedu.2012.11.022. pez, X., & Sepúlveda, M. (2010). A monitoring and evaluation scheme for an ICT-supported education program in schools. Rodríguez, P., Nussbaum, M., Lo Educational Technology & Society, 13, 166e179. Retrieved from http://www.ceppe.cl/images/stories/articulos/tic/2.2-a-monitoring.-rodriguez1.pdf. Roschelle, J., Rafanan, K., Bhanot, R., Estrella, G., Penuel, B., Nussbaum, M., et al. (2010). Scaffolding group explanation and feedback with handheld technology: impact on students' mathematics learning. Educational Technology Research and Development, 58(4), 399e419. http://dx.doi.org/10.1007/ s11423-009-9142-9. Sandberg, J., Maris, M., & De Geus, K. (2011). Mobile English learning: an evidence-based study with ﬁfth graders. Computers & Education, 57, 1334e1347. http://dx.doi.org/10.1016/j.compedu.2011.01.015. Saran, M., Seferoglu, G., & Cagiltay, K. (2009). Mobile assisted language learning: English pronunciation at learners' ﬁngertips. Eurasian Journal of Educational Research, 34, 97e114. Retrieved from http://94.73.145.182/0DOWNLOAD/pdﬂer/tr/902903948.pdf. lu, G., & Cag ıltay, K. (2012). Mobile language learning: contribution of multimedia messages via mobile phones in consolidating voSaran, M., Seferog cabulary. The Asia-Paciﬁc Education Researcher, 21, 181e190. Retrieved from http://www.ejournals.ph/index.php? journal¼TAPER&page¼article&op¼view&path%5B%5D¼4826&path%5B%5D¼5016. Shih, W. C., & Tseng, S. S. (2009). A knowledge-based approach to retrieving teaching materials for context-aware learning. Educational Technology & Society, 12, 82e106. Retrieved from http://www.ifets.info/journals/12_1/8.pdf. Siegle, D., & Foster, T. (2001). Laptop computers and multimedia and presentation software: their effects on student achievement in anatomy and physiology. Journal of Research on Technology in Education, 34, 29e37. http://dx.doi.org/10.1080/15391523.2001.10782331. Skeate, R. C., Wahi, M. M., Jessurun, J., & Connelly, D. P. (2007). Personal digital assistant-enabled report content knowledgebase results in more complete pathology reports and enhances resident learning. Human Pathology, 38, 1727e1735. http://dx.doi.org/10.1016/j.humpath.2007.05.019. Solhaug, T. (2009). Two conﬁgurations for accessing classroom computers: differential impact on students' critical reﬂections and their empowerment. Journal of Computer Assisted Learning, 25, 411e422. http://dx.doi.org/10.1111/j.1365-2729.2009.00318.x. Suhr, K. A., Hernandez, D. A., Grimes, D., & Warschauer, M. (2010). Laptops and fourth grade literacy: assisting the jump over the fourth-grade slump. The Journal of Technology, Learning, and Assessment, 9, 1e45. Retrieved from http://136.167.2.46/ojs/index.php/jtla/article/view/1610/1459. Sung, E., & Mayer, R. E. (2013). Online multimedia learning with mobile devices and desktop computers: an experimental test of Clark's methods-not-media hypothesis. Computers in Human Behavior, 29, 639e647. http://dx.doi.org/10.1016/j.chb.2012.10.022. Sung, Y. T., Chang, K. E., Hou, H. T., & Chen, P. F. (2010). Designing an electronic guidebook for learning engagement in a museum of history. Computers in Human Behavior, 26, 74e83. http://dx.doi.org/10.1016/j.chb.2009.08.004. Sung, Y. T., Chang, K. E., Lee, Y. H., & Yu, W. C. (2008). Effects of a mobile electronic guidebook on visitors' attention and visiting behaviors. Educational Technology & Society, 11, 67e80. Retrieved from http://ifets.info/journals/11_2/7.pdf. Tan, T. H., Liu, T. Y., & Chang, C. C. (2007). Development and evaluation of an RFID-based ubiquitous learning environment for outdoor learning. Interactive Learning Environments, 15, 253e269. http://dx.doi.org/10.1080/10494820701281431. Thornton, P., & Houser, C. (2005). Using mobile phones in English education in Japan. Journal of Computer Assisted Learning, 21, 217e228. http://dx.doi.org/ 10.1111/j.1365-2729.2005.00129.x. Uluyol, C., & Agca, R. K. (2012). Integrating mobile multimedia into textbooks: 2D barcodes. Computers & Education, 59, 1192e1198. http://dx.doi.org/10. 1016/j.compedu.2012.05.018. Wang, S. L., & Wu, C. Y. (2011). Application of context-aware and personalized recommendation to implement an adaptive ubiquitous learning system. Expert Systems with Applications, 38, 10831e10838. http://dx.doi.org/10.1016/j.eswa.2011.02.083. Wei, C. W., Hung, I. C., Lee, L., & Chen, N. S. (2011). A joyful classroom learning system with robot learning companion for children to learn mathematics multiplication. Turkish Online Journal of Educational Technology, 10, 11e23. Retrieved from http://ﬁles.eric.ed.gov/fulltext/EJ932221.pdf. Wessels, A., Fries, S., Horz, H., Scheele, N., & Effelsberg, W. (2007). Interactive lectures: effective teaching and learning in lectures using wireless networks. Computers in Human Behavior, 23, 2524e2537. http://dx.doi.org/10.1016/j.chb.2006.05.001. Wood, C., Jackson, E., Hart, L., Plester, B., & Wilde, L. (2011). The effect of text messaging on 9- and 10-year-old children's reading, spelling and phonological processing skills. Journal of Computer Assisted Learning, 27, 28e36. http://dx.doi.org/10.1111/j.1365-2729.2010.00398.x. Wu, P. H., Hwang, G. J., Su, L. H., & Huang, Y. M. (2012). A context-aware mobile learning system for supporting cognitive apprenticeships in nursing skills training. Educational Technology & Society, 15, 223e236. Retrieved from http://www.ifets.info/journals/15_1/20.pdf. Wu, T. T., Sung, T. W., Huang, Y. M., Yang, C. S., & Yang, J. T. (2011). Ubiquitous English learning system with dynamic personalized guidance of learning portfolio. Educational Technology & Society, 14, 164e180. Retrieved from http://www.ifets.info/journals/14_4/ets_14_4.pdf#page¼169. Yang, C. C., Hwang, G. J., Hung, C. M., & Tseng, S. S. (2013). An evaluation of the learning effectiveness of concept map-based science book reading via mobile devices. Educational Technology & Society, 16, 167e178. Retrieved from http://www.ifets.info/journals/16_3/13.pdf. Yang, C. C., Tseng, S. S., Liao, A. Y. H., & Liang, T. (2013). Situated poetry learning using multimedia resource sharing approach. Educational Technology & Society, 16, 282e295. Retrieved from http://www.ifets.info/journals/16_2/23.pdf. Zhang, B., Looi, C. K., Seow, P., Chia, G., Wong, L. H., Chen, W., et al. (2010). Deconstructing and reconstructing: transforming primary science learning via a mobilized curriculum. Computers & Education, 55, 1504e1523. http://dx.doi.org/10.1016/j.compedu.2010.06.016. Zhang, H., Song, W., & Burston, J. (2011). Reexamining the effectiveness of vocabulary learning via mobile phones. The Turkish Online Journal of Educational Technology, 10, 203e214. Retrieved from http://ﬁles.eric.ed.gov/fulltext/EJ944968.pdf. Zheng, B., Warschauer, M., & Farkas, G. (2013). Digital writing and diversity: the effects of school laptop programs on literacy processes and outcomes. Journal of Educational Computing Research, 48, 267e299. http://dx.doi.org/10.2190/EC.48.3.a. Zurita, G., & Nussbaum, M. (2004a). Computer supported collaborative learning using wirelessly interconnected handheld computers. Computers & Education, 42, 289e314. http://dx.doi.org/10.1016/j.compedu.2003.08.005. Zurita, G., & Nussbaum, M. (2004b). A constructivist mobile learning environment supported by a wireless handheld network. Journal of Computer Assisted Learning, 20, 235e243. http://dx.doi.org/10.1111/j.1365-2729.2004.00089.x.