A Masked Engagement? The Influence of Mask-wearing on Students’ Learning Engagement

: Educational system was dangerously challenged by COVID-19 without preparation. Schools and students cannot afford another strike. Mask-wearing has become a new normal. Scholars researched the potential effects of mask-wearing, but no direct study focuses on students learning engagement under mask-wearing conditions. In our study, we implemented a quantitative method to study students’ learning engagement in four aspects: emotional, behavioral, academic, and cognitive engagement. We received 218 questionnaire responses from native Chinese college students. Then, 90 valid responses were generated after exposure to three different stimuli: a native Chinese instructor lecturing English morphology in a quiet environment (SNR > 15 dB) without wearing a mask, the instructor lecturing the same content in the same acoustic environment (SNR > 15 dB) wearing a mask, and the instructor lecturing in the presence of noise (SNR < 15 dB) with mask-wearing. Our finding indicates that neither mask-wearing nor a noisy environment has a substantial influence on our participants’ emotional, behavioral, and cognitive engagement. However, our participants’ academic engagement was discouraged significantly after being lectured in noise after mask-wearing.


A Masked Engagement? The Influence of Mask-wearing on Students' Learning Engagement
The COVID-19 pandemic has been the most severe threat in the last five decades to the worldwide education system [1]. Schools were forced to reschedule their teaching plans from in-person to online teaching at the onset of the pandemic without full preparation. After the distribution of the COVID-19 vaccination, hospitalization and mortality rates declined [2]. Additionally, previous research suggested that schools' reopening would not increase infection and mortality rates when instructors and students obeyed the mandatory mask-wearing policy [3]. Based on the pandemic's improvement, many schools resumed their in-person teaching under mask-wearing conditions. However, some scholars were concerned that wearing a mask could cause potential problems. Recent studies found three potential negative effects of mask-wearing that may be directly or indirectly detrimental to education practices.
First, fewer emotional cues after wearing a mask degrade learning quality. Emotional cues include from facial expressions to body movements [4]. Recent experiments found that the speaker wearing a mask not only greatly reduced the accuracy and confidence of reading different facial expressions for listeners, but unconsciously distorted facial expression reading and even caused bias [5][6][7]. In the educational setting, the emotional and facial expressions of instructors are more expressive and informative, less these nonverbal expressions negatively impact students' emotions and knowledge perception, and connect with lower students' learning satisfaction [8][9][10].
In addition, obscured speech intelligibility after wearing a mask result in a worse learning experience. Speech intelligibility was defined as the intended message from speaker to listener [11]. Current research pointed out that mask-wearing obscured a speaker's speech intelligibility both in a quiet and noisy environment. In a quiet speaking environment, signal-to-noise ratio (SNR) > 15 dB, speech intelligibility degraded when the speaker was wearing a mask [12][13][14][15]. In the presence of noise, with SNR < 15 dB, the speech intelligibility decreased more than in the quiet scenario [16][17][18]. In the classroom, mask-wearing similarly impedes the instructor's voice transmission which is an indispensable element in classroom management and knowledge conceptualization and, consequently, negatively influences students' learning performance [19][20][21].
Lastly, increased listening effort after wearing a mask distracts students' task performance. Listening effort refers to the required cognitive action for comprehending others' speeches [22]. Given that mask-wearing obscured speech intelligibility, the current study proposed that listening effort also increased especially in a noisy acoustic environment [23]. In classrooms, consequently, the increased listening effort may be detrimental to students' knowledge of recalling-related tasks, academic performance, and overall learning engagement and motivation [24][25][26].

Acoustic Feature of Classroom
Despite being challenged by the three potential problems of mask-wearing, current education still faces another question: an astonishing number of schools ignore or fail to follow the international standard for acoustic features in classrooms that SNR > 15 dB according to the American National Standards Institute [27]. In reality, given the literature above emphasizes the side-effects of maskwearing in noise and quiet situations, however, there is no direct research that explores whether an instructor wearing a mask influences students' learning engagement (SLE).

Definitions of Learning Engagement
Although there are overwhelming amounts of definitions for SLE or namely student engagement, no unified definition in academia was formed. Scholars defined student engagement as the students' inclass learning experience, participation in both inside and outside schools' educational practice, or the high-quality outcomes after educational investment. In our study, we focused on the first definition of SLE as the students' in-class learning experience because we aimed to analyze the SLE and experience in an in-class setting after completing a series of educational activities [28][29][30][31][32][33].
To categorize these definitions, scholars proposed that SLE could be measured on different interactive aspects: emotional engagement, behavior engagement, cognitive engagement, and academic engagement [34][35][36][37][38][39]. Emotional engagement (EE) refers to students' actual affective emotions and learning experiences in the teaching and learning environment, including observable and unobservable emotional feelings, such as enjoyment, happiness, anxiety, boredom, sadness, and mental connection with an instructor as specific and even education as broad. Behavior engagement (BE) describes the necessary behaviors for completing assignments, involvement, and participation in teaching and learning activities concerning completing a task or meeting a requirement. Cognitive engagement (CE) was defined as the students' psychological state where intensive effort is required for finishing classroom activities and the intention to promote future academic performance. For schooling success, academic work completion is another key element [40][41][42][43][44][45][46][47]. Academic engagement (AE) focuses on students' in-class attendance, student teaching activities participation, and assignment completion. Research assessing 194 undergraduate and postgraduate students in India was conducted and it was found that students' learning engagement positively correlated to academic outcome and performance, which corresponded with previous studies [48][49][50][51][52][53].

Proposed Study
Though recent research well studied the potential problems of mask-wearing (less emotional cues, obscured speech intelligibility, and more demanding listening effort), no direct research study focuses on the perspective of students: SLE. We researched two questions in terms of SLE. First, was SLE influenced by which of the following three conditions? The conditions are mask-off in a quiet environment (SNR >15 dB), mask-on in a quiet environment (SNR > 15 dB), and mask-on combined with background noise (SNR < 15 dB). Second, which specific SLE (EE, BE, AE, and CE) was influenced most? And was the influence significant enough?

Materials and Methods
Stimuli, original questionnaire data (pseudonymized), R script, coding files, complete questionnaire samples, PowerPoint slides, and content checklists are available at https://doi.org/10.17605/OSF.IO/Z4CFB.  A total of 218 questionnaires were sent out to native Chinese participants from college WeChat groups in Shenzhen, China (valid questionnaire retrieval rate: 41.3%). This population was chosen because English education is well developed and Shenzhen is one of the standard cities in Covid-19 control. Before our experiment, all participants were notified of their task requirements and gave their consent. Then, all participants were required to take a 13-item Hearing & Demographic Questionnaire (HDQ) collecting necessary basic information: the hearing condition, gender, class level, grade point average (GPA), major, and English proficiency level. See Figure 1 for the flowchart screening questionnaire responses. Figure 1 illustrates the number of responses before and after data screening as well as the data screening criteria. The left blue boxes refer to different experiment stages. The right white boxes and arrows are used to clarify the exact number screened and the data screening sequence.

Participants
After finishing the HDQ and two data screening procedures, our final valid responses were collected: the controlled group as Group 1 (N=30), experimental group A as Group 2 (N=30), and experiment group B as Group 3 (N=30). See Table 1 for detailed demographic information.  [54]. c Morphology Learned was defined as prior systematic knowledge of morphology.

Overview
Three English video recordings were filmed at 720p, 60 FPS as the stimuli for this experiment: (1) lecturing without a mask in a quiet acoustic environment (SNR > 15 dB), (2) lecturing with a blue disposable medical mask in a quiet environment (SNR > 15 dB), and (3) lecturing with the same mask in a noisy environment (SNR < 15 dB). All the videos were recorded via Zoom (version 5.11.1) with an 11-inch MacBook Pro laptop (IOS 12.2.1). Video editing was conducted via an Apple pre-installed app, iMovie (version 10.3.3), and auditory-editing software Cubase (version 10.5). The PowerPoint content examination and Student Learning Engagement Questionnaire (SLEQ) content examination were peer-reviewed to ensure the vocabulary in both contents matched our participants' vocabulary proficiency levels [54]. To specify the stimuli, two major stages were explained including preparing and recording the videos.

Stimuli Preparation Stage
A 21-slide PowerPoint mainly introduces basic concept knowledge about morphology: morphology, morpheme, free morpheme, bound morpheme, derivational morpheme, and inflectional morpheme. See Table 2 for more information about PowerPoint content. Two rationales support this chosen topic: Morphology. First, teaching morphology can facilitate participants' future learning. The association between lecture knowledge and current knowledge repertoire facilitates their future English learning. Second, lecturing morphology increased the face validity of the lecturing content. In other words, teaching real-life related or practical content decreases the risk of potentially dropping engagement due to a low face validity [55][56][57].
In the PowerPoint, we introduced three simple codes consisting of two numbers and two uncapitalized letters. Then, these codes were inserted into the beginning (4th slide), middle (13rd slide), and last (20th slide) parts of the PowerPoint to guarantee that our participants watched the assigned stimuli and to improve the authenticity of the questionnaire data. Last, we peer-reviewed all the words used in the PowerPoint and reached a 0.882 Po score (proportion of observed agreement) to confirm that the content matches our participants' vocabulary level in case of low validity data yielded due to lexical problems.

Stimuli Recording Stage
SNR standard was implemented for creating stimuli to ensure our stimuli effectiveness and that our participants are exposed to a relatively stable ratio of background noise and lecturing sound. Three videos as the stimuli are the same male instructor lecturing the same content in English lasting for 7 minutes and 19 seconds at an average speed of 142 words per minute. The first stimulus was recorded in a quiet environment (average SNR > 15 dB) with a lecturer without a mask. The second stimulus was filmed in a quiet environment (average SNR > 15 dB) and the lecturer was wearing a blue disposable medical mask. The third stimulus was created via Cubase to simulate the noisy authentic lecturing-style classroom with an average SNR < 15 dB [58]. See Figure 2 for the stimuli recording environment and mask type. This video length was considered unlikely to trigger participants' videowatching fatigue, neither encouraging nor discouraging participants [59,60]. In all three stimuli, a script beforehand was written to guide the lecturing content to control the instructor's lecturing speed. All videos were recorded slide by slide, generating 21 video clips under the mask-off condition for Group 1, and another 21 video clips under the mask-on condition for Group 2. Then, two sets of 21 video clips were edited as two complete stimuli. Last, the stimulus from Group 2 after attenuating all non-signal sounds (background noise) was combined with an artificial soundtrack (college classroom debating background sounds) from iMovie as the third stimulus for Group 3.   Figure 2 records three important details: the recording environment of all stimuli, the wearing mask type, and the randomly selected signal and noise samples. Both plot graphs illustrating stimulus 3 are generated by RStudio. The average signal sound in stimulus 3 is 69.5 dB to simulate the actual lecturing sound level [61]. The average noise sound in stimulus 3 is 63.1 dB. According to SNR, the average SNR in stimulus 3 is < 15 dB which simulates the average classroom SNR [58].

SLEQ
SLEQ was introduced to measure our participants' learning engagement after being exposed to the assigned stimuli. Before releasing our SLEQ, we also conducted a peer-review scrutinizing the suitability of the vocabulary in this questionnaire to our participants' lexical proficiency level. The Po score in this examination reached 1.000, indicating the questionnaire content matches our participants' vocabulary proficiency level.
Next, thirty-three questions were designed and classified into six categories: consent question, validating question, emotional engagement, behavioral engagement, academic engagement, and cognitive engagement. See Table 3 for more information about SLEQ. Note. a EE as the emotional engagement can be measured by recording students' emotional reactions to teaching materials and the class [62,63]. b BE as the behavioral engagement was assessed by analyzing students' participation, attention, and behavior in teaching activities [63]. c AE as the academic engagement can be measured in a quantitative method according to students' academic assignment performance [64][65][66][67]. d CE cognitive engagement was evaluated by tracking the willingness for more challenging work and future study intention [46]. e Likert Scale is a 5-option psychometric rating scale: strongly disagree, disagree, just so-so, agree, strongly agree.

Reliability & Validity
To obtain a reliable and valid experiment result, the SLEQ with three groups of data must pass the Cronbach's coefficient alpha (α) test, KMO, and Bartlett test. All the tests were processed with SPSS software. Cronbach's coefficient alpha was implemented to test the reliability of the SLEQ. The test was implemented twice for all three groups for a more reliable questionnaire. For the first test, a total of 18 Likert scale questions (question 6 to 17 and 28 to 33) were assessed according to Cronbach's coefficient reliability test and returned EE (G1: α = 0.613, G2: α = 0.662, and G3: α = 0.635) which all were considered low-reliability α < 0.700. Next, question 8 was deleted from this questionnaire and excluded for further data analysis because it failed to generate a satisfactory reliability result (α >= 0.800) and was considered overlapped in the content assessed. For the second test, 17 questions excluding question 8 were tested and yielded an acceptable result of α > 0.800 (in EE, BE, CE, and overall score) suggesting that SLEQ is a reliable question. See Table 4 for detailed information about the reliability test of SLEQ.
To test the validity of SLEQ, KMO and Bartlett tested the same 18 questions mentioned in the reliability test. All three groups demonstrated KMO > 0.600 and Bartlett's value significance (p-value < 0.05) which indicated SLEQ passed the validity test. See Table 5 for detailed SLEQ validity test data.  result and overlapped content assessed). Besides, the questions from 18 to 27 in academic engagement were excluded because this section is not designed as a Likert scale but as a 5-option multiple choice with only one correct answer assessing participants' concept knowledge about morphology.

Descriptive Statistics of SLEQ
Given three different stimuli were assigned, a descriptive table was designed to quantify the varied SLEQ scores from three groups. See Table 6 for details. According to Table 6, there was a descending trend in SLEQ's average (75.867 to 70.933) and medium (79.000 to 69.000) scores. This declining trend suggested that our participants' learning engagement was impacted to some degree. However, the standard deviation in the test results implied that detailed analyses of each engagement category (EE, BE, AE, and CE) were needed to assess the declining SLEQ between groups.

Result
T-test was utilized to evaluate the SLEQ scores on each engagement (EE, BE, AE, CE) generated by the three groups. First, we contrasted the scores of Group 1 with Group 2. See Table 7. Then, Group 2 was contrasted with Group 3. See Table 8. Last, Group 1 and Group 3 were contrasted.

The Undistracted EE, BE, and CE
In this study, we found that our participants' EE, BE, and CE were neither discouraged by the maskwearing of the instructor nor the presence of background noise (SNR < 15 dB). This finding rejects our original hypothesis that students' engagement was negatively influenced by mask-wearing and the presence of background noise. According to Table 7, the average SLEQ score of Group 2 was lower in EE (19.97 to 19.73), BE (25.83 to 24.67), and AE (8.17 to 7.33) contrasted with Group 1, while the average CE score increased from 21.90 to 22.07. However, all these variations did not demonstrate result significance suggesting no substantial difference in participants' EE (p = 0.833), BE (p =0.415), AE (p =0.139), and CE (p = 0.900) after the mask-wearing interference.
In Table 8, Group 2's average SLEQ score was higher after exposure to stimulus 2 (SNR > 15 dB) in contrast to Group 3's average score exposed to stimulus 3 (SNR < 15 dB). Group Table 9 shows the T-test between Group 1 and Group 3. We noticed Group 3 appeared to be less engaged in this experiment than Group 1. Three types of engagements were influenced without significance: EE (Mean = 19.97 to 18.17, with p = 0.78), BE (Mean = 25.83 to 24.33, with p = 0.210), and CE (Mean = 21.90 to 21.60, with p = 0.814). Nevertheless, the AE reported a lower score (Group 1's Mean = 8.17 to Group 2's Mean = 6.83), meanwhile providing a p-value = 0.021 with significance. In other words, this p-value found that participants' academic engagement was significantly damaged when the instructor wore a mask and taught in a noisy environment (SNR < 15 dB). The impacted AE confirmed prior research findings that students' AE could be discouraged in a noisy classroom at all ages [68][69][70].

Conclusion
In this study, we collected data from 90 qualified participants in three groups in terms of EE, BE, AE, and CE. Our test results highlight two findings. First, students' EE, BE, and CE did not strongly correlate with an instructor wearing a mask or in a noisy teaching environment. Second, students' AE was tested to be significantly vulnerable to a noisy teaching environment, especially after the instructor wore a mask. The findings of this experiment proposed that both instructors and schools should take classroom acoustic standards into consideration because learning in a poor sound environment may directly or indirectly affect students' academic engagement, especially during the COVID-19 pandemic. Though our data demonstrated finding significance, three limitations were bothering this study. First, statistical bias may occur due to online self-report questionnaires. The experiment data may lower the validity in nature which may explain why our standard variations in SLEQ are relatively large. Next, the limited sample size may contribute to false positive or negative results. We only received 90 valid responses mainly from one university in China which may contain potential bias disruptive to the final data analysis. Last, online-based experiments lower the stimuli's effectiveness. We cannot conduct a field study due to the Covid-19 health policy. Our participants received their assigned stimuli online and took their questionnaires online. It is unlikely to know how much the actual sound (in dB) our participants were exposed to. To best secure the stimuli's effectiveness, the SNR standard was introduced to guarantee that our participants were exposed to a relatively stable sound and noise ratio. Covid-19 has cost millions of lives. Now, monkeypox has arrived. What will come next? Scientists do not have an accurate theory predicting what the next pandemic is waiting for humankind, but we all know the intervals between global pandemics are becoming closer. Lessons from Covid-19 that have taught us that we are not fully prepared. The educational system is not fully prepared. Our study sheds light on future studies on: first, what contributed to the dropping academic engagement of students; second, which aspects of learning engagement students dropped most under mask-wearing environments; third, whether learning engagement after mask-wearing is impacted from the perspective of students with disabilities; forth, field study investigates the students' learning engagement and the instructor's teaching experience after both parties under mask-wearing condition; last, whether our findings apply to students from other age groups or countries.