Supporting novice mathematics teachers: The impact of an e-mentoring and video-based professional development program on teachers’ noticing skills

. Supporting novice mathematics teachers: The impact of an e-mentoring and video-based professional development program on teachers ’ noticing skills


INTRODUCTION
In this century, while the learners are expected to have some important skills (e.g., complex problem solving, critical thinking), it is underlined that teachers should have competencies to support development of these skills (Darling-Hammond et al., 2017).In this context, noticing skills, requiring teachers to pay attention to and interpret the learning process of the students, help teachers develop better teaching strategies and focus on student thinking (Santagata & Yeh, 2016).In recent years, noticing as a situation-specific skill was conceptualized as a component of teacher competence (Meschede et al., 2017;Sherin & van Es, 2009), which shows how critical it is for teachers to have noticing skills.
In order to develop noticing skills, as well as learning contemporary pedagogies, teachers may need support.It is acknowledged that particularly teachers in the first years of their careers may need more professional support than others, especially about focusing on and understanding individual student thinking (Darling-Hammond, 2010).Beginning years of the profession is difficult for teachers as they are learning how to teach when they are engaged in teaching activities (Feiman-Nemser, 2001).The support provided for teachers in novice years is critical for the quality of immediate professional experience and their long-term professional learning.There are various models of induction programs for teachers in early years of teaching to support teachers' professional growth and a majority of the programs involve mentorship practices (Hightower et al., 2021).Mentoring practices in general and e-mentoring practices in particular are extremely limited and appear to be mostly carried out in the fields of education management, inspection and planning (Ekinci, 2010;Ozdemir, 2015).
Novice teachers in general and particularly in Turkey are employed in rural areas where physical and social opportunities are limited for professional growth (Ekinci, 2010).In this context, it can be said that different approaches are needed to eliminate the limitations such as time, place and transportation in terms of reaching novice teachers and supporting them professionally.Thus, this study uses e-mentoring (Alemdag & Erdem, 2017;Mihram, 2004) approach to overcome these limitations.Various studies in the literature reported that mentoring in general and e-mentoring in particular are effective in terms of providing professional development (PD) (e.g., Erdogan et al., 2022;Spanorriga et al., 2018).PD literature on e-mentoring consists primarily of research in fields other than educational sciences, and little is known about the effectiveness of this method in education.Furthermore, the impact of e-mentoring on PD of mathematics teachers is not well-documented.To fill this gap, this study reports findings of the project "E-mentoring for mathematics teachers (E-MMT)" aimed at developing, implementing and evaluating an e-mentoring and video-based program for teachers in teaching mathematics in the early years of their profession.In this study we investigated effectiveness of E-MMT in improving novice teachers' noticing skills.

Teacher Noticing
The term teacher noticing has its roots in professional vision (Goodwin, 1994), which refers to how professionals view a certain situation and engage in reasoning specific to their profession.In a model of teacher competence, noticing and professional vision appear to influence how teacher knowledge and dispositions become visible as classroom performance (Meschede et al., 2017).Noticing has been conceptualized in different ways in teacher education literature (Santagata et al., 2021).While some scholars studied noticing as selective attention and knowledge-based reasoning (interpretation) (van Es & Sherin, 2002), others included aspects of decision making in relation to what teachers attended to and made sense of (Barnhart & van Es, 2015;Jacobs et al., 2010;Kaiser et al., 2015).For instance, professional noticing (Jacobs et al., 2010) is conceptualized as three interrelated skills: attending to students' thinking, interpreting student thinking, and deciding how to respond based on the interpretations.Similarly, Kaiser et al. (2015) conceptualized noticing as three components: perceiving, interpreting and decision-making.While Jacobs et al. (2010) focused on individual student thinking in video clips of clinical interviews, other researchers studied noticing related to classroom events (Barnhart & van Es, 2015;Kaiser et al., 2015).We conceptualize noticing as attending to what is noteworthy in classroom, interpreting classroom events and deciding how to respond (decision making in offering alternative instructional strategies) (similar to Barnhart & van Es, 2015).Drawing on aforementioned studies in literature (Barnhardt & van Es, 2015), in our study, attending in general terms refers to the extent that teachers pay attention to significant aspects of teaching (individual student learning, details of instructional actions and mathematics content), interpretation involves analysis of the classroom situations and the extent that the teachers are able to provide evidence for their claims and establish relationships between teaching and learning.Lastly, deciding to respond refers to how teachers would make decisions regarding instructional strategies, the extent that the alternative strategies they offer are in line with the student understanding and classroom situations teachers pay attention to and interpret.
One of the most common ways of enhancing teacher noticing is introducing different frameworks about analyzing lessons in video-based teacher education courses or programs (Santagata & Angelici, 2010;Santagata & Guariano, 2011;Santagata & Yeh, 2016).In a study by Santagata and Yeh (2016), the researchers investigated beginning elementary mathematics teachers' competence during their first two years of teaching.The authors assessed noticing skills by using a classroom video analysis measurement and found that teacher scores increased over time.The authors suggested that improving situation-specific (i.e., noticing) skills of teachers by deliberate practice also enhanced teacher knowledge, beliefs and practices.Similarly, in a study conducted by Santaga and Angelici (2010), it was revealed that teaching the framework of lesson analysis helped increase the quality of the discussions about the videos that the teacher candidates viewed.The lesson analysis is a framework that facilitates thinking about significant aspects of teaching: the aim of the lesson, student thinking, teaching strategies, relationship between teaching and learning, and proposing alternative pedagogical strategies.This framework was found useful in shaping teachers' professional growth, particularly their skills related to noticing (Santagata & Guarino, 2011).Some studies measured growth in teacher noticing dimensions (i.e., attending, interpreting, deciding how to respond) separately as a result of different types of interventions (Jacobs et al., 2010;Krupa et al., 2017;Schack et al., 2013;Ulusoy & Cakiroglu, 2021).There are studies, which documented it was difficult to improve deciding to respond skill, especially compared to attending and interpreting skills (Jacobs et al., 2010;Krupa et al., 2017).Schack et al. (2013) found that pre-service teachers scored the least in the deciding to respond dimension compared to other dimensions in the pre-test.As a result of the intervention by authors, prospective teachers demonstrated growth in all dimensions of teacher noticing but the deciding to respond had the largest increase due to low pre-test results.In contrast with findings of Jacobs et al. (2010) and Krupa et al. (2017), Ulusoy and Cakiroglu (2021) found that their video study helped prospective teachers improve their responding skills, i.e. they started to provide more detailed and mathematically appropriate suggestions as instructional actions in comparison to before intervention.Differences in literature about enhancing particular dimensions of noticing may be caused by the nature of interventions and measurement.There is a need for more research evidence about how to conduct programs in order to support all dimensions of teacher noticing.
The participants in many noticing studies have been prospective teachers (Guler et al., 2020;Krupa et al., 2017;Santagata et al., 2021;Schack et al., 2013).The interventions that helped prospective teachers' noticing skills have taken various forms.For instance, conducting one-to-one clinical interviews (Krupa et al., 2017), participating in long term video analysis of student mathematical thinking (Schack et al., 2013) have been approaches shown as beneficial in developing noticing skills.Although understanding prospective teachers' noticing skills is important to help them prepare for the profession, in-service teachers' need to enhance noticing skills may be more urgent as they may have a hard time to transfer their knowledge of theoretical courses in education to their practice in classrooms (Feiman-Nemser, 2001).
In a review of studies on teacher noticing in the context of video-based programs, Santagata et al. (2021) described different theoretical perspectives in studying teacher noticing: cognitive-psychological, socio-cultural, discipline-specific and expertiserelated.In our study, teacher noticing is investigated with a cognitive-psychological perspective similar to prior conceptualizations such as Jacobs et al. (2010), Kaiser et al., (2015), and van Es and Sherin (2002).This perspective allows teacher educators and researchers to implement video-based programs by considering cognitive processes and with prompts that encourage teachers to focus on student thinking or teaching practices.
In their systematic review on noticing studies in the context of video-based programs, Santagata et al. (2021) reported that only eight studies in 35 studies reviewed focused on in-service teachers.Out of these studies, only four of them utilized both quantitative and qualitative methods.These studies were in the video-club format in face-to-face meetings that took place in the United States (Sherin & van Es, 2009;van Es & Sherin, 2006).The results of the review suggest that there is a need for more videobased noticing studies in different formats and in different contexts.All in all, this study differs from other teacher noticing and video-based programs as (1) the focus is in-service novice teachers, (2) it combines mentoring with video-based content, and (3) mentoring and video-viewing are conducted virtually instead of face-to-face.
Considering these aspects, this study has a potential to contribute to the teacher noticing research, especially considering the necessity of online learning.

E-Mentoring
Commercial and welfare sectors are increasingly including mentoring as a tool for PD and service enhancement (Thompson et al., 2010).Mentoring, on the other hand, is seen as a vehicle for PD of teachers (Darling-Hammond, 2012;Guler & Celik, 2022).According to Ambrosetti et al. (2014), current definitions of mentoring suggest that mentors and mentees are in a hierarchical relationship, where mentors have more experience than mentees or can provide the knowledge and skills needed by mentees.It is more traditional to define mentoring as an inexperienced individual receiving support from an experienced colleague in the field he or she needs (Guler, 2019).Recent definitions of mentoring indicate that both groups (mentors and mentees) can benefit from each other (Gadomska-Lila, 2020).No matter how mentoring is defined, it is a complex activity because it involves many factors, including the mentor-mentee relationship, the needs and goals of the mentee, as well as the context in which the mentoring occurs (Ambrosetti, 2014).
In recent years, with the development of technology, the change seen in almost every field of life has been observed in mentoring activities (Single & Single, 2005) and the concept of e-mentoring, also known as electronic mentoring (otherwise referred to as online mentoring, telementoring, cybermentoring, virtual mentoring), has emerged.According to Mihram (2004), ementoring is a mentoring activity in which the development of individuals with little experience or inexperience is brought together with an experienced person (mentor) through electronic communication tools, without time and geographical limitations.Compared to face-to-face mentoring, e-mentoring has emerged as a growing field that offers enhanced opportunities.E-mentoring practices offer a more comfortable learning environment and can appeal to more participants than face-to-face mentoring (Kahraman & Kuzu, 2016;Spanorriga et al., 2018).It also removes the need for mentors and mentees to be in the same physical space at the same time and overcomes time barriers (Spanorriga et al., 2018).
Studies on e-mentoring practices in general report positive outcomes for participating teachers of different subjects (Erdogan et al., 2022;McAleer & Bangert, 2011;Simonsen et al., 2009).For instance, a study conducted by Erdogan et al. (2022) examined the impact of an e-mentoring-based education program on in-service preschool teachers' PD in terms of pedagogical content knowledge.Using various materials (e.g., e-books, videos, and live lessons) and with the participation of 28 pre-school teachers and nine academics, the content supported the professional knowledge, learning environments and classroom applications of the preschool teachers.On the other hand, only a few studies reported findings of mentoring practices designed for mathematics teachers (Guler & Celik, 2022;Simonsen et al., 2009;Surrette, 2020).In an experimental study conducted by Guler and Celik (2022), an e-mentoring application enriched with both videos and video clips were used to enhance lesson analysis skills of twelve novice mathematics teachers.The researchers highlighted that the intervention provided teacher to think on student mathematical thinking, focus on mathematical content, and generate alternative strategies to improve a lesson.They indicated that similar ementoring approaches can be effective for improving mathematics teachers' lesson analysis skills.
In the aforementioned studies on mentoring, mentors are experienced teachers in the same schools or districts.Different than previous studies on mentoring, E-MMT is designed in order to enhance novice teachers' noticing skills by viewing different kinds of videos and use of lesson analysis in addition to mentoring support.An affordance of E-MMT was to allow novice teachers from different rural areas come together and learn from each other and mentors online.This study is unique in the sense that facilitation of discussions on video records of teaching is embedded in the e-mentoring design.Mentors in our study do not only have teaching experience but also have researcher and teacher educator perspectives, which is different than prior research on working with mentors who are experienced K-12 teachers.A researcher perspective allowed mentors to design a research-based program, which had the potential to support teacher learning.
This paper investigated effects of E-MMT on novice mathematics teachers' noticing skills.More specifically, we consider changes in novice mathematics teachers' attending, interpretation and decision to responding skills.In this study, we address the following research questions: 1. RQ1.Is there a statistically significant difference between novice mathematics teachers' attending skills before and after they attend E-MMT?
2. RQ2.Is there a statistically significant difference between novice mathematics teachers' interpretation skills before and after they attend E-MMT?
3. RQ3.Is there a statistically significant difference between novice mathematics teachers' deciding to respond skills before and after they attend E-MMT?

Significance of the Study
Different than previous studies, our study demonstrates an alternative way to face to face video-based studies in enhancing teacher noticing.Our study informs how E-MMT was implemented in detail and teacher noticing was supported as well as assessed by using video.This study was conducted before COVID-19 pandemic.However, conducting this study online is also relevant and worthwhile to be presented to other researchers and teacher educators particularly after COVID-19.In addition to saving resources and bringing together novice teachers and teacher educators from different geographical locations, programs like E-MMT can be inspiring to future researchers and teacher educators when online learning is necessary.

METHOD
A one-group, pretest-posttest research design was adopted in this paper since it was aimed to investigate the effect of a program on a group of novice mathematics teachers' noticing skills.Although impact studies on a single group have limitations (Knapp, 2016), single group experimental studies are recommended especially in designs that have more than one independent variable whose effect cannot be controlled and explained solely, and the relationship between the variables cannot be investigated (Cakiroglu et al., 2017).

Participants
The study was conducted with 17 middle school novice mathematics teachers employed at different regions of Turkey.Since the aim of the study was to support teachers in the first years of their profession, models in the literature were examined (e.g., Glennie et al., 2016;Mok, 2005) and teachers in their first five years of their careers were included in the study.For instance, Glennie et al. (2016) reported that turnover rate was the highest among novice teachers who were in their first four years in many countries.
In this respect, it can be said that purposeful sampling is used.Researchers sent invitation letters to local schools and seventeen novice mathematics teachers voluntarily participated in the study.The participants were selected from the county or rural areas of the same province with at least one mentor due to the activities carried out during the initial phase of E-MMT (meeting, implementing pre-test, etc.), which will be detailed in the next section.
Teaching experience of the participants varies between 1.5 years to 4,5 years.Schools where teachers work are located in the villages and town centers.Finally, considering educational degrees, it is seen that eleven of the teachers hold a bachelor's degree while six of them continue their graduate studies.There were five mentors to support the of 17 teachers involved in E-MMT.The mentors, who were also the researchers, were all teacher educators from different universities.They came together because of their common research interests in teacher noticing PD and e-mentoring.They had K-12 and university teaching experience at varied levels.

Process
E-MMT was carried out in three cycles.Two different mentors carried out mentoring process in each cycle with different teachers.One teacher educator acted as a mentor in the first two cycles.This mentor participated as an observer in the last cycle in order to maintain consistency between cycles.There were six mathematics teachers in the first two cycles, and five mathematics teachers in the third cycle.The main goal of E-MMT was to ensure the development of novice teachers' pedagogical content knowledge with a special focus on increasing their attending to student thinking, interpreting and decision making in order to improve their teaching.During the implementation of E-MMT, teachers were expected to actively participate, share their reflections on teaching as a group, receive feedback from each other and mentors, and engage in reflection (Darling-Hammond et al., 2017).E-MMT was designed as 10 weeks, composed of three main phases: The initial phase (determining the teachers and the goals), the cultivation phase (implementing the content), and the separation phase (completing the group mentoring and continuing with the one-on-one mentoring).

Initial phase (weeks 1-2)
The mentors informed the teachers (mentees) about E-MMT, and the teachers learned how to use Adobe Connect, the software utilized in the process.

Cultivation phase (weeks 3-8)
The weekly contents were delivered online using the Adobe Connect interface, with simultaneous participation of teachers.Cultivation phase was enriched with videos and reflecting on classroom practice as an approach of learning to teach by using teaching practices (Santagata et al., 2007;Sun & van Es, 2015).This phase of the intervention is as summarized in Figure 1.
Because this phase is the phase where the content is implemented, it includes components that aim to support teachers' PD intensively.The content was mainly video-based.During the selection process of the videos, the goal was to select videos, which demonstrated high levels of teacher-student interaction, the use of rich materials, and the active participation of the students at every stage of the lesson.To achieve this goal, the content of the program included (1) video clips of student thinking, (2) videos of the whole-class teaching of entire lessons (enabling novices to have the opportunity to understand influence of teaching practices on student understanding and model effective practices), and (3) novice teachers' own classroom videos to allow for self-reflection and evaluation.
While the discussions on the video clips focused mostly on the student thinking on a specific case, in whole-class videos, the effectiveness of the course, the teaching approach of teacher, and alternative ways of teaching were discussed in addition to student thinking and learning.To systematize this for all whole-class videos, lesson analysis framework was used.In that way, it was aimed to make associations with all components of pedagogical content knowledge by focusing on the effects of the teacher strategies on student understanding, the importance of the use of materials, the questions asked by the teacher, and learning outcomes considering the goals of the curriculum.Accordingly, discussions were based on the lesson analysis framework.In the third week (which is the first week of the cultivation phase), the components of mathematics knowledge for teaching (MKT) (Ball et al., 2008) were first introduced.Afterwards, in order to benefit from the potential of short videos in focusing directly on student thinking, video clips suggested by Sherin et al. (2009) so as to provide windows into student thinking, the depth of thinking, and the clarity of the thinking were presented to teachers.At the end of the third week, participants were asked to anticipate student responses to questions in a given interview protocol.The fourth week began with a discussion of the extent to which teachers were able to predict student thinking.After a discussion of possible student responses, the teachers watched and discussed video clips of students, which demonstrated actual student responses to the given protocol.In the fourth week and weeks that followed, the video content was entire whole-class lessons and discussions about videos were based on the lesson analysis framework.In this sense, videos were analyzed considering learning goals of the lesson, student learning, effectiveness of the teaching activity, and alternative strategies to improve the viewed lesson.The mentors guided participants' attention to the moments where relationship between the teacher's actions and students' learning were evident in the video and facilitated interpretation of these moments.For the fifth week, teachers were given some homework, including analysis of a video from the third international mathematics and science study (TIMSS) using the lesson analysis framework.Participants were also given feedback for each of their analysis before the fifth week.After viewing the critical moments in TIMSS video the beginning of the fifth week, a wholeclass video was discussed using the lesson analysis framework, similar to the previous week.
In the sixth week, mentors focused on the importance of lesson plans and teachers were engaged in discussions about lesson plans (Li et al., 2009).Following that, mentors introduced a lesson plan and encouraged teachers to analyze strengths and weaknesses of the lesson plan considering the learning goals and the topic.The teachers viewed the video of the implementation of the lesson plan.The teaching in the video was analyzed considering the lesson plan and the lesson analysis framework.In the seventh and eighth weeks teachers analyzed their videos of teaching with their colleagues and mentors.For these two weeks, novice teachers were asked to develop a lesson-plan, implement and record it.Each teacher shared own video records of the teaching with the mentors ahead of the online session.Mentors identified significant aspects of the lesson considering the lesson analysis framework.During the sessions, participants viewed the videos together with their peers and analyzed lessons with the facilitation of the mentors.Each teacher reflected on their lesson plans as well as the effectiveness of the lesson on student learning and alternative pedagogical decisions.

Separation phase (weeks 9-10)
In this phase, one-on-one mentoring was implemented in order to support teachers according to their individual needs.In this context, teachers and mentors prepared a lesson plan together.Teachers videotaped their lessons as they implemented the lesson plan, analyzed their videos, met with their mentor on Adobe Connect and reflected on their teaching.The mentors, on the other hand, examined the records and made some suggestions to teachers on their approach to student thinking, teaching methods and alternative approaches.The meetings lasted about two hours.In the last two weeks, the discussions on the edited records of teachers' classroom teaching lasted about two and a half hours.

Instrument
Considering previous literature on video studies and research questions in this study, we developed a video repository in order to assess the participants' noticing skills.The researchers selected three videos that they thought would serve the purpose, among the videos of different teachers' lessons.A whole-class video of one entire lesson, which all three researchers (experienced in conducting research in use of video in teacher noticing) selected in common and independently from each other was determined as the data source.The video contained rich teacher-student interactions that had the potential to provide opportunities to talk about students' mathematical thinking and assessment of the learning outcomes.
In the previous research studies, dividing video of an entire class period into shorter segments was recommended to support teachers' noticing of important moments (Santagata et al., 2007;Santagata & Angelici, 2010).Similar to this approach, the 30minute video of the chosen whole-class video was divided into four segments that are between 6-10 minutes each.As pre-and post-test, teachers were asked to answer the same questions related to the video segments.Because nature of video may influence teacher noticing, the same video was used for maintaining the validity of the video assessment.In the process of developing these video analysis questions, a draft was created by considering the lesson analysis framework (Santagata & Yeh, 2016) and further developed by gathering opinions of experts.The final version of the video assessment is provided below (Table 1).Video assessment included three noticing components associated with viewing each video clip: attending to significant events, interpretation of student understanding and deciding how to respond based on students' understanding (or decision-making).

Data Collection and Analysis
The pre-test took place in the second meeting and before the implementation of E-MMT.In the pre-test, the teachers were asked to read the video assessment questions (Table 1) before the video segments were reflected on the screen.The teachers could take notes while viewing the videos.After each video segment, the teachers were given time to answer the questions, and the next video segment was viewed after all the participants finished answering questions about the previous video segment.After the implementation of E-MMT, the teachers were invited to the university.The post-test was carried out in a similar way.Participants did not have access to the video content in the time between.Considering three-months of time lag between tests, it is unlikely that post-test answers were affected by the pre-test.
In order to assess the noticing skills of the mathematics teachers, data analysis framework was developed by the researchers considering previous research (Barnhardt & van Es, 2015;van Es, 2011).Accordingly, the responses of the teachers to each video were analyzed separately in dimensions of attending, interpreting and decision-making considering the indicators presented in Table 2.We started data analysis by considering classroom events in each video clip (teacher actions, student understanding, etc.).Aligned with prior conceptualizations (Barnhardt & van Es, 2015), we considered noticing components as attending, interpreting, and decision-making.
Participants' attending was coded at three levels: low evidence, medium evidence and robust evidence.When participants attended to the teacher's actions, pedagogical behaviors or general classroom environment, their answers were coded as "low evidence".When participants' attention focused on the teacher in the video but included mathematical details, their comments were coded as "medium evidence".Similar to previous research (van Es, 2011), extended levels of noticing is associated with considering student thinking.Therefore, participant answers were considered as "robust evidence" when they focused on student thinking or attended to teacher actions in relation to student understanding.
Participants' interpretation was coded at three levels: low evidence, medium evidence and robust evidence.Low evidence codes were assigned to descriptive comments and comments focused on general aspects of teaching and learning.When participants interpreted the teacher's actions based on principles of teaching and learning, their comments were coded as medium evidence.When participants were able to make connections between students' understanding and teacher actions in their interpretations, their responses were coded as robust evidence.
Participants' providing alternative pedagogical suggestions/decisions were also coded as low, medium or robust evidence.When the participants provided no suggestion or the suggestions were too general, then the answers were considered as low evidence.When participants' alternative instructional decision suggestions included mathematical details but failed to consider student understanding or thinking, then their responses were coded as medium evidence.When the instructional decision suggestions included mathematical details by taking into accounts of students' understanding, then the responses were coded as "robust evidence".
It is important to note that we did not expect participants to attend to every aspect of video.We considered the highest level of evidence they were able to provide in their responses in the data analysis process.Additionally, similar to prior research (Barnhart & van Es, 2015), we did not evaluate instructional suggestions based on their appropriateness.
For each teacher, pre-and post-test answers were analyzed by researchers.In total, one teacher's answers needed to be coded in 24 instances.In order to maintain inter-rater reliability, four teachers' pre and post video assessment answers were used for training purposes.After coding for 4 teachers' answers together, the coding dictionary was finalized by the research team.The remaining 13 teachers' answers were coded individually by the research team.Out of 312 total instances of coding, there were disagreements in coding 53 of them.As a result, the inter-rater reliability was found to be 83%.The research team discussed and solved conflicting cases and agreed on coding of all instances.Each noticing component was scored between 1-3 depending on the level of noticing demonstrated by teachers: low evidence (1), medium evidence (2), robust evidence (3).Considering responses of novice teachers separately in each video clip, their minimum scores would be 4, and maximum scores would be 12 for each dimension.In order to reveal the effect of the intervention, the Wilcoxon-Signed rank test was used since the study was carried out with a relatively small and a single sample.

RESULTS
The non-parametric tests carried out revealed that practice E-MMT was effective in terms of improvement in each dimension of noticing.In order to determine the magnitude of these effects, Figure 2 summarizes the results of each dimension, alongside Hedge's g effect size.As the effect sizes indicate, the improvement in noticing skills was moderate but close to high impact for attending and had a high impact for both interpreting and decision-making.
In this section, the results from each dimension of noticing framework is presented by underlying changes in the levels of teacher responses as measured by statistical tests.

Attending Dimension
The analysis of pre-test revealed that participants performed better in attending dimension (m=6.88)compared to the other dimensions.Many of the teachers tended to make evidence-based explanations in the videos.However, they focused on teacher actions instead of focusing on student's understanding.After the implementation of E-MMT, it was seen that the teachers' average score regarding the attending dimension increased to m=9.76.This result indicates that teachers' attention moved from teacher action towards student understanding.Wilcoxon signed-rank test results indicated a statistically significant improvement with

Medium evidence
To explain what teacher does by providing evidence.
When teacher said, "let's draw height", only height of one side was drawn.Teacher said that "this is not height of one side, but height of parallelogram".

Robust evidence
Identifying evidence-based situations related to student's understanding.In fact, teacher should have drawn students' attention to concepts of base & ceiling while constructing height.When students see this shape, they cannot perceive base & ceiling of parallelogram.As soon as shape rotates, it could be specified which side could be considered base or ceiling & height could have been constructed accordingly.
regards to participants' attending skills following the implementation of E-MMT program (Z=-3,075, p=.000).Table 3 includes representative examples of teachers' attending responses before and after E-MMT.
As seen from Table 3, compared to the responses in the pre-test, teachers started to make evidence-based comments on both teachers' instructional actions and on students' understanding in the post-test.For instance, analysis of pre-test revealed that T9 focused on teacher action of assessment.In the post-test, although T9's focus was still on teacher actions, she started to make evidence-based explanations.Similarly, in the pre-test, T8 focused on teacher actions during introduction of the lesson.In the post-test, T8 provided evidence of a specific student understanding (student answer on properties of line).In the pre-test, even though comments of T12 were evidence based, the comments focused on only teacher actions (i.e.constructing definition of perpendicular lines).In the post-test, T12 provided specific evidence on student understanding about definition of perpendicular lines.Considering mean scores of pre-and post-test, a majority of teacher comments went up to medium evidence and robust evidence levels in the attending dimension as a result of the intervention.

Interpreting Dimension
The pre-test average score for this dimension, m=5.65, showed that the teachers had difficulties in making connections between the teacher actions in the videos they viewed with the evidence on the student understanding.In other words, the causeeffect relationship was mostly limited to either the teacher action solely or the general teaching principles and methods.Post-test average scores regarding the interpreting dimension increased to m=8.88.Wilcoxon signed-rank test results also showed significant improvement and confirmed the effectiveness of the intervention (Z=-3,535, p=.000).
Table 4 includes representative examples of teachers' interpretation responses before and after E-MMT.Considering changes in teacher interpretations, T7 moved from a very general comment to providing a specific and mathematics related comment about teacher's actions for video clip 3.In the pre-test, T12 focused on the teacher's mathematical related actions while in the post-test she interpreted a specific student's construction of the height.Another teacher, T14 focused on teacher actions in a general way in the pre-test while she made specific references to a student misconception (confusing side of the parallelogram with the side) in the post-test.Overall, in the pre-test, even though participants made connections between teacher actions and student learning, their responses tended to be general rather than specifically mathematics related (i.e., T14 pre-test).In the posttest, their responses included more specific and mathematics related evidence to support their arguments about connections between teaching and student understanding.Low evidence

Post-test
The teacher could not ask appropriate questions to the students when she requested definition and explanations about the parallelogram.For instance, when a student answered that "extends in both directions infinitely", she could have asked "what does extend infinitely?" (video clip 1).

Robust evidence T12
Pre-test The teacher did not directly define what perpendicular was, instead she constructed the definition by asking students to come up with ideas (video clip 2).

Post-test
The teacher asked how to construct the height.She showed two different examples (one appropriate and one inappropriate) and made sure that students found the correct representation of height.It was the students who stated that the perpendicular angle was 90 degrees, that the height was straight and that "the height is the shortest distance between two lines" (video clip 2).

Decision-Making Dimension
As final indicator of noticing skills, decision-making was the lowest performed dimension in pre-test (m=5.12).The pretest data showed that the teachers' suggestions to improve the quality of instruction were mostly general suggestions without focusing on the specific teacher strategies and student understanding in the lesson they viewed.In other words, while making suggestions, none of the participants focused on the evidence of classroom situations in the videos.Post-test results in this dimension increased to m=8.Although teachers showed the lowest performance in both the pre-and post-tests in decision making, the results showed that E-MMT showed a significant effect in improving this dimension (Z=-3,422, p=.001).
Table 5 includes representative examples of teachers' responses before and after E-MMT.Considering the answers of T13, in the pre-test she did not provide an alternative suggestion whereas in the post-test she made a relevant and mathematics related suggestion (though in a general manner).In the pre-test, T10 suggested using a ruler in order to be precise, which was not necessarily based on student understanding.In the post-test, her suggestion became more specific, mathematics related and based on student understanding.In the pre-test, T7 made a general suggestion of using materials (cutting paper or cardboard models) whereas in the post-test T7 made a very specific mathematics related suggestion based on students' understanding (naming the height of a specific side to be considered).Overall, in the pre-test teachers tended to suggest alternative instructional strategies in general ways without considering student thinking.In the post-test, when teachers provided specific mathematics suggestions based on observations of student understanding, it indicated heightened levels of decision-making skills.

DISCUSSION AND CONCLUSION
The present study aimed to examine the effect of E-MMT on developing novice mathematics teachers' noticing skills.In order to achieve this goal, novice teachers working in different regions came together on an online platform.The results showed that E-MMT contributed positively to the development of teachers' noticing skills.It is important to note that the videos used as Robust evidence fundamental material in the process along with the lesson analysis framework were effective in improving participants' noticing skills.This result confirms the outcomes of the existing PD studies focusing on improvement of noticing skills and using video as a tool for this purpose (Jacobs et al., 2010;Santagata & Yeh, 2016).Noticing focuses on understanding and making sense of teaching through student learning, which is indispensable for teaching environments (Barnhart & van Es, 2015;Sherin & van Es, 2009).By introducing lesson analysis framework in facilitation of video discussions, teachers were able to focus on significant aspects of the lesson, such as teaching strategies and assessment practices as well as evidence of student thinking (Santagata et al., 2007).The teachers had opportunities to examine the effect of teaching on student learning in the observed lessons (Sun & van Es, 2015) and provide suggestions to improve teaching (Santagata et al., 2007).Considering the implementation process as a whole, it can be said that teachers demonstrated noticing skills actively utilizing the components of lesson analysis in the process.
In providing the PD of the in-service teachers, the mentoring practice is mostly carried out in environments where mentors are experienced teachers (Celik & Atik, 2020;Maor & McConney, 2015).In the present study, the program was facilitated by experienced mathematics teacher educators.Active involvement of academicians in the process may have enriched the discussion environment with their expertise in novice teacher learning.Mentors have taken on facilitator roles such as conducting group discussions, evaluating the suggestions given for improving student learning in a viewed lesson, presenting alternative ideas to teachers in order to improve teaching, and drawing teachers' attention to significant aspects of student learning and specialized teaching methods, and examining the lessons in depth.Mentors' efforts to help teachers focus on these points enabled teachers to start and talking about them.As a matter of fact, these reflections are evident in the responses to the pre-test and posttest.These results are also important in terms of eliminating the general perception that faculty-school cooperation is limited to pre-service teachers' school experience (Aydin et al., 2016) and show that productive opportunities for university and school partnership can be achieved.
Attending to critical events within classroom environment is an integral part of noticing (Ozdemir-Baki & Kilicoglu, 2023;van Es & Sherin, 2002).A significant difference was found between the pre-test and post-test scores of the teachers regarding the attending dimension.Attending pre-test mean scores of the novice teachers at the beginning of E-MMT were higher than the average scores of other components.Comparing pre-and post-test scores, the component that teachers improved the least was attending, which may be caused by the relatively high initial scores at this level.Most importantly, when the post-tests were examined, it was seen that teachers started to consider the events that took place in the classroom from the perspective of the students.They started to focus on the student's understanding and thinking.This is in line with findings of the study by Sherin and Han (2004), who reported that attention of the participating teachers started to shift more towards the mathematical ideas of the students rather than teachers in the videos they viewed after participating in the PD program.
Considering the dimension of interpretation, there was a significant difference between pre-test and post-test scores.What is expected from teachers regarding the interpretation skills is to interpret classroom situations by using reasoning and establishing a cause-and-effect relationship.In this study, teachers made the most progress in the interpretation dimension.Before the implementation, the teachers tended to describe general behaviors and practices of the teacher in the videos and make short and general comments without using reasoning or making any connections to student understanding.In contrast, teachers started to consider what the teacher does in terms of contribution to student understanding in the post-test.We believe facilitation discussions that took place during E-MMT based on lesson analysis, i.e. asking teachers to provide reasoning for their opinions, to encourage considering relationships between teachers practices and student learning as well as the learning goals may have been productive in enhancing teachers' interpretation skills (Santagata & Angelici, 2010;Santagata et al., 2007).
Similar to interpretation, decision-making was a dimension in which a significant difference was found between the pre-test scores and the post-test scores.The suggestions of the novice teachers in the pre-test were general rather than focusing on specific mathematical topics.In the post-test, teachers tended to move away from giving general suggestions to more specific suggestions involving students' understanding.Some studies reported giving suggestions as instructional decisions was among the most difficult components for novice teachers to learn (Barnhart & van Es, 2015).Consistent with previous research (Ulusoy & Cakiroglu, 2021), this study showed that if novice teachers are supported, they can improve their ability to give alternative suggestions for improving teaching.The design of the mentoring program and discussions within the online platform regarding how the situations in the classroom affect the students' understanding and proposing alternative instructional decisions may have contributed to the improving of this noticing dimension.In the process, the mentors tried to guide not only pedagogically but also to improve teaching by focusing on specific mathematical situations aiming at enhancing subject-specific pedagogical content knowledge.
Although teachers developed a tendency to provide suggestions for instructional decision-making, the suggestions of teachers were not always in the desired quality.The important thing for us was to help teachers analyze a teaching activity and give suggestions to improve teaching based on student understanding.In our study, we did not expect the teachers to give the most effective suggestions, but the alternatives, which had a potential to make teaching better by considering student understanding and the mathematical instruction.We aimed to see whether the teachers could develop suggestions based on mathematical content and student understanding in the video assessment.Different suggestions can be developed for specific situations, but most appropriate or effective suggestion in such cases is debatable.Future studies can examine teachers' explanations for giving suggestions in depth.In order to give direction for future e-mentoring programs in different contexts, it is important to discuss reasons as to why the program was successful in enhancing novice teachers' noticing skills.In our study, we have incorporated several design decisions that may have led to growth in participants' noticing skills, such as viewing and analyzing both student clips, other teachers' videos and participants' own teaching.This study also showed video-based e-mentoring programs could be considered for future studies as it saved time and travel costs.This design was effective in this study and future research may compare influence of different designs on teacher noticing skills.

First
student who stood up to board drew a line (as in figure)& it was emphasized that this line he drew was actually parallel to AB & DC & therefore could not be perpendicular.I think it is nice to emphasize construction of height not only of base but also of sides.Because students always think that height could be drawn only from base, it is very difficult for them to draw line perpendicular to other sides, no matter how much name of sides is emphasized.InterpretingLow evidence Making descriptive comments.Commenting on general teaching & learning principles.The course contributed greatly to students' understanding of the outcome.Because students have learned by doing and experiencing.Medium evidence To interpret the teacher's actions by making explanations on principles of teaching & learning.Interpreting evidence in a cause & effect relationship.It was nice that teacher indicated heights of different sides of parallelogram.Teacher had students draw heights that belonged to [BC] side.It was good that teacher showed height could also be drawn from extension of side.Robust evidence Making explanations on principles of teaching & learning & interpret what student does together with teacher's actions.To explain evidence in cause & effect relation.To interpret specific situations related to students on evidence.Teacher tried to have students discover height between two parallel lines before moving on to height of parallelogram.The fact that students found the shortest distance between parallel lines was perpendicular & that they were equal, contributed to students' learning.Decision-making Low evidence Giving general suggestions or not giving suggestions.Providing ideas for improving teacher's pedagogical approach.Introduction part of lesson was too long, it could have been shorter.Medium evidence Presenting alternative ideas about what to do next time to improve teaching.She introduced lecture by starting with properties of parallelogram & she put a lot of emphasis on properties.I think she could have started lesson by focusing on construction of height.She could have made students find heights of shapes in daily life, & then move on with parallelogram.Robust evidence Providing suggestions to improve student understanding on what teacher does.Provide alternative ways of teaching on evidence.

Figure 2 .
Figure 2. Mean scores and effect sizes of dimensions (Source: Authors' own elaboration)

Table 4 .
Representative examples, which illustrate differences in teacher's interpretation skills between pre-and post-tests teacher showed heights of different sides of parallelogram.Teacher had students construct height of [BC] side.It was good that teacher showed that height could also be constructed by using extension of side (Teacher had a difficulty in constructing a perpendicular line to a side.This happened partly because teacher tried to get students to construct height themselves (video clip 4).Medium evidencePost-testFirst student who stood up to board drew a line (as in figure)& it was emphasized that this line he drew was actually parallel to AB & DC & thus could not be perpendicular.I think it is nice to emphasize construction of height not only of base but also of sides.Because students always think that height could be drawn only from base, it is very difficult for them to draw line perpendicular to other sides, no matter how much name of sides is emphasized (video clip 4approach is appropriate.She is trying to let kids discover.She continues with examples, but she has trouble with time management.That's why she cannot conclude.I think students will not be able to fully understand topic (video clip 4).Low evidencePost-test Teacher used a meter, but I think she was a bit late to use it.I do not think students fully grasped subject.Because there were students who still showed side of parallelogram as height (video clip 4).Robust evidenceTable5.Representative examples, which illustrate differences in teacher's decision-making skills between pre-and post-tests Participant Representative examples of teachers' decision-making responses Level T3 Pre-test When drawing shapes, teacher uses a ruler, not drawing random shapes.This can be useful for presenting an accurate demonstration to students (video clip 3).Low evidence Post-test Teacher mentioned height of only one side.She could have also showed heights of other sides (video clip 3Construction of height could be made with a ruler.It was shown on a hypothetical figure that students' drawings were equal.Measurements are important in mathematics (video clip 3).Medium evidence Post-test Teacher could have also drawn various parallelograms & determine different heights with correct/precise measurements.It should not be contented with a single parallelogram type.It should be mentioned that different heights can be drawn (video clip 3Maybe students could have cut parallelograms out of cardboard or paper & showed on them (video clip 3height of side [DC] were asked in addition to height of side [BC], it would have awakened in students idea that height of more than only one side could be drawn (video clip 3).

Table 1 .
Video assessment: Questions related to the first video segment Questions You will watch a video clip on 6 th grade lesson objective "drawing (constructing) altitudes of corresponding bases in a parallelogram."When you view following video segments take a note of important moments you find & support your answers by providing evidence (examples) from video.The first video segment (i) Which moments are the most significant moments in the video segment you watched?Why? (ii) What do you think about appropriateness of teachers' instruction methods considering learning goals & contribution to student learning?(iii) (If there is) What would you suggest to the teacher in the video?Note.*The 2 nd , 3 rd , and 4 th video questions are the same with the questions in Table 1

Table 2 .
Dimensions, levels and explanations, and representative examples regarding noticing Teacher makes students say what they know by giving them permission to speak.Teacher stands in same area of classroom & cannot fully manage classroom.