Classification of Test Items Written by Mathematics Pre-Service Teachers

Most teachers assume that asking questions contributes to the effectiveness of their instruction. Because proper questioning techniques are important for the classroom, this study identified the Mathematics pre-service teachers’ classification of test items using the revised Bloom’s Taxonomy (rBT) and the Cunningham’s Levels of Questions (CLQs). It used a group of forty two pre-service Mathematics teachers who were asked to classify each thinking skills in the rBT as to LOT or HOT and then create test items falling under each category of the rBT and CLQ. Results revealed that most of the pre-service teachers have viewed LOT and HOT based on the level of difficulty of the given problem. They found it relatively easier to create test items using the Cunningham’s Levels of Questions, a non-familiar nomenclature for classifying test items, rather than the well-known Bloom’s Taxonomy. Pre-service teachers should be able to identify and classify questions according to their cognitive purposes.


INTRODUCTION
Mathematics contributes directly to effective, intelligent living. Competencies with number relations are essential to every individual beginning at an early age and continuing through old age. Hence, numeracy and problem solving skills have been the focus of the present Mathematics instruction. However, Mathematics teaching has traditionally relied on factual recall and a focus on the use of standardized algorithms with little effort to teach or assess for higher-order thinking skills (Battista, 1994;Kulm, 1990) as cited by Thompson (2008). As a result, students generally learn Mathematics without being able to use their knowledge to solve problems in diverse or non-familiar situations. Furthermore, the result of the 2011 Trends in International Math and Science Study (TIMSS) with 46 participating countries is a dismal scenario where the Philippines emerged as third from the last both in Math and Science after South Africa and Morocco. Educational analysts believe that the performance of the Filipino students in the TIMSS indicates danger signal which do not augur well for the country's attainment of its goals towards industrialization and economic development.
Unfortunately, the blame for the said disappointing scenario is always on the part of the teachers as ineffectiveness in instructional efforts results to the underachievement of students in Mathematics; and that ineffective instruction is a result of being an incompetent teacher. Thus, teachers of Mathematics have been spending more time and effort to find the best possible teaching strategy that could make their students learn. However, evaluation of Mathematics achievement yields the same poor results, if not worse.

/ 10
http://www.iejme.com According to the National Center for Education Statistics (1996), teaching for higher order thinking (HOT) along with professional development in HOT were found to be two of the top five variables positively associated with improved student achievement. Students of teachers who teach for both lower-order-thinking (LOT) and higher-order-thinking outperform students whose teachers only teach for LOT. Thus, Mathematics teachers should have a strong conception of higher-order thinking and sustain it as an integral part of classroom instruction and assessment. With this, students will be able to meaningfully apply methods and concepts to situations previously unfamiliar to them. And because Bloom's Taxonomy is a well-known model for teaching critical thinking skills in any subject area, it is used as a guide in creating test items.
Also, based on the K -12 Mathematics Curriculum Guide (2013), the twin goals of mathematics in the basic education levels are critical thinking and problem solving. Critical thinking, according to Scriven and Paul (1987) as cited in the 2013 Curriculum Guide, is the intellectually disciplined process of actively and skillfully conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, or communication, as a guide to belief and action. On the other hand, according to Polya (1945Polya ( , 1962, mathematical problem solving is finding a way around a difficulty, around an obstacle, and finding a solution to a problem that is unknown. These two goals are to be achieved with organized and rigorous curriculum content, a well-defined set of high-level skills and processes, desirable values and attitudes, and appropriate tools, taking into account the different contexts of Filipino learners. Critical thinking skills have become a generic phase used to encompass many processes involved in learning and problem solving. Several reasons are often given for teaching thinking skills. One reason given by Gough in 1997, which was affirmed by Nisbet andSchucksmith (2000), is that the world is changing rapidly and critical thinking skill is the most important element in today's information age.
Most countries like the Philippines are concerned with raising the educational standards throughout the compulsory school years. The Department of Education (DepEd Order No. 33, s. 2004) has set minimum desired learning competencies and mastery of the "basics" are exposed such as reading, writing, Mathematics and Science. These basics maybe excellently taught, but these are not sufficient to meet the demands of the labor market and active citizenship. It is now recognized that a broader range of competencies, redefined as higher-order thinking skills (HOTS), are required because: a. The "banking theory" of knowledge based upon rote learning has been discredited as it is recognized that individuals cannot "store" sufficient knowledge in their memories for future use.
b. Information is expanding at such a rate that individuals require transferable skills to allow them to address different problems in different contexts at different times through their lives.
c. The complexity of modern jobs requires staff who demonstrates comprehension and judgment as participants in the generation of knowledge or processes.
d. Modern society assumes active citizenship which requires individuals to assimilate information from multiple sources, determine its veracity and make judgments.
The cognitive skills developed by people in a society are profoundly influenced by the ways knowledge and literacy are taught and used. Thinking skills are viewed as crucial for educated persons to cope with a rapidly changing world. Many educators believed that specific knowledge will not be as important to tomorrow's workers and citizens as to the ability to learn and make sense of new information.
According to Robinson (1997), if students are to function successfully in a highly technical society, then they must be equipped with life-long learning and thinking skills necessary to acquire and process in an everchanging world. Moreover, Pascua (1991) mentioned that it is the higher-order thinking skills that enable an individual to learn more, to apply Mathematics in other disciplines and to solve problems throughout life.
Bloom's research as cited by Mergel in 1998, emphasized that in skill mastery, the teacher needs to focus and develop the higher-order thinking of students. There is substantial evidence that directed and controlled intervention programs can improve students' abilities to give more original and unusual solutions to Mathematical problems of higher level.
According to Balmaceda as cited by Laviña (2000), in the knowledge-based and technology-driven society of the next millennium, creativity, analytical skills, logical reasoning and critical thinking will be of paramount importance. These skills will be crucial in coping with the explosion of information and adapting to new and changing demands and challenges. The development of higher-order thinking skills (HOTS) and strategies to improve such has been the focus of educators in recent years, partly in response to the challenges of the 3 / 10 emerging world. Higher-order thinking skills or what others refer to as critical thinking has emerged as a priority for secondary and college curricula in most subject areas.
Since pre-service teachers were being developed to become successful in-service teachers, they were the focus of this study. Thus, Mathematics pre-service teachers must have a strong conception of critical thinking (higher-order thinking) and sustain it as an integral part of classroom instruction and assessment. With this, students will be able to meaningfully apply methods and concepts to situations previously unfamiliar to them. And because Bloom's Taxonomy is a well-known model for teaching critical thinking skills in any subject area, it is used as a guide in creating test items.

THEORETICAL FRAMEWORK
Social constructivism is the primary theoretical framework guiding this paper. Social constructivists believe social interaction, cultural tools, and activity shape individual development and learning (Resnick, 1991;Tudge & Scrimsher, 2003;Wertsch, 1991). Chin (2007) stated that in the classroom "knowledge was constructed through language and other semiotic means" (p. 816). If we take into consideration that much of this language takes place in the form of teachers asking questions (Levin & Long, 1981), then the teachers' questions are an essential part of classroom discourse.
Teacher's questions represent the social constructivist concept of Scaffolding. Scaffolding refers to the assistance provided by more competent peers or adults (Wood, Bruner, & Ross, 1976). Students do not explore the world in isolation. Therefore, student's learning may be facilitated by parents or teachers who are usually more knowledgeable (Vygotsky, 1978). During the classroom discourse, questions can be used as a psychological tool that mediates students' knowledge construction (Chin, 2007), triggers classroom interactions, and aids students in building content knowledge.
In 1978, Vygotsky made a major contribution to the social constructivist theory. Vyogtsky (1978) added the Zone of Proximal Development (ZPD) and defined ZPD as "the distance between the actual developmental level as determined by independent problem solving and the level of potential development as determined through problem solving under adult guidance or in collaboration with more capable peers" (p. 86). Based on this theory, teacher's questions should be neither too easy nor too difficult and new knowledge should be built on student's prior knowledge. The teacher needs to know the current developmental level of a student so that the question asked is appropriate and is located in the student's ZPD. When students interact socially within their ZPD, students are more likely to discover new knowledge and bridge the new knowledge with prior knowledge (Schunk, 2007). In other words asking a confused student a higher cognitive question will not help the student with knowledge construction because the question is not in the student's ZPD. Asking an appropriate follow-up question is more likely to facilitate connections between previous knowledge and the current question. Feedback should enhance the questions teachers ask and the subject being studied and should be asked in such a way that it guides student learning. Moreover, feedback must direct attention to the intended learning, point out strengths and offer specific information for improvement, be timely and occur during the learning, address partial understanding, and limit the corrective information to advice the student can act on (Chappuis, 2009). Effective feedback is important because it is specifically related to achievement no matter the grade level, race, or socioeconomic status (Bellon, Bellon, & Blank, 1991). Vygotsky's ZPD theory and the social constructivist theory provide us with a lens through which we may identify the importance of follow-up questions and feedback.
A question is defined as an utterance that is posed in the form of an interrogation or has a grammatical form which seeks to find out some information about a student's knowledge or thinking (Chin, 2007). Socrates recognized the importance of questioning as early as the fifth century BC (Ellis, 1993;Harrop & Swinson, 2003;Overholser, 1992). When Socrates taught he did not answer students' questions by providing direct answers (Moore & Rudd, 2002), instead he posed further questions to place the responsibility of thinking on the students. This technique became known as the Socratic Method and required students to be active thinkers rather than passive listeners. The Socratic Method of questioning seldom requests factual information, but persuades and permits students to express their opinions and explore the rationale for their responses (Overholser, 1992). Questioning should challenge students to think critically and creatively (Ellis, 1993;Wilen, 1991), stimulate student participation, arouses student interest (Wilen, 1991), identify student abilities (Ellis, 1993;Wilen, 1991) and misconceptions, confirm students' understanding of the material being taught.
Even though the Socratic Method of questioning is a successful model and questioning is widely used in the classroom, the cognitive level and the purpose of questions teachers ask indicate that classroom teachers possess inadequate questioning techniques. There are three reasons why questioning by classroom teachers is failing to meet the rigor of the Socratic Method. First, the majority of the questions teachers ask in the classroom are lower level questions that emphasize rote memory and recall of factual information (Cunningham, 1987;Gall, 1984;Myhill & Dunkin, 2005;Wilen, 1991). A study conducted by Pate and Bremer (1967) revealed that most teachers regarded questioning as a means to check student understanding, diagnose misconceptions, and urge students to recall specific facts. Ten percent of the participants mentioned generalizing and making inferences. However, teachers' questions should go far beyond memorizing. Teachers should understand that authentic thoughts are stimulated by questions that require making inferences, drawing conclusions, and creating meaning (Elder & Paul, 1998).
Most K-12 teachers are familiar with Bloom's Taxonomy as a scaffold for questioning, but few have had substantive training in their teacher education courses that included effective questioning based on Bloom's Taxonomy (Hannel, 2009). Being aware of Bloom's Taxonomy is not enough to persuade classroom teachers to design questions based on the six levels within the cognitive domain. Moreover, pre-service elementary mathematics teachers struggle when listening to and responding to students (Nicol, 1999). Therefore, today's pre-service teacher education courses cannot focus solely on posing questions. Teacher educators must pay equal attention to questioning skills with regard to posing, listening, and responding to students. Because posing questions without listening and responding to students does not build a constructive discussion, teachers must listen to students, make sense of students' answers and questions, and identify students' understanding of the questions posed in order to provide constructive feedback or ask constructive follow-up questions. Questioning should be a two-way interaction between teachers/students and students/students. The categorization of questions and the respective theorists/researchers is summarized and may be reviewed in Table 1.
Bloom's Taxonomy (or its revision), Sander's Taxonomy and Cunningham's Five Levels of Questions provide teacher educators with examples of levels of questioning. Bloom's Taxonomy (Bloom et al., 1956) is the most popular and has been used to categorize questions into hierarchical cognitive levels. Bloom's Taxonomy divides learning objectives into three domains: cognitive, affective, and psychomotor. The cognitive domain includes six hierarchical categories: knowledge, comprehension, application, analysis, synthesis, and evaluation. Knowledge is the lowest cognitive level and evaluation is considered the most complex. Each category of the cognitive domain is associated with specific verbs, which may be used to phrase learning objectives and questions. Sanders' Taxonomy (1966) divides cognition into seven hierarchical categories: memory, translation, interpretation, application, analysis, synthesis, and evaluation.
Bloom's and Sanders' Taxonomies are based on the verbs that describe learning objectives and have been criticized for being too narrow to allow for an accurate measurement of each level (Riegle, 1976). Moreover,

Table 1. Question Levels and corresponding theorist
Theorists/Researcher Question Levels • Bloom, et.al (1956) • Gall (1970) claimed that "a weakness of the cognitive-process approach to question classification is that these processes are inferential constructs" (p.710) that cannot be directly observed. Zhang and Patrick (2012) proposed the use of Cunningham's Five Levels of Questions (1987) as a better choice for observable and measurable categorizing of questions ( Table 2). The lowest level of questioning emphasizes rote memory and the answer to the question is predictable. The middle level of questioning is convergent and is divided into low and high levels. Low-convergent questions require students to put facts together and construct a response using comparing, contrasting, generalizing, transferring form, or explaining. High-convergent questions require students to look for evidence to support the answer, give reasons for behaviors or outcomes, and draw conclusions. However, teachers also look for specific answers at this level. The highest level is composed of divergent questions that and are divided into low and high levels. Divergent questions are usually open-ended. Low-divergent questions require students to find alternative solutions. High-divergent questions promote creative thinking.

RESEARCH QUESTIONS
This study attempted to determine the classification of test items written by Mathematics pre-service teachers using the revised Bloom's Taxonomy and the Cunningham's Level of Questions. Specifically, it aimed to answer the following questions: a. What is the description of the Mathematics pre-service teachers in terms of their academic achievement?
b. How do Mathematics pre-service teachers describe lower-and higher-order thinking skills?
c. Which questions written by mathematics pre-service teachers were representative of: • each thinking skill in the Bloom's Taxonomy?
• each level in the Cunningham's Taxonomy?
d. Which taxonomy is easier to use in preparing test items as perceived by mathematics pre-service teachers?

METHODOLOGY
The participants of the study were composed of 42 mathematics pre-service teachers (MPSTs) from two selected universities in Cavite. Their GPAs in the content and professional courses that were already taken were gathered. The MPSTs were asked to describe lower-and higher-order thinking. When testing for levels of thinking, it is important that teachers should note the familiarity to the problem situation (Thompson, 2008). It is considered as HOT if the problem situation is new, unfamiliar, or in some way different from those used in the instruction. Otherwise, it is deemed LOT.
Pre-service teachers were then asked to classify each thinking skill in Bloom's Taxonomy as either LOT or HOT. Lastly, they were asked to construct a Geometry examination item for each thinking skill in the RBT and CLQ. Since the latter is not familiar with the PSTs, the researcher explained each level and provided them with a handout that consists of the characteristics and examples of each question level. Once they were comfortable with the CLQ, they proceeded to writing test items. The researcher classified each item as to whether it fits with a thinking skill in BT and in the Cunningham's Levels of Questions. Three consultants were asked to classify and validate each test item independent of the researcher to break disagreements on classification.

RESULTS
Of the twenty 42 participating Mathematics pre-service teachers, seventy four percent of them indicated that they were either very familiar (14%) or somewhat familiar (60%) with Bloom's Taxonomy (BT); but none of them indicated familiarity with Cunningham's Levels of Questions. Some of these pre-service teachers also indicated that they had learned BT during their undergraduate courses as part of their curricular program. Others have learned it also through researches and seminars they had attended.

Research Question No. 1: What is the Description of the Mathematics Pre-service Teachers in terms of their Academic Achievement?
MPSTs' final grades in content and professional courses were gathered by computing their GPAs (see Table 3). Dominantly, student's achievements in specialization content courses and professional education courses are interpreted as very satisfactory that falls under 1.76 to 2.00.
Based on these, it is expected that each pre-service teacher is well-versed in writing test items since their content and professional knowledge are both very satisfactory which are both very essential to students taking teacher education program.

Research Question No. 2: How do Mathematics Pre-service Teachers Describe Lowerand Higher-order Thinking Skills?
Some pre-service teachers included as part of their characterizations of LOT or HOT the level of difficulty of the given problem -as to simple or complex processes involved. Several pre-service teachers wrote that LOT problems are easier than HOT problems -they involve logical and critical thinking as well as careful analysis of the given problem.
Furthermore, a few pre-service teachers had mentioned that LOT requires simple recall of information and that these information are too basic and based only on rote memory, whereas, HOT as mentioned is a challenging tool in teaching and learning Mathematics because it enhances the creativity of the students in solving given problems. However, they fail to characterize LOT or HOT problems based on students' familiarity to the given problem.
On the other hand, pre-service teachers were asked to classify each thinking skill in the Bloom's Taxonomy as either LOT or HOT. The results are presented in Table 4.
Teachers' classification of thinking skills indicated that 100% of the teachers correctly classified remembering as to LOT; analyzing and creating as to HOT. Over 75% of the teachers classified understanding correctly as LOT. Applying is considered LOT or HOT in the revised Bloom's Taxonomy. However, about 55% of the teachers classified application as HOT and the rest as LOT. It is interesting to note that most of the teachers identified analyzing items correctly. Evaluating is the second highest level of thinking skill in BT

Research Question No. 3: What are the Categories of Questions Developed by the Preservice Teachers in terms of the Revised Bloom's Taxonomy and the Cunningham's Level of Questions?
Out of 243 test items that were developed under the RBT presented in Table 5, a little more than half (53.50%) were correctly classified according to Bloom's Taxonomy. The remembering and application categories both received the highest percentage of correct classifications (70%) followed by understanding skills (60%). Test items under creating skills had the lowest percentage (21.05%).
Out of 194 test items that were developed under the CLQ presented in Table 6, more than 75% were correctly classified according to Cunninghams' Level of Questions. The high convergent, low convergent and high divergent questions all received a remarkable percentage of correct classifications.
In the analysis of test items written, it is noticeable that common higher-level items were those that will require students to construct and/or criticize proofs given some conditions of the problems, without really establishing their familiarity with the problem. Another is that teachers were likely to categorize as HOT those items that will require students to solve complex procedures where in fact, the procedure has been taught to the students before but is not implied on the problem the solution to be used. Some teachers had interpreted evaluating items as "finding the value of" where in fact; test items within this category must be considered application items. Several pre-service teachers commented that although the test items were familiar to the students, there is still a need to analyze the problems, identify and combine theorems and postulates and their relations to the condition stated in order to arrive at the conclusion. Hence, despite the pre-service teachers' familiarity with the Bloom's Taxonomy, this did not help much in the construction of the test items that are appropriate to their classifications of thinking skills.
Surprisingly, despite that all of the MPSTs were not familiar with Cunningham's Levels of Questions, they successfully wrote test items falling in each category under this classification.

Research Question No. 4: Which Taxonomy is Easier to Use in Preparing Test Items as Perceived by Mathematics Pre-service Teachers?
In classification of test items, it is interesting to note that MPSTs could develop and classify test items more correctly in terms of the Cunninghams' (85.57%) rather than Bloom's Taxonomy (53.50%). Is this  indicative that MPSTs perceived the use of Cunningham's to be easier than the Bloom's Taxonomy? Table 7 presents the results of t-test of difference between two proportions.
Comparison of correct classification of test items using RBT and CLQ indicates that there is significant difference in the way mathematics pre-service teachers classify test items, Z = -7.978, p < 0.05. This indicates that MPSTs relatively find it easier to write test items using Cunningham's Levels of Question, a new nomenclature for them, rather than the revised Bloom's Taxonomy, a very quite familiar nomenclature in classifying test items in cognitive levels.

CONCLUSIONS
Mathematics pre-service teachers have very satisfactory academic achievement which help them in writing test items. In teaching for HOT in Mathematics, pre-service teachers disregarded the concept of familiarity of the student to the situation presented to them. What they thought of was that LOT items were easier to answer and/or solve than HOT items. Most of the pre-service teachers found it difficult to construct test items that fit in the thinking skill described in Bloom's Taxonomy, especially those at the evaluating and creating levels. Teachers also overestimate those "complex problems" and regarded them as HOT where in fact, they are considered as LOT. Also, they tend to underestimate those LOT items which require student to recall theorems and concepts without anticipating that the problems need to be analyzed in order to come up with such theorems. It is interesting to note that although they were not familiar with Cunningham's Levels of Questions, they successfully wrote test items under each category.
To support pre-service teachers in promoting classroom interactions through higher-level questioning, preservice teachers should be aware of the types of questions that encourage higher level thinking. They should be able to identify and classify questions according to their cognitive purposes such as lower level knowledge questions and higher-level evaluation questions. And so, the researcher recommends the use of Cunningham's Levels of Questions first in categorizing test items since most of the pre-service teachers could easily create test items under this classification. However, it is not suggested that the use of Bloom's taxonomy be eliminated in the curriculum as this is essential in classifying questions in the cognitive level. They should go hand in hand to better equip pre-service teachers with adequate knowledge in promoting classroom interactions through higher-level questioning.