The article discusses the challenges of and solutions of developing a common, general standards-referenced student assessment framework for mathematics. Two main challenges are faced. First, the main challenge is the lack of commonly accepted standards as the basis for the criterion- and standards-referenced assessment. Second challenge is that, even if having criteria and standards for the assessment, the descriptions of the standards are, in many cases, so vaguely worded that it is not possible to create unambiguous test items on the basis of a specific level of the standards. An initial common framework for mathematics standards is introduced on the basis of Common European Framework in Reference for Languages (CEFR) – a common framework for mathematics (CFM).

• Acharya, D. R., & Metsämuuronen, J. (2014). Nepali Achievement at Grade 3 in NASA 2012. In ERO, Student achievement in Mathematics, Nepali and English at grade 3 and 5 in 2011. Ministry of Education, Kathmandu, Nepal.
• Acharya, D. R., Metsämuuronen, J., & Adhikari, H. (2013). Nepali Achievement in NASA 2011. In J. Metsämuuronen & B. R. Kafle (Eds.), Where Are We Now? Student achievement in Mathematics, Nepali and Social Studies in 2011 (pp. 113–170). Ministry of Education, Kathmandu, Nepal.
• AERA, APA, & NCME. (2014). Standards for Educational and Psychological Testing. Washington DC: American Educational Research Association, American Psychological Association. National Council of Measurement in Education.
• Angoff, W. H. (1974). Criterion referencing, norm referencing and the SAT. College Board Review, 92, pp. 3–5, 21.
• Béguin, A. (2000). Robustness of Equating High-Stake Tests. Enschede: Febodruk B.V.
• Brown, S. (1988). Criterion referenced assessment: what role for research? In H. D. Black, & W. D. Dockerell, New developments in educational assessment, British Journal of Educational Psychology, Monograph series, 3, 1–14.
• Bruner, J. S. (1986). Actual Minds, Possible Worlds, Cambridge, Massachusetts: Harvard University Press.
• Bruner, J. S. (1990) Culture and Human Development: A New Look. Human Development 33 (6), 344–355. https://doi.org/10.1159/000276535
• Bruner, J. S. (1996). The Culture of Education, Cambridge, London: Harvard University Press.
• CLS. (2014). Research impact case study. The impact of adult literacy and numeracy research based on the 1970 British Cohort Study. Centre for Longitudinal Studies. Institute of Education, University of London. Retrieved on October 18, 2016 from http://www.cls.ioe.ac.uk/library-media%5Cdocuments%5CFINAL%20-%20CLS%20impact%20case%20study%20-%20Adult%20literacy%20and%20numeracy.pdf
• Council of Europe. (2011). Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Council of Europe. Retrieved on October 18, 2016 from http://www.coe.int/t/dg4/linguistic/Source/Framework_EN.pdf
• Cox, B. (1995). The Battle for the English Curriculum. London: Hodder & Stoughton.
• Criterion-referenced Test. (2014). In S. Abbott (Ed.), The glossary of education reform. Retrieved on October 18, 2016 from http://edglossary.org/criterion-referenced-test/
• Daro, P., McCallum, W., & Zimba, J. (2012). The Structure is the Standard. In Tools for the Common Core Standards. Retrieved on October 18, 2016 form http://commoncoretools.me/2012/02/16/the-structure-is-the-standards/
• Dziuban, C. D., & Vickery, K. V. (1973). Criterion-referenced Measurement: Some Recent Developments. Educational Leadership, February 1973, 483–486. Retrieved on October 18, 2016 from http://www.ascd.org/ASCD/pdf/journals/ed_lead/el_197302_dziuban.pdf
• ERO. (2015). Unpublished output of a workshop of creating National standards-reference assessment system for Nepali, English, Science and Mathematics at 4. – 22.12.2015 in Educational Review Office (ERO), Sanothimi, Bakhtapur, Nepal.
• FNBE. (2003). National Core Curriculum for Upper secondary Schools 2003. Finnish Board of Education. Vammala: Vammalan Kirjapaino Oy. Retrieved on October 18, 2016 from http://www.opetushallitus.fi/download/47678_core_curricula_upper_secondary_education.pdf
• FNBE. (2004). National Core Curriculum for Basic Education 2004. Finnish Board of Education. Helsinki: Yliopistopaino. Retrieved on October 18, 2016 from http://www.opetushallitus.fi/english/curricula_and_qualifications/basic_education/curricula_2004
• FNBE. (2014). National Core Curriculum 2014. Finnish Board of Education. Määräykset ja ohjeet 2014:96. Helsinki: Next Print Oy. Retrieved on October 18, 2016 from http://www.opetushallitus.fi/download/163777_perusopetuksen_opetussuunnitelman_perusteet_2014.pdf [In Finnish]
• FNBE. (2015). National Core Curriculum for Upper secondary Schools 2015. Finnish Board of Education. Määräykset ja ohjeet 2015:48. Helsinki: Next Print Oy. Retrieved on October 18, 2016 from http://www.opetushallitus.fi/download/172124_lukion_opetussuunnitelman_perusteet_2015.pdf [In Finnish]
• Glaser, R. (1963). Instructional Technology and the Measurement of Learning Outcomes. American Psychologist, 18, 519–521. Reprinted in Educational Measurement: Issues and Practice, 13(4), 6 – 8. https://doi.org/10.1111/j.1745-3992.1994.tb00561.x
• Glaser, R., & Klaus, D. J. (1962). Proficiency measurements: Assessing human performance. In R. M. Gagne (Ed.), Psychological Principles in Systems Development (pp. 419 – 474). New York: Holt, Reinhart & Winston.
• Glaser, R., & Nitko, A. (1971). Measurement in Learning and Instruction. In R. L. Thorndike (Ed.), Educational Measurement (pp. 625 –670). Washington DC: American Council of Education.
• Glass, G. V. (1978). Standards and criteria, Journal of Educational Measurement, 15, 237–261. https://doi.org/10.1111/j.1745-3984.1978.tb00072.x
• Gray, W. M. (1978). A comparison of Piagetian theory and criterion-referenced measurement. Review of Educational Research, 48, 223–249. https://doi.org/10.3102/00346543048002223
• Green, S. (2002). Criterion Referenced Assessment as a Guide to Learning – The Importance of Progression and Reliability. A paper presented at the Association for the Study of Evaluation in Education in Southern Africa International Conference Johannesburg, 10 July – 12 July 2002. Retrieved on October 18, 2016 from http://www.cambridgeassessment.org.uk/images/109693-criterion-referenced-assessment-as-a-guide-to-learning-the-importance-of-progression-and-reliability.pdf
• Hambleton, R. K. (1980). Contributions to Criterion-referenced Testing Technology: An Introduction. Applied Psychological Measurement, 4(4), 421–424. Doi: http://dx.doi.org/10.1177/014662168000400401. Retrieved from https://conservancy.umn.edu/bitstream/handle/11299/100198/v04n4p421.pdf?sequence=1
• Hambleton, R. K. (1981). Advances in criterion-referenced testing technology. In C. R. Reynolds & T. B. Gutkin (Eds.), Handbook of school psychology (pp. 351–379). New York: Wiley.
• Hannula, M. (2015). Email discussion at November 6th, 2015.
• K-8 Publishers. (2013). K–8 Publishers’ Criteria for the Core State Standards for Mathematics. Spring 2013 Release 04/09/2013. Retrieved on October 18, 2016 from http://achievethecore.org/content/upload/Math_Publishers_Criteria_K-8_Spring_2013_FINAL.pdf
• Kaftandjieva, F. (2004). Standard Setting. Reference Supplement, Section B. In S. Takala (Ed.) (2009), Relating Language Examinations to the Common European Framework of Reference for Languages: Learning, Teaching, Assessment (CEFR). A Manual. Language Policy Division, Strasbourg. Retrieved on October 18, 2016 from https://www.coe.int/t/dg4/linguistic/CEF-refSupp-SectionB.pdf
• Kane, M. T. (1986). The role of reliability in criterion-referenced tests. Journal of Educational Measurement, 23(3), 221–224. https://doi.org/10.1111/j.1745-3984.1986.tb00247.x
• Kuukka, K., & Metsämuuronen, J. (2016). Perusopetuksen päättövaiheen suomi toisena kielenä (S2) -oppimäärän oppimistulosten arviointi 2015 [Assessment of the learning outcomes of the L2 Finnish 2015.] Kansallinen koulutuksen arviointikeskus. Julkaisut 13:2016. Tampere: Juvenes Print – Suomen Yliopistopaino Oy. https://doi.org/10.13140/RG.2.1.2569.0005 Retrieved on October 18, 2016 from http://karvi.fi/app/uploads/2016/05/KARVI_1316.pdf [In Finnish].
• Linn, R. L. (1993). Linking results of distinct assessment. Applied Measurement in Education, 6(1), 83–102. https://doi.org/10.1207/s15324818ame0601_5
• Linn, R. L., & Gronlund, N. E. (2000). Measurement and assessment in teaching (8th ed.). Upper Saddle River, NJ: Prentice Hall.
• Lithner, J., Bergqvist, E., Bergqvist, T., Boesen, J., Palm, T. & Palmberg, B. (2010). Mathematical competencies: A research framework. In C. Bergsten, E. Jablonka, & T. Wedege (Eds.), Mathematics and mathematics education: Cultural and social dimensions. Proceedings of MADIF 7, the Seventh Mathematics Education Research Seminar, Stockholm, January 26-27 (pp. 157–167). Linköping, Sweden: Svensk förening för matematikdidaktisk forskning, SMDF.
• MEB. (2016). The Finnish Matriculation Examination. The Matriculation Examination Board. Retrieved on October 18, 2016 from https://www.ylioppilastutkinto.fi/fi/english
• Mehrens, W. A., & Lehmann, I. J. (1991). Measurement and Evaluation in Education and Psychology. 4th Ed. Fort Worth, TX: Harcourt Brace College Publishers.
• Metsämuuronen, J. & Mattsson, M. (2013). Effect of repeated testing to the development of Vocabulary, Nominal Structures, and Verbal morphology. Journal of Educational and Psychological Development, 3(2), 89–101. https://doi.org/10.5539/jedp.v3n2p89 Retrieved on October 18, 2016 from http://www.ccsenet.org/journal/index.php/jedp/article/view/28330
• Metsämuuronen, J. (2009). Ratkaisuprosentin muuntaminen taitotasoksi – A-Finska-oppiaineen aineiston erityiskysymyksiä [Changing the percentage of correct answers into proficiency level – Specific questions of A-Finnish assessment]. Unpublished memorandum 28.12.2009. [In Finnish]
• Metsämuuronen, J. (2010). Omvandlingen av poängtal till färdighetsnivåer inom det receptive delområdet av provet i A-lärokursen och i den modersmålsinriktade lärokursen. [Changing the percentage of correct answers into proficiency level in Finnish language for Swedish speakers A-course and native speakers’ course test]. Appendix VII. In O. Toropainen (2010). Utvärdering av läroämnet finska i den grundläggande utbildningen. Inlärningsresultaten i finska enligt A-lärokursen och den modersmålsinriktade lärokursen i årskurs 9 våren 2009 (pp. 164–168). Uppföljningsrapporter 2010:1. Utbildningsstyrelsen. Helsinki. Retrieved from http://www.oph.fi/publikationer/2010/utvardering_av_laroamnet_finska_i_den_grundlaggande_utbildningen [In Swedish].
• Metsämuuronen, J. (2013). A New Method to Setting Standard for the Wide Range of Language Proficiency Levels. International Education Research, 1(1). https://doi.org/10.12735/ier.v1i1p01 Retrieved on October 18, 2016 from http://www.todayscience.org/IER/v1/Jari%20Metsamuuronen.pdf
• Metsämuuronen, J. (2014). English Achievement at Grade 5 in NASA 2012. In ERO, Student achievement in Mathematics, Nepali and English at grade 3 and 5 in 2011. Ministry of Education, Kathmandu, Nepal.
• Metsämuuronen, J. (2016a). Challenges in the National Level Criterion-referenced assessment. Nepalese Journal of Educational Assessment, 1(1), 17 – 36.
• Metsämuuronen, J. (2016b). Oppia ikä kaikki. Matemaattinen osaaminen toisen asteen koulutuksen lopulla. [Learning all my days. Mathematical achievement at the end of the upper secondary education.] Kansallinen koulutuksen arviointikeskus. Julkaisut 2016. Tampere: Juvenes Print – Suomen Yliopistopaino Oy. [forthcoming] [in Finnish]
• Metsämuuronen, J., & Salonen, R. V. (2016). Student achievement in Mathematics in Vocational Education. Kansallinen koulutuksen arviointikeskus. Julkaisut 2016. Tampere: Juvenes Print – Suomen Yliopistopaino Oy. [forthcoming] [In Finnish]
• Metsämuuronen, J., Acharya, S. P., & Aryal, H. (2013). Setting Standard in Nepali Language – CEFR levels. In J. Metsämuuronen, & B. R. Kafle (Eds). Where Are We Now? Student achievement in Mathematics, Nepali and Social Studies in 2011 (pp. 393–422). Ministry of Education, Kathmandu, Nepal.
• Mislevy, R. J. (1992). Linking educational assessments: Concepts, issues, methods, and prospects. Princeton, NJ: ETS Policy Information Center.
• Niss, M. & Jensen, T. H. (Eds.). (2002). Kompetencer og matematiklæring: Ideer og inspiration til udvikling af matematikundervisning i danmark. København: Undervisningsministeriets forlag.
• Niss, M., & Højgaard, T. (Eds.) (2011). Competencies and Mathematical Learning. Ideas and Inspiration for the development of mathematics teaching and learning in Denmark. English Edition, October 2011. Roskilde University, Department of Science, Systems and Models, IMFUFA. nr. 485 – 2011. Retrieved on October 18, 2016 from http://rudar.ruc.dk//bitstream/1800/7375/1/IMFUFA_485.pdf
• Nitko, A. J. (1980). Distinguishing the many varieties of criterion-referenced tests. Review of Educational Research, 50, 461–485. https://doi.org/10.3102/00346543050003461
• OECD. (2014). PISA 2012 Technical report. Paris: OECD. Retrieved on October 18, 2016 from http://www.oecd.org/pisa/pisaproducts/PISA-2012-technical-report-final.pdf
• Oxford English Dictionary (1983). Oxford University Press.
• Parsons, S. (2012). User guide to accompany. The 1970 British Cohort Study 2004 adult literacy and numeracy assessment data. Centre for Longitudinal Studies. Institute of Education, University of London. Retrieved on October 18, 2016 from www.cls.ioe.ac.uk/bcs70_userguide_literacynumeracy
• Poldrack, R. A. & Packard, M. G. (2003). Competition among multiple memory systems: Converging evidence from animal and human brain studies. Neuropsychologia, 41(3), 245–251. https://doi.org/10.1016/S0028-3932(02)00157-4
• Pollitt, A. (1994). Measuring and evaluating reliability in national curriculum assessments. In D. Hutchinson & I. Schagen, Eds, How reliable is national curriculum assessment? (pp. 60 – 70). London: NFER.
• Popham, W. J. (1978a). Well-crafted criterion-referenced tests. Educational Leadership, 36(2), 91–95.
• Popham, W. J. (1978b). Criterion-referenced measurement. Eaglewood Cliffs, NJ: Prentice-Hall.
• Popham, W. J. (1980). Domain specification strategies. In Berk, R.A. (Ed,), Criterion-referenced measurement: the state of the art (pp. 15–31). Baltimore and London: John Hopkins University Press.
• Popham, W. J. (1981). Modern educational measurement. Englewood Cliffs, NJ: Prentice-Hall.
• Popham, W. J. (2014). Criterion-referenced Measurement: Half a Century Wasted? Educational Leadership, 71(6), 62–66.
• Popham, W. J., & Husek, T. R. (1969). Implications of criterion-referenced measurement. Journal of Educational Measurement, 6(1), 1–9. https://doi.org/10.1111/j.1745-3984.1969.tb00654.x
• QCA. (2006). The National Qualifications Framework. Qualifications and Curriculum Authority, Qualifications, Curriculum and Assessment Authority for Wales, and Council for the Curriculum, Examinations and Assessment. Retrieved on October 18, 2016 from http://web.archive.org/web/20090126170205/http://qca.org.uk/libraryAssets/media/qca-06-2298-nqf-web.pdf
• Räkköläinen, M., & Metsämuuronen, J. (2016). Student achievement in Sustainable Development in Vocational Education. FINEEC. [Forthcoming]
• Räsänen, P. (2015). Email discussion at 2015 November 9th.
• Raynolds, C. R., & Fletcher-Janzen, E. (Eds.) (2004). Criterion-referenced Testing. In Concise Encyclopedia of Special Education: A Reference for the Education of the Handicapped and Other Exceptional Children and Adults. 2nd Edition. New York, NY: Wiley.
• Sadler, D. R. (1987). Specifying and Promulgating Achievement Standards. Oxford Review of Education, 13(2), 191—209. https://doi.org/10.1080/0305498870130207 Retrieved on October 18, 2016 from http://www.clinteach.com.au/assets/Specifying-and-Promulgating-Achievement-Standards.pdf
• Sadler, D. R. (2009) Indeterminacy in the use of preset criteria for assessment and grading in higher education. Assessment and Evaluation in Higher Education, 34, 159–179. https://doi.org/10.1080/02602930801956059
• Sadler, D. R. (2012). Assessment, evaluation and quality assurance: Implications for integrity in reporting academic achievement in higher education. Education Inquiry. 3(2), 201–216. Doi: https://doi.org/10.3402/edui.v3i2.22028 Retrieved on October 18, 2016 from http://www.education-inquiry.net/index.php/edui/article/view/22028
• Säftström, A. I. (2013). Exercising Mathematical Competence. Practising Representation Theory and Representing Mathematical Practice. Thesis for the Degree of Doctor of Philosophy. Göteborgs Universitet. Retrieved on October 18, 2016 from http://hdl.handle.net/2077/32484
• Shakya, S., & Metsämuuronen, J. (2014). Nepali Achievement at Grade 5 in NASA 2012. In ERO, Where Are the grade 3 and 5 students now? Student achievement in Mathematics, Nepali and English at grade 3 and 5 in 2011. Ministry of Education, Kathmandu, Nepal.
• Squire, L. R. (2009). Memory and brain systems: 1969–2009. The Journal of Neuroscience, 29(41), 12711–12716. https://doi.org/10.1523/JNEUROSCI.3575-09.2009 Retrieved on October 18, 2016 from http://www.jneurosci.org/content/29/41/12711.full
• Takala, S., & Kaftandjieva, F. (2009). Reseptiivisen kielitaidon kokeen muuntaminen taitotasoiksi. [Changing the receptive language test to proficiency levels]. In E Tuokko, Miten Ruotsia osataan peruskoulussa? Perusopetuksen päättövaiheen ruotsin kielen B-oppimäärän oppimistulosten kansallinen arviointi 2008 (p. 118). [How Swedish language is spoken in general education? National assessment 2008 of the student achievement in Swedish language B-course at the end of basic education]. Oppimistulosten arviointi 2/2009. Opetushallitus. Helsinki: Edita Prima Oy. [in Finnish]. Retrieved on October 18, 2016 from http://www.oph.fi/julkaisut/2009/miten_ruotsia_osataan_peruskoulussa
• Toropainen, O. (2010). Utvärdering av läroämnet finska i den grundläggande utbildningen. Inlärningsresultaten i finska enligt A-lärokursen och den modersmålsinriktade lärokursen i årskurs 9 våren 2009. Uppföljningsrapporter 2010:1. Utbildningsstyrelsen. Helsinki. [In Swedish]. Retrieved on October 18, 2016 from http://www.oph.fi/publikationer/2010/utvardering_av_laroamnet_finska_i_den_grundlaggande_utbildningen
• Tulving, E. (1983). Elements of Episodic Memory, New York: Oxford University Press.
• Ullman, M.T. (2004) Contributions of memory circuits to language: the declarative/procedural model. Cognition. 92, 231–270.
• Wiliam, D. (1993) Validity, dependability and reliability in National Curriculum assessment, The Curriculum Journal, 4(3), 335–350. https://doi.org/10.1080/0958517930040303

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.