The nature and extent of learning outcomes to be achieved at different levels of the general education system, and the means through which they should be achieved, is usually articulated in the curriculum or education programme. The curriculum, on the other hand, will usually receive its cue from national development goals and priorities.
Teaching and learning processes operationalise these outcomes and give them effect. Assessment verifies if stipulated outcomes have been achieved, although it can also be an input for learning to occur and/or be directed. The extent to which stipulated outcomes have been achieved remains a dominant1, though not exclusive signal of the quality of education, as well as of the effectiveness of curriculum implementation, teaching and learning. That is to say, assessment procedures will normally only be able to capture limited elements of learning that has occurred, in specifically defined areas, for example, literacy and numeracy.
Assessment in itself is a varied education process. It varies by purpose, forms of assessment and area of assessment. An initial distinction has to be made between assessment for learning and assessment of learning. The former is concerned with the function of assessment as an educational process. For this, feedback to the learner is essential3. Nevertheless, on a systemic level, assessment of learning is essential in order to monitor achievement of the education system as a whole. Assessment of learning, on the systemic level, can also result in (policy) lessons to improve systemic performance and, in this sense, on this level as well, ‘assessment for learning.’ can take place (although this expression is not usually used to refer to systemic learning). To this end, such large scale assessments usually use instruments for assessment of factors associated with learning in addition to the actual tests, which are normally grounded in a framework such as the generic ‘CIPP’-model (CIPP stands for Context, Inputs, Process and Product, see adjacent Figure 2), which is used, for example, by the Latin American Laboratory for Assessment of the Quality of Education (LLECE).
This Analytical Tool aims at assisting users to diagnose if, and to what extent, the existing assessment system is part of the impediments to reaching the desired and/or stated goals of education quality. The paramount question in the diagnosis of our assessment systems is how assessments can contribute to improving the quality of our education system and learning effectiveness. The diagnosis addresses this paramount question by posing some key questions with regard to assessment policies, frameworks and methods in place, the implementation mechanisms, and the systems for drawing appropriate lessons from assessment results and using the results from assessments to improve the different aspects of education processes and outcomes.
Diagnosis and analysis
assessment policies, frameworks and methods
1. Do we have a national strategy / policy / position paper on educational assessment? If yes, how recent is this? Which educational levels (both in terms of ISCED and in terms of location (local – regional – national) and subjects are covered by this? Has it been evaluated?
2. To what extent is the choice of purposes, targets and subject matters for assessment, for example in national assessments, related directly to what the country thinks of as important in terms of learning outcomes for its learners and not only in terms of what is easy to assess? (See Articles on Assesment)
3. What have been the criteria used to determine the coverage of the assessment and the level at which national assessments are conducted? Are these criteria linked to clear objectives and goals of the assessment? Is there evidence that the coverage and the levels at which the assessments are made contributed to improvement of education system quality?
4. In general, to what extent is assessment in this country effective? To what ends? Is it inclusive? In what way? What evidence do we have for this? Do we know where the system stands in terms of achievement outcomes at every level?
Implementation of assessment
1. If there is an educational assessment policy has it been implemented/enacted? How do we know? At what levels is assessment implemented? What are the objectives of this?
2. Is there evidence that the implementation of the assessments is according to rules of good practice, incl. inclusiveness? What is this based on? [Analytical Tool on Equity and Inclusion]
3. Who implements assessments? How does this vary by types of assessment?
4. How are tests conceptualised (i.e. how are test items developed) and what is the conceptual basis for this (for example, a curriculum/syllabus analysis or rather an orientation of ‘life skills’)? What psychometric methods and techniques are used to classify items, and to what extent are these item characteristics taken into account in the development of achievement tests? Are open and closed items used? In terms of test conceptualisation, is there a good mix of standardised and non-standardised testing available?
5. Are assessments also measuring ‘associated factors’ that facilitate analysis (e.g. looking at age, gender, socioeconomic status and other background information)?
6. If applicable, how are data processed and fed into a centralised information system?
7. What is the evidence that participation in international quality assessment (LLECE, PISA, SACMEQ and others) help us to bench mark the quality of our education system? What has been our and others experience of international assessments? If we have not participated, was it a deliberate decision and, if so, why?
Utilisation of assessment results
1. What mechanisms do we have for making the evaluation of the assessment results inform education policy and practice (at classroom, school, regional and national level)? How often do we use these mechanisms? What is the evidence that we do such evaluation in a purposeful and systematic way? 2. How do we interpret the findings from evaluations of assessment results findings, and how do we make sure that educational assessments have the intended impact of improving the education system quality and learning effectiveness? How do we communicate our evaluation so as to focus on how we can do better? How are outcomes data linked to other variables, such as finance data, which permits rigorous analyses?
3. Are assessment results made public, and to whom (for example, individual student results to parents / carers; school rankings to the general public, etc.)?
Priorities for action
1. What are the key areas to be addressed urgently to make assessment contribute to the quality of our education system?
2. What are the knowledge gaps which need to be filled for an evidence-based policy and practice of school-based and national assessments?
3. What are the required actions to deal with the priority constraints and the identified knowledge gaps?