Your browser is no longer supported

For the best possible experience using our website we recommend you upgrade to a newer version or another browser.

Your browser appears to have cookies disabled. For the best experience of this website, please enable cookies in your browser

We'll assume we have your consent to use cookies, for example so you won't need to log in each time you visit our site.
Learn more

'A'-LEVELS FINAL REPORT PUBLISHED

  • Comment
I am pleased to attach the final report on my inquiry into the...
I am pleased to attach the final report on my inquiry into the

conduct of the A level system. This phase 2 report makes proposals

not only to secure the examining process in 2003, but also on how the

system might be developed in the future.

I firmly believe the present operation can, and should, be improved.

I hope you will find my recommendations helpful in this regard.

MIKE TOMLINSON

A LEVEL INQUIRY: PHASE 2 REPORT

CONTENTS/ Page

Summary of conclusions and major recommendations 5

Introduction 8

The purpose and design of GCE A levels 10

Reducing burdens on the system 20

Processes for grading students' performance 25

QCA and its relationships with Ministers and the awarding bodies 30

Awarding bodies and the administration of the examination system 37

Professionalisation of the examinations system 43

The use of ICT in qualifications 50

ANNEX: steps in the examining and awarding process 57

Schedule of evidence presented to the inquiry 58

SUMMARY OF CONCLUSIONS AND MAJOR RECOMMENDATIONS

1. This inquiry has sought to resolve the major concerns expressed

this summer with the grading of A levels and propose arrangements

which will secure the examinations in the future and provide

assurances that the A level standard is being maintained year on

year.

2. I remain convinced that my interim report and the subsequent

review of grade boundaries dealt effectively with the major concerns

and allegations about manipulation of the grading process. Some of

the remaining concerns are not a consequence of grading but relate to

subject syllabuses (technically known as 'specifications') or the

marking and moderation of students' work. These concerns are being

dealt with separately by the QCA and the relevant awarding bodies.

3. Action by the QCA and the other regulatory bodies on my earlier

recommendations, allied to further proposals made in this final

report will, in my view, secure the standards and integrity of next

year's examinations. That has been, and remains, my priority. The

action taken includes defining the standard and levels of demand to

be associated with the AS and A2 examinations, supported by

exemplification of performance at grades A and E. This will, I

believe, deal with the major concern I had with implementation of the

Curriculum 2000 reforms.

4. Changes to the Code of Practice will ensure in future that a more

appropriate balance is struck than was always the case in the past

between professional judgement and statistical data in setting mark

grade boundaries, and that late changes to these boundaries will

require either the agreement of Chairs of Examiners or a report to

QCA and the governing body of the relevant awarding body. Through

these and other measures in hand, I hope that teachers, students,

parents and users of A levels will have their confidence restored.

However, it is vital that all the changes are communicated

effectively and speedily to all the relevant groups; if nothing else,

this inquiry has revealed the need for far better communication of

changes to our qualification system. In this context I fully endorse

the communications plans which the QCA is currently developing to

improve professional and public understanding.

5. Once the above changes have been implemented over the coming

months, a period of consolidation is necessary before further

evolution of the AS and A2 system is undertaken. Any further changes

must be carefully planned, piloted and introduced over a sensible

period. My recommendations must be seen as evolutionary and should be

considered in many cases within existing policy developments,

including reduction of bureaucratic burdens on schools and colleges

and the 14-19 proposals due to be published shortly.

6. The major recommendations cover:

Medium Term

- Systematic reform of the administrative requirements for the AS

and A2 examinations to reduce the demands placed on schools and

colleges by the awarding bodies' differing requirements and practices

(paragraph 112).

- Professionalisation of examining. This should include high quality

training for examiners and examination officers linked to career

development (paragraphs 134-136).

- Clarifying and making more transparent the relationship between

the QCA, the DfES and the awarding bodies, through a memorandum of

understanding. In addition, there should be changes to the

responsibilities of QCA (paragraphs 95-96).

- Arrangements to ensure, and reinforce confidence, that standards

over time are being safeguarded (paragraph 29).

- Simplification to the rules governing re-sits and 'cashing-in' of

AS units (Paragraph 43.)

- Changes to the timetable for publication of A level results to

give more time for marking and awarding (paragraph 58).

- Increasing the use of ICT in the administration and marking of

public examinations and eventually in the examining process itself.

(Paragraph 163.)

Longer term

- De-coupling of AS and A2 to create two free-standing

qualifications as part of the 14-19 policy developments.

Consideration should be given at the same time to other changes in

the design of A level assessment (paragraphs 42 and 53).

- Further work on the practicality of introducing a

post-qualifications admission (PQA) system for entry to Higher

Education (paragraph 59).

7. This inquiry arose out of concerns expressed by schools in England

about the processes followed by Awarding Bodies based in England.

However, a number of my recommendations have impact and relevance not

just in England but also in Wales and Northern Ireland. It is vitally

important that the QCA and its partner bodies, ACCAC in Wales and

CCEA in Northern Ireland, continue to work closely together to ensure

the smooth and consistent delivery of qualifications throughout the

three countries.

8. Finally I believe it to be vital that there is greater public

understanding of the examination process and that as a consequence

there is an end to the annual argument about A level results. The

standard has not been lowered if an increased proportion of students

meet it as a consequence of improved teaching and hard work.

INTRODUCTION

9. This inquiry was set up following concerns expressed about the

setting of A level standards in July 2002. The terms of reference for

the inquiry are:

- To investigate allegations about the setting of standards for

A-level grades this year. In particular, to make sure that the

conversion from marks to grades was determined according to proper

standards and procedures. A first report on this will be provided to

the Secretary of State by Friday 27 September.

- To investigate the arrangements at QCA and the awarding bodies for

setting, maintaining and judging A level standards, which are

challenging, and ensuring their consistency over time; and to make

recommendations by November to the Secretary of State and Ken Boston,

Chief Executive of the QCA, for action with the aim of securing the

credibility and integrity of these exams.

10. The first point was covered in my interim report published on 27

September. This led to the review of grades awarded to candidates for

one or more units in 31 separate A level subjects. The outcomes of

the review were that 9,800 candidate entries had unit grades improved

and of these 1,945 candidates received higher overall AS and A level

grades in at least one subject. Other recommendations in the interim

report have been the subject of work by QCA, supported by a programme

board. I have been encouraged by the progress made so far.

11. This final report deals primarily with point 2 of the remit

though necessary reference has been made to the action proposed in

relation to the recommendations made in my interim report. The

priority of this final report has been to secure the examination

process for 2003 so that there can be no repetition of the concerns

expressed this year. In addition, proposals are made for the medium

term once the standards and levels of demand of AS and A2 have been

firmly established in the system and the revised arrangements in the

Code of Practice have become fully operational. There should be no

major upheavals of Curriculum 2000 before the above have been

achieved.

12. The medium and longer term changes need to be considered as part

of other significant policy developments such as the drive to reduce

bureaucracy and simplify demands on schools and colleges and the 14

-19 proposals due to be announced shortly. The latter provides an

ideal basis for considering a number of my recommendations

13. I have been greatly helped in preparing this final report by the

many written submissions from bodies and individuals (over 100) and

by the three regional meetings I have held with teachers, head

teachers, college principals, governors, pupils and students. A

schedule of evidence presented to the inquiry is included as an

annex. The views offered have always been constructive and insightful

and I am grateful to each and everyone for their input, particularly

given the pressures upon their time.

14. These meetings have also raised some continuing concerns with the

2002 examination process. A significant number of these relate to the

quality of marking, coursework, promptness and extent of responses of

the Awarding Bodies and to the effectiveness of the assessment

requirements in some subjects. The latter are being revised for 2003

and some other concerns are covered in this final report.

15. Many of my recommendations will involve not only the Awarding

Bodies in England but also those in Wales and Northern Ireland. I

have appreciated the cooperation of colleagues from ACCAC in Wales

and CCEA in Northern Ireland.

16. I also wish to record my sincere thanks to members of the

reference group: Mike Cresswell, Sue Kirkham, Judith Norrington,

David Raffe, Sue Singer and Leslie Wagner. Together they have

provided a sounding board and have interrogated the evidence. This

report is, however, mine. None of the views, nor any flaws, are

attributable to the group. Finally I must thank the team who have

supported my inquiry: Audrey Beckford, Marcus Chrysostomou, Darren

Goff, Nancy McLean, Georgina Nolan, Peter O'Connor, Kate Taylor and

Matthew White.

THE PURPOSE AND DESIGN OF GCE A LEVELS

What are A levels for?

17. Ever since their introduction, A levels have been associated with

entry to higher education. This remains a valid and useful

application. But over time they have also acquired a broader

significance as a precursor to employment and as one strand in a

qualifications framework which is designed to recognise the full

range of advanced achievement of which young people are capable,

ranging from the purely academic and theoretical learning through to

the skills and knowledge associated with specific jobs. This trend is

embodied in the Curriculum 2000 reforms which increased the

flexibility of, and broadened the range of subjects and types of

learning within, the A level strand, for instance by establishing A

levels in vocational subjects.

18. During my inquiry, I have found broad commitment to this wider

purpose of A levels as a means of recognising young people's

achievement, and strong support for the principles of the Curriculum

2000 reforms. Students in particular value the possibility of

studying a broad range of subjects in their first year of A levels

and achieving a recognised AS qualification at that point for those

subjects which they do not pursue to full A level.

19. There is also strong support for the existing A level design

principle that the achievement required for an A level should remain

the same from year to year and reflect predetermined standards of

attainment, irrespective of how many students achieve the necessary

standards. This is often thought of as 'criterion referencing',

although paragraphs 63 to 65 describe some of the difficulties of

applying pure criterion referencing to A level examinations and

assessment. I have encountered very little systematic support for a

return to grading in which fixed quotas of grades would be awarded to

students according to rank order rather than performance against a

fixed standard of achievement (broadly, 'norm referencing'). This

conclusion is not incompatible with their traditional role as a

filter for entry into higher education or employment. But it

nonetheless has significant implications for the design of the A

level framework.

20. Even in relation to university entry, A levels must constitute

more than a simple means of ranking students to help HE admissions

authorities choose between applicants. They must also contain an

assurance that students have acquired the specific skills and

knowledge that they need in order to embark on their chosen degree

course.

21. This role of accrediting specific levels of knowledge and skills

is even more important if A levels are to act as a consistent and

reliable indicator of students' achievements from one cohort to the

next. In this wider context it is the accreditation of known

standards of achievement which is important rather than the ranking

of individual students against their peers. This reinforces the

weight of argument in favour of judging students' performance against

a fixed standard. If 100% of students reach the standard then 100%

should pass, and that outcome should not be seen as a 'lowering of

standards'.

22. The design implications arising from this have been significant

in the recent developments of the A level system. Confident matching

of students' performance to a fixed standard requires not just

accurate and consistent marking but also the ability to define and

maintain standards of grading from year to year and between subjects,

and the means to judge the performance of individual candidates

against that standard. Standards over time

23. Ever since the 1996 Standards Over Time study undertaken by QCA's

predecessor body, SCAA, and OFSTED, QCA has sought input from

independent experts to its work and to verify its conclusions on the

maintenance of standards over time. Such work has in the past been

hampered by the absence of definitive evidence about students'

performance over the past three decades. For more recent years, the

systematic archiving of examination material will ensure that such

evidence is available to inform future work.

24. Earlier this year, an independent report by a panel of three

international experts, Chaired by Professor Eva Baker, commended

QCA's work in monitoring A level standards, whilst also highlighting

the difficulty of answering with certainty the question of whether

standards have been maintained.

25. Whilst changes to qualification structure, syllabus content and

assessment methods can make the task more difficult, I believe it is

important that QCA should maintain efforts to investigate

comparability from year to year and across the A level system, and

should do so in a way that commands credibility with education

professionals and the public. In doing so, QCA should build both on

its existing rolling programme of five-yearly subject reviews and its

use of external expertise and verification. The annual debate about A

level standards undermines and devalues demonstrable improvements in

teaching and learning over time, and the hard work and commitment of

students. Action is needed to ensure that the evidence is available

to rebut ill-founded claims both about the value of qualifications

and about year-on-year comparisons. But, equally, it is vital that

where legitimate concerns exist about standards, these must be looked

into and appropriate remedial action identified and taken. Setting

and Maintaining the standard

26. The crucial first step is to establish a common understanding of

what the standards should be. In my initial report, I highlighted the

absence of a clear understanding of the standards or levels of demand

for either AS or A2 assessment which lay at the heart of the concerns

this year about the grading of A level performance.

27. QCA, the other regulators and the awarding bodies have been

working to rectify this deficiency. By the January 2003 examinations

there will be in place:

- a description in broad terms of the separate AS and A2 standards

and levels of demand, and their relationship with the overall

standard of the former A level system. This makes it clear that the

level of AS is equivalent to the first half of a two year traditional

A level course, and that A2 covers the more demanding material found

in the second half;

- descriptions of the performance that merits an E grade and A grade

for both AS and A2 in the 27 A level subjects for which QCA has

established subject criteria;

- exemplar material illustrating, through students' actual scripts

and other material, the level of performance at A and E grades for

both AS and A2 in the twelve most popular A level subjects, covering

some 90% of the January entries.

28. By June 2003 the exemplar material will be extended to cover all

A level subjects. In the longer term the material will be

strengthened, refined and reviewed in the light of further

experience.

29. I now recommend that QCA should establish an independent

committee whose role would be to review and, if necessary, advise QCA

publicly on whether or not standards are being maintained - advising

on a limited number of subjects each year - using all the available

evidence including subject syllabuses, students' work, mark schemes

and question papers. The group should also be able to review and

verify other aspects of QCA's regulatory work, as requested by the

QCA board. This committee will help provide reassurance that

standards are being kept continuously under review and that, where

necessary, action will be identified and taken to safeguard standards

over time.

30. I am satisfied that these measures will provide a secure

foundation for the grading of A level entries; help ensure that

individual candidates' performance can be consistently and reliably

graded; and provide credible ongoing evidence that standards are

being maintained.

31. In the longer term, however, I believe that there are some

further steps which can reduce the complexity of the AS/A2 system and

provide a more readily understood basis for grading. Relationship

between AS and A2

32. The annex to this report shows the key steps in the examining and

grading process. Students' scripts and other work for AS and A2 units

are marked and graded separately and then combined to determine the

candidate's overall A level grade. The process to arrive at the final

grade is complex.

33. First, a sample of students' work and other information is used

to determine the mark which should define each grade boundary for

each unit. Those grade boundaries are then translated into fixed

points on a uniform mark scale (UMS). The range covered by this scale

varies between units. For a 0-100 scale, the ranges of scores for

each grade are: E=40-49, D=50-59, C=60-69, B=70-79, A=80-100. The

scores for each grade boundary on scales which are less or greater

than 100 will be proportionately lower or higher. From that,

individual students' marks for each unit are translated into a UMS

score for the unit. Marks which lie between two grade boundaries on

the raw mark scale will translate into intermediate points between

the same two boundaries on the UMS scale. The UMS scores for

individual units are then added together to give an overall UMS score

for the subject as a whole. The maximum total UMS score is always

600. The final A level grade is awarded based on where the overall

score lies in relation to fixed points on this 600 point scale (ie

E=240 to 299, D=300 to 359).

34. This summer's experience, reinforced by much of the evidence

submitted to my inquiry, demonstrates clearly the lack of clarity

among teachers, students and parents about the way in which students'

marks for individual units are converted into UMS scores for each

unit and overall grades. In particular, there is no clear

understanding of the relationship between marks awarded for

individual units by markers and the UMS scores on which the final

grade is based.

35. A further layer of complexity is added by the range of choice

with which students are faced about the options for re-sitting units

and 'cashing-in' their AS units in exchange for an AS award part-way

through their A level course. Broadly, after a student has completed

the three AS units they may:

- exchange their unit UMS scores for an AS qualification in the

relevant subject - an option known as 'cashing-in';

- elect to move straight on to A2 units without cashing in their AS

units; or

- re-sit some or all of the AS units in an attempt to improve either

their AS grade before cashing-in or their overall A level grade.

36. Even after AS units have been cashed-in, the student may re-sit

some or all of the same AS units in order to improve their overall A

level grade.

37. These arrangements give students a great deal of flexibility. But

the awarding bodies' current guidance on the 'entry, aggregation and

certification' of A levels acknowledges that: 'For many centres and

students the decision whether or not to cash in can be a difficult

one.' That appraisal is certainly borne out by the remainder of the

guidance, which, for instance, offers as a worked example a situation

in which for a single subject the student sits 4 units (ie including

1 re-sit) to achieve their 3 unit AS award and a total of 11

(including 5 re-sits) to complete their full 6 unit A level course.

One of the relevant AS units is repeated 3 times.

38. There are other potential complications. For example:

- if an AS award is cashed in at the end of the first year, the AS

result must be declared on any UCAS application form. If they are not

cashed in, the student may report the unit results to UCAS but does

not have to;

- because a student may re-sit AS units after cashing-in, the AS

unit results which comprise the original AS award may be different

from the AS unit results which contribute to the overall A level

grade.

39. None of this implies that the system is technically flawed. I am

satisfied that operated in this way, and applying the correct AS and

A2 standards, assessment will be robust, and that students will

receive grades that are technically accurate and which reflect their

overall performance. Action is needed over time to simplify the

awarding arrangements and reduce the complexity and lack of

transparency which affects perceptions about the system's reliability

and fairness.

40. This might partly be rectified by more intensive efforts to

provide accessible information about the grading process and the

options open to students. But as it stands the system is unlikely

ever to attract the levels of public and professional understanding

which would prevent recurring confusion and dissatisfaction. In my

view, the complexity of the current arrangements will continue to

undermine the extent to which A level results are understood and

trusted, even though the actual outcomes accurately reflect students'

achievement.

41. Nor am I persuaded that the existing processes for sitting and

re-sitting units and for combining AS and A2 scores are necessary to

the effective accreditation of students' achievement. The rationale

initially has been to provide a single overall grade which is

directly comparable to those awarded for the former A level. As the

new system becomes embedded and stands in its own right, the need to

relate all current achievement to the previous A level outcomes will

diminish. In effect the continuity of A level standards would be

carried forward in the separate AS and A2 standards and levels of

demand and the design of the qualifications rather than in direct

comparability between students' past and present results.

42. I therefore recommend that in time, and as part of the 14-19

policy development process, the AS/A2 system should be simplified as

follows:

- AS and A2 should be assessed,graded and awarded separately. They

should become distinct and separate qualifications;

- as now, AS should cover the first half of a two year A level

programme; and A2 should cover the more demanding second half. No

element of assessment of either should be at a level which would have

been outside the scope of a traditional A level course;

- before any new arrangements are introduced, the separate standards

for AS and A2 should be securely established and embedded within the

existing A level framework; and time must also be allowed for the

necessary design, development and testing of the new arrangements,

and of any further changes along the lines illustrated in paragraph

46, below. 43. As an interim measure, in the shorter term, I

recommend that the QCA and Ministers should look urgently at the

scope for simplifying the rules governing re-sits and the cashing-in

of AS units, with a view to introducing changes for students

embarking upon AS in September 2003 for examinations in 2004.

44. Separation of AS and A2 would build on the existing system.

Examinations could continue to be set, taught, assessed and graded as

now, using the established AS and A2 standards. It would also retain

the consistency in the levels of demand inherited from the 'legacy' A

level which the current AS/A2 design is intended to secure, and it

would retain the possibility of students receiving a qualification

for the subjects which they drop after AS. At the same time, it would

remove the complexities and scope for confusion within the process

for re-sitting units, cashing-in, and combining AS and A2 unit

results to give a single overall grade.

45. There are nonetheless some important policy and design issues

which would need to be resolved, including:

- the extent to which completion of a relevant AS course should be a

precondition of entry onto A2 courses;

- the extent to which A2 should include 'synoptic' assessment -

whether it should not only assess the material covered in the A2

units but also ensure that students can relate what they have learned

across both the AS and A2;

- retaining flexibility for adults and other non-traditional

learners;

- ensuring consistency with the structure and evolution of

vocational A levels;

- maintaining the value and currency of the AS award in its own

right.

46. Separating the two awards in this way would also provide an

opportunity to review other aspects of the syllabus and assessment

methods, for instance to reflect the nature of learning in specific

subjects; to reduce burdens on the examination system by changing the

weight of, and balance between, coursework and examinations; or to

change the ways in which synoptic assessment is integrated into the

course. The factors that would influence final decisions on these

issues go well beyond the boundaries of my remit. I make no specific

recommendations on them except that in following up the

recommendation for separate AS and A2 awards, consideration should be

given to these issues.

47. There is a range of potentially significant changes arising from

this work. Reform should not be rushed. Implementation should take

account of emerging timetables for wider 14-19 policy development.

Equally crucially, sufficient time must be allowed for the necessary

design, development and testing, and for schools and colleges to

familiarise themselves with any changes. In my view it should be at

least five years before significant changes are fully introduced,

although the preparatory work could begin almost immediately.

Relevance for Northern Ireland and Wales

48. Maintenance of the national qualifications framework covering all

three countries requires application of the A level standard and

design changes on a three country basis. The recommendations in this

chapter should therefore be taken forward jointly.

Reducing Burdens on the System

49. The evidence presented to me suggests that the public

examinations system is operating at, or even beyond, capacity. Last

year, the number of examination scripts and coursework assignments

produced at GCSE, AS and A level totalled 24 million. In GCE

advanced, the number of entries across the UK has increased 158% in

the last 20 years; from 656,838 A level entries in 1982 to 1,696,784

A and AS entries in 2002. Full A level (ie not AS) accounted for

about 700,000 of the 2002 entries. This increase is a result of the

higher numbers staying on post-16, changes to the design of GCE

advanced qualifications and of the popularity of AS. Candidates are

now examined for AS at the end of Y12, often in more subjects than

they would traditionally have taken at A level, and assessment across

Y12 and Y13 is done on a modular, rather than a linear basis. There

has been a corresponding increase in the complexity of the system and

in the volume of decision-making at the awarding stage.

50. GCSEs and national tests also contribute very significantly to

strains on students and on the examination system. In 2002 there were

around 6 million GCSE entries and nearly 2 million children sat Key

Stage tests, including over 600,000 at Key Stages 3. Students are

externally assessed in at least four out of the five years from age

13 (ie year 9) to age 18 - to the detriment of some wider educational

and developmental activity. The need to deliver the rising quantity

of examination-related tasks stretches awarding bodies, schools,

colleges and teachers

51. Some of the recommendations in this report will simplify the

system, improve its efficiency and potentially reduce the burdens on

schools, students and awarding bodies. Action to improve the supply

of examiners will also help to increase the capacity of the system to

deliver. However, I am convinced that the issue of burdens,

particularly that which arises from external, examination-type,

assessment can only be addressed effectively by looking more widely

than A levels. Addressing this issue in the long-term implies a

review of assessment at all levels,with a view to reducing the

number of external examinations and tests pupils must take. That

breadth of activity is beyond my remit.

Options relevant to A level include:

- reducing the number of assessment units; and/or

- making greater use of internal assessment and coursework.

52. Neither of these options is necessarily incompatible with the

AS/A2 model recommended in paragraph 42. There is no prima facie

reason why a modular curriculum requires separate assessment of each

module and it may be that three curricular modules could be assessed

in two assessment units. The burden of external examination could

also be reduced by relying more on internal assessment and

coursework, providing care is exercised to ensure that the total

burden of internal assessment does not become excessive. There are

issues that would first need resolving about coursework standards and

how it is assessed, and about the training and professionalisation of

examiners, some of which are explored in paragraphs 120 to 136. It

has not, however, been possible in the time available for me to come

to any firm conclusions about the amount and type of assessment

appropriate to the curriculum and the maintenance of standards.

53. I recommend that, as part of the development of the 14-19

reforms, the Government and its partners examine carefully the scope

for reducing the total burden of external assessment derived from

GCSEs and A levels.

Relationship between the timetables for A level examinations and

university admissions

54. The timetable for the marking and grading of A levels has been

driven largely by the need to issue results in time to ensure that

students may confirm their university places before the start of the

academic year. Although there have been huge changes in the volume of

A level entries, the time available for marking, grading and issuing

results has hardly changed in 50 years.

55. The availability of technology has removed much of the burden of

administration and data processing, but time pressure remains a

significant factor and can still have a notable impact on the quality

of the marking and awarding processes. For example, post examination

meetings to standardise marking and determine grade boundaries

sometimes have to take place with only a limited amount students'

work available on which to make judgements.

56. The timing issue has also adversely affected some of my own

recommendations. In particular, it is my view that considerable

benefits could be gained in the judging and maintaining of standards

from the presence at awarding meetings of a Chair of Examiners from

another board. As set out in paragraph 70, however, given current

time constraints and capacity issues I do not believe that this is

possible in the short-term.

57. It would in my view be beneficial to create additional space in

the academic calendar for awarding to reinforce arrangements for

guaranteeing standards and to enhance public confidence that

important procedures are not being rushed. This needs, however, to be

balanced against other demands, not least the time available for

teaching. Consideration of these issues is well beyond the scope of

my remit, though I do believe there are steps that can be taken which

will bring about immediate improvements.

58. As an interim measure, I believe that delaying the publication of

AS and A level results by 1-2 weeks is possible if consequential

adjustments are made to the timing of GCSE results - perhaps with

GCSE results coming out later in the same week as those for A levels.

The impact on university admissions procedures, including clearing,

should be manageable and offset by the potential benefits. I

recommend that this is done to take effect as soon as can

realistically be managed without risk to the effective administration

of the systems for examinations and for entry to higher education.

59. For the longer term, I recommend that consideration be given to

structural reforms to the timetables for A levels and university

admissions to extend the time available for examining and awarding,

without reducing the teaching time for A levels. This should, in my

opinion, include looking at the possibility of reform of the school

year and the university admissions timetable with a view to moving to

a post-qualifications admission (PQA) system.

60. Though there are practical issues that need to be addressed, I

have been persuaded in the course of my inquiry that PQA has much to

commend it. The movement to PQA would increase accuracy in the

admissions process as well as removing much of the burden of

clearing. Students would be relieved of the stress of applying for

universities during year 13 and have more time to focus on gaining

good results. I think therefore it is worth considering how a PQA

system might be established. This might include looking at:

- delaying the start of the first year of university. Moving the

start of the first year of undergraduate study back to, say, the end

of October might provide the additional space necessary for robust

quality assurance and PQA. The impact on first year teaching might,

if universities wished, be offset by a narrower break between the

first and second years. This is a matter for universities to resolve,

and they may want to consider how it would relate to other changes,

such as the trend in many universities towards the adoption of a

semester system, and the balance between the potential costs and the

benefits of PQA in simplifying university admissions.

I urge UUK to continue their work in this area;

- the LGA recommendation for a six term year. If this model were

adopted on a national basis, examinations could perhaps be taken in

term five, giving awarding bodies substantially more time to carry

out the awarding process. The knock-on effect for earlier years would

need careful evaluation, as an earlier start to the school year would

be necessary to ensure no reduction in the teaching time that

teachers and students already feel is too compressed, especially for

AS. The impact on the availability of markers and the present

twice-yearly examination sessions would also need to assessed.

Relevance to Northern Ireland and Wales

61. Action to tackle burdens of assessment may be a matter of

national policy and need not necessarily be common to all three

countries. Nevertheless, any reforms which affect the design of

national qualifications must be developed on a three country basis.

Equally, reform of the A level and school and university timetables

would need to apply across all three countries. The impact on

Scotland, especially of changes to the university year, would require

appropriate involvement by Scottish institutions in consideration of

any changes.

PROCESSES FOR GRADING STUDENTS' PERFORMANCE

62. The basis for any system of assessment intended to judge

students' achievement using a fixed standard should in principle lie

in a comparison of individual students' work against that standard.

Many of the perceptions this year about the quality of the A level

grading processes have been based on the perception that there can

and should be a direct and fixed relationship between the marks for

individual students' work and the grades they are awarded.

63. In practice, the system cannot be this simple. Neither the

setting of standards, nor the judgement of individuals' work can be

sufficiently precise to produce consistent and accurate grades

directly from the marking of work. Most notably, the assessment

process involves subjective judgements of individual examiners in the

setting, marking and grading of students' work. Without intervention,

variations in those judgements would lead to inconsistency of grading

within and between subjects.

64. The awarding process needs to compensate for these

inconsistencies. The awarding bodies have therefore developed

sophisticated techniques for the use of historical and other

comparative data to cross-check and modify judgements based on

students' work. The purpose is not to substitute grading against a

fixed standard with some form of norm-referencing but to produce

consistency and accuracy of grading over time and between subjects:

if 100% of students meet the standard then 100% should pass.

Effective use of statistical information will provide results which

are closer to those that would result from effective

criterion-referencing than could be achieved by considering

individual students' work in isolation from statistical and other

comparative information.

65. I must stress that the use of statistical and other information

in this way is a wholly legitimate and necessary part of the process

of maintaining standards. But it is clear that part of the problem in

2002 was the perception that too much weight was accorded to

statistical information in the grading process for some subjects, and

particularly in modifications made in the latter stages of the

awarding process to the grade boundaries recommended to awarding body

Accountable Officers by their Chairs of Examiners.

66. It is in my view essential to restore confidence in the grading

process as a whole, and particularly the legitimate use of

statistical information.

67. The regrading process following my initial report reviewed late

changes by the Accountable Officers to mark grade boundaries which

fell outside the scale which had been normal for such changes in the

past. This review involved independent observers nominated by the

organisations representing teachers, schools and colleges. I believe

that it has helped restore confidence in the 2002 A level results.

68. But longer term action is needed to ensure that similar concerns

do not arise in 2003 and subsequently. QCA, together with the other

regulatory bodies, is currently revising the Code of Practice

governing A level grading in time for the January 2003 examinations

to: give greater emphasis to examiners' judgements about the quality

of candidates' work and to clarify the role for statistical evidence

in awarding; to ensure consultation between Accountable Officers and

Chairs of Examiners before grade boundaries are finalised; and to

ensure that QCA can monitor grade boundary decisions made by

Accountable Officers. These are important steps towards improving the

transparency and consistency of the awarding process.

69. In addition, I now recommend that:

- as a short term measure, to help restore confidence in the system,

for the January and summer 2003 examinations only, the QCA, in

consultation with the Secretary of State, should appoint an

appropriately qualified individual to observe and report publicly to

the QCA board on the awarding process;

- permanent procedures should be put in place for QCA to scrutinise

all stages of the awarding process, including Accountable Officers'

and other consideration of the grade boundaries recommended by Chairs

of Examiners;

- as with the re-grading exercise, nominated representatives of

teachers, schools and colleges should be invited, and given the

necessary training, to observe and report each year in confidence to

QCA on the application of the processes governing the setting of mark

grade boundaries;

- QCA should take steps to ensure that Chairs of Examiners are, and

are seen to be, impartial in giving their professional opinion during

the awarding process.

70. I have considered very carefully recommending that the awarding

processes of each of the awarding bodies should involve examiners

from another awarding body. I believe that this cross-referencing of

standards strengthened the re-grading in October. In the longer term,

too, it would help reinforce the mechanisms for maintaining

consistent standards. However, on the basis of the evidence presented

to me, I am forced to conclude that it is not a practical proposition

whilst the existing time and resource pressures persist during the

period when A levels are marked and graded. However, I believe this

matter should be kept under review.

Other issues relating to this year's concerns

71. In the context of phase 2 of my inquiry, I have looked carefully

at the circumstances that gave rise to some of the individual cases

which have been highlighted as a cause for concern this year, such as

the appearance of results for individual units which appeared to be

out of line with the relevant students' pattern of results in other

units ('the AAU problem').

72. Most of the concerns raised by those cases related to the overall

standard setting and grading processes, which I have addressed in my

two reports. However, there are other cases which highlight issues

related to the design and quality assurance of individual subject

syllabuses and marking schemes; and the consistency and transparency

of marking.

73. For instance in one coursework unit, the design of the syllabus

and marking scheme made it inevitable that students' marks would be

compressed into a very narrow range. The result was 9 marks (from a

total of 50 available) separating the A and U grades. Where the grade

boundaries are this close together it is difficult for the grading

process to differentiate effectively between the performance of

individual candidates. QCA and the awarding bodies are taking the

necessary steps to correct such flaws for examinations in 2003.

74. Some continuing concerns have been expressed to me about the

consistency of marking. Many of the case histories submitted to my

inquiry, including some that have fuelled concern about 'rogue'

grades, appear to have affected only individual candidates or groups

of students within a school. I am persuaded that such results cannot

be the result of systematic manipulation of the grading process since

that would downgrade all of the students taking the relevant units

and in extreme cases would have led to up to 80% of them receiving a

U grade. I have no evidence that this happened in any unit. I can

therefore conclude that such cases are either a proper reflection of

lower-than-expected performance by the candidate or inconsistent

marking and/or coursework moderation. I have no clear evidence on

which to judge a conclusion about the balance between these two

explanations.

75. I am concerned also about the quality of communication and

feedback to schools and colleges. For example, a number of case

histories were submitted to me in which specific concerns about

students' grades were based on significant misunderstanding of the A

level marking and awarding process (e.g. about the translation from

raw marks to UMS scores). Despite correspondence between the relevant

school or college and the awarding body, these misunderstandings were

often not corrected or explained, leaving teachers and students

uncertain about whether their work has been treated fairly. It is

essential that schools, colleges and students have access to high

quality feedback and clear explanations of the processes which have

applied to their entries.

76. I am clear that all these problems are separate from the

allegations which I investigated in my first report that the grading

process was systematically manipulated. Similar issues can, and have,

arisen in previous years. They have nevertheless added weight to the

current concerns about the credibility of the system.

77. I do not believe that changes are needed at the level of

system-wide procedures and practices to prevent such problems

occurring again: they can and should be detected and tackled on a

case-by-case basis as they arise. I am content that the existing

regulatory framework, if properly applied, offers the correct

mechanism to achieve this. Nevertheless, it is necessary for the

regulators and the awarding bodies to keep their quality assurance

and communication procedures under review in the light of experience.

This year's events offer a good deal of experience on which to draw.

Relevance for Northern Ireland and Wales

78. The Code of Practice and other key regulatory processes are

common to all three countries. Changes to this framework should

therefore be taken forward jointly. Any other arrangements for

scrutiny of the awarding process are a matter for individual

countries to consider in the light of their own circumstances and

current practices.

QCA AND ITS RELATIONSHIPS WITH MINISTERS AND THE AWARDING BODIES

79. The QCA, in close collaboration with the other regulators - ACCAC

in Wales and CCEA in Northern Ireland - plays a pivotal role in the

development and regulation of national qualifications. It:

- advises the Secretary of State for Education and Skills on the

development and implementation of qualification policy;

- works with the awarding bodies to implement qualifications policy;

- regulates the operation of the qualifications system, including

accrediting the awarding bodies' A level, GCSE and GNVQ courses and

monitoring the standards of marking and grading of students' work.

80. The Quinquennial Review of QCA earlier this year reported in

detail on QCA's work. I have not sought to reproduce the scope of

this review. Moreover, I have not sought to investigate in detail the

internal management and operation of QCA. I have every confidence

that the new leadership at QCA will take whatever action is needed to

ensure that the Authority can perform its functions effectively and

credibly. In particular, they will need to ensure that the regulatory

role is clearly delineated and that it functions separately from

QCA's wider business.

81. In this inquiry I have focused on concerns highlighted by recent

events about the role of QCA and its relationship with Ministers and

the awarding bodies. This chapter examines those concerns and sets

out my recommendations relating to the roles of, and relationships

between, these key players in England.

Relationship between Ministers and the QCA

Developing and implementing qualifications policy

82. It is self-evident that Ministers should be responsible for the

key decisions which shape the qualifications system andits

relationship with wider education and skills policy. In exercising

that responsibility they should be able to look to the QCA for expert

advice. I fully accept that the QCA should have a clear duty to

respond to requests from Ministers for advice. In this context I note

also that, whilst QCA is the Secretary of State's principal advisory

body on qualifications, others have an interest in the development of

qualifications policy and should be able to make their views known

directly to Ministers if they so wish.

83. The normal mechanism for seeking QCA advice is a remit from the

Secretary of State or other education Minister to the Chairman of

QCA, asking for advice on specific aspects of qualifications policy,

and setting out any parameters - such as timescale or wider policy

constraints - which the QCA should take into account in preparing its

advice. QCA officers will normally then prepare a response for

approval by the QCA board, consulting other relevant interests, such

as the awarding bodies or teachers. It is usual practice for QCA's

advice to be published. There is no fixed timing for such

publication, but the pattern is often for publication to take place

alongside announcement of any decisions by Ministers resulting from

the advice which QCA has offered. In general, QCA will have a

responsibility to manage the implementation of those decisions in

collaboration with the other regulators and the awarding bodies.

Regulating qualifications

84. The QCA's regulatory function is further removed from the

Secretary of State's direct responsibilities than its role in

advising Ministers on qualifications policy. In its regulatory role,

QCA's responsibilities include:

- putting in place the framework of rules and processes to ensure

that standards are maintained - for instance the Code of Practice

governing A levels, GCSEs and GNVQs;

- monitoring the application of this framework by the awarding

bodies;

- approving individual qualifications, including all A level

courses, proposed by awarding bodies;

- reviewing, and taking whatever steps are necessary to maintain,

standards over time between individual qualifications and across

awarding bodies;

- monitoring and taking steps to maintain the overall health and

effectiveness of the qualifications system.

85. Ongoing responsibility for maintaining and operating the

regulatory system rests with QCA and the other regulatory bodies.

>From my inquiry it is clear that there is a wide consensus that

Ministers have a legitimate interest in assuring themselves through

contact with the QCA that these functions are being carried out

effectively. Nevertheless, they should not intervene in the ongoing

operation of the regulatory framework nor in the details of the QCA's

relationship with the awarding bodies.

Does the relationship between Ministers and the QCA work?

86. I have received a range of views on the relationship between

Ministers and the QCA. Many of these turn on whether QCA is, and is

seen to be, sufficiently independent of Ministers. In particular,

there is concern:

- that QCA's advice may be informally constrained in order to

produce an outcome that is acceptable to Ministers;

- that lines of accountability are unclear. Even among some of those

most closely involved in the delivery of qualifications, the

decision-making processes can be unclear, and they are unsure where

responsibility for some key decisions lies;

- that the DfES may become involved too closely in monitoring and

influencing implementation and regulation activity which should

properly be the responsibility of QCA.

87. The perception that the relationship might function in these ways

damages the reputations both of Ministers and of QCA and affects the

credibility of the decisions and actions which both take legitimately

to maintain and develop the qualifications system.

QCA and the awarding bodies

88. Some similar issues arise in perceptions of the QCA's

relationship with the awarding bodies. In particular there is a

significant lack of clarity about the boundaries between QCA's role

in overseeing and ensuring the health of the qualifications system

and the awarding bodies' responsibility for operational management of

that system and the qualifications outcomes to which it leads. In my

earlier report, I highlighted a specific difference in perceptions

between the Authority and the awarding bodies about the way in which

QCA was carrying out its functions as regulator. That difference was

indicative, at least in part, of a blurring of the boundaries between

the respective roles of regulator and awarding bodies.

89. It is essential that the QCA acts as a regulator, and is rigorous

and consistent in its actions. But it should not become involved in

managing the detail of the awarding bodies' responsibilities in

relation to the setting, marking and grading of A levels and other

qualifications.

90. There is one aspect of the relationship between QCA and the

awarding bodies which requires particular attention. QCA acts, in

effect, as the awarding body for key skills qualifications and the

national curriculum Key Stage tests in schools. Both of these

activities fall outside my remit. Nonetheless, they involve a

different relationship between QCA and the awarding bodies, because

the delivery of significant parts of the key skills and key stage

assessment process is carried out by the awarding bodies under

contract to the QCA. The Quinquennial Review of the QCA published

earlier this year expresses concern about this additional dimension

to the relationship. Those concerns should be given significant

weight. It appears to me that there is potentially a conflict of

interest between the contractual and the regulatory relationships;

QCA's management of contracts with the awarding bodies to deliver the

tests is not compatible with the Authority's role in regulating those

same awarding bodies' own qualifications.

Conclusions

91. Some have argued the case forgiving QCA a greater measure of

statutory independence. The QCA's status as a non-departmental public

body gives greater formal powers for the Secretary of State to steer

its work than would be the case if, for instance, the Authority were

to become a non-ministerial government department like OFSTED.

However, whilst a change of status might reduce some of the formal

powers which Ministers have to influence and direct QCA's operations,

I am not persuaded that it would address effectively the concerns

which have been highlighted this year; and there is a risk that the

necessary legislation and disruption arising from major institutional

change would be a distraction from the immediate priority of managing

the delivery and standards of A levels.

92. Nor am I persuaded that the relationship of necessarily close

co-operation between the DfES and QCA across a wide range of

curriculum and qualifications matters would be enhanced by a greater

degree of separation.

93. I have also considered whether the QCA's regulatory functions

should be separated from its other roles and accorded a separate

independent status. I agree that there is a clear need to ensure that

the regulatory function is effectively ring-fenced from QCA's wider

activities. But the evidence presented to me suggests that the

benefits of separating QCA's regulatory role would be largely

presentational, and would not outweigh the risks inherent in

organisational change, and in dividing the qualifications expertise

currently available within QCA.

94. On this basis, I am persuaded that a change in the status of QCA

or some if its component parts is not justified. There is nonetheless

an early need to tackle the perceptions which appear to have grown up

about the ways in which the DfES (Ministers and officials), QCA and

the awarding bodies actually behave in practice. The key to this is

to inject a greater degree of clarity and transparency into the roles

and responsibilities of each, and into the ways in which they work

together. I note that the Quinquennial Review recommended a

memorandum of understanding setting out the guiding principles which

should underpin the relationship. I support that approach.

95. I recommend that the DfES, QCA and the awarding bodies jointly

draw up an agreement which describes their roles and relationship and

establishes clear lines of responsibility and accountability. This

agreement should also, wherever appropriate, set out ways of working

to ensure that the relationship is transparent. In particular, the

agreement should ensure that:

- requests to QCA from Ministers for fresh advice or new work should

set out the matters which the advice should cover or the action

required, who will have responsibility for subsequent decisions and

follow up action, and which organisations or individuals should

contribute to that work, including where appropriate DfES Ministers

or officials;

- such remits and QCA advice to Ministers should be published at or

near the time when they are issued - unless there are strong reasons

why they need to remain confidential;

- implementation timetables for new policies and initiatives are

established and published as early as possible in the development

process, and fully reflect the need, amply illustrated by this year's

experience, to allow realistic time for development and testing. In

my view, major changes to the qualifications system should have a

lead time of at least five years;

- as clear a distinction as possible is maintained between (a)

QCA's responsibility for monitoring and coordinating delivery, and

for overseeing and guaranteeing standards and the general health of

the qualifications system and (b) the awarding bodies' individual

responsibility for managing effective delivery of their own

qualifications to students, schools and colleges;

- the awarding bodies and others with an interest can make

representations directly to DfES on matters of qualifications policy;

- that application of the agreement is monitored, and that mechanisms

exist for resolving inconsistencies or uncertainty about its

application in specific cases. 96. I share the misgivings expressed

in the Quinquennial Review about QCA's responsibilities for delivery

of the key and basic skills qualifications and Key Stage tests, and

recommend that these should be separated from QCA's other

responsibilities and placed in the hands of a separate body. QCA

should retain its regulatory oversight of these activities.

97. In making these recommendations, I am conscious of the extent to

which these bodies work together formally and informally across a

wide range of policy and delivery issues. This is a proper dimension

to their relationship and plays a crucial part in helping to foster

effective collaboration. A clearer delineation of roles and

responsibilities should take care not to cut across the day-to-day

working partnerships which are necessary to the smooth operation of

the qualifications system.

Relevance for Northern Ireland and Wales

98. I have focused on the QCA and relationships within the English

qualifications system. These recommendations apply only in that

context. The institutions and their roles within the Wales and

Northern Ireland systems are different. It is necessary that there

should be a common understanding, and thus a joint approach across

all three countries, about the regulators' roles and responsibilities

in relation to those of the awarding bodies. My recommendations

should be taken forward on that basis.

Awarding Bodies and the Administration of the Examination System

99. In 1997, the Government sought views on arrangements to underpin

the new qualifications framework and to ensure that high and

consistent standards were maintained across its qualifications. As

part of the Guaranteeing Standards consultation, the case for

rationalising the number of awarding bodies in England was examined

with a view to ensuring comparability across qualifications.

Rationalisation was intended to reduce variations between syllabuses,

assessment, administration and customer service.

100. The consultation revealed broad support for the idea of

rationalising the number of awarding bodies. There was also agreement

that the development of qualifications provision should be

underpinned by the principles of quality of content and assessment;

consistent standards of awards; choice; cost effectiveness and

accountability; and quality of service. Among these, choice and

consistent standards of award were thought to be of prime importance.

The decision was taken that three awarding bodies in England was an

appropriate number to secure these and to tackle inconsistencies.

Having three awarding bodies also limits the risk of system failure

present where there is only one.

101. The evidence presented to me suggests that these arguments are

still valid today. There is a wide range of support, including from

schools and colleges, for the choice of syllabuses and potential for

innovation provided by the current system. For instance, schools are

able to select syllabuses which meet their preferences from the wide

range of periods of history offered across the awarding bodies.

Equally, alternative approaches to science and mathematics such as

Nuffield, Salters, and MEI syllabuses offer a variety of approaches

to the same subjects. Continued vigilance is needed by the regulators

and the awarding bodies to ensure that the variety of assessment

objectives is controlled and that a diversity of content in the same

subjects from each of the awarding bodies does not become a diversity

of standards. Overall, however, I see no strong reason to challenge

present arrangements.

102. I have, on the other hand, been persuaded that differences

between awarding bodies in their administrative practices contribute

significantly to the administrative complexity and burden on schools

and colleges, without offering significant counterbalancing benefits.

103. Centres' concerns about awarding bodies' different requirements

are two-fold:

- Examination fee structures

- Administrative procedures.

Fee structures

104. Standard unit entry fees currently range from #9.20 to #10,

while late entries are charged at the standard rate plus #4.60, #9.40

or #10, depending on the board. For very late entries the additional

charge ranges from #9.20 to #20, meaning that costs range from #18.40

to #30. There is in schools' and colleges' view no obvious

correlation between the charges made and the service they receive.

Competition regulation outlaws the possibility of the awarding bodies

collaborating to standardise the level of fees they charge. The

prices charged are a matter for the awarding bodies, although I note

that the QCA has some statutory powers to cap entry fee levels. More

significantly, differences in the types of service for which the

awarding bodies charge and the pricing structure for those services

make direct price comparisons very difficult, and complicate the

administration for schools and colleges. Consideration should be

given to ensuring greater consistency and transparency in pricing

structures.

Administration

105. Progress has been made by the awarding bodies since the

Guaranteeing Standards consultation in improving the commonality of

their systems, but significant differences remain, particularly in

the administrative process through which they communicate with

centres. These differences often appear to have no clear

justification and seem to be the result of custom and practice within

the individual awarding bodies rather than of overriding operational

need. I offer some examples of these below.

Entry procedures

106. The system for administering entries varies between boards.

Advances have been made in recent years in standardising the

terminology used, but there are still considerable differences in the

codes used to identify candidates, series and syllabuses.

107. Not all of these differences are easily diminished, nor are

they necessarily issues that the awarding bodies themselves can

resolve. Some of the problems around the use of unique identification

numbers for candidates, for example, arise from provision in the 1998

Data Protection Act, which also has a major bearing on the way

educational data can be captured, stored, reported and exchanged.

Efforts must continue to resolve these issues, though the process of

doing so may not be quick.

108. Short-term progress does, however, seem possible in areas like

the awarding bodies codes for units and examination sessions. It is

unclear, for instance, why the GCE January 2003 examinations are

given the code E IE03 by AQA, 1C by Edexcel and 1B by OCR. This

complicates and extends the entry administration process. Similar

inconsistencies are evident throughout, including in the format and

provision of envelopes and labels and in the use of forms.

109. Over the longer term, important opportunities to simplify

administration are presented by ICT. These are explored in paragraphs

145 to 164. The need is not to impose uniform systems on awarding

bodies, but to simplify their interface with users. Standardising how

centres input and amend their entry data would bring significant

efficiency gains.

Issuing results

110. At present awarding bodies deliver results to centres in

different formats. For example, one supplies results in alphabetical

order, and another in candidate number order. Centres must then spend

time cross-referencing information in disparate formats to collate

results for candidates. Certificates for students also differ in the

supplementary information they show. One awarding body, for instance,

issues grades at unit level, whilst others do not.

Appeals procedures

111. Although the processes for making inquiries upon results for

individuals or groups of candidates are common between A level

awarding bodies, the awarding bodies' own appeals processes, which

may be followed where the school or collegeremains dissatisfied,

appear to differ significantly in the demands they make on schools

and colleges.

112. I recommend that the awarding bodies work together and with QCA

to undertake a systematic review of such differences in

administration and take urgent action to eliminate unjustified

differences.

OVERNMENT NEWS NETWORK News Release () issued by the Government

News Network on 3 December 2002

ON BEHALF OF INQUIRY INTO A-LEVEL STANDARDS

PART FOUR OF FOUR PARTS

The Joint Council

113. The Joint Council for General Qualifications (JCGQ) could do

much to enhance commonality between the awarding bodies. It has

already produced a wide range of common documentation and identified

common procedures and guidance covering many aspects of the

examination process. I have already referred above to the need to

carry this work further, to standardise any aspect of awarding

bodies' work and requirements where there is no good reason for

differences to be maintained.

114. JCGQ has the potential to act as a self-regulatory body,

defining and encouraging minimum standards of service and defining

the way in which award standards are to be set, judged and

maintained. Mechanisms already exist for this, though so far groups

like the Compliance Committee have limited their activity to sharing

common practice. I believe that standards of both service and awards

could be greatly enhanced if work was extended into actively seeking

to define and enforce common standards, in areas such as monitoring

examiners, dealing with enquiries about results, reviewing awarding

arrangements and, in consultation with QCA, placing strict limits on

the late notification of changes to syllabus and assessment

requirements. The latter is a particular problem for teachers and

lecturers at present, adding significantly to the complications of

delivering A level courses to students.

115. JCGQ might also provide a forum for enhanced co-operation

between the awarding bodies - for instance in examiner training and

the sharing of data more effectively during the awarding process -

with potential benefits for the overall consistency and quality of

the system.

116. I recommend that awarding bodies strengthen the self-regulatory

role of JCGQ, and review the council's structure, to:

- take forward the work recommended in paragraph 112 to improve

commonality of administrative requirements between boards;

- identify and take forward further areas where it might usefully

self-regulate, such as defining minimum service standards;

- enhance co-operation between the awarding bodies in the awarding

process.

117. Whilst I recommend that the JCGQ should be the vehicle for

enhanced collaboration and commonality between the awarding bodies, I

should make it clear that I consider that the work itself must be

taken forward urgently. If the JCGQ cannot be strengthened in the way

I envisage then its purpose must be seriously questioned. In these

circumstances I recommend that the QCA and the other regulators

should step in to progress this necessary work.

Implications for Wales and Northern Ireland

118. Students in Wales and Northern Ireland sit qualifications

offered by English awarding bodies, while students in England and

Northern Ireland take qualifications offered by WJEC. In 2002, for

example, English awarding bodies received over 30,000 A and AS

subject entries from students at Welsh centres, while WJEC had around

27,500 subject entries from students at English centres. This

cross-border traffic of qualifications means that it is important

that any approach to standardisation across boards is common to all

three countries. My recommendations should be taken forward jointly.

119. The JCGQ represents the awarding bodies in all three countries.

Any development of its role would need to be taken forward on a three

country basis.

Professionalisation of the examinations system

120. Any system is only as good as those operating it. During my

inquiry I have looked closely at the supply and status of two key

groups whose skills and professionalism are essential for the

examination system to operate effectively.

- Examiners - those, mainly teachers, who mark examinations and

coursework, and, at more senior levels, help ensure consistent

standards and participate in the grade boundary-setting process.

- Examinations officers - the staff in schools and colleges who

handle the administration of their students' examination entries and

results.

The supply and training of examiners

121. Examiner shortages are not a new problem. But are a growing

cause for concern, not least to the English awarding bodies. With the

introduction of Curriculum 2000, the number of examination entries

more than doubled, compounding an 8-fold increase in subject entries

between 1951 and 2001. There was a 41% increase in GCE examiners

(across the UK) between 1999 and 2002 and even that is thought to be

insufficient to meet the demands of Curriculum 2000 with its modular

structure and expectation that students will take more subjects.

122. Moreover, it is unclear what impact this year's problems with A

levels will have on the supply of examiners in 2003. There are some

concerns that the result will be a shortage owing to demoralisation

and examiners' unwillingness to associate themselves with a system in

which they have lost faith.

123. In the short term, it is necessary to guarantee examiner

capacity for January and June 2003. The QCA-led Examinations

Taskforce is currently considering a variety of measures to secure

the supply of examiners for next year. These include extensive use of

marking centres, release of teachers from their regular school or

college duties (something some institutions already allow

voluntarily, supported by the existing teacher release scheme),

engaging the teacher associations and reviewing examiner pay.

124. The Taskforce's work should also build on the findings of

pilots conducted by Edexcel and OCR involving the use of graduates -

particularly new PGCE graduates - as examiners. The Edexcel pilot

used PGCE graduates to address a shortage in GCSE history examiners,

while OCR relied on graduates with relevant subject expertise to

redress a shortfall in ICT examiners.

125. Contrary to received wisdom that examiners should have lengthy

classroom experience, the PGCE and graduate examiners were found to

mark consistently and effectively. In the Edexcel pilot, the

performance profile of the PGCE graduates was the same as for all

examiners in 2002 - with 83% of A- or B-grade performance - and

better than that for other new examiners. Factors contributing to

this success were the extra training and regular monitoring the PGCE

graduates were subject to; their deployment to specific papers; and

the support they received from the senior examiner team. The initial

allocation of papers they received was however about 25% lower than

usual for new examiners to minimise pressure and risk, and the

benefits need to be balanced against the cost and administrative

implications.

126. At least one awarding body is already planning to make

extensive use of PGCE graduates to address the examiner shortage for

2003.

127. In the OCR pilot, GCSE foundation scripts were marked by

newly-graduated students in a supervised environment. The pilot met

with great success, the level of accuracy being higher than for the

teams of assistant examiners who marked at home in the normal manner.

The reasons for this were considered to be:

- the ability to confer or seek guidance, which added significantly

to the consistency of the marking;

- the controlled and supervised environment in which marking took

place. This was seen to improve the quality of checking and

administrative compliance.

128. On the basis of the evidence presented to me, and subject to

rigorous supervision and quality assurance procedures, I am confident

in fully endorsing further development of this approach to the

marking of public examinations.

The supply and training of examiners - longer-term action

129. In the longer term, I believe that the key to securing the

supply of high quality examiners is to raise the professional status

of what they do and to ensure that they, and the schools and colleges

which provide them, are recognised for participation in the

examinations system.

130. Professionalisation of marking and examining, including the

marking of coursework, would have clear system-wide benefits. In my

inquiry, I have found wide support for professionalisation, including

from the awarding bodies and teachers' associations.

131. Professionalisation promises benefits for both institutions and

individuals in the potentially greater recognition and rewards for

both examiners and the institutions supplying them. Over the longer

term, this could enable some centres or federations of centres to

earn the right to take more control over assessment, enhancing

perceptions of the reliability of internal assessment models, which

would at the same time reduce the pressures on the public examination

system.

132. The availability and take-up of good quality training is

essential to professionalising the examining workforce. I am

concerned by anecdotal reports received in conducting my inquiry

about the quality of training and the attendance of staff from

centres. I believe that additional resources are needed to tackle

these issues, and that take-up of available training should be a

necessary part of any schemes to enhance teachers' career prospects

and status as a result of their participation in the examination

system.

133. I am equally concerned by anecdotal reports about the variable

quality of the training available. The QCA, working with the Joint

Council and other relevant bodies, like the NCSL, should have a role

in ensuring training provided is fit for purpose, timely and of a

good standard. This should include making use of the exemplar

material and other materials currently being gathered by QCA to

define AS and A2 standards. Further efforts should also be made to

offer common provision as far as possible, with bolt-on sessions

offering training specific to awarding bodies as necessary.

134. I recommend that the use of graduates, and especially PGCE

graduates, as markers in public examinations should be extended,

subject to effective training, supervision, evaluation and public

reporting.

135. I recommend a thorough professionalisation of the

role of markers and examiners, including coursework markers.

Implementation should be taken forward swiftly by the Secretary of

State in collaboration with the regulatory and awarding bodies and

the teaching profession, and should examine:

- linking service and training as an examiner to professional

recognition and career and reward structures, including through

provision made by the National College for School Leadership (NCSL)

and the proposed Post-16 Leadership College;

- improving the take-up of training, including by making available

central government funding, distributed through the schools and FE

Standards Funds or other appropriate funding mechanisms. Provision

should also be made for examiners supplied by the independent sector;

- improving the quality of training, through rigorous accreditation

procedures, drawing on work already underway within the awarding

bodies to develop a qualification for examiners;

- working towards implementation of proposals, like those put forward

by the Secondary Heads Associations for 'Chartered Examiners', which

would establish a high status professional role for experienced

examiners, and would give the schools and colleges in which they are

based the right to take more control over examination assessment.

136. In addition, I also recommend that schools and colleges should

be rewarded with 'beacon' or similar status for significant

contributions to the examination system, including examiner

recruitment and administrative efficiency. As well as kudos derived

from achieving a publicly recognised symbol of excellence, this could

bring with it a lower level of scrutiny in relation to the

administration and conduct of examinations, and possibly, financial

or other rewards.

Examination officers

137. Examination officers have a critical role to play in the

effective administration of the system. Perceptions of this role vary

widely and it may be the responsibility of teaching or administrative

staff. What has been clear in the evidence submitted to my inquiry is

that the role of examination officer is time-consuming and cannot be

done without training and mastery of the variety of systems used.

138. There is a need for common training across the awarding bodies

where commonalities in systems and procedures already exist, and

these should increase over time. All training should be to common

standards and criteria specified by QCA, with required attendance

supported by the availability of money from the Standards Fund. It

should be accredited and recognised in career and development and

reward structures. The difficulties most schools and colleges face if

they lose their examination officer only underlines the invaluable

nature of their contribution.

139. Raising standards of examination administration is significant

to guaranteeing standards of awards, as well as to the smooth running

of the system, and in particular to combating problems arising from

the huge number of late entries and entries made on the day of the

examination ('pirate' entries).

140. This year AQA received 73,241 late, 48,674 very late and about

11,000 'pirate' (entries on the day using photocopied question papers

and answer booklets) GCE and VCE entries. These amounted to nearly 4%

of all entries. The corresponding figures for Edexcel GCE entries

were 57,432 very late and 6,784 pirate entries, nearly 5% of the

total, and OCR had 24,235 late entries, nearly 2% of the total. The

awarding bodies are clear that they should not damage candidates'

life chances by refusing to accept these entries. Nevertheless, late

and pirate entries place a significant additional strain on awarding

bodies, which are already working in an extremely tight time window,

including by hindering the planning of the numbers of markers and

examiners.

141. More than this, however, late entries can hinder efforts to

judge and maintain standards. This is particularly true of the late

submission of coursework. Ideally an awarding body should have

received 75% of coursework by the first deadline to inform

standardisation meetings. It has been reported to me that this year

by the first deadline one board had received only about 20% of

coursework, and only about 40% by a second deadline 2-3 weeks later.

This has impacts on the judgement of standards and on whether

candidates receive the grade they deserve. Action must be taken to

rectify the situation.

142. I recommend that by January 2004, the awarding bodies, in

consultation with schools and colleges, the regulatory bodies and the

Secretary of State, should develop and publish a strategy to:

- ensure good quality training for examination officers, who should

gain professional recognition for their role;

- promote and reward the efficiency of schools' and colleges'

administration as part of the inspection process;

- encourage good practice. In my view that might include using the

fee structure to provide incentives for timely examination entries.

143. I further recommend that OFSTED and ALI should be asked to look

at and report on the quality of schools' and colleges' examinations

administration as part of the institutional inspection frameworks.

Implications for Wales and Northern Ireland

144. Examiner shortages are less acute in Northern Ireland and

Wales. Indeed CCEA has successfully addressed an examiner supply

problem in recent years. Nevertheless, the measures in this chapter

would be of benefit more widely than just England, and the principles

which underpin them could usefully be applied on a three country

basis.

THE USE OF ICT IN QUALIFICATIONS

145. Consideration of how to improve the reliability, efficiency and

quality of the qualifications system inevitably raised questions

about the potential impact of ICT. I have looked at the potential for

ICT in three aspects of the system:

a. administration and data processing;

b. to improve the quality and efficiency of the marking processes;

c. as a medium for examining - ie ICT based examinations and

assessment tasks.

Administration

146. Each of the awarding bodies uses computer systems to support

their administration and data processing. Indeed, the current scale

of the examinations task would not be manageable in any other way.

>From candidate entries at one end of the system through to the

issuing of final results at the other, computing power is central to

the vast majority of data processes needed to collect information

about the performance of individual students, convert their raw marks

into grades and issue their results and collate data across subjects,

awarding bodies and the system as a whole.

147. All three of the English awarding bodies have recently made, or

are just about to embark upon, major investment to update their

administrative ICT systems. Nothing in the evidence put to me

suggests that in relation to these administrative tasks the computer

systems already operated or planned by the awarding bodies are

fundamentally inadequate; nor that significant improvements in the

quality or efficiency of qualifications administration could be

achieved simply by additional investment in ICT beyond that already

planned. It is certainly possible that the current position could be

improved by continuing investment to extend the capacity of the

systems to take on new tasks or manage existing tasks more

efficiently. However, further work would be needed to establish the

scope and business case for further investment of this kind.

148. Nevertheless, there is some evidencethat additional investment

elsewhere is needed to make best use of the current ICT capacity in

the awarding bodies. This is particularly the case in relation to the

commercially- operated Electronic Data Interchange (EDI) systems,

which enable schools and colleges to make their examination entries

on line and to exchange information with the awarding bodies about

their candidates' entries and results.

149. Used to full potential, the EDI system would do much to

eliminate differences in administrative procedures between awarding

bodies. I gather that a significant obstacle to this is that some

schools and colleges lack the necessary hardware and software

packages to make full use of the EDI system. I have not had time to

investigate this situation in detail, but if this understanding is

correct it should be rectified as a matter of urgency.

ICT in marking

150. Based on the evidence presented to this inquiry, I am persuaded

that ICT has the potential to bring about a step change in the

efficiency and quality of the marking process. Students' scripts are

currently distributed via the postal service to markers and the

awarding bodies head offices. Typically they spend about five days

in transit during the marking stage. The marking process is almost

wholly manual. Markers work directly onto the written papers and

collate their records manually.

151. The technology already exists to translate students' written

performance into electronic form. Once digitised, scripts can be

distributed and marked on-line, often in ways that would be difficult

or impossible now. For instance, each script can be broken down into

component answers and different answers can be distributed in

different ways, for example by giving relatively straightforward

short-answer questions to clerical or inexperienced markers.

Questions which require more complex judgements can be sent to more

experienced examiners. Marks awarded during the process can be

collated and monitored continuously whilst marking is in progress.

So, for instance, the performance of individual markers can be

monitored during the marking process and inconsistencies between them

can be identified and corrected much more quickly and accurately than

is possible currently. Estimates given to me suggest that a 10-20%

increase in the efficiency of marking could be achieved, accompanied

by a significant increase in the consistency and reliability of

marking.

152. All of this would be of significant benefit. But it can not be

a quick fix. The awarding bodies have piloted systems along these

lines. Whilst some of the potential benefits were apparent in these

projects, there are some problems still to be overcome. Significant

among these is managing the natural human response of markers and

examiners to what would amount to full-scale industrialisation of the

marking process and the need to adapt their working practices to

reflect this. I am confident that this and other issues, such as the

adaptation of thecurrently available systems to the specific needs

of the awarding bodies and their qualifications, can be overcome

given time and investment.

153. I am not, however, persuaded of the benefits of using ICT to

replace human markers. Whilst there are systems which can produce

reasonable marking of multiple choice and short answer questions, I

do not have evidence of the capability to machine mark answers which

require complex judgement. There would be some efficiency gains to be

had from more extensive use of machine marking of the simpler answers

in existing examination scripts. But such systems would embed

reliance on these question and answer formats, and could

significantly constrain longer term innovation in the examination

system. On this basis, significant investment in such systems would

be misguided.

ICT as a medium for examining and assessment

154. The systems described above would not change significantly the

nature of examinations as experienced by students themselves. If

anything, the need for a common format for script and students'

responses in order to ensure that they can be digitised accurately,

for instance by scanning, could impose an additional constraint on

the scope for innovation in the examinations process.

155. A more radical longer term solution would be the use of ICT as

the means by which students take examinations and submit work for

assessment. For instance, video, photographs, other images, and

sound, could supplement traditional written work. Work could be

submitted electronically and assessed over time to build up a

portfolio of assessed work by the end of the course, rather than

relying largely on one-off examinations. More flexible ways of

assessing students' work could change radically the nature of the

assessment process and the extent to which it promotes key attributes

such as creativity and communication skills.

156. A further development of this approach would be to bring

students' performance data - for instance, test and examination

results, and higher education performance - and their work together

in a single national database to provide a permanent and

comprehensive record of their progress and achievements through

school, college, training and higher education. This would be

information which they could access themselves, for instance to

support applications for university or employment - to supplement or

replace paper certification of their results.

Investment

157. Any new ICT systems would carry a significant cost. It is not

clear that any of the awarding bodies have the capacity for

investment on the scale necessary. Indeed, even some of the limited

piloting of new ICT systems has been curtailed simply on the grounds

of costs.

158. Nor is it necessarily the case that awarding bodies should be

expected to bear the whole cost. Many of the potential advantages of

more extensive or consistent use of ICT would be for schools,

colleges and students rather than producing efficiency savingsor

other benefits for the awarding bodies themselves. Indeed, for some

of the possibilities outlined in this chapter it would be very

difficult for the awarding bodies individually or collectively to

justify the investment solely in terms of a cost-benefit analysis for

their activities. The benefits are spread widely across the system

and would contribute directly to wider policy objectives such as

reducing administrative burdens on schools. For systems which use ICT

as the medium through which students undertake examination and

assessment tasks, there would be a very substantial element of

investment directly in schools and colleges to provide the necessary

equipment and expertise to ensure students' access.

159. On this basis there is a strong case for suggesting that at

least some of the costs should be met directly from public funds

rather than specifically from the awarding bodies' own resources.

Conclusions

1. From the evidence presented to me, I am persuaded that ICT offers

the prospect of significant improvements in the efficiency, quality

and flexibility of the examinations system.

2. Although such a conclusion goes well beyond my remit, I am also

persuaded that ICT offers in time the prospect of linking students'

achievement information with examples of their work and other

information to provide students with an individually accessible

comprehensive portfolio which illustrates their learning much more

broadly than simply examination results.

3. Most of the technology to achieve all these things is available

now - albeit at a price. But there are still some important technical

issues which need to be resolved. Most notably, in the context of

work submitted on-line, it is essential that the identity of students

and their responsibility for the work they are submitting can be

verified. Equally, reliable security would be needed to ensure

privacy of personal records and control individuals' access to

personal portfolios and other personal data.

4. I recommend that:

- resources should be made available from public funds to the

awarding bodies, schools and colleges to ensure that full use can be

made of the existing EDI systems, and for other ICT developments to

improve the commonality of administration and reduce the

administrative burdens on schools;

- an expert working group should be established by the regulators and

the awarding bodies to examine and develop the use of ICT to improve

the efficiency and quality of administration and marking, whilst

ensuring such developments do not constrain innovation by individual

awarding bodies or the options for longer term changes in the nature

and format of assessment;

- the Government should prepare a longer term strategy for developing

the use of ICT as the medium by which students take examinations and

submit material for assessment, taking account of wider developments

of ICT in relation to the curriculum and teaching and learning;

- resources for developing, piloting and implementing new ICT-based

systems in the qualifications framework should be provided directly

from public funds to an extent which properly reflects the public

benefits of such systems and the financial circumstances of the

awarding bodies.

Relevance for Northern Ireland and Wales

5. The issues of consistency between awarding bodies which I have set

out in this chapter are equally relevant to Wales and Northern

Ireland and to those schools and colleges which draw on

qualifications from Welsh or Northern Irish awarding bodies alongside

those from England. This work should be taken forward on a three

country basis. Longer term development of the potential of ICT is

also potentially relevant to all three countries.

ANNEX: STEPS IN THE EXAMINING AND AWARDING PROCESS

SCHEDULE OF EVIDENCE PRESENTED TO THE INQUIRY

During the course of my inquiry I took oral submissions from:

Head Masters Conference (HMC)

Girls' Schools Association (GSA)

Secondary Heads Association SHA)

National Association of Head Teachers (NAHT)

National Union of Teachers (NUT)

National Association Schoolmasters/Union of Women Teachers (NASUWT)

Professional Association of Teachers (PAT)

Association of Teachers and Lecturers (ATL)

National Association of Teachers in Further and Higher Education

(MATFHE)

Association of Colleges (AoC)

National Confederation of Parent Teacher Associations (NCPTA)

OFSTED

Universities and Colleges Admissions Services (UCAS)

Universities UK

Standing Conference of Principals (SCOP)

Qualifications Curriculum and Assessment Authority for Wales (ACCAC)

Council for the Curriculum Examination and Assessment (CCEA)

International Baccalaureate Organisation

The Joint Council

CBI

The Chambers of Commerce

Institute of Directors

The Chief Executives of the three English Awarding Bodies: AQA,

Edexcel and OCR.

Ken Boston, Chief Executive of the QCA, Sir Alan Greener and QCA

officers

Nick Stuart

Roger Porkess

Professor Steve Heppell - Ultralab

NCS Pearson

Professor Carol Fitz-Gibbon

Professor David Melville

Dianne Francombe

Dr Brenda Cross

Damien Green MP

Phil Willis MP

Celia Johnson, DfES

Rob Hull, DfES

I have also received written evidence and submissions from:

SHA

HMC

GSA

NAHT

NUT

ATL

NATFHE

NASUWT

NCPTA

AoC

The General Teaching Council for England

Universities UK

UCAS

SCOP

QCA

OFSTED

AQA

Edexcel

OCR

Damien Green MP

Phil Willis MP

In addition, I have received helpful written submissions from a very

wide range of individuals, schools, teachers, examiners and others.

I held three regional meetings with teachers, parents, governors and

students and would like to thank all that took part and who helped in

their organisation, particularly: Professor Robin Millar, York

University; Kath James, York University; David Moores, Deputy Head,

Fulford School; Roger Dancey, Chief Master, King Edward's School;

Sarah Evans, Head, King Edward's High School; Alan Jenkins,

Principal, Varndean College; and Dr Russell Strutt, Principal,

Haywards Heath College.

  • Comment

Have your say

You must sign in to make a comment

Please remember that the submission of any material is governed by our Terms and Conditions and by submitting material you confirm your agreement to these Terms and Conditions.

Links may be included in your comments but HTML is not permitted.