Explanation and Impacted Examinations
Which exams have been impacted?
Overall 65 examinations had been potentially impacted by the early publication of the examination material.
This includes an additional 9 exams that were identified after the conclusion of the exams and the publication of the list of 56 impacted exams.
The analysis conducted by the independent Expert Advisory Panel has concluded that 25 of the 65 exams either had sufficient changes made to them or the published material was such that no advantage could be obtained from it.
The remaining 40 examinations were subject to extensive analysis by the Expert Advisory Panel who found no evidence in the data that any student had obtained any unusual advantage in 35 of the examinations. These exams were recommended to be processed as normal.
After extensive analysis, only 5 exams were found to have been impacted by the early publication of examination material. These were Business Management, Legal Studies, Product Design and Technologies, Philosophy and Visual Communication Design.
A full list of impacted and cleared exams is available
above.
Were any examinations released in full?
No examinations were released in full. In many cases only partial prompts or questions were released. No answers were released.
Does this confirm that 69 students accessed the exam material in advance?
There is no indication that any of these 69 students had accessed any examination material prior to sitting the exam.
None of these students were clustered by school or any other relationship that could be determined, suggesting that there was no co-ordinated behaviour to access examination material.
The statistical analysis revealed anomalies in the data where a small number of students in 5 exams performed significantly better on the impacted questions compared to their performance in the rest of the paper.
Statistical anomalies can and do occur in any exam and this could be for a number of reasons.
VCAA took a conservative approach to the impacted examinations and applied a process to ensure that no student was disadvantaged from students
potentially having access to published examination materials ahead of the exam.
Did students have marks taken away?
No. For the impacted material in the 5 examinations, all 69 students were awarded their marks consistent with the marking guides.
No students were marked down as a result of this process.
Were marks given as part of the process and did that advantage students who were part it?
No student has had an advantage over another as a result of this process.
The calculation of study scores for each subject is independent of all other subjects, meaning that each subject is marked fairly, in its own right.
The independent Expert Advisory Panel led by Professor John Firth advised on a process to ensure valid and reliable scores were applied to all students across subjects impacted.
What treatment was applied to the 5 exams that weren’t cleared to be assessed as normal?
To avoid any consequent disadvantage for other students, the Panel recommended that those students were removed from the standard study score, including the distribution of study scores. The scores of impacted students were then inserted into the overall distribution at a matched point after the study scores had been distributed.
This has ensured that no student lost a mark through this process and no student was disadvantaged as a result of this process.
You previously said that only 56 exams were impacted, why did the list grow to 65?
After the exams were completed and during the examination marking process, the VCAA became aware of an additional 9 studies where it may have been possible for students to gain early access to examination material through the use of LibreOffice software, a different program than was used to gain access to the original material.
This prompted an independent forensic analysis on all exam cover sheets and the total number of potentially impacted exams was found to be 65.
After extensive scrutiny and analysis by the independent Expert Advisory Panel only 5 examinations were found to have been impacted by anomalous results for 69 students.
The other examinations were cleared because either the exams had been sufficiently rewritten, the published information was of no material benefit or there was no evidence in the data that any student had obtained an unusual advantage.
Do you know which questions were impacted and to what extent?
The impact of the breach varies across these exams.
It ranges from the partial publishing of prompts with no questions published, to the publication of some full questions. No examinations were published in full. No answers were published.
The early publication of exam material only represented a
potential advantage to students who may have accessed it.
Whether there was an
actual advantage, and the impact on each exam, was only able to be determined once the Expert Advisory Panel had completed its analysis.
From the analysis of the Expert Advisory Panel, the VCAA was advised that only 5 exams had anomalous results for a total of 69 students on questions that could have been accessed prior to the examination.
There is no indication that any of these students had accessed any examination material prior to sitting the exam.
Reviewing exams impacted and the expert advisory panel
Who is on the expert advisory panel?
Former VCAA CEO Professor John Firth is Chair of an expert advisory panel.
Professor Firth is supported by Dr Nathan Zoanetti, Research Director, Methodology and Measurement with the Australian Council for Educational Research (ACER) and the internationally regarded educational measurement expert and former Chair of the Australian Curriculum, Assessment and Reporting Authority, Professor Barry McGaw AO.
How did the anomalous grade check process work?
The expert advisory panel:
- Reviewed student marks in the affected exams and identified any anomalies in how students have responded to the affected questions, including how the affected question related to the rest of the exam.
- Analysed if any discrepancies were identified. If so, the panel conducted further analysis.
How does the VCAA know this process worked?
The anomalous grade check process is not new and is not exclusive to Victoria. Similar processes are used across Australia and internationally to check and validate exam results periodically.
In applying the process, the VCAA used tested adjustment techniques that we know work and we know are the best ones to apply. The work of the VCAA was overseen by the independent expert panel led by Professor John Firth.
All techniques that the VCAA considered applying are based on well-founded practice here in Australia and internationally.
The VCAA is confident the process has left no student advantaged or disadvantaged as the VCAA has applied post-exam modification processes recently. During COVID-19, a considerably enhanced Consideration of Disadvantage process was implemented for students who sat exams after disruption to learning and to exam scheduling; the process resulted in students receiving fair, accurate and valid results that year.
Who oversaw the process?
An independent expert advisory panel chaired by Professor John Firth. Professor Firth is a former CEO of VCAA who supported development of the Consideration of Disadvantage Process during COVID-19 and is familiar with the anomalous grade check process.
Professor Firth was joined by Dr Nathan Zoanetti, Research Director, Methodology and Measurement with the Australian Council for Educational Research (ACER) and the internationally regarded educational measurement expert and former Chair of the Australian Curriculum, Assessment and Reporting Authority, Professor Barry McGaw.
Were all VCE students subject to the process?
Every year, VCE exams across all subjects are checked in the anomalous grade check process as standard procedure. This year, the process was extended to include a detailed analysis of all exam questions impacted by the production error, and the results of this were scrutinised by the expert advisory panel.
Results
What if a student disagrees with their results?
If a student would like more information on their marks, the first step is to request a Statement of Marks, which provides the marks for each question or criteria of the exam compared to the total marks available. To apply, students fill out the application form included with their December results package. There is no fee for this process.
As with every year, students can also request an inspection of their exam response material. This is where students can see a copy of the examination paper submitted. The fee for this process has been waived for 2024. This inspection is completed with a teacher present to answer any questions the student has.
If at this point a student and teacher believe there has been a clear error in the marking of the student’s examination, the student can apply for a VCE Examination Score Review . This is made on behalf of a student by a school principal and supported by their subject teacher.
More information on this process is available in the
VCE Administrative Handbook on page 136.
What will students see if they apply to inspect their examination papers?
If students wish to see their examination papers, they can apply to complete an Inspection of Examination Response Material.
Inspections take place at the student’s school, with the principal’s permission. The principal must sign the Inspection of Scripts application, which is forwarded to the VCAA. Material will be sent to the principal. An appointment must be made at the school for the inspection. When schools are closed, inspections may take place at the VCAA, in which case the VCAA will make an appointment with the student.
The VCAA allows 15 minutes for the inspection of each examination. Students attend the inspection accompanied by their study teacher. If the study teacher is not available, students should contact their school to discuss a possible suitable substitute.
The student will see a copy of the paper they submitted on the exam day. There are no notes, mark ups or additional comments on the paper, just what the student wrote during the exam.
Why does a teacher need to be present when the examination inspection takes place?
The subject teacher is present to answer any questions the student may have on their written response and their marks. As a subject expert, the teacher assists the student in reviewing their response and associated marks and can provide explanations on how marks have been awarded according to the marking criteria.
What is involved in a VCE Examination Score Review?
A VCE Examination Score Review can take place after a Statement of Marks and Inspection of Examination Response Material are completed. If a student believes there has been an error in the way their examination response was marked, and this is endorsed by the Principal, the school can make an application to be considered by the VCAA.
What if a student thinks their result was impacted by another student or students having early access to exam material?
The process we have outlined shows clearly that no student could have been disadvantaged by another student having early access to exam material. However, if a student wishes to raise a particular case or circumstance related to their exams, they can contact the Post Results and ATAR Service (PRAS) through the VCAA on 1800 653 080 (free call) or via email at:
pras@education.vic.gov.au
How long will the appeals process take?
The VCE Examination Score Review process takes several months each year.
The first step is to apply for an application for a statement of marks and inspection of examination response materials. Applications are open until Friday 7 February 2025.
The second step is to apply for score review which is open until 21 February 2025.
Outcomes of a score review are not available before university offers.
Future VCE examinations
What is the purpose and scope of the independent review?
A ‘root and branch’ review of the VCAA has been commissioned and announced by the Victorian Government to provide full confidence to Victorian students, families and schools in the VCAA. The Review will examine the structure and operations of the VCAA. The review will commence in 2025.
What is the scope and purpose of the independent monitor?
This independent monitor will work with and provide guidance to the VCAA and provide advice to the Government of the VCAA's role regarding examination preparation, management and production from next year. Terms of reference will be developed to outline the monitor's role and responsibilities.
Who will it be and when will they start?
A person of appropriate standing and expertise will be appointed to commence as the independent monitor in the role early in 2025.
What is the difference between the panel and the independent monitor?
The expert advisory panel will oversee the current process to address any potential advantage from the publication of the 2024 examination material. The independent monitor will commence in January 2025 to oversee the 2025 VCE examination process.