Reviewer Tools and Processes
Understand the tools available to a reviewer to support the quality assurance process
The role of the reviewer is one of quality assurance. Someone who supervises and reviews the work of assessors to ensure a fair and robust assessment process has been applied.
To support these processes, there are three options for access privileges for the reviewer: Basic Access, View Marking Information and Decide Final Grades which will be explained in more detail below.
In this article:
Basic Access
By default, the reviewer has access to the participant's submission, the similarity report (if relevant), annotations and feedback (if shared with co-assessors). It is not possible to view the rubric, scores or the final grade.
- See the assignment and additional material for assessors
- See start and end dates of the participation and marking periods
- Click to see a list of manager(s) on the flow
- This shows which marking procedure is used on the flow
- This shows the percentage value of the similarity analysis and links directly to the detailed similarity report (only where applicable)
- Access the submission via the marking tool
View Marking Information
All flow types
- See the assignment and additional material for assessors
- See start and end dates of the participation and marking periods
- Click to see a list of flow managers
- This shows which marking procedure is used on the flow
- Filter the list by assessors
- Download the submissions as a ZIP file (FLOWassign and FLOWlock only)
- This shows the percentage value of the similarity analysis and links directly to the detailed similarity report (FLOWassign and FLOWlock only)
- Access the submission via the marking tool or go directly to the summary for the participant
- This column shows the grade given by the assessors assigned to the participant
- If several assessors have been assigned to a flow and they have submitted the same grade, this will be shown here
It is possible to have this setting on your licence as the default for all reviewers. Contact Support for further information.
FLOWmulti Only
Additionally, reviewers with access to marking information can also view the statistics of a FLOWmulti exam:
For more details on flow statistics please see the statistics article.
Grade Audit Trail
Only submitted grades are visible in the Grade Audit Trail. Entered grades will not be displayed.
If the grades from multiple assessors have been submitted but there is a conflict between the co-assessors, the conflicting grades will be displayed in the Grade Audit Trail.
Any comments submitted to accompany the submitted grades will be displayed. So too will any Alternative Grades.
The Grade Audit Trail will also show if an assessor has withdrawn their submitted grade, or if a reviewer or manager has unlocked a submitted grade.
Any actions completed prior to the release of this new feature on Friday 8 October 2022 will not be recorded in the Grade Audit Trail.
Decide Final Grades
All flow types
Further details on this process can be found in the article: Marker Agreement vs Reviewer Approval.
FLOWmulti Item-Based Marking
If you are added as a reviewer on a FLOWmulti with item-based marking and have been allowed to see or submit the grades, it will look a bit different than the other flow types:
- See the assignment and additional material for assessors
- See start and end dates of the assessment period
- Click to see a list of flow managers
- This shows the marking procedure for the flow
- Filters the list by assessors
- View Statistics of a FLOWmulti exam
- Here you can either open the paper in the marking tool or go directly to the summary for the participant
- This column shows the sum of marks given by either the automated scoring or the assessor. Those indicated by a padlock have been submitted manually by an assessor.
- By clicking on the marks in the column, you can open a small overview of the item, and here you can cancel the assessors’ submitted points by clicking the unlock icon but only if the final scores have not been submitted.
- This section will only be available if you are allowed to submit the final grades
- Opens the grade scaler, which will be used to convert marks to grades
- Submits all the available marks
- This column shows all the marks that were converted from the point in the previous column
- Here you can see all the marks that are submitted