Providing Feedback

In FLOWmulti, there are two views: View and Annotate. In the View tab, you are presented with each item of the assignment in turn to which you have been allocated (item-based marking). Remember: you may not have been allocated to a whole assignment but only individual items(s). If you have been allocated to the whole assignment, you can add inline annotations to any of these items, even if they have been auto-scored. To add an annotation, select Annotate (a PDF version of the assignment); you can then use any of the tools listed below.

The new marking tool offers three different inline annotation options: an annotate tool, a shape tool and an insert tool.

The Annotate Tool

Click Annotate, and choose from the tools available: highlight, underline, strikeout, squiggly underline, note or a text box. Select the area of text you wish to draw attention to and the desired colour. Once the tool and the text has been selected, an annotation box will appear in the right-hand window where you can add your comment. By using the icon to select text first, you can also choose to copy the text or link to a URL. Additional functions include undo, redo and erase.

 

The Shapes Tool

Click Shapes and choose from the shapes available (see screenshot below) including a free hand tool. Once the chosen shape has been inserted, (and a preferred colour chosen, if applicable), an annotation box will appear in the right-hand window where you can add your comment. By using the icon to select text first, you can also choose to copy the text or link to a URL. Additional functions include undo, redo and erase.

 

The Insert Tool

Click Insert and choose to insert a signature, a rubber stamp or today's date. Once the chosen tool has been inserted, (and a preferred colour chosen, if applicable), an annotation box will appear in the right-hand window where you can add your comment. By using the icon to select text first, you can also choose to copy the text or link to a URL. Additional functions include undo, redo and erase.

 

The Annotation Editor Window

Within each new annotation window, comments can be added via the text box. Click on the coloured box in the top right corner to change with whom the comment is to be shared: participant, co-assessor, private or all. Your comment will be save automatically.

Once saved, the three dot icon can be used to edit or delete the annotation. 

At the top of the annotations window there is a search box that allows you to search the annotations for text. It is also possible to sort the annotations by position, created date, modified date, sharing state, author, type or colour.

The three-strip icon can be used to filter the annotations using any of the annotations' attributes, such as author, sharing state, annotation type or colour.

The double   icon can be used select multiple annotations and the All icon to select all annotations. This enables bulk action to be performed: (1) update the sharing status, (2) update the format or (3) delete the selected annotations.

Sharing Individual Annotations

There are four possible options for sharing annotations:

1. Private: Comments and annotations are visible only to the assessor (This is unless a reviewer has been given the privilege to access all assessment information which includes all comments made by an assessor, even if they are set to private).
2. Share with participant: Comments and annotations are visible to the assessor and the participant (upon the release of grades and feedback) only.
3. Share with co-assessor: Comments and annotations are visible to the assessor and any co-assessors that are also assigned to the same participant.
4. Share with all: Comments and annotations are available to the assessor, and co-assessors that are also assigned to the same participant and the participant (upon the release of grades and feedback).

These options can be configured per annotation in order to offer maximum flexibility to assessors. To change the setting on a single annotation, click on the coloured box in the top right corner to change with whom the comment is to be shared: participant, co-assessor, private or all.

The default WISEflow behaviour is that comments are private unless the assessor decides to share them. However, it is possible to configure this at an individual licence level or at the flow level to make the default behaviour be any of the four listed options.

For example, if the default behaviour is that annotations are shared with all, assessors would need to deselect the groups with whom they did not want to share the annotation. Please see the article on Licence Settings (licence level) or Managing Assessors, Reviewers and Marking Settings (flow level) for more information.

Submission Feedback

Alongside the annotation tools which allow assessors to make comments linked to a specific position in the submission, WISEflow also makes it possible to provide general comments and feedback at both the submission level as well as to the cohort an assessor has been allocated to. The latter can be useful if assessors, for example, start to notice the same errors appearing in several submissions.

Submission feedback can be added by clicking on the feedback icon  and opening the common feedback window from the left-hand navigation pane in the marking tool.

Add your text to the text box and format. Select your preferred sharing option i.e. with participants, co-assessors or any combination thereof. Add a file if you wish and click Submit. Any previously entered feedback is listed below in the same window. The list of feedback can also be filtered and sorted.

1. Add new feedback
2. Filter and sort the list of feedback comments
3. View or change the sharing status for the feedback
4. Identifies the assessor and, by hovering over the name, when the comment was last edited
5. Add additional files to the feedback
6. Delete the feedback
7. Available formatting options for the feedback text
8. Edit the feedback text. Clicking on the text box will open the feedback comment for further editing.
9. Indication if feedback is edited or saved

Reviewer Moderating Submission Feedback

Once a reviewer has been given Moderator level status, they are able to copy and adjust any comment of submission feedback provided by an assessor.

To copy an assessor's submission feedback comment, click on the copy icon. The comment will be duplicated with the reviewer's name instead of the assessor (unless anonymised).

The comment text can now be adjusted.

It is also possible for the reviewer to change the status of the original (or any) submission feedback comment.

This means that if an assessor's submission feedback comment needs adjusting or re-writing, the reviewer can copy the comment, make the relevant changes, and change the status of the original comment so that it is not visible to the participant.

The sharing settings are described above.

Common Feedback

Common feedback can be added by clicking on the speech bubble icon and opening the common feedback window from the left-hand navigation pane in the marking tool.

Add your text to the text box and format. Select your preferred sharing option i.e. with participants, co-assessors or any combination thereof (more information on sharing settings in the annotations article). Add a file if you wish and submit. Any previously entered feedback is listed below in the same window. The list of feedback can also be filtered and sorted.

1. Add new feedback
2. Filter and sort the list of feedback comments
3. View or change the sharing status for the feedback
4. Identifies the assessor and, by hovering over the name, when the comment was last edited
5. Add additional files to the feedback
6. Delete the feedback
7. Available formatting options for the feedback text
8. Edit the feedback text. Clicking on the text box will open the feedback comment for further editing.
9. Indication if feedback is edited or saved

Reviewer Moderating Common Feedback

Once a reviewer has been given Moderator level status, they are able to copy and adjust any comment of common feedback provided by an assessor.

To copy an assessor's common feedback comment, click on the copy icon. The comment will be duplicated with the reviewer's name instead of the assessor (unless anonymised).

The comment text can now be adjusted.