Commit 83ae4ee6 by srpearce

Merge pull request #2021 from edx/sylvia/documentation/ORA-279

Add instructor notes to document
parents f5f24fcc de9b8bcb
......@@ -8,6 +8,15 @@ Open Response Assessments for Students
Introduction to Open Response Assessments
-----------------------------------------
.. warning::
Modify this section according to your course. For example, you
can delete sentences such as "For more information, see :ref:`ORA Peer Assessment`"
and "For more information, see :ref:`ORA AI Assessment`" if your ORA problem doesn't
contain peer assessments or AI assessments and you want to delete these sections from
this document.
Open response assessments allow you to submit a short written answer,
an essay, or a file such as an image or computer code file.
......@@ -37,8 +46,8 @@ assessment types run in the order in which they appear in the problem.
see :ref:`ORA AI Assessment`.
An open response assessment problem doesn't have to use all assessment types. For example, one problem
may use self assessment and AI assessment, while another problem uses self assessment
and peer assessment, and another problem uses only peer assessment.
may use self assessment and AI assessment, while another problem may use self assessment
and peer assessment, and another problem may use only peer assessment.
You'll answer open response assessment problems in much the same way that you answer other
problems. For more information about how to submit responses, see :ref:`ORA Submit a Response`.
......@@ -79,7 +88,7 @@ or uploading a file.
#. Click **Submit**, and then click **OK** in the dialog box to continue.
**Note** If you want to save your response and work on it again later, click **Save**.
.. note:: If you want to save your response and work on it again later, click **Save**.
An "Answer saved, but not yet submitted" message appears directly under the **Save** and
**Submit** buttons.
......@@ -92,6 +101,11 @@ see :ref:`ORA Self Assessment`, :ref:`ORA Peer Assessment`, or :ref:`ORA AI Asse
Self Assessment
---------------
.. warning::
You can delete this section if your ORA problem doesn't use self assessments.
In a self assessment, the rubric for the problem appears below your response immediately
after you submit the response. You then assess your response based on the rubric.
......@@ -115,6 +129,11 @@ Perform a Self Assessment
Peer Assessment
---------------
.. warning::
You can delete this section if your ORA problem doesn't use peer assessments.
In a peer assessment, several students in the course grade your response while you grade
other students' responses. You have to grade a number of your peers' responses before
you receive your score. (After you grade the minimum number of responses required to
......@@ -154,13 +173,22 @@ below.
Step 1: Access responses from other students
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. warning::
Modify the content in this section according to
your course. For example, if your students can only grade by using the **Open
Ended Console** page, change the introductory sentence below, and delete the
second and third bullets.
**Note** *You can only grade a response if you've submitted a response to the
question, an instructor has already graded at least 20 responses, and
there are more essays from other students left to grade. If you haven't submitted
a response or no responses are available for grading, you see a yellow message in the
interface.*
.. image:: /Images/PAStudent_NoSubmissions.gif
.. image:: /Images/PAStudent_NoSubmissions.gif
There are several ways to access other students' responses, depending on
the way that the course is set up.
......@@ -265,31 +293,34 @@ If you want to grade additional responses at any time, you can go back
to the **Peer Grading** page and click the name of the problem that you want
to continue grading.
**Note**
When a response opens for you to grade, it leaves the current "grading pool"
that other instructors or students are grading from, which prevents other
instructors or students from
grading the response while you are working on it. If you do not submit a score
for this response within 30 minutes, the response returns to the grading pool
(so that it again becomes available for others to grade), even if you still have
the response open on your screen.
.. note:: When a response opens for you to grade, it leaves the current "grading pool"
that other instructors or students are grading from, which prevents other
instructors or students from
grading the response while you are working on it. If you do not submit a score
for this response within 30 minutes, the response returns to the grading pool
(so that it again becomes available for others to grade), even if you still have
the response open on your screen.
If the response returns to the grading pool (because the 30 minutes have passed),
but the response is still open on your screen, you can still submit feedback for
that response. If another instructor or student grades the response after it returns to the
grading pool but before you submit your feedback, the response receives two grades.
If the response returns to the grading pool (because the 30 minutes have passed),
but the response is still open on your screen, you can still submit feedback for
that response. If another instructor or student grades the response after it returns to the
grading pool but before you submit your feedback, the response receives two grades.
If you click your browser’s **Back** button to return to the problem list before you
click **Submit** to submit your feedback for a response, the response stays outside
the grading pool until 30 minutes have passed. When the response returns to the
grading pool, you can grade it.
If you click your browser's **Back** button to return to the problem list before you
click **Submit** to submit your feedback for a response, the response stays outside
the grading pool until 30 minutes have passed. When the response returns to the
grading pool, you can grade it.
.. _ORA AI Assessment:
Artificial Intelligence (AI) Assessment
---------------------------------------
.. warning::
You can delete this section if your ORA problem doesn't use AI assessments.
In an AI assessment, an instructor grades a sample set of student responses to the
open response assessment problem. A machine learning algorithm then creates a model
based on the instructor's scores and grades the remaining students' responses.
......@@ -311,6 +342,11 @@ For more information about accessing your scores, see :ref:`ORA Access Scores`.
Access Scores and Feedback
--------------------------
.. warning::
Modify the text in this section to apply to your course.
For *self assessments*, the score that you give yourself appears as soon as you submit
the score.
......@@ -344,9 +380,9 @@ graders.
If you want to see the full rubric for either an AI or peer assessment,
click **Toggle Full Rubric**.
**Note** *For a peer assessment, if you haven't yet graded enough
problems to see your score, you receive a message that lets you know how
many problems you still need to grade.*
.. note:: For a peer assessment, if you haven't yet graded enough
problems to see your score, you receive a message that lets you know how
many problems you still need to grade.
.. image:: /Images/FeedbackNotAvailable.gif
......@@ -355,6 +391,10 @@ For more information about grading peer assessments, see :ref:`ORA Peer Assessme
Resubmitting a Response
-----------------------
.. warning::
You can delete this section if you don't allow students to submit multiple responses.
Some open response assessments allow multiple attempts. For these
problems, a **New Submission** button appears below your original
response.
......@@ -362,5 +402,3 @@ response.
If you want to answer the question again, click **New Submission** to
clear your former response, and click **OK** in the dialog box that
appears. You can then enter a new response for the problem.
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment