Commit 83ae4ee6 by srpearce

Merge pull request #2021 from edx/sylvia/documentation/ORA-279

Add instructor notes to document
parents f5f24fcc de9b8bcb
...@@ -8,6 +8,15 @@ Open Response Assessments for Students ...@@ -8,6 +8,15 @@ Open Response Assessments for Students
Introduction to Open Response Assessments Introduction to Open Response Assessments
----------------------------------------- -----------------------------------------
.. warning::
Modify this section according to your course. For example, you
can delete sentences such as "For more information, see :ref:`ORA Peer Assessment`"
and "For more information, see :ref:`ORA AI Assessment`" if your ORA problem doesn't
contain peer assessments or AI assessments and you want to delete these sections from
this document.
Open response assessments allow you to submit a short written answer, Open response assessments allow you to submit a short written answer,
an essay, or a file such as an image or computer code file. an essay, or a file such as an image or computer code file.
...@@ -37,8 +46,8 @@ assessment types run in the order in which they appear in the problem. ...@@ -37,8 +46,8 @@ assessment types run in the order in which they appear in the problem.
see :ref:`ORA AI Assessment`. see :ref:`ORA AI Assessment`.
An open response assessment problem doesn't have to use all assessment types. For example, one problem An open response assessment problem doesn't have to use all assessment types. For example, one problem
may use self assessment and AI assessment, while another problem uses self assessment may use self assessment and AI assessment, while another problem may use self assessment
and peer assessment, and another problem uses only peer assessment. and peer assessment, and another problem may use only peer assessment.
You'll answer open response assessment problems in much the same way that you answer other You'll answer open response assessment problems in much the same way that you answer other
problems. For more information about how to submit responses, see :ref:`ORA Submit a Response`. problems. For more information about how to submit responses, see :ref:`ORA Submit a Response`.
...@@ -79,7 +88,7 @@ or uploading a file. ...@@ -79,7 +88,7 @@ or uploading a file.
#. Click **Submit**, and then click **OK** in the dialog box to continue. #. Click **Submit**, and then click **OK** in the dialog box to continue.
**Note** If you want to save your response and work on it again later, click **Save**. .. note:: If you want to save your response and work on it again later, click **Save**.
An "Answer saved, but not yet submitted" message appears directly under the **Save** and An "Answer saved, but not yet submitted" message appears directly under the **Save** and
**Submit** buttons. **Submit** buttons.
...@@ -92,6 +101,11 @@ see :ref:`ORA Self Assessment`, :ref:`ORA Peer Assessment`, or :ref:`ORA AI Asse ...@@ -92,6 +101,11 @@ see :ref:`ORA Self Assessment`, :ref:`ORA Peer Assessment`, or :ref:`ORA AI Asse
Self Assessment Self Assessment
--------------- ---------------
.. warning::
You can delete this section if your ORA problem doesn't use self assessments.
In a self assessment, the rubric for the problem appears below your response immediately In a self assessment, the rubric for the problem appears below your response immediately
after you submit the response. You then assess your response based on the rubric. after you submit the response. You then assess your response based on the rubric.
...@@ -115,6 +129,11 @@ Perform a Self Assessment ...@@ -115,6 +129,11 @@ Perform a Self Assessment
Peer Assessment Peer Assessment
--------------- ---------------
.. warning::
You can delete this section if your ORA problem doesn't use peer assessments.
In a peer assessment, several students in the course grade your response while you grade In a peer assessment, several students in the course grade your response while you grade
other students' responses. You have to grade a number of your peers' responses before other students' responses. You have to grade a number of your peers' responses before
you receive your score. (After you grade the minimum number of responses required to you receive your score. (After you grade the minimum number of responses required to
...@@ -154,13 +173,22 @@ below. ...@@ -154,13 +173,22 @@ below.
Step 1: Access responses from other students Step 1: Access responses from other students
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. warning::
Modify the content in this section according to
your course. For example, if your students can only grade by using the **Open
Ended Console** page, change the introductory sentence below, and delete the
second and third bullets.
**Note** *You can only grade a response if you've submitted a response to the **Note** *You can only grade a response if you've submitted a response to the
question, an instructor has already graded at least 20 responses, and question, an instructor has already graded at least 20 responses, and
there are more essays from other students left to grade. If you haven't submitted there are more essays from other students left to grade. If you haven't submitted
a response or no responses are available for grading, you see a yellow message in the a response or no responses are available for grading, you see a yellow message in the
interface.* interface.*
.. image:: /Images/PAStudent_NoSubmissions.gif .. image:: /Images/PAStudent_NoSubmissions.gif
There are several ways to access other students' responses, depending on There are several ways to access other students' responses, depending on
the way that the course is set up. the way that the course is set up.
...@@ -265,31 +293,34 @@ If you want to grade additional responses at any time, you can go back ...@@ -265,31 +293,34 @@ If you want to grade additional responses at any time, you can go back
to the **Peer Grading** page and click the name of the problem that you want to the **Peer Grading** page and click the name of the problem that you want
to continue grading. to continue grading.
**Note** .. note:: When a response opens for you to grade, it leaves the current "grading pool"
that other instructors or students are grading from, which prevents other
When a response opens for you to grade, it leaves the current "grading pool" instructors or students from
that other instructors or students are grading from, which prevents other grading the response while you are working on it. If you do not submit a score
instructors or students from for this response within 30 minutes, the response returns to the grading pool
grading the response while you are working on it. If you do not submit a score (so that it again becomes available for others to grade), even if you still have
for this response within 30 minutes, the response returns to the grading pool the response open on your screen.
(so that it again becomes available for others to grade), even if you still have
the response open on your screen.
If the response returns to the grading pool (because the 30 minutes have passed), If the response returns to the grading pool (because the 30 minutes have passed),
but the response is still open on your screen, you can still submit feedback for but the response is still open on your screen, you can still submit feedback for
that response. If another instructor or student grades the response after it returns to the that response. If another instructor or student grades the response after it returns to the
grading pool but before you submit your feedback, the response receives two grades. grading pool but before you submit your feedback, the response receives two grades.
If you click your browser’s **Back** button to return to the problem list before you If you click your browser's **Back** button to return to the problem list before you
click **Submit** to submit your feedback for a response, the response stays outside click **Submit** to submit your feedback for a response, the response stays outside
the grading pool until 30 minutes have passed. When the response returns to the the grading pool until 30 minutes have passed. When the response returns to the
grading pool, you can grade it. grading pool, you can grade it.
.. _ORA AI Assessment: .. _ORA AI Assessment:
Artificial Intelligence (AI) Assessment Artificial Intelligence (AI) Assessment
--------------------------------------- ---------------------------------------
.. warning::
You can delete this section if your ORA problem doesn't use AI assessments.
In an AI assessment, an instructor grades a sample set of student responses to the In an AI assessment, an instructor grades a sample set of student responses to the
open response assessment problem. A machine learning algorithm then creates a model open response assessment problem. A machine learning algorithm then creates a model
based on the instructor's scores and grades the remaining students' responses. based on the instructor's scores and grades the remaining students' responses.
...@@ -311,6 +342,11 @@ For more information about accessing your scores, see :ref:`ORA Access Scores`. ...@@ -311,6 +342,11 @@ For more information about accessing your scores, see :ref:`ORA Access Scores`.
Access Scores and Feedback Access Scores and Feedback
-------------------------- --------------------------
.. warning::
Modify the text in this section to apply to your course.
For *self assessments*, the score that you give yourself appears as soon as you submit For *self assessments*, the score that you give yourself appears as soon as you submit
the score. the score.
...@@ -344,9 +380,9 @@ graders. ...@@ -344,9 +380,9 @@ graders.
If you want to see the full rubric for either an AI or peer assessment, If you want to see the full rubric for either an AI or peer assessment,
click **Toggle Full Rubric**. click **Toggle Full Rubric**.
**Note** *For a peer assessment, if you haven't yet graded enough .. note:: For a peer assessment, if you haven't yet graded enough
problems to see your score, you receive a message that lets you know how problems to see your score, you receive a message that lets you know how
many problems you still need to grade.* many problems you still need to grade.
.. image:: /Images/FeedbackNotAvailable.gif .. image:: /Images/FeedbackNotAvailable.gif
...@@ -355,6 +391,10 @@ For more information about grading peer assessments, see :ref:`ORA Peer Assessme ...@@ -355,6 +391,10 @@ For more information about grading peer assessments, see :ref:`ORA Peer Assessme
Resubmitting a Response Resubmitting a Response
----------------------- -----------------------
.. warning::
You can delete this section if you don't allow students to submit multiple responses.
Some open response assessments allow multiple attempts. For these Some open response assessments allow multiple attempts. For these
problems, a **New Submission** button appears below your original problems, a **New Submission** button appears below your original
response. response.
...@@ -362,5 +402,3 @@ response. ...@@ -362,5 +402,3 @@ response.
If you want to answer the question again, click **New Submission** to If you want to answer the question again, click **New Submission** to
clear your former response, and click **OK** in the dialog box that clear your former response, and click **OK** in the dialog box that
appears. You can then enter a new response for the problem. appears. You can then enter a new response for the problem.
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment