Skip to content
Projects
Groups
Snippets
Help
This project
Loading...
Sign in / Register
Toggle navigation
E
edx-platform
Overview
Overview
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
edx
edx-platform
Commits
de9b8bcb
Commit
de9b8bcb
authored
Dec 23, 2013
by
spearce
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Update formatting for notes to instructor
Change formatting for notes to instructor from table to warning
parent
4af4e2e0
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
99 additions
and
130 deletions
+99
-130
docs/course_authors/source/ora_students.rst
+99
-130
No files found.
docs/course_authors/source/ora_students.rst
View file @
de9b8bcb
.. _ORA for Students:
.. _ORA for Students:
Open Response Assessments for Students
Open Response Assessments for Students
======================================
======================================
.. _ORA Introduction:
.. _ORA Introduction:
Introduction to Open Response Assessments
Introduction to Open Response Assessments
-----------------------------------------
-----------------------------------------
.. list-table::
.. warning::
:widths: 80
:header-rows: 1
Modify this section according to your course. For example, you
can delete sentences such as "For more information, see :ref:`ORA Peer Assessment`"
* - To instructors:
and "For more information, see :ref:`ORA AI Assessment`" if your ORA problem doesn't
contain peer assessments or AI assessments and you want to delete these sections from
* - Modify this section according to your course. For example, you
this document.
can delete sentences such as "For more information, see :ref:`ORA Peer Assessment`"
and "For more information, see :ref:`ORA AI Assessment`" if your ORA problem doesn't
contain peer assessments or AI assessments and you want to delete these sections from
Open response assessments allow you to submit a short written answer,
this document.
an essay, or a file such as an image or computer code file.
Open response assessments allow you to submit a short written answer,
an essay, or a file such as an image or computer code file.
When you come to an open response assessment problem, you see the name of the
When you come to an open response assessment problem, you see the name of the
problem, the assessment types, the text of the question, the field where you'll
problem, the assessment types, the text of the question, the field where you'll
enter your response, and the **Save** and **Submit** buttons.
enter your response, and the **Save** and **Submit** buttons.
.. image:: /Images/ExampleORA.gif
.. image:: /Images/ExampleORA.gif
...
@@ -36,17 +32,17 @@ that you'll click to upload your file.
...
@@ -36,17 +32,17 @@ that you'll click to upload your file.
.. image:: /Images/ExampleORA_File.gif
.. image:: /Images/ExampleORA_File.gif
The *assessment types* can include *self assessment*, *peer assessment*, and *artificial intelligence (AI) assessment*. The
The *assessment types* can include *self assessment*, *peer assessment*, and *artificial intelligence (AI) assessment*. The
assessment types run in the order in which they appear in the problem.
assessment types run in the order in which they appear in the problem.
- In a self assessment, you assess your response according a rubric that the
- In a self assessment, you assess your response according a rubric that the
instructor has created. For more information, see :ref:`ORA Self Assessment`.
instructor has created. For more information, see :ref:`ORA Self Assessment`.
- In a peer assessment, you grade
- In a peer assessment, you grade
responses that your peers have submitted while several of your peers
responses that your peers have submitted while several of your peers
grade your response. For more information, see
grade your response. For more information, see
:ref:`ORA Peer Assessment`.
:ref:`ORA Peer Assessment`.
- In an AI assessment, a computer algorithm grades your response. For more information,
- In an AI assessment, a computer algorithm grades your response. For more information,
see :ref:`ORA AI Assessment`.
see :ref:`ORA AI Assessment`.
An open response assessment problem doesn't have to use all assessment types. For example, one problem
An open response assessment problem doesn't have to use all assessment types. For example, one problem
...
@@ -55,8 +51,8 @@ and peer assessment, and another problem may use only peer assessment.
...
@@ -55,8 +51,8 @@ and peer assessment, and another problem may use only peer assessment.
You'll answer open response assessment problems in much the same way that you answer other
You'll answer open response assessment problems in much the same way that you answer other
problems. For more information about how to submit responses, see :ref:`ORA Submit a Response`.
problems. For more information about how to submit responses, see :ref:`ORA Submit a Response`.
When you submit a response to an open response assessment, the next step
When you submit a response to an open response assessment, the next step
depends on the type of assessment that the problem uses. For more information,
depends on the type of assessment that the problem uses. For more information,
see :ref:`ORA Self Assessment`, :ref:`ORA Peer Assessment`, and :ref:`ORA AI Assessment`.
see :ref:`ORA Self Assessment`, :ref:`ORA Peer Assessment`, and :ref:`ORA AI Assessment`.
...
@@ -66,12 +62,12 @@ see :ref:`ORA Access Scores`.
...
@@ -66,12 +62,12 @@ see :ref:`ORA Access Scores`.
If you want to experiment with open response assessments, you can try out the open
If you want to experiment with open response assessments, you can try out the open
assessment problems in the `EdX Demo <https://courses.edx.org/courses/edX/DemoX/Demo_Course/info>`_
assessment problems in the `EdX Demo <https://courses.edx.org/courses/edX/DemoX/Demo_Course/info>`_
course. To get started, go
course. To get started, go
to the `Self-Assessed Essay <https://courses.edx.org/courses/edX/DemoX/Demo_Course/courseware/graded_interactions/machine_grading/2>`_
to the `Self-Assessed Essay <https://courses.edx.org/courses/edX/DemoX/Demo_Course/courseware/graded_interactions/machine_grading/2>`_
unit, and then enter a response in the **Response** field under the
unit, and then enter a response in the **Response** field under the
question. You can enter your own response, or you can use one of the sample
question. You can enter your own response, or you can use one of the sample
responses in the `Sample Answers <https://courses.edx.org/courses/edX/DemoX/Demo_Course/courseware/graded_interactions/machine_grading/6/>`_
responses in the `Sample Answers <https://courses.edx.org/courses/edX/DemoX/Demo_Course/courseware/graded_interactions/machine_grading/6/>`_
unit.
unit.
.. _ORA Submit a Response:
.. _ORA Submit a Response:
...
@@ -83,19 +79,19 @@ or uploading a file.
...
@@ -83,19 +79,19 @@ or uploading a file.
#. Enter the response that you want to submit.
#. Enter the response that you want to submit.
- If you're submitting a written response, type your response in the
- If you're submitting a written response, type your response in the
**Response** field.
**Response** field.
- If you're uploading a file, click **Choose File** under the **Response**
- If you're uploading a file, click **Choose File** under the **Response**
field. In the dialog box that opens, select the file that you want to upload,
field. In the dialog box that opens, select the file that you want to upload,
and then click **Open**.
and then click **Open**.
#. Click **Submit**, and then click **OK** in the dialog box to continue.
#. Click **Submit**, and then click **OK** in the dialog box to continue.
.. note:: If you want to save your response and work on it again later, click **Save**.
.. note:: If you want to save your response and work on it again later, click **Save**.
An "Answer saved, but not yet submitted" message appears directly under the **Save** and
An "Answer saved, but not yet submitted" message appears directly under the **Save** and
**Submit** buttons.
**Submit** buttons.
After you submit your response, the assessment types start running in the order in which they
After you submit your response, the assessment types start running in the order in which they
appear in the problem. For more information,
appear in the problem. For more information,
see :ref:`ORA Self Assessment`, :ref:`ORA Peer Assessment`, or :ref:`ORA AI Assessment`.
see :ref:`ORA Self Assessment`, :ref:`ORA Peer Assessment`, or :ref:`ORA AI Assessment`.
...
@@ -105,15 +101,11 @@ see :ref:`ORA Self Assessment`, :ref:`ORA Peer Assessment`, or :ref:`ORA AI Asse
...
@@ -105,15 +101,11 @@ see :ref:`ORA Self Assessment`, :ref:`ORA Peer Assessment`, or :ref:`ORA AI Asse
Self Assessment
Self Assessment
---------------
---------------
.. list-table::
.. warning::
:widths: 80
:header-rows: 1
You can delete this section if your ORA problem doesn't use self assessments.
* - To instructors:
* - You can delete this section if your ORA problem doesn't use self assessments.
In a self assessment, the rubric for the problem appears below your response immediately
In a self assessment, the rubric for the problem appears below your response immediately
after you submit the response. You then assess your response based on the rubric.
after you submit the response. You then assess your response based on the rubric.
...
@@ -122,7 +114,7 @@ Perform a Self Assessment
...
@@ -122,7 +114,7 @@ Perform a Self Assessment
#. Submit a response to a self-assessed ORA problem.
#. Submit a response to a self-assessed ORA problem.
#. When the rubric appears, compare your response with the rubric, and select the
#. When the rubric appears, compare your response with the rubric, and select the
option that you think is appropriate for each category.
option that you think is appropriate for each category.
.. image:: /Images/Rubric1.gif
.. image:: /Images/Rubric1.gif
...
@@ -133,25 +125,21 @@ Perform a Self Assessment
...
@@ -133,25 +125,21 @@ Perform a Self Assessment
yourself.
yourself.
.. _ORA Peer Assessment:
.. _ORA Peer Assessment:
Peer Assessment
Peer Assessment
---------------
---------------
.. list-table::
.. warning::
:widths: 80
:header-rows: 1
You can delete this section if your ORA problem doesn't use peer assessments.
* - To instructors:
* - You can delete this section if your ORA problem doesn't use peer assessments.
In a peer assessment, several students in the course grade your response while you grade
In a peer assessment, several students in the course grade your response while you grade
other students' responses. You have to grade a number of your peers' responses before
other students' responses. You have to grade a number of your peers' responses before
you receive your score. (After you grade the minimum number of responses required to
you receive your score. (After you grade the minimum number of responses required to
receive your score, you can grade as many additional responses as you want.)
receive your score, you can grade as many additional responses as you want.)
After you submit your response for grading, the following
After you submit your response for grading, the following
message appears under your response.
message appears under your response.
**Your response has been submitted. Please check back later for your grade.**
**Your response has been submitted. Please check back later for your grade.**
...
@@ -185,53 +173,49 @@ below.
...
@@ -185,53 +173,49 @@ below.
Step 1: Access responses from other students
Step 1: Access responses from other students
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. list-table::
.. warning::
:widths: 80
:header-rows: 1
* - To instructors:
Modify the content in this section according to
your course. For example, if your students can only grade by using the **Open
* - Modify the content in this section according to
Ended Console** page, change the introductory sentence below, and delete the
your course. For example, if your students can only grade by using the **Open
second and third bullets.
Ended Console** page, change the introductory sentence below, and delete the
second and third bullets.
**Note** *You can only grade a response if you've submitted a response to the
**Note** *You can only grade a response if you've submitted a response to the
question, an instructor has already graded at least 20 responses, and
question, an instructor has already graded at least 20 responses, and
there are more essays from other students left to grade. If you haven't submitted
there are more essays from other students left to grade. If you haven't submitted
a response or no responses are available for grading, you see a yellow message in the
a response or no responses are available for grading, you see a yellow message in the
interface.*
interface.*
.. image:: /Images/PAStudent_NoSubmissions.gif
.. image:: /Images/PAStudent_NoSubmissions.gif
There are several ways to access other students' responses, depending on
There are several ways to access other students' responses, depending on
the way that the course is set up.
the way that the course is set up.
- Through the **Open Ended Console** page. This option is always
- Through the **Open Ended Console** page. This option is always
available for every course. To access the **Open Ended Console** page,
available for every course. To access the **Open Ended Console** page,
click the **Open Ended Panel** tab at the top of any page in the course.
click the **Open Ended Panel** tab at the top of any page in the course.
When you see the list of problems that have responses available to grade,
When you see the list of problems that have responses available to grade,
click the name of the problem that you want to open it.
click the name of the problem that you want to open it.
.. image:: /Images/PGI_FromOEC_2Problems.gif
.. image:: /Images/PGI_FromOEC_2Problems.gif
- Through the courseware, in a specific unit. This option is only available if the
- Through the courseware, in a specific unit. This option is only available if the
instructor has included a peer grading interface for the problem in the body of
instructor has included a peer grading interface for the problem in the body of
the course. To access responses in the courseware, go to the unit that contains
the course. To access responses in the courseware, go to the unit that contains
the open response assessment problem. Scroll down past the response that you
the open response assessment problem. Scroll down past the response that you
submitted until you see the peer grading interface that appears below the problem.
submitted until you see the peer grading interface that appears below the problem.
.. image:: /Images/PGI_InUnitComposite.gif
.. image:: /Images/PGI_InUnitComposite.gif
- Through the courseware, in a separate section. This option may not be available
- Through the courseware, in a separate section. This option may not be available
for your course. If it is, you'll see the section for peer grading in the
for your course. If it is, you'll see the section for peer grading in the
course accordion on the left side of your screen. For example, MIT's 6.00x:
course accordion on the left side of your screen. For example, MIT's 6.00x:
Introduction to Computer Science and Programming course has a separate section
Introduction to Computer Science and Programming course has a separate section
that holds all the course peer grading interfaces. To access peer grading for
that holds all the course peer grading interfaces. To access peer grading for
a problem, you click the problem name.
a problem, you click the problem name.
.. image:: /Images/PGI_Multiple-600x.gif
.. image:: /Images/PGI_Multiple-600x.gif
...
@@ -242,9 +226,9 @@ Step 2: Learn to grade
...
@@ -242,9 +226,9 @@ Step 2: Learn to grade
Before you grade your peers' responses, you must learn to grade
Before you grade your peers' responses, you must learn to grade
the same way that an instructor would. In this process, called
the same way that an instructor would. In this process, called
*calibration*, you'll grade several responses that an instructor has already
*calibration*, you'll grade several responses that an instructor has already
graded. If your grading is similar to the instructor's, you can begin grading
graded. If your grading is similar to the instructor's, you can begin grading
other students' responses to the question.
other students' responses to the question.
#. Click the name of the problem. When the **Learning to grade** page
#. Click the name of the problem. When the **Learning to grade** page
opens, click **Start learning to grade**.
opens, click **Start learning to grade**.
...
@@ -274,22 +258,22 @@ Step 3: Grade responses
...
@@ -274,22 +258,22 @@ Step 3: Grade responses
^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^
When you grade a peer assessment response, you can not only select
When you grade a peer assessment response, you can not only select
options in the rubric, but also provide additional feedback for the
options in the rubric, but also provide additional feedback for the
student who submitted the response.
student who submitted the response.
#. When the response opens, select the options in the rubric that you
#. When the response opens, select the options in the rubric that you
feel best apply to the response, as you did in the calibration process.
feel best apply to the response, as you did in the calibration process.
If you have concerns about the response, you can select other
If you have concerns about the response, you can select other
options to flag the response for instructor review. You don't have to fill
options to flag the response for instructor review. You don't have to fill
out the rubric before you select these options.
out the rubric before you select these options.
- If you aren't sure how to grade the response, select the **I am unsure about
- If you aren't sure how to grade the response, select the **I am unsure about
the scores I have given above** check box.
the scores I have given above** check box.
- If the response is offensive, or if you suspect that it contains plagiarized
- If the response is offensive, or if you suspect that it contains plagiarized
material, select the **This submission has explicit, offensive, or (I suspect)
material, select the **This submission has explicit, offensive, or (I suspect)
plagiarized content** check box.
plagiarized content** check box.
#. Under **Written Feedback**, write a comment about the score that you
#. Under **Written Feedback**, write a comment about the score that you
gave the response.
gave the response.
...
@@ -310,50 +294,46 @@ to the **Peer Grading** page and click the name of the problem that you want
...
@@ -310,50 +294,46 @@ to the **Peer Grading** page and click the name of the problem that you want
to continue grading.
to continue grading.
.. note:: When a response opens for you to grade, it leaves the current "grading pool"
.. note:: When a response opens for you to grade, it leaves the current "grading pool"
that other instructors or students are grading from, which prevents other
that other instructors or students are grading from, which prevents other
instructors or students from
instructors or students from
grading the response while you are working on it. If you do not submit a score
grading the response while you are working on it. If you do not submit a score
for this response within 30 minutes, the response returns to the grading pool
for this response within 30 minutes, the response returns to the grading pool
(so that it again becomes available for others to grade), even if you still have
(so that it again becomes available for others to grade), even if you still have
the response open on your screen.
the response open on your screen.
If the response returns to the grading pool (because the 30 minutes have passed),
If the response returns to the grading pool (because the 30 minutes have passed),
but the response is still open on your screen, you can still submit feedback for
but the response is still open on your screen, you can still submit feedback for
that response. If another instructor or student grades the response after it returns to the
that response. If another instructor or student grades the response after it returns to the
grading pool but before you submit your feedback, the response receives two grades.
grading pool but before you submit your feedback, the response receives two grades.
If you click your browser's **Back** button to return to the problem list before you
If you click your browser's **Back** button to return to the problem list before you
click **Submit** to submit your feedback for a response, the response stays outside
click **Submit** to submit your feedback for a response, the response stays outside
the grading pool until 30 minutes have passed. When the response returns to the
the grading pool until 30 minutes have passed. When the response returns to the
grading pool, you can grade it.
grading pool, you can grade it.
.. _ORA AI Assessment:
.. _ORA AI Assessment:
Artificial Intelligence (AI) Assessment
Artificial Intelligence (AI) Assessment
---------------------------------------
---------------------------------------
..
list-table
::
..
warning
::
:widths: 80
:header-rows: 1
You can delete this section if your ORA problem doesn't use AI assessments.
* - To instructors:
* - You can delete this section if your ORA problem doesn't use AI assessments.
In an AI assessment, an instructor grades a sample set of student responses to the
In an AI assessment, an instructor grades a sample set of student responses to the
open response assessment problem. A machine learning algorithm then creates a model
open response assessment problem. A machine learning algorithm then creates a model
based on the instructor's scores and grades the remaining students' responses.
based on the instructor's scores and grades the remaining students' responses.
After you submit your response to an AI assessment, the following message appears under your
After you submit your response to an AI assessment, the following message appears under your
response.
response.
**Your response has been submitted. Please check back later for your grade.**
**Your response has been submitted. Please check back later for your grade.**
Depending on the time that it takes for the instructor to grade a sample set of
Depending on the time that it takes for the instructor to grade a sample set of
responses, you may receive your grade within minutes, or you may have to wait
responses, you may receive your grade within minutes, or you may have to wait
a few days. You won't receive a notification when your score is ready, so keep
a few days. You won't receive a notification when your score is ready, so keep
checking back.
checking back.
For more information about accessing your scores, see :ref:`ORA Access Scores`.
For more information about accessing your scores, see :ref:`ORA Access Scores`.
...
@@ -362,17 +342,13 @@ For more information about accessing your scores, see :ref:`ORA Access Scores`.
...
@@ -362,17 +342,13 @@ For more information about accessing your scores, see :ref:`ORA Access Scores`.
Access Scores and Feedback
Access Scores and Feedback
--------------------------
--------------------------
..
list-table
::
..
warning
::
:widths: 80
:header-rows: 1
Modify the text in this section to apply to your course.
* - To instructors:
* - Modify the text in this section to apply to your course.
For *self assessments*, the score that you give yourself appears as soon as you submit
For *self assessments*, the score that you give yourself appears as soon as you submit
the score.
the score.
For *peer assessments* and *AI assessments*, you'll access your scores through the **Open Ended Console** page.
For *peer assessments* and *AI assessments*, you'll access your scores through the **Open Ended Console** page.
...
@@ -386,12 +362,12 @@ For *peer assessments* and *AI assessments*, you'll access your scores through t
...
@@ -386,12 +362,12 @@ For *peer assessments* and *AI assessments*, you'll access your scores through t
see whether your responses have been graded. The status for each problem is
see whether your responses have been graded. The status for each problem is
either **Waiting to be Graded** or **Finished**.
either **Waiting to be Graded** or **Finished**.
#. If **Finished** appears in the **Status** column for the problem you want,
#. If **Finished** appears in the **Status** column for the problem you want,
click the name of the problem to see your score for that problem. When you
click the name of the problem to see your score for that problem. When you
click the name of the problem, the problem opens in the courseware.
click the name of the problem, the problem opens in the courseware.
For both AI and peer assessments, the score appears below your response
For both AI and peer assessments, the score appears below your response
in an abbreviated version of the rubric.
in an abbreviated version of the rubric.
.. image:: /Images/AIScoredResponse.gif
.. image:: /Images/AIScoredResponse.gif
...
@@ -415,14 +391,10 @@ For more information about grading peer assessments, see :ref:`ORA Peer Assessme
...
@@ -415,14 +391,10 @@ For more information about grading peer assessments, see :ref:`ORA Peer Assessme
Resubmitting a Response
Resubmitting a Response
-----------------------
-----------------------
..
list-table
::
..
warning
::
:widths: 80
:header-rows: 1
You can delete this section if you don't allow students to submit multiple responses.
* - To instructors:
* - You can delete this section if you don't allow students to submit multiple responses.
Some open response assessments allow multiple attempts. For these
Some open response assessments allow multiple attempts. For these
problems, a **New Submission** button appears below your original
problems, a **New Submission** button appears below your original
response.
response.
...
@@ -430,5 +402,3 @@ response.
...
@@ -430,5 +402,3 @@ response.
If you want to answer the question again, click **New Submission** to
If you want to answer the question again, click **New Submission** to
clear your former response, and click **OK** in the dialog box that
clear your former response, and click **OK** in the dialog box that
appears. You can then enter a new response for the problem.
appears. You can then enter a new response for the problem.
\ No newline at end of file
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment