Skip to content

E
edx-ora2
Commit 5f1184ae by srpearce
Options

Merge pull request #394 from edx/sylvia/docs/TIM-597 

Add student training assessment information for course authors
parents b08f8119 dc548c61
Showing with 255 additions and 65 deletions
apps/openassessment/templates/openassessmentblock/student_training/student_training.html
......@@ -64,7 +64,7 @@
<div class="step__content">
<article class="student-training__display" id="student-training">
<header class="student-training__display__header">
<h3 class="student-training__display__title">{% trans "Training Essay #" %}
<h3 class="student-training__display__title">{% trans "Training Assessment #" %}
<span class="student-training__number--current">{{ training_num_current }}</span> of
<span class="student-training__number--required">{{ training_num_available }}</span>
</h3>
......
apps/openassessment/xblock/defaults.py
......@@ -64,7 +64,16 @@ DEFAULT_RUBRIC_FEEDBACK_PROMPT = """
(Optional) What aspects of this response stood out to you? What did it do well? How could it improve?
"""
DEFAULT_EXAMPLE_ANSWER = "Replace this text with a sample response for this assignment. You'll assess this sample response in the courseware, and students will then learn to assess responses by assessing this response and comparing the options that they select in the rubric with the options that you selected."
DEFAULT_EXAMPLE_ANSWER = """
(optional) Replace this text with your own sample response for this assignment. Below, list the names of the criteria for this assignment, and then specify the name of the option that you would select for this response. Students will learn to assess responses by assessing this response and comparing the rubric options that they select with the rubric options that you specified.
If you don't want to provide sample responses and scores, delete the entire 'assessment name="student-training"' element.
"""
DEFAULT_EXAMPLE_ANSWER_2 = """
(optional) Replace this text with another sample response, and then specify the options that you would select for this response below. To provide more sample responses, copy an "example" element and paste as many as you want before the closing "assessment" tag.
"""
DEFAULT_STUDENT_TRAINING = {
"name": "student-training",
"start": None,
......@@ -84,7 +93,7 @@ DEFAULT_STUDENT_TRAINING = {
]
},
{
"answer": DEFAULT_EXAMPLE_ANSWER,
"answer": DEFAULT_EXAMPLE_ANSWER_2,
"options_selected": [
{
"criterion": "Ideas",
......
docs/en_us/course_authors/source/PeerAssessment.rst
.. _Peer Assessments:
########################
Peer Assessments
########################
#########################
Open Response Assessments
#########################
********************************
Introduction to Peer Assessments
********************************
*****************************************
Introduction to Open Response Assessments
*****************************************
.. note:: **The peer assessment feature is in limited release.** If you're at an edX consortium university and you plan to include peer assessments in a MOOC, you'll work with your edX project manager (PM) to enable the peer assessment feature and create peer assessment assignments.
.. note:: **The open response assessments feature is in limited release.** If you're at an edX consortium university and you plan to include open response assessments in a MOOC, you'll work with your edX project manager (PM) to enable the open response assessments feature and create assignments.
Peer assessments allow instructors to assign questions that may not have definite answers. Students submit a response to a question, and then compare their peers' responses to a rubric that you create. Peer assessment problems also include self assessments, in which students compare their own responses to the rubric.
Open response assessments allow instructors to assign questions that may not have definite answers. Students submit a response to the question, and then that student and the student's peers compare the response to a rubric that you create.
Open response assessments include peer assessments and self assessments. In peer assessments, students compare their peers' responses to a rubric that you create. In self assessments, students compare their own responses to the rubric.
For more information, see the following sections:
......@@ -21,15 +23,15 @@ For more information, see the following sections:
.. _PA Elements:
********************************
Elements of a Peer Assessment
********************************
*****************************************
Elements of an Open Response Assessment
*****************************************
When you create a peer assessment problem, you'll specify several elements:
When you create an open response assessment problem, you include several elements:
* The number of responses and assessments.
* The assessment type or types.
* The due dates for each step (optional).
* One or more assessment types. Assessment types include **student training**, **peer**, and **self**.
* (Optional) The due dates for each step.
* The question.
* The rubric.
......@@ -51,15 +53,15 @@ For more information, see :ref:`PA Specify Name and Assessment Types`.
Assessment Type
=====================
In your problem, you'll also specify the **assessment type or types**. You can see the type and order of the assessments when you look at the problem. In the following example, after students submit a response, they complete peer assessments on other students' responses ("Assess Peers") and then complete self assessments ("Assess Yourself").
In your problem, you'll also specify the **assessment type or types**. You can see the type and order of the assessments when you look at the problem. In the following example, after students submit a response, they complete peer assessments on other students' responses ("Assess Peers") and then complete self assessments ("Assess Your Response").
.. image:: /Images/PA_AsmtWithResponse.png
:alt: Image of peer assessment with assessment steps and status labeled
:width: 600
You can set the assignment to include a peer assessment only, a self assessment only, or a peer assessment followed by a self assessment.
You can set the assignment to include a peer assessment only, a self assessment only, or both a peer assessment and a self assessment. You can also include a student training assessment that students will complete before they perform peer and self assessments. Student training assessments contain sample responses and scores that you create. They help students learn to grade their peers' responses.
For more information, see :ref:`PA Specify Name and Assessment Types`.
For more information, see :ref:`PA Specify Name and Assessment Types` and :ref:`PA Student Training Assessments`.
===================================
Start and Due Dates (optional)
......@@ -97,7 +99,7 @@ Rubrics are made of *criteria* and *options*.
* Each criterion has a *name*, a *prompt*, and two or more *options*.
* The name is a very short summary of the criterion, such as Ideas or Content. Criterion names generally have just one word. **The name for each criterion must be unique.** The system uses the criterion name for identification. Criterion names do not appear in the rubric that students see when they are completing peer assessments, but they do appear on the page that shows the student's final grade.
* The name is a very short summary of the criterion, such as Ideas or Content. Criterion names generally have just one word. Because the system uses criteria names for identification, **the name for each criterion must be unique.** Criterion names do not appear in the rubric that students see when they are completing peer assessments, but they do appear on the page that shows the student's final grade.
.. image :: /Images/PA_CriterionName.png
:alt: A final score page with call-outs for the criterion names
......@@ -109,7 +111,7 @@ Rubrics are made of *criteria* and *options*.
.. image:: /Images/PA_Rubric_LMS.png
:alt: Image of a rubric in the LMS with call-outs for the criterion prompt and option names, explanations, and points
You can see both criterion and option names when you access assignment information for an individual student. For more information, see :ref:`Accessing PA Information`.
You can see both criterion and option names when you access assignment information for an individual student. For more information, see :ref:`PA Accessing Assignment Information`.
.. image:: /Images/PA_Crit_Option_Names.png
......@@ -153,6 +155,53 @@ Note that different criteria in the same assignment can have different numbers o
For more information, see :ref:`PA Add Rubric`.
.. _PA Student Training Assessments:
========================================
Student Training Assessments (optional)
========================================
When you create a peer assessment assignment, you can create one or more student training assessments to help students learn to perform their own assessments. A student training assessment contains one or more sample responses that you write, together with the scores that you would give the sample responses. Students review these responses and try to score them the way that you scored them.
In a student training assessment, the **Learn to Assess Responses** step opens immediately after a student submits a response. The student sees one of the sample responses that you created, along with the rubric. The scores that you gave the response do not appear. The student also sees the number of sample responses that he or she will assess.
.. image:: Images/PA_TrainingAssessment.png
:alt: Sample training response, unscored
:width: 500
The student selects an option for each of the assignment's criteria, and then clicks **Compare your selections with the instructor's selections**. If all of the student's selections match the instructor's selections, the next sample response opens automatically.
If any of the student's selections differs from the instructor's selections, the student sees the response again, and the following message appears above the response:
.. code-block:: xml
Learning to Assess Responses
Your assessment differs from the instructor's assessment of this response. Review the
response and consider why the instructor may have assessed it differently. Then, try
the assessment again.
For each of the criteria, the student sees one of the following two messages, depending on whether the student's selections matched those of the instructor:
.. code-block:: xml
Selected Options Differ
The option you selected is not the option that the instructor selected.
.. code-block:: xml
Selected Options Agree
The option you selected is the option that the instructor selected.
For example, the following student chose one correct option and one incorrect option.
.. image:: /Images/PA_TrainingAssessment_Scored.png
:alt: Sample training response, scored
:width: 500
The student continues to try scoring the sample response until the student's scoring for all criteria matches the instructor's scoring.
For more information, see :ref:`PA Add a Student Training Assessment`.
.. _PA Scoring:
***********************
......@@ -232,9 +281,16 @@ Step 1. Create the Component
#. Under **Add New Component**, click **Advanced**, and then click **Peer Assessment**.
#. In the Problem component that appears, click **Edit**.
The component editor opens, and you can see sample code that includes the assignment's title, the assessment type or types, the number of assessments that students must complete, a sample question ("prompt"), and a rubric. You'll replace this sample content with the content for your problem in the next steps.
When the component editor opens, you can see sample code that includes the following. You'll replace this sample content with the content for your problem:
* The assignment's title.
* The training responses for the assignment.
* The assessment type or types.
* The number of assessments that students must complete.
* A sample question ("prompt").
* A sample rubric.
Note that you won't use the **Settings** tab in the component editor when you create peer assessments.
Note that you won't use the **Settings** tab in the component editor when you create peer assessments.
.. _PA Specify Name and Assessment Types:
......@@ -242,23 +298,35 @@ Step 1. Create the Component
Step 2. Specify the Problem Name and Assessment Types
========================================================
To specify the name and assessment types for the problem, you'll work with the XML at the top of the problem.
To specify the name and assessment types for the problem, you'll work with the XML near the top of the problem.
Locate the following XML near the top of the component editor:
In the component editor, locate the following XML:
.. code-block:: xml
<openassessment>
<title></title>
<assessments>
<assessment name="student-training">
<example>
<answer>Replace this text with a sample response for this assignment. You'll assess this sample response in the courseware, and students will then learn to assess responses by assessing this response and comparing the options that they select in the rubric with the options that you selected.</answer>
<select criterion="Ideas" option="Fair"/>
<select criterion="Content" option="Good"/>
</example>
<example>
<answer>Replace this text with a sample response for this assignment. You'll assess this sample response in the courseware, and students will then learn to assess responses by assessing this response and comparing the options that they select in the rubric with the options that you selected.</answer>
<select criterion="Ideas" option="Poor"/>
<select criterion="Content" option="Good"/>
</example>
</assessment>
<assessment name="peer-assessment" must_grade="5" must_be_graded_by="3"/>
<assessment name="self-assessment"/>
</assessments>
This code specifies four elements:
This code includes several elements:
* **The title of the assignment**. In this example, because there is no text between the ``<title>`` tags, the assignment does not have a specified title.
* **The type and order of the assessments**. This information is in the **name** attribute in the two ``<assessment>`` tags. Assessments run in the order in which they're listed. In this example, the peer assessment runs, and then the student performs a self assessment.
* **The type and order of the assessments**. This information is in the **name** attribute in the ``<assessment>`` tags. Assessments run in the order in which they're listed. In this example, students complete the student training assessment, the peer assessment, and the self assessment, in that order.
* **Two sample responses for student training**, together with the options that you select for each of the criteria for the assignment. This information is between the two sets of ``<example> </example>`` tags. Step-by-step instructions for creating student training responses appear in :ref:`PA Add a Student Training Assessment`.
* **The number of responses that each student must assess** (for peer assessments). This information is in the **must_grade** attribute in the ``<assessment>`` tag for the peer assessment. In this example, each student must grade five peer responses before he receives the scores that his peers have given him.
* **The number of peer assessments each response must receive** (for peer assessments). This information is in the **must_be_graded_by** attribute in the ``<assessment>`` tag for the peer assessment. In this example, each response must receive assessments from three students before it can return to the student who submitted it.
......@@ -266,7 +334,9 @@ To specify the name and assessment types, follow these steps.
#. Between the ``<title>`` tags, add a name for the problem.
#. Specify the type of assessments you want students to complete.
#. Specify the type of assessments you want students to complete. Assessments run in the order in which they're listed.
.. note:: If you include both peer and self assessments, the peer assessment must precede the self assessment. If you include a student training assessment, the student training assessment must precede the peer and self assessments. You can also include a student training assessment paired with either a peer assessment only or a self assessment only.
- If you want students to complete a peer assessment only, delete the ``<assessment name="self-assessment"/>`` tag.
......@@ -274,10 +344,53 @@ To specify the name and assessment types, follow these steps.
- If you want students to complete a peer assessment and then a self assessment, leave the default tags.
- If you include a student training assessment, make sure you add the ``<assessment name="student-training">`` tag *before* the ``<assessment name="peer-assessment">`` and ``<assessment name="self-assessment">`` tags.
#. If your students will complete a peer assessment, replace the values for **must_grade** and **must_be_graded_by** in the ``<assessment name="peer-assessment">`` tag with the numbers that you want.
.. note:: The value for **must_grade** must be greater than or equal to the value for **must_be_graded_by**.
.. _PA Add a Student Training Assessment:
========================================================
Step 3. Include a Student Training Assessment (optional)
========================================================
To include a student training assessment, which contains both sample responses and scores, you'll work with the following XML:
.. code-block:: xml
<assessment name="student-training">
<example>
<answer>Replace this text with a sample response for this assignment. You'll assess this sample response in the courseware, and students will then learn to assess responses by assessing this response and comparing the options that they select in the rubric with the options that you selected.</answer>
<select criterion="Ideas" option="Fair"/>
<select criterion="Content" option="Good"/>
</example>
<example>
<answer>Replace this text with a sample response for this assignment. You'll assess this sample response in the courseware, and students will then learn to assess responses by assessing this response and comparing the options that they select in the rubric with the options that you selected.</answer>
<select criterion="Ideas" option="Poor"/>
<select criterion="Content" option="Good"/>
</example>
</assessment>
.. note:: If you don't want to include a student training assessment, delete all of this XML.
This code includes several elements:
* The ``<assessment name="student-training">`` tag indicates that this assessment is a student training assessment.
* Each set of ``<example>`` tags contains one set of ``<answer>`` tags and two or more ``<select/>`` tags.
* The set of ``<answer>`` tags contains the text of a sample response that you've created.
* Each ``<select/>`` tag contains the name of one of the assignment's criteria, as well as the option that you select for the criterion. (For more information about criteria and options, see :ref:`PA Rubric`.)
To add student training responses and scores:
#. Replace the placeholder text between the ``<answer>`` tags with the text of your response. To include paragraph breaks, include a blank line between paragraphs. You don't have to add any other formatting tags to include paragraph breaks.
#. Replace the criterion name in each ``<select/>`` tag with the name of one of the criteria in your assignment. To add more criteria, copy and paste as many ``<select/>`` tags as you need. You must include one ``<select/>`` tag for each of the assignment's criteria.
#. In the ``<select/>`` tag for each criterion, replace the placeholder option name with the name of the option that you would select for the sample response.
#. Copy and paste as many sets of ``<example>`` tags as you need to cover all the criteria for your assignment.
For more information, see :ref:`PA Student Training Assessments`.
.. _PA Add Due Dates:
......
docs/en_us/course_authors/source/PeerAssessment_Students.rst
.. _PA for Students:
#############################
Peer Assessments for Students
#############################
###########################################
Open Response Assessments for Students
###########################################
You may want to let your students know what to expect when they complete peer assessments. This guide walks students through each step of the process.
You may want to let your students know what to expect when they complete open response assessments. This guide walks students through each step of the process.
********************
**************************************************
Student Introduction to Open Response Asssessments
**************************************************
In an open response assessment, you'll provide a response to a question that may not have a simple or definitive answer. Some open response assessments have asked students to submit written responses, videos of speeches, and computer code.
Open response assessments may include a peer assessment, a self assessment, or both. With a peer assessment, you'll assess, or grade, responses that several of your peers have submitted, and several of your peers will assess your response. With a self assessment, you'll assess your own response. To assess a response, you'll compare the response to a rubric that the instructor provides.
A *rubric* is a list of expectations that a response should meet. Rubrics are made of *criteria* and *options*. *Criteria* describe characteristics that the response should have, such as topics the response should cover. The *options* for each of the criteria describe how well the response satisfies the criteria. In the following image, you can see a rubric with two criteria. Each of the criteria has several options.
.. image:: /Images/PA_S_Rubric.png
:alt: Rubric showing criteria and options
:width: 500
When you assess a response, you'll select the option that best describes the response for each of the criteria.
************************
Student Instructions
********************
************************
Peer assessment problems have several steps--you'll submit a response, then assess (grade) peer responses, and then assess your own response. When you come to a peer assessment problem in the course, you'll see the question and a response field, followed by grayed-out steps that show you what you'll do next.
When you come to an open response assessment in the course, you'll see the question and a response field. After you submit your response, you'll assess some of your peers' responses, your own response, or both, depending on the assignment. You can see the steps that your assignment includes below the response field.
.. image:: /Images/PA_AsmtWithResponse_Students.png
:alt: Peer assessment example with question, response field, and assessment types and status labeled
.. image:: /Images/PA_S_AsmtWithResponse.png
:alt: Open response assessment example with question, response field, and assessment types and status labeled
:width: 550
Here, we'll walk you through the process of performing a peer assessment:
Here, we'll walk you through the process of completing an open response assessment that includes a peer assessment and a self assessment:
#. Submit your response to a question.
#. Assess, or grade, responses that other students have submitted.
#. Learn to assess responses.
#. Assess responses that other students have submitted.
#. Assess your own response to the question.
#. Receive your score and provide feedback on the peer assessment.
At any time during the assessment, you can see your status at the bottom of the page under **Your Grade**. A message tells you the steps that you still have to perform before you can receive your grade. For example, you may see the following message:
Not Completed
You have not completed the Peer Assessment step and Self Assessment step of this problem.
.. code-block:: xml
Not Completed
You have not completed the peer assessment step and self assessment step of this problem.
=====================
Submit Your Response
********************
=====================
Read the question carefully. Some instructors include important information in the question, such as how long your response must be or specific topics your response must cover.
......@@ -39,13 +57,16 @@ Read the question carefully. Some instructors include important information in t
After you compose a response, type it into the response field under **Your Response**, and then click **Submit your response and move to the next step**. If you can't finish your response all at once, you can click **Save Your Progress** to save a draft of your response, and then come back and submit it later.
After you submit your response, if no other students have submitted responses yet, you'll see the following message:
After you submit your response, if other students have already submitted responses, the peer assessment step starts immediately. However, you don't have to start grading right away. If you want to stop working and come back later, just refresh or reopen your browser when you come back. New peer responses will be available for you to grade.
**Waiting for Peer Responses**
If no other students have submitted responses yet, you'll see the following message:
**All submitted peer responses have been assessed. Check back later to see if more students have submitted responses. You'll receive your grade after you've completed all the steps for this problem and your peers have assessed your response.**
.. code-block:: xml
If other students have already submitted responses, the peer assessment step starts immediately after you submit your response. However, you don't have to start grading right away. If you want to stop working and come back later, just refresh or reopen your browser when you come back. New peer responses will be available for you to grade.
Waiting for Peer Responses
All submitted peer responses have been assessed. Check back later to see if more students
have submitted responses. You'll receive your grade after you complete the peer assessment
and self assessment steps, and after your peers have assessed your response.
Note that you can view your response at any time after you submit it. To do this, click the **Your Response** heading to expand the response field.
......@@ -53,43 +74,89 @@ Note that you can view your response at any time after you submit it. To do this
:alt: Image of the Response field collapsed and then expanded
:width: 550
============================
Learn to Assess Responses
============================
Assess Peer Responses
**********************
In this step, you'll learn to assess responses effectively by reviewing and assessing sample responses that the instructor has provided. You'll try to select the same options for the response that the instructor selected.
When peer assessment starts, you'll see the original question followed by another student's response and a rubric. Above the rubric you can also see how many responses you'll assess, how many you've already assessed, and the amount of time you can expect to spend on each assessment. Below the rubric is a field where you can provide comments about the student's response.
.. note:: Not all instructors provide sample responses for training. If the instructor doesn't provide sample responses, this step won't appear in the assignment.
The rubric contains *criteria*, each of which has several *options*. Read the student's response and the rubric carefully, and then select the option that you think best reflects the student's response.
After you submit your response, one of the sample responses opens, together with the rubric for the assignment. Read the sample response and the rubric carefully, select the options that you think best reflect the response, and then click **Compare your selections with the instructor's selections**.
.. image:: /Images/PA_LMS-PARubric.png
:alt: Image of the question, response, and rubric
:width: 550
If all of your selections match the instructor's selections, the next sample response opens automatically.
If any your selections doesn't match the instructor's selections, you'll see the response again, and the following message appears above the response:
.. code-block:: xml
Learning to Assess Responses
Your assessment differs from the instructor's assessment of this response. Review the
response and consider why the instructor may have assessed it differently. Then, try
the assessment again.
For each of the criteria, you'll see one of the following two messages, depending on whether your selections matched those of the instructor:
When you've selected options in the rubric, provide additional comments about the response in the field below the rubric. When you're done, click **Submit your assessment and move to response #<number>**.
.. code-block:: xml
Selected Options Differ
The option you selected is not the option that the instructor selected.
.. code-block:: xml
Selected Options Agree
The option you selected is the option that the instructor selected.
In the following example, the student chose one correct option and one incorrect option.
.. image:: /Images/PA_TrainingAssessment_Scored.png
:alt: Sample training response, scored
:width: 500
You'll continue to assess the sample response until the options you select for all criteria match the options the instructor selected.
When you've successfully assessed all of the sample responses, you'll move to the next step in the assignment.
=====================
Assess Peer Responses
=====================
When peer assessment starts, you'll see the original question, another student's response, and the rubric for the assignment. Above the response you can see how many responses you'll assess and how many you've already assessed.
.. image:: /Images/PA_S_PeerAssmt.png
:alt: In-progress peer assessment
:width: 500
You'll assess these responses by selecting options in the rubric, the same way you assessed the sample responses in the "learn to assess responses" step. Additionally, this step has a field below the rubric where you can provide comments about the student's response.
.. note:: Some assessments may have an additional **Comments** field for one or more of the assessment's individual criteria. You can enter up to 300 characters in these fields. In the following image, the first of the criteria has a separate **Comments** field, but the second does not.
.. image:: /Images/PA_Students_CommentBoxes.png
.. image:: /Images/PA_S_CommentBoxes.png
:alt: Rubric with call-outs for comment boxes
:width: 500
When you submit your assessment of the first student's response, another response opens for you. Assess this response in the same way that you assessed the first response, and then submit your assessment. You'll repeat these steps until you've assessed the required number of responses. The number in the upper-right corner of the rubric updates as you assess each response.
After you've selected options in the rubric and provided additional comments about the response in this field, click **Submit your assessment and move to response #<number>**.
When you submit your assessment of the first student's response, another response opens for you. Assess this response in the same way that you assessed the first response, and then submit your assessment. You'll repeat these steps until you've assessed the required number of responses. The number in the upper-right corner of the step is updated as you assess each response.
=====================
Assess Your Response
********************
=====================
When you've completed enough peer assessments, your self assessment opens. You'll see your response along with the same rubric that you used in the peer assessment step. Assess your response, and then click **Submit Your Assessment**.
==========================================
Receive Your Score and Provide Feedback
****************************************
==========================================
After you submit your self assessment, if other students are still assessing your response, you'll see the following message under the **Assess Your Response** step.
**Your Grade: Waiting for Peer Assessment**
.. code-block:: xml
Your Grade: Waiting for Peer Assessment
**Your response is still undergoing peer assessment. After your peers have assessed your response, you'll see their feedback and receive your final grade.**
Your response is still undergoing peer assessment. After your peers have assessed your
response, you'll see their feedback and receive your final grade.
If you see this message, keep checking back periodically until peer assessment is complete.
......@@ -101,8 +168,9 @@ When peer assessment is complete, you can see the scores you received from all o
If you want to, you can provide feedback on the scores that you received under **Provide Feedback on Peer Assessments**.
=================================================
Assess Additional Peer Responses (optional)
*******************************************
=================================================
If you've assessed the required number of peer responses and completed your self assessment, you can assess additional peer responses. To do this, click the **Assess Peers** heading. If any responses remain to be assessed, a new response opens.
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment