Skip to content

E
edx-ora2
Commit e2151c65 by srpearce
Options

Merge pull request #573 from edx/sylvia/docs/ORA-65 

Sylvia/docs/ora 65
parents 80017a21 6d1b09a3
Showing with 394 additions and 511 deletions
docs/en_us/course_authors/source/CreatePeerAssessment.rst 0 → 100644
.. _PA Create a PA Assignment:
#############################################
Create an Open Response Assessment Assignment
#############################################
Creating an open response assessment is a multi-step process:
* :ref:`PA Create Component`
* :ref:`PA Add Prompt`
* :ref:`PA Add Rubric`
* :ref:`PA Specify Name and Dates`
* :ref:`PA Select Assignment Steps`
* :ref:`PA Specify Step Settings`
* :ref:`PA Show Top Responses`
* :ref:`PA Test Assignment`
Each of these steps is covered in detail below.
For more information about the components of an open response assessment, see :ref:`Peer Assessments`.
.. _PA Create Component:
******************************
Step 1. Create the Component
******************************
To create the component for your open response assessment:
#. In Studio, open the unit where you want to create the open response assessment.
#. Under **Add New Component**, click **Problem**, click the **Advanced** tab, and then click **Peer Assessment**.
#. In the Problem component that appears, click **Edit**.
.. _PA Add Prompt:
******************************
Step 2. Add the Prompt
******************************
To add the prompt, or question:
#. In the open response assessment component editor, click the **Prompt** tab.
#. Add the text of your question in the text field. Replace any default text, if necessary.
========================================
Add Formatting or Images to the Prompt
========================================
Currently, you cannot add text formatting or images inside the Peer Assessment component. If you want to include formatting or images in the text of your prompt, you can add an HTML component that contains your text above the Peer Assessment component and leave the text field in the **Prompt** tab blank. The instructions for the peer assessment still appear above the **Your Response** field.
.. image:: /Images/PA_HTMLComponent.png
:alt: A peer assessment that has an image in an HTML component
:width: 500
.. _PA Allow Images:
============================================
Allow Students to Submit Images (optional)
============================================
To allow students to submit an image with a response:
#. In the component editor, click the **Settings** tab.
#. Next to **Allow Image Responses**, select **True**.
.. note::
* Currently, course teams cannot see any of the images that students submit. Images are not visible in the body of the assignment in the courseware, and they are not included in the course data package.
* You can allow students to upload an image, but you cannot require it.
* Students can only submit one image with a response.
* All responses must contain text. Students cannot submit a response that contains only an image.
.. _PA Add Rubric:
******************************
Step 3. Add the Rubric
******************************
In this step, you'll add your rubric and provide your students with feedback options.
For each step below, replace any default text with your own text.
.. note:: All open response assessments include a feedback field below the rubric so that students can provide written feedback on a peer's overall response. You can also allow or require students to provide feedback for individual criteria. See step 2.4 below for instructions. For more information, see :ref:`Feedback Options`.
To add the rubric:
#. In the open response assessment component editor, click the **Rubric** tab.
#. In the first **Criterion** section, enter the name and prompt text of your first criterion.
#. In the first **Option** section, enter the name, explanation, and point value for the first option.
#. In the next **Option** section, enter the name, explanation, and point value for the next option.
#. Repeat step 4 for each option. If you need to add more options, click **Add Option**.
#. Next to **Feedback for This Criterion**, select a value in the drop-down list.
* If you don't want students to provide feedback for this individual criterion, select **None**.
* If you want to require students to provide feedback, select **Required**.
* If you want to allow students to provide feedback, but not require it, select **Optional**.
7. Follow the instructions in steps 2-6 to add your remaining criteria. If you need to add more criteria, click **Add Criterion** at the end of the list of criteria.
#. Include instructions for students to provide overall written feedback on their peers' responses. You can leave the default text in the **Feedback Instructions** field or replace it with your own text.
.. _PA Criteria Comment Field Only:
==========================================================
Provide Only Comment Fields for Individual Criteria
==========================================================
When you add a comment field to a criterion, the comment field appears below the options for the criterion. You can also provide a comment field, but no options.
In the following image, the first criterion has a comment field but no options. The second includes options, but does not have a comment field.
.. image:: /Images/PA_0_Option_Criteria.png
To provide a comment field without options:
#. In the criterion, click **Remove** to remove, or delete, all the options.
#. Next to **Feedback for This Criterion**, select **Required** in the drop-down list.
.. _PA Specify Name and Dates:
************************************************************
Step 4. Specify the Assignment Name and Response Dates
************************************************************
To specify a name for the assignment as well as start and due dates for all student responses:
#. In the component editor, click the **Settings** tab.
#. Next to **Display Name**, type the name you want to give the assignment.
#. Next to **Response Start Date** and **Response Start Time**, enter the date and time when you want students to be able to begin submitting responses. Note that all times are in Universal Coordinated Time (UTC).
#. Next to **Response Due Date** and **Response Due Time**, enter the date and time by which all student responses must be submitted. Note that all times are in Universal Coordinated Time (UTC).
.. note:: We recommend that you set the response due date and time at least two days before the peer assessment due date and time. If the response due time and peer assessment due time are close together, and a student submits a response just before responses are due, other students may not have time to perform peer assessments before peer assessments are due.
.. _PA Select Assignment Steps:
****************************************
Step 5. Select Assignment Steps
****************************************
Open response assessment assignments can include student training, peer assessment, and self assessment steps. You can include a peer assessment step before a self assessment step and vice versa.
If you include a student training step, you **must** include a peer assessment step. You can also include a self assessment step. The student training step must come before both the peer assessment and the self assessment step.
To add steps to the assignment:
#. In the component editor, click the **Settings** tab.
#. Scroll down past the **Allow Image Responses** field.
#. Locate the following headings:
* **Step: Student Training**
* **Step: Peer Assessment**
* **Step: Self Assessment**
Select the check boxes for the steps that you want the assignment to include.
#. (optional) If you want to change the order of the steps, drag the steps into the order that you want. If you include a student training step, make sure it is the first step in the assignment.
.. _PA Specify Step Settings:
******************************
Step 6. Specify Step Settings
******************************
After you select the steps that you want, you'll specify settings for those steps.
.. note:: If you make changes to a step, but then you clear the check box for that step, the step will no longer be part of the assignment and your changes will not be saved.
.. _PA Student Training Step:
========================
Student Training
========================
For the student training step, you'll enter one or more responses that you have created, then select an option for each criterion in your rubric.
.. note:: You must enter your complete rubric on the **Rubric** tab before you can select options for the student training responses. If you later change one of your criteria or any of its options, you'll also have to update the student training step.
To add and score student training responses:
#. Under **Step: Student Training**, locate the first **Scored Response** section.
#. In the **Response** field, enter the text of your example response.
#. Under **Response Score**, select the option that you want for each criterion.
For more information, see :ref:`PA Student Training Assessments`.
============================
Peer Assessment
============================
For the peer assessment step, you'll specify the number of responses that each student must grade, the number of students that must grade each response, and start and due dates. All fields are required.
To specify peer assessment settings:
#. Locate the **Step: Peer Assessment** heading.
#. Next to **Must Grade**, enter the number of responses that each student must grade.
#. Next to **Graded By**, enter the number of students that must grade each response.
#. Next to **Start Date** and **Start Time**, enter the date and time when students can begin assessing their peers' responses. All times are in Universal Coordinated Time (UTC).
#. Next to **Due Date** and **Due Time**, enter the date and time by which all peer assessments must be complete. All times are in UTC.
============================
Self Assessment
============================
For the self assessment step, you'll specify when the step starts and ends.
#. Locate the **Step: Self Assessment** heading.
#. Next to **Start Date** and **Start Time**, enter the date and time when students can begin assessing their peers' responses. All times are in Universal Coordinated Time (UTC).
#. Next to **Due Date** and **Due Time**, enter the date and time by which all peer assessments must be complete. All times are in UTC.
.. _PA Show Top Responses:
******************************
Step 7. Show Top Responses
******************************
To allow students to see the top-scoring responses for the assignment, you'll specify a number on the **Settings** tab.
#. In the component editor, click the **Settings** tab.
#. In the **Top Responses** field, specify the number of responses that you want to appear in the **Top Responses** section below the student's final score. If you don't want this section to appear, set the number to 0. The maximum number is 100.
.. note:: Because each response can be up to 300 pixels in height, we recommend that you set this number to 20 or lower to prevent the page from becoming too long.
For more information, see :ref:`PA Top Responses`.
.. _PA Test Assignment:
******************************
Step 8. Test the Assignment
******************************
To test your assignment, set up the assignment in your course, set the section or subsection date in the future, and ask a group of beta users to submit responses and grade each other. The beta testers can then let you know if they found the question and the rubric easy to understand or if they had any problems with the assignment.
For more information about beta testing, see :ref:`Beta_Testing`.
docs/en_us/course_authors/source/PeerAssessment.rst
......@@ -12,7 +12,9 @@ Open response assessments allow instructors to assign questions that may not hav
Open response assessments include peer assessments and self assessments. In peer assessments, students compare their peers' responses to a rubric that you create. In self assessments, students compare their own responses to the rubric.
For more information, see the following sections:
In open response assessments, students usually only see their own responses and any peer responses they assess. You can also allow students to see the top-scoring responses that their peers have submitted. For more information, see :ref:`PA Top Responses`.
For more information about creating open response assessments, including step-by-step instructions, see the following sections:
* :ref:`PA Elements`
* :ref:`PA Scoring`
......@@ -21,98 +23,80 @@ For more information, see the following sections:
.. _PA Elements:
*****************************************
==========================================
Elements of an Open Response Assessment
*****************************************
==========================================
When you create an open response assessment assignment, you include several elements:
* The number of responses and assessments.
* One or more assessment types. Assessment types include **student training**, **peer**, and **self**.
* (Optional) The due dates for each step.
* The question.
* The prompt, or question.
* The rubric.
* One or more assessment steps. Assignments can include a student training step, a peer assessment step, and a self assessment step.
For step-by-step instructions, see :ref:`PA Create a PA Assignment`.
=======================================
Number of Responses and Assessments
=======================================
In the assignment code, you'll indicate the **number of responses** each student has to assess and the **number of peer assessments** each response has to receive.
.. note:: Because some students may submit a response but not complete peer assessments, some responses may not receive the required number of assessments. To increase the chance that all responses will receive enough assessments, you must set the number of responses that students have to assess to be higher than the number of assessments that each response must undergo. For example, if you require each response to receive three assessments, you could require each student to assess five responses.
If all responses have received assessments, but some students haven't completed the required number of peer assessments, those students can assess responses that other students have already assessed. The student who submitted the response sees the additional peer assessments when he sees his score. However, the additional peer assessments do not count toward the score that the response receives.
For more information, see :ref:`PA Specify Name and Assessment Types`.
.. note:: If you include a student training step, you must also add a peer assessment step. The student training step must be the first step.
=====================
Assessment Type
=====================
In your assignment, you'll also specify the **assessment type or types**. You can see the type and order of the assessments when you look at the assignment. In the following example, after students submit a response, they complete peer assessments on other students' responses ("Assess Peers") and then complete self assessments ("Assess Your Response").
.. image:: /Images/PA_AsmtWithResponse.png
:alt: Image of peer assessment with assessment steps and status labeled
:width: 600
For step-by-step instructions for creating an open response assessment, see :ref:`PA Create a PA Assignment`.
You can set the assignment to include a peer assessment only, a self assessment only, or both a peer assessment and a self assessment. You can also include a student training assessment that students will complete before they perform peer and self assessments. Student training assessments contain sample responses and scores that you create. They help students learn to grade their peers' responses.
************************
Prompt
************************
For more information, see :ref:`PA Specify Name and Assessment Types` and :ref:`PA Student Training Assessments`.
The **prompt**, or question that you want your students to answer, appears near the top of the page, followed by a field where the student enters a response. You can require your students to enter text as a response, or you can allow your students to both enter text and upload an image.
===================================
Start and Due Dates (optional)
===================================
You can specify **start dates** and **due dates** for students to submit responses, perform peer assessments, and perform self assessments.
.. image:: /Images/PA_QandRField.png
:width: 500
:alt: ORA question and blank response field
You can set different dates for each step, and these dates can overlap. For example, you can allow students to submit responses and complete peer and self assessments starting on March 1. You can require all responses to be submitted by March 7, but allow students to continue peer and self assessments until March 14, a week after all responses are due.
If you don't specify dates, the deadline for all elements--responses, peer assessments, and self assessments--is the due date that you set for the subsection that contains the peer assessment. If you do specify dates, those dates take precedence over the subsection due date.
When you write your question, you can include helpful information for your students, such as what students can expect after they submit responses and the approximate number of words or sentences that a student's response should have. (A response cannot have more than 10,000 words.)
.. note:: We don't recommend that you use the same due date and time for response submissions and assessments. If a student submits a response immediately before the due date, other students will have very little time to assess the response before peer assessment closes. In this case, a student's response may not receive a score.
For more information, see :ref:`PA Add Prompt`.
For more information, see :ref:`PA Add Due Dates`.
==========================================
Asking Students to Upload Images
==========================================
==============
Question
==============
You can ask your students to upload an image as part of their response. If you do this, however, keep the following in mind:
You'll also specify the **question** that you want your students to answer. This appears near the top of the component, followed by a field where the student enters a response. You can require your students to enter text as a response, or you can require your students to both enter text and upload an image. (All student responses must include text. You cannot require students to only upload an image.)
* Currently, you cannot require your students to upload an image. You can only allow it.
.. note:: Currently, course teams cannot see images that students upload. Images do not appear in information that course teams can access about individual students, and they are not included in the course data package.
* All responses must include some text. Students cannot submit a response that only contains an image.
When you write your question, you can include helpful information for your students, such as what students can expect after they submit responses and the approximate number of words or sentences that a student's response should have. (A response cannot have more than 10,000 words.)
* Students can only submit one image with their response.
For more information, see :ref:`PA Add Question`.
.. note:: Currently, course teams cannot see any of the images that students submit. Images are not visible in the body of the assignment in the courseware, and they are not included in the course data package.
.. _PA Rubric:
=======
************************
Rubric
=======
************************
Your assignment must include a **rubric** that you design. The same rubric is used for peer and self assessments, and the rubric appears when students begin grading. Students compare their peers' responses to the rubric.
Rubrics are made of *criteria* and *options*.
* Each criterion has a *name*, a *prompt*, and two or more *options*.
* Each criterion has a *name*, a *prompt*, and one or more *options*.
* The name is a very short summary of the criterion, such as Ideas or Content. Criterion names generally have just one word. Because the system uses criteria names for identification, **the name for each criterion must be unique.** Criterion names do not appear in the rubric that students see when they are completing peer assessments, but they do appear on the page that shows the student's final grade.
* The name is a very short summary of the criterion, such as "Ideas" or "Content". Criterion names generally have just one word. Because the system uses criterion names for identification, **the name for each criterion must be unique.** Criterion names do not appear in the rubric that students see when they are completing peer assessments, but they do appear on the page that shows the student's final grade.
.. image :: /Images/PA_CriterionName.png
:alt: A final score page with call-outs for the criterion names
* The prompt is a description of the criterion.
* The prompt is a description of the criterion.
* Options describe how well the response satisfies the criterion.
* Each option has a *name*, an *explanation*, and a *point value*.
.. image:: /Images/PA_Rubric_LMS.png
:alt: Image of a rubric in the LMS with call-outs for the criterion prompt and option names, explanations, and points
You can see both criterion and option names when you access assignment information for an individual student. For more information, see :ref:`PA Accessing Assignment Information`.
Different criteria in the same assignment can have different numbers of options. For example, in the image above, the first criterion has three options and the second criterion has four options.
.. note:: You can also include criteria that do not have options, but that do include a field where students can enter feedback. For more information, see :ref:`PA Criteria Comment Field Only`.
You can see both criterion and option names when you access assignment information for an individual student. For more information, see :ref:`PA Accessing Assignment Information`.
.. image:: /Images/PA_Crit_Option_Names.png
:width: 600
......@@ -151,17 +135,31 @@ Prompt: Does this response explain the origins of the Hundred Years' War? (5 poi
For more information about writing effective rubrics, see Heidi Goodrich Andrade's `Understanding Rubrics <http://learnweb.harvard.edu/alps/thinking/docs/rubricar.htm>`_.
Note that different criteria in the same assignment can have different numbers of options. For example, in the image above, the first criterion has three options and the second criterion has four options.
For more information, see :ref:`PA Add Rubric`.
************************
Assessment Steps
************************
In your assignment, you'll also specify the **assessment steps**. You can set the assignment to include a student training step, a peer assessment step, and a self assessment step.
You can see the type and order of the assessments when you look at the assignment. In the following example, after students submit a response, they complete a student training step ("Learn to Assess Responses"), complete peer assessments on other students' responses ("Assess Peers"), and then complete self assessments ("Assess Your Response").
.. image:: /Images/PA_AsmtWithResponse.png
:alt: Image of peer assessment with assessment steps and status labeled
:width: 600
.. note:: If you include a student training step, you must also include a peer assessment step. The student training step must come before peer and self assessment steps.
.. _PA Student Training Assessments:
========================================
Student Training Assessments (optional)
========================================
========================
Student Training Step
========================
When you create a peer assessment assignment, you can create one or more student training assessments to help students learn to perform their own assessments. A student training assessment contains one or more sample responses that you write, together with the scores that you would give the sample responses. Students review these responses and try to score them the way that you scored them.
When you create a peer assessment assignment, you can include one or more student training assessments to help students learn to perform their own assessments. A student training assessment contains one or more sample responses that you write, together with the scores that you would give the sample responses. Students review these responses and try to score them the way that you scored them.
.. note:: If you include a student training step, you must also include a peer assessment step. The student training step must come before peer and self assessment steps.
In a student training assessment, the **Learn to Assess Responses** step opens immediately after a student submits a response. The student sees one of the sample responses that you created, along with the rubric. The scores that you gave the response do not appear. The student also sees the number of sample responses that he or she will assess.
......@@ -200,11 +198,41 @@ For example, the following student chose one correct option and one incorrect op
The student continues to try scoring the sample response until the student's scoring for all criteria matches the instructor's scoring.
For more information, see :ref:`PA Add a Student Training Assessment`.
For more information, see :ref:`PA Student Training Step`.
=====================
Peer Assessment Step
=====================
In the peer assessment step, students review other students' responses and select an option for each criterion in your rubric based on the response. Students can also provide text feedback, or comments, on the response.
Number of Responses and Assessments
************************************
When you specify a peer assessment step, you'll specify the **number of responses** each student has to assess and the **number of peer assessments** each response has to receive.
.. note:: Because some students may submit a response but not complete peer assessments, some responses may not receive the required number of assessments. To increase the chance that all responses will receive enough assessments, you must set the number of responses that students have to assess to be higher than the number of assessments that each response must undergo. For example, if you require each response to receive three assessments, you could require each student to assess five responses.
If all responses have received assessments, but some students haven't completed the required number of peer assessments, those students can assess responses that other students have already assessed. The student who submitted the response sees the additional peer assessments when he sees his score. However, the additional peer assessments do not count toward the score that the response receives.
.. _Feedback Options:
Feedback Options
****************
By default, students see a single comment field below the entire rubric. You can also add a comment field to an individual criterion or to several individual criteria. This comment field can contain up to 300 characters.
The comment field appears below the options for the criterion. In the following image, both criteria have a comment field. There is also a field for overall comments on the response.
.. image:: /Images/PA_CriterionAndOverallComments.png
:alt: Rubric with comment fields under each criterion and under overall response
:width: 600
For more information, see :ref:`Add Rubric` and :ref:`PA Criteria Comment Field Only`.
.. _PA Scoring:
***********************
Peer Assessment Scoring
***********************
......@@ -246,458 +274,44 @@ To calculate the final score, add the median scores for each criterion:
Note, again, that final scores are calculated by criteria, not by individual assessor. Thus the response's score is not the median of the scores that each individual peer assessor gave the response.
.. _PA Create a PA Assignment:
************************************
Create a Peer Assessment Assignment
************************************
To create a peer assessment assignment, you'll edit XML code in a Problem component, similar to the way you create other assignments. The following image shows what a peer assessment component looks like when you edit it in Studio, as well as the way that students see that peer assessment in the courseware.
.. image:: /Images/PA_XML_LMS_All.png
:alt: Image of a peer assessment in Studio and LMS views
:width: 800
Creating a peer assessment is a multi-step process:
* :ref:`PA Create Component`
* :ref:`PA Specify Name and Assessment Types`
* :ref:`PA Add a Student Training Assessment`
* :ref:`PA Add Due Dates`
* :ref:`PA Add Question`
* :ref:`PA Add Rubric`
* :ref:`PA Provide Comment Options`
* :ref:`PA Test Assignment`
Each of these steps is covered in detail below.
.. _PA Create Component:
============================
Step 1. Create the Component
============================
#. In Studio, open the unit where you want to create the assessment.
#. Under **Add New Component**, click **Problem**, click the **Advanced** tab, and then click **Peer Assessment**.
#. In the Problem component that appears, click **Edit**.
When the component editor opens, you can see sample code that includes the following. You'll replace this sample content with the content for your assignment:
* The assignment's title.
* The training responses for the assignment.
* The assessment type or types.
* The number of assessments that students must complete.
* A sample question ("prompt").
* A sample rubric.
Note that you won't use the **Settings** tab in the component editor when you create peer assessments.
.. _PA Specify Name and Assessment Types:
========================================================
Step 2. Specify the Assignment Name and Assessment Types
========================================================
To specify the name and assessment types for the assignment, you'll work with the XML near the top of the component editor.
In the component editor, locate the following XML:
.. code-block:: xml
<title></title>
<assessments>
<assessment name="student-training">
<example>
<answer>
(optional) Replace this text with your own sample response for this assignment. Below, list the names of the criteria for this assignment, and then specify the name of the option that you would select for this response. Students will learn to assess responses by assessing this response and comparing the rubric options that they select with the rubric options that you specified.
If you don't want to provide sample responses and scores, delete the entire 'assessment name="student-training"' element.
</answer>
<select criterion="Ideas" option="Fair"/>
<select criterion="Content" option="Good"/>
</example>
<example>
<answer>
(optional) Replace this text with another sample response, and then specify the options that you would select for this response below. To provide more sample responses, copy an "example" element and paste as many as you want before the closing "assessment" tag.
</answer>
<select criterion="Ideas" option="Poor"/>
<select criterion="Content" option="Good"/>
</example>
</assessment>
<assessment name="peer-assessment" must_grade="5" must_be_graded_by="3"/>
<assessment name="self-assessment"/>
</assessments>
This code includes several elements:
* **The title of the assignment**. In this example, because there is no text between the ``<title>`` tags, the assignment does not have a specified title.
* **The type and order of the assessments**. This information is in the **name** attribute in the ``<assessment>`` tags. Assessments run in the order in which they're listed. In this example, students complete the student training assessment, the peer assessment, and the self assessment, in that order.
* **Two sample responses for student training**, together with the options that you select for each of the criteria for the assignment. This information is between the two sets of ``<example> </example>`` tags. Step-by-step instructions for creating student training responses appear in :ref:`PA Add a Student Training Assessment`.
* **The number of responses that each student must assess** (for peer assessments). This information is in the **must_grade** attribute in the ``<assessment>`` tag for the peer assessment. In this example, each student must grade five peer responses before he receives the scores that his peers have given him.
* **The number of peer assessments each response must receive** (for peer assessments). This information is in the **must_be_graded_by** attribute in the ``<assessment>`` tag for the peer assessment. In this example, each response must receive assessments from three students before it can return to the student who submitted it.
To specify the name and assessment types, follow these steps.
#. Between the ``<title>`` tags, add a name for the assignment.
#. Specify the type of assessments you want students to complete. Assessments run in the order in which they're listed.
.. note:: If you include both peer and self assessments, the peer assessment must precede the self assessment. If you include a student training assessment, the student training assessment must precede the peer and self assessments. You can also include a student training assessment paired with either a peer assessment only or a self assessment only.
- If you want students to complete a peer assessment only, delete the ``<assessment name="self-assessment"/>`` tag.
- If you want students to complete a self assessment only, delete the ``<assessment name="peer-assessment" must_grade="5" must_be_graded_by="3""/>`` tag.
- If you want students to complete a peer assessment and then a self assessment, leave the default tags.
- If you include a student training assessment, make sure you add the ``<assessment name="student-training">`` tag *before* the ``<assessment name="peer-assessment">`` and ``<assessment name="self-assessment">`` tags.
#. If your students will complete a peer assessment, replace the values for **must_grade** and **must_be_graded_by** in the ``<assessment name="peer-assessment">`` tag with the numbers that you want.
.. note:: The value for **must_grade** must be greater than or equal to the value for **must_be_graded_by**.
.. _PA Add a Student Training Assessment:
========================================================
Step 3. Include a Student Training Assessment (optional)
========================================================
To include a student training assessment, which contains both sample responses and scores, you'll work with the following XML:
.. code-block:: xml
<assessment name="student-training">
<example>
<answer>Replace this text with a sample response for this assignment. You'll assess this sample response in the courseware, and students will then learn to assess responses by assessing this response and comparing the options that they select in the rubric with the options that you selected.</answer>
<select criterion="Ideas" option="Fair"/>
<select criterion="Content" option="Good"/>
</example>
<example>
<answer>Replace this text with a sample response for this assignment. You'll assess this sample response in the courseware, and students will then learn to assess responses by assessing this response and comparing the options that they select in the rubric with the options that you selected.</answer>
<select criterion="Ideas" option="Poor"/>
<select criterion="Content" option="Good"/>
</example>
</assessment>
.. note:: If you don't want to include a student training assessment, delete all of this XML.
Assessing Additional Responses
********************************
This code includes several elements:
Students can assess more than the required number of responses. After a student completes the peer assessment step, the step "collapses" so that just the **Assess Peers** heading is visible.
* The ``<assessment name="student-training">`` tag indicates that this assessment is a student training assessment.
* Each set of ``<example>`` tags contains one set of ``<answer>`` tags and two or more ``<select/>`` tags.
* The set of ``<answer>`` tags contains the text of a sample response that you've created.
* Each ``<select/>`` tag contains the name of one of the assignment's criteria, as well as the option that you select for the criterion. (For more information about criteria and options, see :ref:`PA Rubric`.)
To add student training responses and scores:
#. Replace the placeholder text between the ``<answer>`` tags with the text of your response. To include paragraph breaks, include a blank line between paragraphs. You don't have to add any other formatting tags to include paragraph breaks.
#. Replace the criterion name in each ``<select/>`` tag with the name of one of the criteria in your assignment. To add more criteria, copy and paste as many ``<select/>`` tags as you need. You must include one ``<select/>`` tag for each of the assignment's criteria.
#. In the ``<select/>`` tag for each criterion, replace the placeholder option name with the name of the option that you would select for the sample response.
#. Copy and paste as many sets of ``<example>`` tags as you need to cover all the criteria for your assignment.
For more information, see :ref:`PA Student Training Assessments`.
.. _PA Add Due Dates:
==========================================
Step 4. Add Start and Due Dates (optional)
==========================================
Setting start and due dates is optional. If you don't specify dates, the deadline for all student responses and assessments is the due date that you set for the subsection that contains the peer assessment. If you do specify dates, those dates take precedence over the subsection due date.
To specify due dates and times, you'll add code that includes the date and time inside the XML tags for the assignment and for each specific assessment. The date and time must be formatted as ``YYYY-MM-DDTHH:MM:SS``.
.. note:: You must include the "T" between the date and the time, with no spaces. All times are in universal coordinated time (UTC).
* To specify a due date for response submissions, add the ``submission_due`` attribute with the date and time to the ``<openassessment>`` tag (this is the first tag in your assignment).
``<openassessment submission_due="YYYY-MM-DDTHH:MM:SS">``
* To specify start and end times for an assessment, add ``start`` and ``due`` attributes with the date and time to the ``<assessment>`` tags for the assessment.
``<assessment name="peer-assessment" must_grade="5" must_be_graded_by="3" start="YYYY-MM-DDTHH:MM:SS" due="YYYY-MM-DDTHH:MM:SS"/>``
``<assessment name="self-assessment" start="YYYY-MM-DDTHH:MM:SS" due="YYYY-MM-DDTHH:MM:SS"/>``
For example, the code for your assignment may resemble the following.
.. code-block:: xml
<openassessment submission_due="2014-03-01T00:00:00">
<assessments>
<assessment name="peer-assessment" must_grade="5" must_be_graded_by="3" start="2014-02-24T00:00:00" due="2014-03-08T00:00:00"/>
<assessment name="self-assessment" start="2014-02-24T00:00:00" due="2014-03-08T00:00:00"/>
</assessments>
In this example, the assignment is set at the subsection level to open on February 24, 2014 at midnight UTC. (This information does not appear in the code.) Additionally, the code specifies the following:
* Students can begin submitting responses on February 24, 2014 at midnight UTC, and must submit all responses before March 1, 2014 at midnight UTC:
``<openassessment submission_due="2014-03-01T00:00:00">``
* Students can begin peer assessments on February 24, 2014 at midnight UTC, and all peer assessments must be complete by March 8, 2014 at midnight UTC:
``<assessment name="peer-assessment" must_grade="5" must_be_graded_by="3" start="2014-02-24T00:00:00" due="2014-03-08T00:00:00"/>``
* Students can begin self assessments on February 24, 2014 at midnight UTC, and all self assessments must be complete by March 8, 2014 at midnight UTC:
``<assessment name="self-assessment" start="2014-02-24T00:00:00" due="2014-03-08T00:00:00"/>``
.. note:: We don't recommend that you use the same due date and time for response submissions and peer assessments. If a student submits a response immediately before the due date, other students will have very little time to assess the response before peer assessment closes. In this case, a student's response may not receive a score.
.. _PA Add Question:
============================
Step 5. Add the Question
============================
The following image shows a question in the component editor along with the way the question appears to students.
.. image:: /Images/PA_Question_XML-LMS.png
:alt: Image of question in XML and the LMS
:width: 800
To add the question:
#. In the component editor, locate the first set of ``<prompt>`` tags. The opening ``<prompt>`` tag appears directly below the opening ``<rubric>`` tag.
#. Replace the sample text between the ``<prompt>`` tags with the text of your question. Note that the component editor respects paragraph breaks and new lines inside the ``<prompt>`` tags. You don't have to add ``<p>`` tags to create individual paragraphs.
Require Students to Upload an Image
****************************************
If you want your students to upload an image as a part of their response, change the very first tag in the assignment from ``<openassessment allow_file_upload="False">`` to ``<openassessment allow_file_upload="True">``. This action adds the **Choose File** and **Upload Your Image** buttons below the student response field.
.. image:: /Images/PA_Upload_ChooseFile.png
:alt: Open response assessment example with Choose File and Upload Your Image buttons circled
.. image:: /Images/PA_PAHeadingCollapsed.png
:width: 500
:alt: The peer assessment step with just the heading visible
.. note:: Currently, course teams cannot see images that students upload. Images do not appear in information that course teams can access about individual students, and they are not included in the course data package.
Add Formatting or Images to the Question
****************************************
In this initial release, you cannot add text formatting or images in the Peer Assessment component. If you want to include formatting or images in the text of your prompt, you can add an HTML component that contains your text above the Peer Assessment component, and then remove the prompt from the Peer Assessment component. The instructions for the peer assessment still appear above the **Your Response** field.
.. image:: /Images/PA_HTMLComponent.png
:alt: A peer assessment that has an image in an HTML component
:width: 500
To remove the prompt from the Peer Assessment component, open the component editor, and then delete the first set of ``<prompt>`` tags together with all the text between the tags. The first few lines of XML for the assignment will then resemble the following.
.. code-block:: xml
<openassessment>
<title></title>
<assessments>
<assessment name="peer-assessment" must_grade="5" must_be_graded_by="3"/>
<assessment name="self-assessment"/>
</assessments>
<rubric>
<criterion feedback="optional">
<name>Ideas</name>
<prompt>Determine if there is a unifying theme or main idea.</prompt>
<option points="0">
.. _PA Add Rubric:
============================
Step 6. Add the Rubric
============================
To add the rubric, you'll create your criteria and options in XML. The following image shows a highlighted criterion and its options in the component editor, followed by the way the criterion and options appear to students.
.. image:: /Images/PA_RubricSample_XML-LMS.png
:alt: Image of rubric in XML and the LMS, with call-outs for criteria and options
For more information about criteria and options, see :ref:`PA Elements`.
If the student clicks the **Assess Peers** heading, the step expands. The student can then click **Continue Assessing Peers**.
To add the rubric:
#. In the component editor, locate the following XML. This XML contains a single criterion and its options. You'll replace the placeholder text with your own content.
.. code-block:: xml
<criterion>
<name>Ideas</name>
<prompt>Determine if there is a unifying theme or main idea.</prompt>
<option points="0">
<name>Poor</name>
<explanation>Difficult for the reader to discern the main idea.
Too brief or too repetitive to establish or maintain a focus.</explanation>
</option>
<option points="3">
<name>Fair</name>
<explanation>Presents a unifying theme or main idea, but may
include minor tangents. Stays somewhat focused on topic and
task.</explanation>
</option>
<option points="5">
<name>Good</name>
<explanation>Presents a unifying theme or main idea without going
off on tangents. Stays completely focused on topic and task.</explanation>
</option>
</criterion>
.. note:: The placeholder text contains indentations and line breaks. You don't have to preserve these indentations and line breaks when you replace the placeholder text.
#. Under the opening ``<criterion>`` tag, replace the text between the ``<name>`` tags with the name of your criterion. Then, replace the text between the ``<prompt>`` tags with the description of that criterion.
Note that **every criterion must have a unique name.** The system uses the criterion name for identification. For more information about criteria, see :ref:`PA Rubric`.
#. Inside the first ``<option>`` tag, replace the value for ``points`` with the number of points that you want this option to receive.
#. Under the ``<option>`` tag, replace the text between the ``<name>`` tags with the name of the first option. Then, replace the text between the ``<explanation>`` tags with the description of that option.
#. Use this format to add as many options as you want.
You can use the following code as a template:
.. code-block:: xml
<criterion>
<name>NAME</name>
<prompt>PROMPT TEXT</prompt>
<option points="NUMBER">
<name>NAME</name>
<explanation>EXPLANATION</explanation>
</option>
<option points="NUMBER">
<name>NAME</name>
<explanation>EXPLANATION</explanation>
</option>
<option points="NUMBER">
<name>NAME</name>
<explanation>EXPLANATION</explanation>
</option>
</criterion>
.. _PA Provide Comment Options:
=============================================
Step 7. Provide Comment Options (optional)
=============================================
After students fill out the rubric, they can provide additional comments for the responses they've assessed. By default, students see a field for comments below the rubric.
.. image:: /Images/PA_CommentsField.png
:alt: Contents field
.. image:: /Images/PA_ContinueGrading.png
:width: 500
:alt: The peer assessment step expanded so that "Continue Assessing Peers" is visible
You can change the text that appears above this comment field. Additionally, you can provide a comment field for each individual criterion.
.. _PA Change Comments Prompt:
Change the Default Prompt Text
*******************************
By default, the prompt text for the comment field is the following:
``(Optional) What aspects of this response stood out to you? What did it do well? How could it improve?``
You can replace this default text with your own text.
To change this text:
#. Locate the ``<feedbackprompt>`` tags between the last closing ``</criterion>`` tag for the rubric and the closing ``</rubric>`` tag for the assignment:
.. code-block:: xml
<option points="3">
<name>Excellent</name>
<explanation>Includes in-depth information and exceptional supporting details that are fully developed. Explores all facets of the topic.</explanation>
</option>
</criterion>
<feedbackprompt>(Optional) What aspects of this response stood out to you? What did it do well? How could it improve?</feedbackprompt>
</rubric>
</openassessment>
=====================
Self Assessment Step
=====================
2. Change the text between the ``<feedbackprompt>`` tags to the text that you want.
In self assessments, the student sees his response followed by your rubric. As with peer assessments, the student compares the rubric to his response and selects an option for each of the criteria.
.. _PA Add Individual Criterion Comments:
If you include both peer and self assessments, we recommend that you include the peer assessment before the self assessment.
Provide a Comment Field for an Individual Criterion
***************************************************
.. _PA Top Responses:
By default, students see only a single comment field below the entire rubric. However, you can add a comment field to an individual criterion or to several individual criteria. The comment field can contain up to 300 characters.
*****************************
Top Responses
*****************************
The comment field appears below the options for the criterion. In the following image, the first criterion has a comment field, but the second does not.
You can include a **Top Responses** section that shows the top-scoring responses that students have submitted for the assignment, along with the scores for those responses. The **Top Responses** section appears below the student's score information after the student finishes every step in the assignment.
.. image:: /Images/PA_Comments_Criterion.png
:alt: Comment box under an individual criterion
.. image:: /Images/PA_TopResponses.png
:alt: Section that shows the text and scores of the top three responses for the assignment
:width: 500
To add a comment field:
#. Locate the opening ``<criterion>`` tag for the criterion that you want to change.
#. Add the ``feedback`` attribute to this tag. Make sure to set a value for this attribute:
* If you want to make comments optional for students, use ``feedback="optional"``.
* If you want to require students to provide comments, use ``feedback="required"``.
The XML for a criterion that has a comment field as well as options resembles the following.
.. code-block:: xml
<criterion feedback="optional">
<name>NAME</name>
<prompt>PROMPT TEXT</prompt>
<option points="NUMBER">
<name>NAME</name>
<explanation>EXPLANATION</explanation>
</option>
<option points="NUMBER">
<name>NAME</name>
<explanation>EXPLANATION</explanation>
</option>
</criterion>
If you want to provide a comment field below any additional criteria, add the ``feedback="optional"`` or ``feedback="required"`` attribute to the opening tag for each criterion.
.. _PA Zero Option Criteria:
Provide Only Comment Fields for Individual Criteria
****************************************************
When you add a comment field to a criterion, the comment field appears below the options for the criterion. You can also provide a comment field, but no options.
In the following image, the first criterion has a comment field but no options. The second includes options, but does not have a comment field.
.. image:: /Images/PA_0_Option_Criteria.png
To provide a comment field without options:
#. Locate the opening ``<criterion>`` tag for the criterion that you want to change.
#. Add the ``feedback="required"`` attribute to this tag.
.. note:: If you don't include options for the criterion, you must include the ``feedback="required"`` attribute. Don't use the ``feedback="optional"`` attribute.
#. If the criterion has options, delete the options.
The XML for a criterion that has a comment field but no options resembles the following.
.. code-block:: xml
<criterion feedback="required">
<name>NAME</name>
<prompt>PROMPT TEXT</prompt>
</criterion>
.. _PA Test Assignment:
============================
Step 8. Test the Assignment
============================
You can allow the **Top Responses** section to show between 1 and 100 responses. Keep in mind, however, that each response may be up to 300 pixels in height in the list. (For longer responses, students can scroll to see the entire response.) We recommend that you specify 20 or fewer responses to prevent the page from becoming too long.
To test your assignment, set up the assignment in a test course, and ask a group of beta users to submit responses and grade each other. The beta testers can then let you know if they found the question and the rubric easy to understand or if they had any problems with the assignment.
.. note:: It may take up to an hour for a high-scoring response to appear in the **Top Responses** list.
For more information, see :ref:`PA Show Top Responses`.
docs/en_us/course_authors/source/PeerAssessment_Students.rst
......@@ -22,6 +22,8 @@ A *rubric* is a list of expectations that a response should meet. Rubrics are ma
When you assess a response, you'll select the option that best describes the response for each of the criteria.
Some instructors create a **Top Responses** section that shows the top-scoring responses for the assignment and the scores that these responses received. If an instructor creates this section, you can see it below your score after you've completed each step of the assignment.
************************
Student Instructions
************************
......@@ -32,7 +34,7 @@ When you come to an open response assessment in the course, you'll see the quest
:alt: Open response assessment example with question, response field, and assessment types and status labeled
:width: 550
Here, we'll walk you through the process of completing an open response assessment that includes a peer assessment and a self assessment:
Here, we'll walk you through the process of completing an open response assessment that includes a student training step, a peer assessment, and a self assessment:
#. Submit your response to a question.
#. Learn to assess responses.
......@@ -77,7 +79,7 @@ Note that you can view your response at any time after you submit it. To do this
Submit an Image with Your Response
***********************************
Some assignments require you to submit an image with your text response. If you have to submit an image, you'll see buttons that you'll use to upload your image.
Some assignments allow you to submit an image with your text response. If you can submit an image, you'll see buttons that you'll use to upload your image.
.. image:: /Images/PA_Upload_ChooseFile.png
:alt: Open response assessment example with Choose File and Upload Your Image buttons circled
......@@ -89,7 +91,7 @@ To upload your image:
#. In the dialog box that opens, select the file that you want, and then click **Open**.
#. When the dialog box closes, click **Upload Your Image**.
Your image appears below the response field, and the name of the image file appears next to the **Choose File** button. If you want to change the image, follow steps 1-3 again.
Your image appears below the response field, and the name of the image file appears next to the **Choose File** button. If you want to change the image, follow steps 1-3 again. You can only upload one image.
.. image:: /Images/PA_Upload_WithImage.png
:alt: Example response with an image of Paris
......@@ -152,16 +154,32 @@ When peer assessment starts, you'll see the original question, another student's
You'll assess these responses by selecting options in the rubric, the same way you assessed the sample responses in the "learn to assess responses" step. Additionally, this step has a field below the rubric where you can provide comments about the student's response.
.. note:: Some assessments may have an additional **Comments** field for one or more of the assessment's individual criteria. You can enter up to 300 characters in these fields. In the following image, the first of the criteria has a separate **Comments** field, but the second does not.
.. note:: Some assessments may have an additional **Comments** field for one or more of the assessment's individual criteria. You can enter up to 300 characters in these fields. In the following image, both criteria have a **Comments** field. There is also a field for overall comments on the response.
.. image:: /Images/PA_S_CommentBoxes.png
:alt: Rubric with call-outs for comment boxes
:width: 500
.. image:: /Images/PA_CriterionAndOverallComments.png
:alt: Rubric with comment fields under each criterion and under overall response
:width: 600
After you've selected options in the rubric and provided additional comments about the response in this field, click **Submit your assessment and move to response #<number>**.
When you submit your assessment of the first student's response, another response opens for you. Assess this response in the same way that you assessed the first response, and then submit your assessment. You'll repeat these steps until you've assessed the required number of responses. The number in the upper-right corner of the step is updated as you assess each response.
Assess Additional Peer Responses
********************************
You can assess more peer responses if you want to. After you assess the required number of responses, the step "collapses" so that just the **Assess Peers** heading is visible.
.. image:: /Images/PA_PAHeadingCollapsed.png
:width: 500
:alt: The peer assessment step with just the heading visible
To assess more responses, click the **Assess Peers** heading to expand the step. Then, click **Continue Assessing Peers**.
.. image:: /Images/PA_ContinueGrading.png
:width: 500
:alt: The peer assessment step expanded so that "Continue Assessing Peers" is visible
=====================
Assess Your Response
=====================
......@@ -201,9 +219,9 @@ If you've assessed the required number of peer responses and completed your self
Peer Assessment Scoring
***********************
Peer assessments are scored by criteria. An individual criterion's score is the median of the scores that each peer assessor gave that criterion. For example, if the Ideas criterion in a peer assessment receives a 10 from one student, a 7 from a second student, and an 8 from a third student, the Ideas criterion's score is 8.
Peer assessments are scored by criteria. An individual criterion's score is the *median*, not average, of the scores that each peer assessor gave that criterion. For example, if the Ideas criterion in a peer assessment receives a 10 from one student, a 7 from a second student, and an 8 from a third student, the Ideas criterion's score is 8.
A student's final score for a peer assessment is the sum of the median scores for each individual criterion.
Your final score for a peer assessment is the sum of the median scores for each individual criterion.
For example, a response may receive the following scores from peer assessors:
......@@ -237,4 +255,18 @@ To calculate the final score, the system adds the median scores for each criteri
**Ideas median (8/10) + Content median (8/10) + Grammar median (4/5) = final score (20/25)**
Note, again, that final scores are calculated by criteria, not by individual assessor. Thus the response's score is not the median of the scores that each individual peer assessor gave the response.
Note, again, that final scores are calculated by criteria, not by assessor. Thus your score is not the median of the scores that each individual peer assessor gave the response.
==================================
View Top Responses (optional)
==================================
If the instructor has included a **Top Responses** section, you can see the highest-scoring responses that your peers have submitted. This section only appears after you've completed all the steps of the assignment.
.. image:: /Images/PA_TopResponses.png
:alt: Section that shows the text and scores of the top three responses for the assignment
:width: 500
docs/en_us/course_authors/source/index.rst
......@@ -12,5 +12,6 @@ Creating Peer Assessments
:maxdepth: 2
PeerAssessment
PeerAssessment_Students
Access_PA_Info
\ No newline at end of file
CreatePeerAssessment
Access_PA_Info
PeerAssessment_Students
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment