Skip to content

E
edx-platform
Commit 10deae91 by srpearce
Options

Merge pull request #2751 from edx/markhoeber/documentation/stud-1260 

WIP:  Content Experiments Documentation
parents c08d43b1 fb1ed6ed
Showing with 529 additions and 4 deletions
docs/en_us/course_authors/source/change_log.rst
......@@ -16,6 +16,8 @@ August, 2014
- Updated documentation in :ref:`Developing Your Course Index`
for new capabilities to publish or hide multiple units
from the outline.
* -
- Added documentation for :ref:`Creating Content Experiments`
* - 08/22/14
- Added warnings about the effects of changing the course grading policy,
graded subsections, or graded components after the course begins.
......
docs/en_us/course_authors/source/content_experiments/content_experiments_add.rst 0 → 100644
.. _Add Content Experiments to Your Course:
#########################################
Add Content Experiments to Your Course
#########################################
After you :ref:`enable content experiments <Enable Content Experiments>` and
:ref:`set up group configurations in Studio <Set up Group Configurations in edX
Studio>` or :ref:`in XML <Set up Group Configurations in an XML Course>`, you
can add content experiments to your course. See:
* :ref:`Add a Content Experiment in Studio`
* :ref:`Add a Content Experiment in XML`
.. _Add a Content Experiment in Studio:
********************************************
Add a Content Experiment in Studio
********************************************
You can add a content experiment in a unit or container page. In Studio, you
create and view content for all groups in the content experiment in a container
page for the experiment, as shown in `Create Content for Groups in the Content
Experiment`_.
When a student views the unit with the content experiment, she has no
indication there is a content experiment in the unit, and the content
experiment display name is not shown. She sees only the content that you
configure for the group she is assigned to. To the student, the unit with the
content experiment is no different than any other unit.
To configure a content experiment in Studio, you:
#. `Create the content experiment`_.
#. `Create content for groups in the content experiment`_.
After you configure a content experiment, you can `change the
group configuration <Change the Group Configuration for a Content
Experiment>`_.
===============================
Create the Content Experiment
===============================
#. In the unit page, under **Add New Component**, click **Advanced**.
#. Select **Content Experiment**.
A new content experiment is added to the unit:
.. image:: ../Images/content_experiment_block.png
:width: 800
:alt: The content experiment component in a unit page
You can work with the content experiment as you can any other component.
See :ref:`Components` for more information.
#. Click either **Select a Group Configuration** or **Edit** to open the content
experiment component.
.. image:: ../Images/content_experiment_editor.png
:alt: The content experiment editor
#. Next to **Group Configuration**, select a group configuration.
#. In the **Display Name** field, enter the name that the component will use in Studio. The
display name is only used in Studio; students do not see this value.
#. Click **Save**.
The content experiment is displayed as a component that contains other
components. See :ref:`Components that Contain Other Components` for more
information.
.. note:: You cannot duplicate a content experiment.
You can now create content for the groups in the experiment.
=====================================================
Create Content for Groups in the Content Experiment
=====================================================
After you select a group configuration, in the content experiment component,
click **View**.
The content experiment page that opens automatically includes a container for
each group that is defined in the group configuration you selected. For
example, if you select a group configuration that defines Group A and Group B,
you see the following page:
.. image:: ../Images/content_experiment_container.png
:alt: The content experiment page with two groups
You add content for both groups as needed, just as you would add content to any
container page. See :ref:`Components that Contain Other Components` for more
information.
For example, you can add an HTML component and a video to Group A:
.. image:: ../Images/a_b_test_child_expanded.png
:alt: Image of an expanded A/B test component
.. note::
It is valid, and can be useful, to have no content for a group in the
experiment. For example, if one group has a video and another group has no
content, you can analyze the effect of the video on student performance.
========================================================
Change the Group Configuration for a Content Experiment
========================================================
You can change the group configuration for a content experiment. When you
change the group configuration, you must add components to any new groups that you create. You
can use the components from the previous groups, as well as create new
components.
.. warning::
Changing the group configuration of a student-visible experiment will affect
the experiment data.
To change the group configuration:
#. Open the unit page of the unit that contains the content experiment.
#. In the content experiment component, click **Edit**.
.. image:: ../Images/content_experiment_editor_group2.png
:alt: The content experiment editor with a group configuration selected
#. Select a different group configuration.
#. Click **Save**.
#. You must now add components to the new groups in the experiment. Click
**View** to open the content experiment.
You see that groups for the new configuration are empty, and any components
that you had added to groups in the previous configuration are now moved to
a section called **Inactive Groups**.
.. image:: ../Images/inactive_groups.png
:alt: Components in inactive groups
#. Drag and drop components from the **Inactive Groups** section into the new
groups. You can also create new components in the new groups.
.. _Add a Content Experiment in XML:
****************************************
Add a Content Experiment in XML
****************************************
If you are developing your course in XML, you work with multiple XML files to
configure a content experiment. This section steps through the files involved
in a content experiment that shows different content to two different groups of
students.
For more information about working with your course's XML files, including
information about terminology, see the `edX XML Tutorial
<http://edx.readthedocs
.org/projects/devdata/en/latest/course_data_formats/course_xml.html>`_.
=====================================================
Define the Content Experiment in the Sequential File
=====================================================
You reference a content experiment in the file for the subsection in the
``sequential`` directory. For example:
.. code-block:: xml
...
<vertical url_name="name for the unit that contains the A/B test" display_name="A/B Test Unit">
<split_test url_name="name of A/B test file in the split_test folder"/>
</vertical>
.....
The ``<split_test>`` element's ``url_name`` value references the name of the
A/B test file in the ``split_test`` directory.
.. _Define the Experiment Content in the Split Test File:
=====================================================
Define the Experiment Content in the Split Test File
=====================================================
After you define the content experiment in the sequential file, you define the
course content you want to test in the file in the ``split_test`` directory.
This is the file referenced in the ``<split_test>`` element in the sequential
file, as shown above.
In the content experiment file, you add elements for the experiment content.
For this example, you add two ``<vertical>`` elements to compare the two
different sets of content.
.. code-block:: xml
<split_test url_name="AB_Test.xml" display_name="A/B Test" user_partition_id="0"
group_id_to_child='{"0": "i4x://path-to-course/vertical/group_a",
"1": "i4x://path-to-course/vertical/group_b"}'>
<vertical url_name="group_a" display_name="Group A">
<html>Welcome to group A.</html>
<video url_name="group_a_video"/>
</vertical>
<vertical url_name="group_b" display_name="Group B">
<html>Welcome to group B.</html>
<problem display_name="Checkboxes">
<p>A checkboxes problem presents checkbox buttons for student input.
Students can select more than one option presented.</p>
<choiceresponse>
<checkboxgroup direction="vertical" label="Select the answer that matches">
<choice correct="true">correct</choice>
<choice correct="false">incorrect</choice>
<choice correct="true">correct</choice>
</checkboxgroup>
</choiceresponse>
</problem>
</vertical>
</split_test>
In this example:
* The ``user_partition_id`` value references the ID of the experiment defined
in the ``policy.json`` file.
* The ``group_id_to_child`` value references the IDs of the groups defined in
the ``policy.json`` file and maps the group IDs to specific content.
For example, the value for group ``0``, ``i4x://path-to-
course/vertical/group_a``, maps to the ``<vertical>`` element with the
``url_name`` equal to ``group_a``. Therefore, students in group 0 see the
content in that vertical.
For information about the ``policy.json`` file, see :ref:`Set up Group
Configurations in an XML Course`.
\ No newline at end of file
docs/en_us/course_authors/source/content_experiments/content_experiments_overview.rst 0 → 100644
.. _Overview of Content Experiments:
#################################
Overview of Content Experiments
#################################
You use content experiments to show different course content to different
groups of students.
Also known as A/B or split tests, content experiments enable you to
research and compare the performance of students in different groups to gain
insight into the relative effectiveness of your course content.
.. caution::
If you have graded problems within a content experiment, grade exports do not
work for your course. This issue is currently being addressed.
For more information, see:
* :ref:`Configure Your Course for Content Experiments`
* :ref:`Add Content Experiments to Your Course`
* :ref:`Test Content Experiments`
.. _Courses with Multiple Content Experiments:
******************************************
Courses with Multiple Content Experiments
******************************************
You can run multiple content experiments in your course. You can set up each
experiment to use the same groups of students, or you can set up each
experiment to be independent and use a different grouping.
.. important::
If your course has multiple experiments, it is critical that you decide
up front if the experiments share the same groups of students or if each
experiment has its own unique grouping. If two experiments share the same
grouping, then any student that is in Group A for the first experiment will
also be in Group A for the second one. If you want the experiments to be
independent, then the experiments must use different groupings so that
students are randomly assigned for each experiment.
To determine the available groupings of students, you :ref:`set up group
configurations in Studio <Set up Group Configurations in edX Studio>` or
:ref:`in XML <Set up Group Configurations in an XML Course>`.
You then select which grouping to use when you :ref:`add a content experiment
in Studio <Add a Content Experiment in Studio>` or :ref:`in XML <Add a Content
Experiment in XML>`.
\ No newline at end of file
docs/en_us/course_authors/source/content_experiments/content_experiments_view.rst 0 → 100644
.. _Test Content Experiments:
##########################################
Test Content Experiments
##########################################
You should test content experiments in your course before the course starts to
ensure that content is delivered to groups as you intended.
When you view a unit that contains a content experiment in the LMS in the Staff
view, you select one of the groups from a drop-down list. The unit page then shows the
content for that group of students.
For example, in the following page, Group 0 is selected, and the HTML component
and video that is part of Group 0 is displayed:
.. image:: ../Images/a-b-test-lms-group-0.png
:alt: Image of a unit page with Group 0 selected
You can change the group selection to view the problem that a different group of
students sees:
.. image:: ../Images/a-b-test-lms-group-2.png
:alt: Image of a unit page with Group 1 selected
.. note::
The example course content in this chapter uses content experiment
terminology to make the functionality clear. Typically, you would not use
terminology in course content that would make students aware of the
experiment.
\ No newline at end of file
docs/en_us/course_authors/source/content_experiments/index.rst 0 → 100644
.. _Creating Content Experiments:
#############################
Creating Content Experiments
#############################
.. toctree::
:maxdepth: 2
content_experiments_overview
content_experiments_configure
content_experiments_add
content_experiments_view
\ No newline at end of file
docs/en_us/course_authors/source/creating_content/index.rst
......@@ -20,4 +20,3 @@ Index`.
create_video
create_discussion
create_problem
docs/en_us/course_authors/source/developing_course/course_components.rst
......@@ -11,6 +11,7 @@ This chapter describes how you develop course components. See:
* `Edit a Component`_
* `Duplicate a Component`_
* `Delete a Component`_
* `Components that Contain Other Components`_
For more information about specific component types, see:
......@@ -125,13 +126,15 @@ Then follow instructions for the type of component you are editing.
After you duplicate a component, the new component is not visible to students
until you :ref:`publish the unit<Publish a Unit>`.
.. note:: You cannot duplicate a content experiment.
.. _Delete a Component:
**********************
Delete a Component
**********************
.. note::
.. caution::
Be sure you want to delete the component. You can not undo the deletion.
To delete a component:
......@@ -144,4 +147,187 @@ To delete a component:
2. When you receive the confirmation prompt, click **Yes, delete this
component**.
After you delete a component, the component remians visible to students until you :ref:`publish the unit<Publish a Unit>`.
\ No newline at end of file
After you delete a component, the component remians visible to students until you :ref:`publish the unit<Publish a Unit>`.
.. _Components that Contain Other Components:
******************************************
Components that Contain Other Components
******************************************
For specific use cases, you configure course content so that components contain
other components. For example, if you want to include conditional components
or content experiments, you have to create components inside components. See
:ref:`Creating Content Experiments` for more information.
The component that contains other components is referred to as the *parent*;
the contained components are referred to as *children*.
In the unit page, a parent component appears with the display name and a
**View** link. For example:
.. image:: ../Images/component_container.png
:alt: Image of a unit page with a parent component
==================================================
Edit a Parent Component
==================================================
A parent component does not directly contain content. Content such as HTML,
videos, or problems are in the child components.
A parent component has a display name. When the unit is private or in draft,
click **Edit** in the parent component to change the display name.
.. note::
Parent components of a specific type, such as content experiments, have
additional attributes that you edit.
======================================
View Child Components
======================================
When you click **View** in the parent component, the parent component page
opens, showing all child components. In this example, Child Component A
contains an HTML component and a video:
.. image:: ../Images/child-components-a.png
:alt: Image of an expanded child component
Click the arrow next to a child component name to collapse it and hide the
component's contents:
.. image:: ../Images/child-components.png
:alt: Image of a child component page
Click the arrow again to expand the component.
See:
* `Edit a Component`_
* `Set the Display Name for a Component`_
* `Duplicate a Component`_
* `Delete a Component`_
======================================
Add a Child Component
======================================
If the containing unit is private or in draft, you can add a child component in
its parent component.
To add a child component, open and expand the parent component. Then click the
component type that you want under **Add New Component** within the parent
component.
.. image:: ../Images/AddNewComponent.png
:alt: Image of adding a new component
For more information, see the documentation for the specific component type that
you want:
- :ref:`Working with Discussion Components`
- :ref:`Working with HTML Components`
- :ref:`Working with Problem Components`
- :ref:`Working with Video Components`
======================================
XML for Parent and Child Components
======================================
You develop parent and child components in XML, then import the XML course into
Studio to verify that the structure is as you intended.
For more information about working with your course's XML files, including
information about terminology, see the `edX XML Tutorial <http://edx.readthedoc
s.org/projects/devdata/en/latest/course_data_formats/course_xml.html>`_.
The following examples show the XML used to create the unit and components
shown in Studio above.
The XML for the unit is:
.. code-block:: xml
<vertical display_name="Unit 1">
<html url_name="6a5cf0ea41a54b209e0815147896d1b2"/>
<vertical url_name="131a499ddaa3474194c1aa2eced34455"/>
</vertical>
The ``<vertical url_name="131a499ddaa3474194c1aa2eced34455"/>`` element above
references the parent component file that contains the child components:
.. code-block:: xml
<vertical display_name="Parent Component">
<vertical url_name="2758bbc495dd40d59050da15b40bd9a5"/>
<vertical url_name="c5c8b27c2c5546e784432f3b2b6cf2ea"/>
</vertical>
The two verticals referenced by the parent component refer to the child
components, which contain the actual content of your course:
.. code-block:: xml
<vertical display_name="Child Component A">
<html url_name="4471618afafb45bfb86cbe511973e225"/>
<video url_name="fbd800d0bdbd4cb69ac70c47c9f699e1"/>
</vertical>
.. code-block:: xml
<vertical display_name="Child Component B">
<html url_name="dd6ef295fda74a639842e1a49c66b2c7"/>
<problem url_name="b40ecbe4ed1b4280ae93e2a158edae6f"/>
</vertical>
Theoretically, there is no limit to the levels of component nesting you can use
in your course.
======================================
The Student View of Nested Components
======================================
For students, all parent and child components are displayed on the unit page.
The following example shows the student view of the unit described above:
.. image:: ../Images/nested_components_student_view.png
:alt: Image of the student's view of nested components
.. note::
The visibility of nested components depends on the visibility of the
parent unit. The parent unit must be public for students to see nested
components. For more information, see :ref:`Visibility of Components
Contained by Other Components`.
*******************************
Reorganizing Child Components
*******************************
You can reorganize child components through the same drag and drop process you
use for other objects in your course outline. You hover over the element handle
on the right side of the screen until the mouse pointer changes to a four-
headed arrow. Then, click and drag the element to the location that you want.
Furthermore, when you have multiple levels of nesting, you can drag a child
component into a different parent component, if both parents are expanded. For
example, you can select the video component that is in Child Component A and
drag it into Child Component B. Select the video component, and as you drag it
into Child Component B, release the mouse button when a dashed outline of the
component you are moving appears in the new location:
.. image:: ../Images/drag_child_component.png
:alt: Image of dragging a child component to a new location
You can also drag a child component outside of a parent, so that the child
moves to the same level as the parent.
.. note::
For content experiments, you cannot drag a child component outside of a test
group.
\ No newline at end of file
docs/en_us/course_authors/source/index.rst
......@@ -19,6 +19,7 @@ Building and Running an edX Course
developing_course/index
creating_content/index
exercises_tools/index
content_experiments/index
releasing_course/index
running_course/index
students/index
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment