Skip to content

E
edx-platform
Commit 2bc6f1d8 by Alison Hodges
Options

ORA2 events

parent ec5a2a1d
Showing with 445 additions and 120 deletions
docs/en_us/data/source/internal_data_formats/change_log.rst
......@@ -11,6 +11,8 @@ Change Log
* - Date
- Change
* - 08/12/14
- Added the :ref:`ora2` section to the :ref:`Tracking Logs` chapter.
* - 08/01/14
- Added the :ref:`Package` chapter with information to help data czars
locate and download data package files.
......
docs/en_us/data/source/internal_data_formats/event_list.rst
.. _event_list:
##############
Event List
##############
#######################
Alphabetical Event List
#######################
.. list-table::
:widths: 30 30
:widths: 40 40
:header-rows: 1
* - Event
......@@ -22,8 +22,6 @@ Event List
- :ref:`Instructor_Event_Types`
* - ``assigned_user_to_partition``
- :ref:`AB_Event_Types`
* - ``peer_grading_show_question``
- :ref:`ora`
* - ``book``
- :ref:`pdf`
* - ``calculate_grades_csv``
......@@ -32,7 +30,17 @@ Event List
- :ref:`AB_Event_Types`
* - ``delete-student-module-state``
- :ref:`Instructor_Event_Types`
* - ``dump_grades``
* - ``dump-answer-dist-csv``
- :ref:`Instructor_Event_Types`
* - ``dump-graded-assignments-config``
- :ref:`Instructor_Event_Types`
* - ``dump-grades``
- :ref:`Instructor_Event_Types`
* - ``dump-grades-csv``
- :ref:`Instructor_Event_Types`
* - ``dump-grades-csv-raw``
- :ref:`Instructor_Event_Types`
* - ``dump-grades-raw``
- :ref:`Instructor_Event_Types`
* - ``edx.course.enrollment.activated``
- :ref:`enrollment` and :ref:`instructor_enrollment`
......@@ -68,23 +76,39 @@ Event List
- :ref:`Instructor_Event_Types`
* - ``modify_access``
- :ref:`Instructor_Event_Types`
* - ``oe_feedback_response_selected``
* - ``oe_feedback_response_selected`` (deprecated)
- :ref:`ora`
* - ``oe_hide_question``
* - ``oe_hide_question`` (deprecated)
- :ref:`ora`
* - ``oe_show_full_feedback``
* - ``oe_show_full_feedback`` (deprecated)
- :ref:`ora`
* - ``oe_show_question``
* - ``oe_show_question`` (deprecated)
- :ref:`ora`
* - ``oe_show_respond_to_feedback``
* - ``oe_show_respond_to_feedback`` (deprecated)
- :ref:`ora`
* - ``openassessmentblock.get_peer_submission``
- :ref:`ora2`
* - ``openassessmentblock.peer_assess``
- :ref:`ora2`
* - ``openassessmentblock.self_assess``
- :ref:`ora2`
* - ``openassessmentblock.submit_feedback_on_assessments``
- :ref:`ora2`
* - ``openassessment.create_submission``
- :ref:`ora2`
* - ``openassessment.save_submission``
- :ref:`ora2`
* - ``openassessment.student_training_assess_example``
- :ref:`ora2`
* - ``openassessment.upload_file``
- :ref:`ora2`
* - ``page_close``
- :ref:`navigational`
* - ``pause_video``
- :ref:`video`
* - ``peer_grading_hide_question``
* - ``peer_grading_hide_question`` (deprecated)
- :ref:`ora`
* - ``peer_grading_show_question``
* - ``peer_grading_show_question`` (deprecated)
- :ref:`ora`
* - ``play_video``
- :ref:`video`
......@@ -122,7 +146,7 @@ Event List
- :ref:`problem`
* - ``reset-student-attempts``
- :ref:`Instructor_Event_Types`
* - ``rubric_select``
* - ``rubric_select`` (deprecated)
- :ref:`ora`
* - ``save_problem_fail``
- :ref:`problem`
......@@ -140,9 +164,9 @@ Event List
- :ref:`problem`
* - ``speed_change_video``
- :ref:`video`
* - ``staff_grading_hide_question``
* - ``staff_grading_hide_question`` (deprecated)
- :ref:`ora`
* - ``staff_grading_show_question``
* - ``staff_grading_show_question`` (deprecated)
- :ref:`ora`
* - ``textbook.pdf.chapter.navigated``
- :ref:`pdf`
......
docs/en_us/data/source/internal_data_formats/tracking_logs.rst
......@@ -290,11 +290,13 @@ outside the Instructor Dashboard.
* :ref:`problem`
* :ref:`ora`
* :ref:`forum_events`
* :ref:`ora2`
* :ref:`AB_Event_Types`
* :ref:`forum_events`
* :ref:`ora`
The descriptions that follow include what each event represents, the system
component it originates from, the history of any changes made to the event over
......@@ -414,8 +416,8 @@ Example
Students who enroll with a ``student_courseenrollment.mode`` of 'audit' or
'honor' in a course that has a verified certificate option see a **Challenge
Yourself** link for the course on their dashboards. The browser emits this event
when a student clicks this option, and the process of upgrading the
Yourself** link for the course on their dashboards. The browser emits this
event when a student clicks this option, and the process of upgrading the
``student_courseenrollment.mode`` for the student to 'verified' begins. See
:ref:`student_courseenrollment`.
......@@ -1536,7 +1538,8 @@ The browser emits ``problem_show`` events when a problem is shown.
- Details
* - ``problem``
- string
- ID of the problem being shown. For example, i4x://MITx/6.00x/problem/L15:L15_Problem_2).
- ID of the problem being shown. For example,
i4x://MITx/6.00x/problem/L15:L15_Problem_2).
``reset_problem``
------------------------------------------------
......@@ -1700,34 +1703,84 @@ for a problem and it is graded successfully.
- Details
* - ``[answers, contents]``
- array
- ``answers`` provides the value checked by the user. ``contents`` delivers
HTML using data entered for the problem in Studio, including the display
name, problem text, and choices or response field labels. The array
includes each problem in a problem component that has multiple problems.
- ``answers`` provides the value checked by the user. ``contents``
delivers HTML using data entered for the problem in Studio, including
the display name, problem text, and choices or response field labels.
The array includes each problem in a problem component that has multiple
problems.
.. _ora:
.. _forum_events:
==========================
Forum Events
==========================
``edx.forum.searched``
----------------------------------
After a user executes a text search in the navigation sidebar of the Discussion
tab of a course, the server emits an ``edx.forum.searched`` event.
**Component**: Discussion Tab
**Event Source**: Server
**History**: Added 16 May 2014. The ``corrected_text`` field was added 5
Jun 2014.
``event`` **Fields**:
.. list-table::
:widths: 15 15 60
:header-rows: 1
* - Field
- Type
- Details
* - ``query``
- string
- The text entered into the search box by the user.
* - ``page``
- integer
- Results are returned in sets of 20 per page. Identifies the page of
results requested by the user.
* - ``total_results``
- integer
- The total number of results matching the query.
* - ``corrected_text``
- string
- A re-spelling of the query, suggested by the search engine, which was
automatically substituted for the original one. This happens only when
there are no results for the original query, but the index contains
matches for a similar term or phrase. Otherwise, this field is null.
.. _ora2:
======================================
Open Response Assessment Events
======================================
**History**: The events in this section record interactions with the prototype
implementation of open response assessment (ORA) problem types. As of May 2014,
new courses are not using this implementation of ORA and a complete redesign of
this feature is in limited release.
An open response assessment is a type of assignment that may not have
definitively correct or incorrect answers. Students review a question and
submit a text response and, optionally, an image file. To evaluate these
responses to the questions, students use an instructor-definfed scoring rubric
to assess different aspects of their own and one or more other students'
essays. For more information about open response assessments, see `Creating a
Peer Assessment`_.
``oe_hide_question`` and ``oe_show_question``
---------------------------------------------------------------------------
**Component**: Open Response Assessments
The browser emits ``oe_hide_question`` and ``oe_show_question`` events when the
user hides or redisplays a combined open-ended problem.
**History:** Limited release of this open response assessment feature began in
April 2014.
**History**: These events were previously named ``oe_hide_problem`` and
``oe_show_problem``.
openassessmentblock.get_peer_submission
----------------------------------------
**Component**: Combined Open-Ended
The server emits these events when an essay is delivered to a student for
evaluation. After students submit their own essays for evaluation, they use the
scoring rubric to evaluate the essays of other course participants.
**Event Source**: Browser
**Event Source**: Server
``event`` **Member Fields**:
......@@ -1738,18 +1791,41 @@ user hides or redisplays a combined open-ended problem.
* - Field
- Type
- Details
* - ``location``
* - ``course_id``
- string
- The location of the question whose prompt is being shown or hidden.
- The identifier of the course that includes this assessment. For open
response assessment problems, the course ID is stated in org/course/run
format.
``rubric_select``
----------------------
(For courses created after mid-2014, the course ID is converted to this
format for open response assessment problems only.)
**Component**: Combined Open-Ended
* - ``item_id``
- string
- The i4x:// style locator that identifies the problem in the course.
* - ``submission_returned_uuid``
- string
- The unique identifer of the submission that the student retrieved for
assessment.
**Event Source**: Browser
If no assessment is available, this is set to "None".
``event`` **Member Fields**:
* - ``requesting_student_id``
- string
- The course-specific anonymized user ID of the student who requested the
essay.
**History**: Added 3 April 2014.
openassessmentblock.peer_assess and openassessmentblock.self_assess
----------------------------------------------------------------------
The server emits these events when a student either submits an assessment of a
peer's essay or submits a self-assessment of her own essay.
**Event Source**: Server
``event`` **Fields**:
.. list-table::
:widths: 15 15 60
......@@ -1758,33 +1834,52 @@ user hides or redisplays a combined open-ended problem.
* - Field
- Type
- Details
* - ``location``
* - ``scorer_id``
- string
- The location of the question whose rubric is being selected.
* - ``selection``
- integer
- Value selected on rubric.
* - ``category``
- integer
- Rubric category selected.
- The course-specific anonymized user ID of the student who submitted this
assessment.
* - ``feedback``
- string
- The student's comments about the submitted essay.
* - ``submission_uuid``
- string
- The unique identifier for the submitted essay.
* - ``score_type``
- string
- "PE" for a peer evaluation, "SE" for a self evaluation.
* - ``parts: [criterion, option, feedback]``
- array
- The ``parts`` field contains member fields for each ``criterion`` in the
rubric, the ``option`` that the student selected for it, and any
``feedback`` comments that the student supplied.
``oe_show_full_feedback`` and ``oe_show_respond_to_feedback``
------------------------------------------------------------------
These member fields are repeated in an array to include all of the
rubric's criteria.
**Component**: Combined Open-Ended
* ``criterion`` (object) contains ``points possible`` and ``name``
member fields
* ``option`` (string)
* ``feedback`` (string)
**Event Source**: Browser
* - ``rubric``
- dict
- This field contains the member field ``contenthash``, which identifies
the rubric that the student used to assess the essay.
* - ``scored_at``
- datetime
- Timestamp for when the assessment was submitted.
``event`` **Member Fields**: None.
**History**: Added 3 April 2014.
``oe_feedback_response_selected``
--------------------------------------------
openassessmentblock.submit_feedback_on_assessments
----------------------------------------------------
**Component**: Combined Open-Ended
The server emits these events when a student submits a suggestion, opinion, or
other feedback about the assessment process.
**Event Source**: Browser
**Event Source**: Server
``event`` **Member Fields**:
``event`` **Fields**:
.. list-table::
:widths: 15 15 60
......@@ -1793,26 +1888,71 @@ user hides or redisplays a combined open-ended problem.
* - Field
- Type
- Details
* - ``value``
- integer
- Value selected in the feedback response form.
* - ``feedback_text``
- string
- The student's comments about the assessment process.
* - ``submission_uuid``
- string
- The unique identifier of the feedback submission.
* - ``options``
- array
- The label of each check box option that the student selected to evaluate
the assessment process.
``peer_grading_hide_question`` and ``peer_grading_show_question``
---------------------------------------------------------------------
**History**: Added 3 April 2014.
.. I couldn't find these names in any js file. peer_grading_problem.js includes oe_hide or show_question.
openassessment.create_submission
--------------------------------
The browser emits ``peer_grading_hide_question`` and
``peer_grading_show_question`` events when the user hides or redisplays a
problem that is peer graded.
The server emits these events each time a student submits an essay. The same
event is emitted when a student submits an essay for peer assessment or for
self assessment.
**History**: These events were previously named ``peer_grading_hide_problem`` and ``peer_grading_show_problem``.
**Event Source**: Server
**Component**: Peer Grading
``event`` **Fields**:
**Event Source**: Browser
.. list-table::
:widths: 15 15 60
:header-rows: 1
``event`` **Member Fields**:
* - Field
- Type
- Details
* - ``answer``
- dict
- This field contains a ``text`` (string) member field for the essay.
For submissions that also include an image file, this field contains a
``file_upload_key`` (string) member field with the AWS S3 key that
identifies the location of the image file on the Amazon S3 storage
service.
* - ``created_at``
- datetime
- Timestamp for when the student submitted the essay.
* - ``attempt_number``
- int
- This value is currently always set to 1.
* - ``submission_uuid``
- string
- The unique identifier of the submission.
* - ``submitted_at``
- datetime
- Timestamp for when the student submitted the essay. This value is
currently always the same as ``created_at``.
**History**: Added 3 April 2014.
openassessment.save_submission
-------------------------------
The server emits these events when a student saves a response essay. Note that
students save responses before they submit them for assessment.
**Event Source**: Server
``event`` **Fields**:
.. list-table::
:widths: 15 15 60
......@@ -1821,26 +1961,64 @@ problem that is peer graded.
* - Field
- Type
- Details
* - ``location``
- string
- The location of the question whose prompt is being shown or hidden.
* - ``saved_response``
- dict
- This field contains a ``text`` (string) member field for the essay.
``staff_grading_hide_question`` and ``staff_grading_show_question``
-----------------------------------------------------------------------
For submissions that also include an image file, this field contains a
``file_upload_key`` (string) member field with the AWS S3 key that
identifies the location of the image file on the Amazon S3 storage
service.
.. staff_grading.js
**History**: Added 3 April 2014.
The browser emits ``staff_grading_hide_question`` and
``staff_grading_show_question`` events when the user hides or redisplays a
problem that is staff graded.
openassessment.student_training_assess_example
-----------------------------------------------
**History**: These events were previously named ``staff_grading_hide_problem`` and ``staff_grading_show_problem``.
The server emits these events when a student submits an assessment for an
example essay. To assess the example essay, the student uses a scoring rubric
provided by the instructor. These events include the options the student
selected to assess the example, and identifies any criteria that the student
scored differently than the instructor.
**Component**: Staff Grading
**Event Source**: Server
``event`` **Fields**:
.. list-table::
:widths: 15 15 60
:header-rows: 1
* - Field
- Type
- Details
* - ``corrections``
- object
- A set of name/value pairs that identify criteria for which the student
selected a different option than the instructor, in the format
``criterion_name: instructor-defined_option_name``.
* - ``submission_uuid``
- string
- The unique identifier of the submission. Identifies the student who
is undergoing training.
* - ``options_selected``
- object
- A set of name/value pairs that identify the option that the student
selected for each criterion in the rubric, in the format
``'criterion_name': 'option_name'``.
**History**: Added 6 August 2014.
openassessment.upload_file
-----------------------------
The browser emits these events when a student successfully uploads an image
file as part of a response. Note that students complete the upload process
before they submit the response.
**Event Source**: Browser
``event`` **Member Fields**:
``event`` **Fields**:
.. list-table::
:widths: 15 15 60
......@@ -1849,9 +2027,19 @@ problem that is staff graded.
* - Field
- Type
- Details
* - ``location``
* - ``fileType``
- string
- The location of the question whose prompt is being shown or hidden.
- The MIME type of the uploaded file. Reported by the student's browser.
* - ``fileName``
- string
- The name of the uploaded file, as stored on the student's client
machine.
* - ``fileSize``
- int
- The size of the uploaded file in bytes. Reported by the student's
browser.
**History**: Added 6 August 2014.
.. _AB_Event_Types:
......@@ -1862,8 +2050,8 @@ A/B Testing Events
Course authors can configure course content to present modules that contain
other modules. For example, a parent module can include two child modules with
content that differs in some way for comparison testing. When a student
navigates to a module that is set up for A/B testing in this way, the student is
randomly assigned to a group and shown only one of the child modules.
navigates to a module that is set up for A/B testing in this way, the student
is randomly assigned to a group and shown only one of the child modules.
* Internally, a *partition* defines the type of experiment: comparing the
effectiveness of video alone to text alone, for example. A course can include
......@@ -1918,7 +2106,8 @@ If the student does not yet have an assignment, the server emits an
- Name of the group.
* - ``partition_id``
- integer
- Identifier for the partition, in the format ``xblock.partition_service.partition_ID`` where ID is an integer.
- Identifier for the partition, in the format
``xblock.partition_service.partition_ID`` where ID is an integer.
* - ``partition_name``
- string
- Name of the partition.
......@@ -1947,25 +2136,31 @@ the child module that was shown to the student.
- string
- ID of the module that displays to the student.
.. _forum_events:
.. _ora:
==========================
Forum Events
==========================
============================================
Open Response Assessment Events (Deprecated)
============================================
``edx.forum.searched``
----------------------------------
**History**: The events in this section recorded interactions with the
prototype implementation of open response assessment (ORA) problem types. As of
May 2014, new courses no longer used this implementation for open response
assessments.
After a user executes a text search in the navigation sidebar of the Discussion tab of a course, the server emits an ``edx.forum.searched`` event.
``oe_hide_question`` and ``oe_show_question``
---------------------------------------------------------------------------
**Component**: Discussion Tab
The browser emits ``oe_hide_question`` and ``oe_show_question`` events when the
user hides or redisplays a combined open-ended problem.
**Event Source**: Server
**History**: These events were previously named ``oe_hide_problem`` and
``oe_show_problem``.
**History**: Added 16 May 2014. The ``corrected_text`` field was added 5
Jun 2014.
**Component**: Combined Open-Ended
``event`` **Fields**:
**Event Source**: Browser
``event`` **Member Fields**:
.. list-table::
:widths: 15 15 60
......@@ -1974,22 +2169,122 @@ Jun 2014.
* - Field
- Type
- Details
* - ``query``
* - ``location``
- string
- The text entered into the search box by the user.
* - ``page``
- The location of the question whose prompt is being shown or hidden.
``rubric_select``
----------------------
**Component**: Combined Open-Ended
**Event Source**: Browser
``event`` **Member Fields**:
.. list-table::
:widths: 15 15 60
:header-rows: 1
* - Field
- Type
- Details
* - ``location``
- string
- The location of the question whose rubric is being selected.
* - ``selection``
- integer
- Results are returned in sets of 20 per page. Identifies the page of
results requested by the user.
* - ``total_results``
- Value selected on rubric.
* - ``category``
- integer
- The total number of results matching the query.
* - ``corrected_text``
- Rubric category selected.
``oe_show_full_feedback`` and ``oe_show_respond_to_feedback``
------------------------------------------------------------------
**Component**: Combined Open-Ended
**Event Source**: Browser
``event`` **Member Fields**: None.
``oe_feedback_response_selected``
--------------------------------------------
**Component**: Combined Open-Ended
**Event Source**: Browser
``event`` **Member Fields**:
.. list-table::
:widths: 15 15 60
:header-rows: 1
* - Field
- Type
- Details
* - ``value``
- integer
- Value selected in the feedback response form.
``peer_grading_hide_question`` and ``peer_grading_show_question``
---------------------------------------------------------------------
.. I couldn't find these names in any js file. peer_grading_problem.js includes oe_hide or show_question.
The browser emits ``peer_grading_hide_question`` and
``peer_grading_show_question`` events when the user hides or redisplays a
problem that is peer graded.
**History**: These events were previously named ``peer_grading_hide_problem``
and ``peer_grading_show_problem``.
**Component**: Peer Grading
**Event Source**: Browser
``event`` **Member Fields**:
.. list-table::
:widths: 15 15 60
:header-rows: 1
* - Field
- Type
- Details
* - ``location``
- string
- A re-spelling of the query, suggested by the search engine, which was
automatically substituted for the original one. This happens only when
there are no results for the original query, but the index contains
matches for a similar term or phrase. Otherwise, this field is null.
- The location of the question whose prompt is being shown or hidden.
``staff_grading_hide_question`` and ``staff_grading_show_question``
-----------------------------------------------------------------------
.. staff_grading.js
The browser emits ``staff_grading_hide_question`` and
``staff_grading_show_question`` events when the user hides or redisplays a
problem that is staff graded.
**History**: These events were previously named ``staff_grading_hide_problem``
and ``staff_grading_show_problem``.
**Component**: Staff Grading
**Event Source**: Browser
``event`` **Member Fields**:
.. list-table::
:widths: 15 15 60
:header-rows: 1
* - Field
- Type
- Details
* - ``location``
- string
- The location of the question whose prompt is being shown or hidden.
.. _Instructor_Event_Types:
......@@ -2000,8 +2295,8 @@ Instructor Events
This section lists the events that the server emits as a result of course team
interaction with the Instructor Dashboard in the LMS.
The schema definitions of each of these events include only the JSON fields that
are common to all events.
The schema definitions of each of these events include only the JSON fields
that are common to all events. See :ref:`common`.
* ``dump-answer-dist-csv``
* ``dump-graded-assignments-config``
......@@ -2274,3 +2569,6 @@ members also generate enrollment events.
server emits a ``edx.course.enrollment.deactivated`` for each unenrollment.
For details about the enrollment events, see :ref:`enrollment`.
.. _Creating a Peer Assessment: http://edx.readthedocs.org/projects/edx-open-response-assessments/en/latest/
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment