Merge branch 'master' into jonahstanley/add-gradingsettings-tests

AUTHORS

@@ -75,3 +75,4 @@ Frances Botsford <frances@edx.org>
 Jonah Stanley <Jonah_Stanley@brown.edu>
 Slater Victoroff <slater.r.victoroff@gmail.com>
 Peter Fogg <peter.p.fogg@gmail.com>
+Renzo Lucioni <renzolucioni@gmail.com>
\ No newline at end of file

CHANGELOG.rst

+Change Log
+----------
+
+These are notable changes in edx-platform.  This is a rolling list of changes,
+in roughly chronological order, most recent first.  Add your entries at or near
+the top.  Include a label indicating the component affected.
+
+LMS: Forums.  Added handling for case where discussion module can get `None` as
+value of lms.start in `lms/djangoapps/django_comment_client/utils.py`
+
+Studio, LMS: Make ModelTypes more strict about their expected content (for
+instance, Boolean, Integer, String), but also allow them to hold either the
+typed value, or a String that can be converted to their typed value. For example,
+an Integer can contain 3 or '3'. This changed an update to the xblock library.
+
+LMS: Courses whose id matches a regex in the COURSES_WITH_UNSAFE_CODE Django
+setting now run entirely outside the Python sandbox.
+
+Blades: Added tests for Video Alpha player.
+
+Blades: Video Alpha bug fix for speed changing to 1.0 in Firefox.
+
+Blades: Additional event tracking added to Video Alpha: fullscreen switch, show/hide
+captions.
+
+CMS: Allow editors to delete uploaded files/assets
+
+XModules: `XModuleDescriptor.__init__` and `XModule.__init__` dropped the
+`location` parameter (and added it as a field), and renamed `system` to `runtime`,
+to accord more closely to `XBlock.__init__`
+
+LMS: Some errors handling Non-ASCII data in XML courses have been fixed.
+
+LMS: Add page-load tracking using segment-io (if SEGMENT_IO_LMS_KEY and
+SEGMENT_IO_LMS feature flag is on)
+
+Blades: Simplify calc.py (which is used for the Numerical/Formula responses); add trig/other functions.
+
+LMS: Background colors on login, register, and courseware have been corrected
+back to white.
+
+LMS: Accessibility improvements have been made to several courseware and
+navigation elements.
+
+LMS: Small design/presentation changes to login and register views.
+
+LMS: Functionality added to instructor enrollment tab in LMS such that invited
+student can be auto-enrolled in course or when activating if not current
+student.
+
+Blades: Staff debug info is now accessible for Graphical Slider Tool problems.
+
+Blades: For Video Alpha the events ready, play, pause, seek, and speed change
+are logged on the server (in the logs).
+
+Common: all dates and times are not time zone aware datetimes. No code should create or use struct_times nor naive
+datetimes.
+
+Common: Developers can now have private Django settings files.
+
+Common: Safety code added to prevent anything above the vertical level in the
+course tree from being marked as version='draft'. It will raise an exception if
+the code tries to so mark a node. We need the backtraces to figure out where
+this very infrequent intermittent marking was occurring. It was making courses
+look different in Studio than in LMS.
+
+Deploy: MKTG_URLS is now read from env.json.
+
+Common: Theming makes it possible to change the look of the site, from
+Stanford.
+
+Common: Accessibility UI fixes.
+
+Common: The "duplicate email" error message is more informative.
+
+Studio: Component metadata settings editor.
+
+Studio: Autoplay for Video Alpha is disabled (only in Studio).
+
+Studio: Single-click creation for video and discussion components.
+
+Studio: fixed a bad link in the activation page.
+
+LMS: Changed the help button text.
+
+LMS: Fixed failing numeric response (decimal but no trailing digits).
+
+LMS: XML Error module no longer shows students a stack trace.
+
+Blades: Videoalpha.
+
+XModules: Added partial credit for foldit module.
+
+XModules: Added "randomize" XModule to list of XModule types.
+
+XModules: Show errors with full descriptors.
+
+XQueue: Fixed (hopefully) worker crash when the connection to RabbitMQ is
+dropped suddenly.
+
+XQueue: Upload file submissions to a specially named bucket in S3.
+
+Common: Removed request debugger.
+
+Common: Updated Django to version 1.4.5.
+
+Common: Updated CodeJail.
+
+Common: Allow setting of authentication session cookie name.
+

README.md

@@ -115,7 +115,7 @@ CMS templates. Fortunately, `rake` will do all of this for you! Just run:
 
 If you are running these commands using the [`zsh`](http://www.zsh.org/) shell,
 zsh will assume that you are doing
-[shell globbing](https://en.wikipedia.org/wiki/Glob_(programming)), search for
+[shell globbing](https://en.wikipedia.org/wiki/Glob_%28programming%29), search for
 a file in your directory named `django-adminsyncdb` or `django-adminmigrate`,
 and fail. To fix this, just surround the argument with quotation marks, so that
 you're running `rake "django-admin[syncdb]"`.

cms/CHANGELOG.md

-Instructions
-============
-For each pull request, add one or more lines to the bottom of the change list. When
-code is released to production, change the `Upcoming` entry to todays date, and add
-a new block at the bottom of the file.
-
-    Upcoming
-    --------
-
-Change log entries should be targeted at end users. A good place to start is the
-user story that instigated the pull request.
-
-
-Changes
-=======
-
-Upcoming
---------
-* Fix: Deleting last component in a unit does not work
-* Fix: Unit name is editable when a unit is public
-* Fix: Visual feedback inconsistent when saving a unit name change

cms/djangoapps/contentstore/features/advanced-settings.feature

@@ -28,11 +28,18 @@ Feature: Advanced (manual) course policy
 
   Scenario: Test how multi-line input appears
     Given I am on the Advanced Course Settings page in Studio
-    When I create a JSON object as a value
+    When I create a JSON object as a value for "discussion_topics"
     Then it is displayed as formatted
     And I reload the page
     Then it is displayed as formatted
 
+  Scenario: Test error if value supplied is of the wrong type
+    Given I am on the Advanced Course Settings page in Studio
+    When I create a JSON object as a value for "display_name"
+    Then I get an error on save
+    And I reload the page
+    Then the policy key value is unchanged
+
   Scenario: Test automatic quoting of non-JSON values
     Given I am on the Advanced Course Settings page in Studio
     When I create a non-JSON value not in quotes

cms/djangoapps/contentstore/features/advanced-settings.py

@@ -2,13 +2,8 @@
 #pylint: disable=W0621
 
 from lettuce import world, step
-from common import *
-from nose.tools import assert_false, assert_equal
-
-"""
-http://selenium.googlecode.com/svn/trunk/docs/api/py/webdriver/selenium.webdriver.common.keys.html
-"""
-from selenium.webdriver.common.keys import Keys
+from nose.tools import assert_false, assert_equal, assert_regexp_matches
+from common import type_in_codemirror
 
 KEY_CSS = '.key input.policy-key'
 VALUE_CSS = 'textarea.json'
@@ -38,13 +33,7 @@ def press_the_notification_button(step, name):
 
 @step(u'I edit the value of a policy key$')
 def edit_the_value_of_a_policy_key(step):
-    """
-    It is hard to figure out how to get into the CodeMirror
-    area, so cheat and do it from the policy key field :)
-    """
-    world.css_find(".CodeMirror")[get_index_of(DISPLAY_NAME_KEY)].click()
-    g = world.css_find("div.CodeMirror.CodeMirror-focused > div > textarea")
-    g._element.send_keys(Keys.ARROW_LEFT, ' ', 'X')
+    type_in_codemirror(get_index_of(DISPLAY_NAME_KEY), 'X')
 
 
 @step(u'I edit the value of a policy key and save$')
@@ -52,9 +41,9 @@ def edit_the_value_of_a_policy_key_and_save(step):
     change_display_name_value(step, '"foo"')
 
 
-@step('I create a JSON object as a value$')
-def create_JSON_object(step):
-    change_display_name_value(step, '{"key": "value", "key_2": "value_2"}')
+@step('I create a JSON object as a value for "(.*)"$')
+def create_JSON_object(step, key):
+    change_value(step, key, '{"key": "value", "key_2": "value_2"}')
 
 
 @step('I create a non-JSON value not in quotes$')
@@ -82,7 +71,12 @@ def they_are_alphabetized(step):
 
 @step('it is displayed as formatted$')
 def it_is_formatted(step):
-    assert_policy_entries([DISPLAY_NAME_KEY], ['{\n    "key": "value",\n    "key_2": "value_2"\n}'])
+    assert_policy_entries(['discussion_topics'], ['{\n    "key": "value",\n    "key_2": "value_2"\n}'])
+
+
+@step('I get an error on save$')
+def error_on_save(step):
+    assert_regexp_matches(world.css_text('#notification-error-description'), 'Incorrect setting format')
 
 
 @step('it is displayed as a string')
@@ -124,12 +118,9 @@ def get_display_name_value():
 
 
 def change_display_name_value(step, new_value):
+    change_value(step, DISPLAY_NAME_KEY, new_value)
+
 
-    world.css_find(".CodeMirror")[get_index_of(DISPLAY_NAME_KEY)].click()
-    g = world.css_find("div.CodeMirror.CodeMirror-focused > div > textarea")
-    display_name = get_display_name_value()
-    for count in range(len(display_name)):
-        g._element.send_keys(Keys.END, Keys.BACK_SPACE)
-        # Must delete "" before typing the JSON value
-    g._element.send_keys(Keys.END, Keys.BACK_SPACE, Keys.BACK_SPACE, new_value)
+def change_value(step, key, new_value):
+    type_in_codemirror(get_index_of(key), new_value)
     press_the_notification_button(step, "Save")

cms/djangoapps/contentstore/features/common.py

@@ -169,3 +169,14 @@ def open_new_unit(step):
     step.given('I have added a new subsection')
     step.given('I expand the first section')
     world.css_click('a.new-unit-item')
+
+
+def type_in_codemirror(index, text):
+    world.css_click(".CodeMirror", index=index)
+    g = world.css_find("div.CodeMirror.CodeMirror-focused > div > textarea")
+    if world.is_mac():
+        g._element.send_keys(Keys.COMMAND + 'a')
+    else:
+        g._element.send_keys(Keys.CONTROL + 'a')
+    g._element.send_keys(Keys.DELETE)
+    g._element.send_keys(text)

cms/djangoapps/contentstore/features/problem-editor.feature

@@ -3,65 +3,71 @@ Feature: Problem Editor
 
   Scenario: User can view metadata
     Given I have created a Blank Common Problem
-    And I edit and select Settings
+    When I edit and select Settings
     Then I see five alphabetized settings and their expected values
     And Edit High Level Source is not visible
 
   Scenario: User can modify String values
     Given I have created a Blank Common Problem
-    And I edit and select Settings
+    When I edit and select Settings
     Then I can modify the display name
     And my display name change is persisted on save
 
   Scenario: User can specify special characters in String values
     Given I have created a Blank Common Problem
-    And I edit and select Settings
+    When I edit and select Settings
     Then I can specify special characters in the display name
     And my special characters and persisted on save
 
   Scenario: User can revert display name to unset
     Given I have created a Blank Common Problem
-    And I edit and select Settings
+    When I edit and select Settings
     Then I can revert the display name to unset
     And my display name is unset on save
 
   Scenario: User can select values in a Select
     Given I have created a Blank Common Problem
-    And I edit and select Settings
+    When I edit and select Settings
     Then I can select Per Student for Randomization
     And my change to randomization is persisted
     And I can revert to the default value for randomization
 
   Scenario: User can modify float input values
     Given I have created a Blank Common Problem
-    And I edit and select Settings
+    When I edit and select Settings
     Then I can set the weight to "3.5"
     And my change to weight is persisted
     And I can revert to the default value of unset for weight
 
   Scenario: User cannot type letters in float number field
     Given I have created a Blank Common Problem
-    And I edit and select Settings
+    When I edit and select Settings
     Then if I set the weight to "abc", it remains unset
 
   Scenario: User cannot type decimal values integer number field
     Given I have created a Blank Common Problem
-    And I edit and select Settings
+    When I edit and select Settings
     Then if I set the max attempts to "2.34", it displays initially as "234", and is persisted as "234"
 
   Scenario: User cannot type out of range values in an integer number field
     Given I have created a Blank Common Problem
-    And I edit and select Settings
-    Then if I set the max attempts to "-3", it displays initially as "-3", and is persisted as "1"
+    When I edit and select Settings
+    Then if I set the max attempts to "-3", it displays initially as "-3", and is persisted as "0"
 
   Scenario: Settings changes are not saved on Cancel
     Given I have created a Blank Common Problem
-    And I edit and select Settings
+    When I edit and select Settings
     Then I can set the weight to "3.5"
     And I can modify the display name
     Then If I press Cancel my changes are not persisted
 
   Scenario: Edit High Level source is available for LaTeX problem
     Given I have created a LaTeX Problem
-    And I edit and select Settings
+    When I edit and select Settings
     Then Edit High Level Source is visible
+
+  Scenario: High Level source is persisted for LaTeX problem (bug STUD-280)
+    Given I have created a LaTeX Problem
+    When I edit and compile the High Level Source
+    Then my change to the High Level Source is persisted
+    And when I view the High Level Source I see my changes

cms/djangoapps/contentstore/features/problem-editor.py

@@ -3,6 +3,7 @@
 
 from lettuce import world, step
 from nose.tools import assert_equal
+from common import type_in_codemirror
 
 DISPLAY_NAME = "Display Name"
 MAXIMUM_ATTEMPTS = "Maximum Attempts"
@@ -41,7 +42,9 @@ def i_see_five_settings_with_values(step):
 
 @step('I can modify the display name')
 def i_can_modify_the_display_name(step):
-    world.get_setting_entry(DISPLAY_NAME).find_by_css('.setting-input')[0].fill('modified')
+    # Verifying that the display name can be a string containing a floating point value
+    # (to confirm that we don't throw an error because it is of the wrong type).
+    world.get_setting_entry(DISPLAY_NAME).find_by_css('.setting-input')[0].fill('3.4')
     verify_modified_display_name()
 
 
@@ -133,12 +136,12 @@ def set_the_max_attempts(step, max_attempts_set, max_attempts_displayed, max_att
 
 @step('Edit High Level Source is not visible')
 def edit_high_level_source_not_visible(step):
-    verify_high_level_source(step, False)
+    verify_high_level_source_links(step, False)
 
 
 @step('Edit High Level Source is visible')
-def edit_high_level_source_visible(step):
-    verify_high_level_source(step, True)
+def edit_high_level_source_links_visible(step):
+    verify_high_level_source_links(step, True)
 
 
 @step('If I press Cancel my changes are not persisted')
@@ -151,13 +154,33 @@ def cancel_does_not_save_changes(step):
 @step('I have created a LaTeX Problem')
 def create_latex_problem(step):
     world.click_new_component_button(step, '.large-problem-icon')
-    # Go to advanced tab (waiting for the tab to be visible)
-    world.css_find('#ui-id-2')
+    # Go to advanced tab.
     world.css_click('#ui-id-2')
     world.click_component_from_menu("i4x://edx/templates/problem/Problem_Written_in_LaTeX", '.xmodule_CapaModule')
 
 
-def verify_high_level_source(step, visible):
+@step('I edit and compile the High Level Source')
+def edit_latex_source(step):
+    open_high_level_source()
+    type_in_codemirror(1, "hi")
+    world.css_click('.hls-compile')
+
+
+@step('my change to the High Level Source is persisted')
+def high_level_source_persisted(step):
+    def verify_text(driver):
+        return world.css_find('.problem').text == 'hi'
+
+    world.wait_for(verify_text)
+
+
+@step('I view the High Level Source I see my changes')
+def high_level_source_in_editor(step):
+    open_high_level_source()
+    assert_equal('hi', world.css_find('.source-edit-box').value)
+
+
+def verify_high_level_source_links(step, visible):
     assert_equal(visible, world.is_css_present('.launch-latex-compiler'))
     world.cancel_component(step)
     assert_equal(visible, world.is_css_present('.upload-button'))
@@ -172,7 +195,7 @@ def verify_modified_randomization():
 
 
 def verify_modified_display_name():
-    world.verify_setting_entry(world.get_setting_entry(DISPLAY_NAME), DISPLAY_NAME, 'modified', True)
+    world.verify_setting_entry(world.get_setting_entry(DISPLAY_NAME), DISPLAY_NAME, '3.4', True)
 
 
 def verify_modified_display_name_with_special_chars():
@@ -185,3 +208,8 @@ def verify_unset_display_name():
 
 def set_weight(weight):
     world.get_setting_entry(PROBLEM_WEIGHT).find_by_css('.setting-input')[0].fill(weight)
+
+
+def open_high_level_source():
+    world.css_click('a.edit-button')
+    world.css_click('.launch-latex-compiler > a')

cms/djangoapps/contentstore/features/section.py

@@ -9,34 +9,34 @@ from nose.tools import assert_equal
 
 
 @step('I click the new section link$')
-def i_click_new_section_link(step):
+def i_click_new_section_link(_step):
     link_css = 'a.new-courseware-section-button'
     world.css_click(link_css)
 
 
 @step('I enter the section name and click save$')
-def i_save_section_name(step):
+def i_save_section_name(_step):
     save_section_name('My Section')
 
 
 @step('I enter a section name with a quote and click save$')
-def i_save_section_name_with_quote(step):
+def i_save_section_name_with_quote(_step):
     save_section_name('Section with "Quote"')
 
 
 @step('I have added a new section$')
-def i_have_added_new_section(step):
+def i_have_added_new_section(_step):
     add_section()
 
 
 @step('I click the Edit link for the release date$')
-def i_click_the_edit_link_for_the_release_date(step):
+def i_click_the_edit_link_for_the_release_date(_step):
     button_css = 'div.section-published-date a.edit-button'
     world.css_click(button_css)
 
 
 @step('I save a new section release date$')
-def i_save_a_new_section_release_date(step):
+def i_save_a_new_section_release_date(_step):
     set_date_and_time('input.start-date.date.hasDatepicker', '12/25/2013',
         'input.start-time.time.ui-timepicker-input', '00:00')
     world.browser.click_link_by_text('Save')
@@ -46,35 +46,35 @@ def i_save_a_new_section_release_date(step):
 
 
 @step('I see my section on the Courseware page$')
-def i_see_my_section_on_the_courseware_page(step):
+def i_see_my_section_on_the_courseware_page(_step):
     see_my_section_on_the_courseware_page('My Section')
 
 
 @step('I see my section name with a quote on the Courseware page$')
-def i_see_my_section_name_with_quote_on_the_courseware_page(step):
+def i_see_my_section_name_with_quote_on_the_courseware_page(_step):
     see_my_section_on_the_courseware_page('Section with "Quote"')
 
 
 @step('I click to edit the section name$')
-def i_click_to_edit_section_name(step):
+def i_click_to_edit_section_name(_step):
     world.css_click('span.section-name-span')
 
 
 @step('I see the complete section name with a quote in the editor$')
-def i_see_complete_section_name_with_quote_in_editor(step):
+def i_see_complete_section_name_with_quote_in_editor(_step):
     css = '.section-name-edit input[type=text]'
     assert world.is_css_present(css)
     assert_equal(world.browser.find_by_css(css).value, 'Section with "Quote"')
 
 
 @step('the section does not exist$')
-def section_does_not_exist(step):
+def section_does_not_exist(_step):
     css = 'h3[data-name="My Section"]'
     assert world.is_css_not_present(css)
 
 
 @step('I see a release date for my section$')
-def i_see_a_release_date_for_my_section(step):
+def i_see_a_release_date_for_my_section(_step):
     import re
 
     css = 'span.published-status'
@@ -83,26 +83,32 @@ def i_see_a_release_date_for_my_section(step):
 
     # e.g. 11/06/2012 at 16:25
     msg = 'Will Release:'
-    date_regex = '[01][0-9]\/[0-3][0-9]\/[12][0-9][0-9][0-9]'
-    time_regex = '[0-2][0-9]:[0-5][0-9]'
-    match_string = '%s %s at %s' % (msg, date_regex, time_regex)
+    date_regex = r'(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec) \d\d?, \d{4}'
+    if not re.search(date_regex, status_text):
+        print status_text, date_regex
+    time_regex = r'[0-2]\d:[0-5]\d( \w{3})?'
+    if not re.search(time_regex, status_text):
+        print status_text, time_regex
+    match_string = r'%s\s+%s at %s' % (msg, date_regex, time_regex)
+    if not re.match(match_string, status_text):
+        print status_text, match_string
     assert re.match(match_string, status_text)
 
 
 @step('I see a link to create a new subsection$')
-def i_see_a_link_to_create_a_new_subsection(step):
+def i_see_a_link_to_create_a_new_subsection(_step):
     css = 'a.new-subsection-item'
     assert world.is_css_present(css)
 
 
 @step('the section release date picker is not visible$')
-def the_section_release_date_picker_not_visible(step):
+def the_section_release_date_picker_not_visible(_step):
     css = 'div.edit-subsection-publish-settings'
     assert not world.css_visible(css)
 
 
 @step('the section release date is updated$')
-def the_section_release_date_is_updated(step):
+def the_section_release_date_is_updated(_step):
     css = 'span.published-status'
     status_text = world.css_text(css)
     assert_equal(status_text, 'Will Release: 12/25/2013 at 00:00 UTC')

cms/djangoapps/contentstore/features/video.feature

@@ -8,3 +8,8 @@ Feature: Video Component
   Scenario: Creating a video takes a single click
     Given I have clicked the new unit button
     Then creating a video takes a single click
+
+  Scenario: Captions are shown correctly
+    Given I have created a Video component
+    And I have hidden captions
+    Then when I view the video it does not show the captions

cms/djangoapps/contentstore/features/video.py

@@ -16,3 +16,13 @@ def video_takes_a_single_click(step):
     assert(not world.is_css_present('.xmodule_VideoModule'))
     world.css_click("a[data-location='i4x://edx/templates/video/default']")
     assert(world.is_css_present('.xmodule_VideoModule'))
+
+
+@step('I have hidden captions')
+def set_show_captions_false(step):
+    world.css_click('a.hide-subtitles')
+
+
+@step('when I view the video it does not show the captions')
+def does_not_show_captions(step):
+    assert world.css_find('.video')[0].has_class('closed')

cms/djangoapps/contentstore/management/commands/empty_asset_trashcan.py

+from django.core.management.base import BaseCommand, CommandError
+from xmodule.course_module import CourseDescriptor
+from xmodule.contentstore.utils import empty_asset_trashcan
+from xmodule.modulestore.django import modulestore
+from .prompt import query_yes_no
+
+
+class Command(BaseCommand):
+    help = '''Empty the trashcan. Can pass an optional course_id to limit the damage.'''
+
+    def handle(self, *args, **options):
+        if len(args) != 1 and len(args) != 0:
+            raise CommandError("empty_asset_trashcan requires one or no arguments: |<location>|")
+
+        locs = []
+
+        if len(args) == 1:
+            locs.append(CourseDescriptor.id_to_location(args[0]))
+        else:
+            courses = modulestore('direct').get_courses()
+            for course in courses:
+                locs.append(course.location)
+
+        if query_yes_no("Emptying trashcan. Confirm?", default="no"):
+            empty_asset_trashcan(locs)

cms/djangoapps/contentstore/management/commands/restore_asset_from_trashcan.py

+from django.core.management.base import BaseCommand, CommandError
+from xmodule.contentstore.utils import restore_asset_from_trashcan
+
+
+class Command(BaseCommand):
+    help = '''Restore a deleted asset from the trashcan back to it's original course'''
+
+    def handle(self, *args, **options):
+        if len(args) != 1 and len(args) != 0:
+            raise CommandError("restore_asset_from_trashcan requires one argument: <location>")
+
+        restore_asset_from_trashcan(args[0])
+

cms/djangoapps/contentstore/tests/test_checklists.py

@@ -19,6 +19,24 @@ class ChecklistTestCase(CourseTestCase):
         modulestore = get_modulestore(self.course.location)
         return modulestore.get_item(self.course.location).checklists
 
+
+    def compare_checklists(self, persisted, request):
+        """
+        Handles url expansion as possible difference and descends into guts
+        :param persisted:
+        :param request:
+        """
+        self.assertEqual(persisted['short_description'], request['short_description'])
+        compare_urls = (persisted.get('action_urls_expanded') == request.get('action_urls_expanded'))
+        for pers, req in zip(persisted['items'], request['items']):
+            self.assertEqual(pers['short_description'], req['short_description'])
+        self.assertEqual(pers['long_description'], req['long_description'])
+        self.assertEqual(pers['is_checked'], req['is_checked'])
+        if compare_urls:
+            self.assertEqual(pers['action_url'], req['action_url'])
+        self.assertEqual(pers['action_text'], req['action_text'])
+        self.assertEqual(pers['action_external'], req['action_external'])
+
     def test_get_checklists(self):
         """ Tests the get checklists method. """
         checklists_url = get_url_reverse('Checklists', self.course)
@@ -31,9 +49,9 @@ class ChecklistTestCase(CourseTestCase):
         self.course.checklists = None
         modulestore = get_modulestore(self.course.location)
         modulestore.update_metadata(self.course.location, own_metadata(self.course))
-        self.assertEquals(self.get_persisted_checklists(), None)
+        self.assertEqual(self.get_persisted_checklists(), None)
         response = self.client.get(checklists_url)
-        self.assertEquals(payload, response.content)
+        self.assertEqual(payload, response.content)
 
     def test_update_checklists_no_index(self):
         """ No checklist index, should return all of them. """
@@ -43,7 +61,8 @@ class ChecklistTestCase(CourseTestCase):
             'name': self.course.location.name})
 
         returned_checklists = json.loads(self.client.get(update_url).content)
-        self.assertListEqual(self.get_persisted_checklists(), returned_checklists)
+        for pay, resp in zip(self.get_persisted_checklists(), returned_checklists):
+            self.compare_checklists(pay, resp)
 
     def test_update_checklists_index_ignored_on_get(self):
         """ Checklist index ignored on get. """
@@ -53,7 +72,8 @@ class ChecklistTestCase(CourseTestCase):
                                                            'checklist_index': 1})
 
         returned_checklists = json.loads(self.client.get(update_url).content)
-        self.assertListEqual(self.get_persisted_checklists(), returned_checklists)
+        for pay, resp in zip(self.get_persisted_checklists(), returned_checklists):
+            self.compare_checklists(pay, resp)
 
     def test_update_checklists_post_no_index(self):
         """ No checklist index, will error on post. """
@@ -78,13 +98,18 @@ class ChecklistTestCase(CourseTestCase):
                                                            'course': self.course.location.course,
                                                            'name': self.course.location.name,
                                                            'checklist_index': 2})
+
+        def get_first_item(checklist):
+            return checklist['items'][0]
+
         payload = self.course.checklists[2]
-        self.assertFalse(payload.get('is_checked'))
-        payload['is_checked'] = True
+        self.assertFalse(get_first_item(payload).get('is_checked'))
+        get_first_item(payload)['is_checked'] = True
 
         returned_checklist = json.loads(self.client.post(update_url, json.dumps(payload), "application/json").content)
-        self.assertTrue(returned_checklist.get('is_checked'))
-        self.assertEqual(self.get_persisted_checklists()[2], returned_checklist)
+        self.assertTrue(get_first_item(returned_checklist).get('is_checked'))
+        pers = self.get_persisted_checklists()
+        self.compare_checklists(pers[2], returned_checklist)
 
     def test_update_checklists_delete_unsupported(self):
         """ Delete operation is not supported. """
@@ -93,4 +118,4 @@ class ChecklistTestCase(CourseTestCase):
                                                            'name': self.course.location.name,
                                                            'checklist_index': 100})
         response = self.client.delete(update_url)
-        self.assertContains(response, 'Unsupported request', status_code=400)
+        self.assertContains(response, 'Unsupported request', status_code=400)
\ No newline at end of file

cms/djangoapps/contentstore/tests/test_course_settings.py

@@ -54,6 +54,7 @@ class CourseDetailsTestCase(CourseTestCase):
     def test_virgin_fetch(self):
         details = CourseDetails.fetch(self.course_location)
         self.assertEqual(details.course_location, self.course_location, "Location not copied into")
+        self.assertIsNotNone(details.start_date.tzinfo)
         self.assertIsNone(details.end_date, "end date somehow initialized " + str(details.end_date))
         self.assertIsNone(details.enrollment_start, "enrollment_start date somehow initialized " + str(details.enrollment_start))
         self.assertIsNone(details.enrollment_end, "enrollment_end date somehow initialized " + str(details.enrollment_end))
@@ -67,7 +68,6 @@ class CourseDetailsTestCase(CourseTestCase):
         jsondetails = json.dumps(details, cls=CourseSettingsEncoder)
         jsondetails = json.loads(jsondetails)
         self.assertTupleEqual(Location(jsondetails['course_location']), self.course_location, "Location !=")
-        # Note, start_date is being initialized someplace. I'm not sure why b/c the default will make no sense.
         self.assertIsNone(jsondetails['end_date'], "end date somehow initialized ")
         self.assertIsNone(jsondetails['enrollment_start'], "enrollment_start date somehow initialized ")
         self.assertIsNone(jsondetails['enrollment_end'], "enrollment_end date somehow initialized ")
@@ -76,6 +76,23 @@ class CourseDetailsTestCase(CourseTestCase):
         self.assertIsNone(jsondetails['intro_video'], "intro_video somehow initialized")
         self.assertIsNone(jsondetails['effort'], "effort somehow initialized")
 
+    def test_ooc_encoder(self):
+        """
+        Test the encoder out of its original constrained purpose to see if it functions for general use
+        """
+        details = {'location': Location(['tag', 'org', 'course', 'category', 'name']),
+            'number': 1,
+            'string': 'string',
+            'datetime': datetime.datetime.now(UTC())}
+        jsondetails = json.dumps(details, cls=CourseSettingsEncoder)
+        jsondetails = json.loads(jsondetails)
+
+        self.assertIn('location', jsondetails)
+        self.assertIn('org', jsondetails['location'])
+        self.assertEquals('org', jsondetails['location'][1])
+        self.assertEquals(1, jsondetails['number'])
+        self.assertEqual(jsondetails['string'], 'string')
+
     def test_update_and_fetch(self):
         # # NOTE: I couldn't figure out how to validly test time setting w/ all the conversions
         jsondetails = CourseDetails.fetch(self.course_location)
@@ -116,11 +133,8 @@ class CourseDetailsViewTest(CourseTestCase):
         self.compare_details_with_encoding(json.loads(resp.content), details.__dict__, field + str(val))
 
     @staticmethod
-    def convert_datetime_to_iso(datetime):
-        if datetime is not None:
-            return datetime.isoformat("T")
-        else:
-            return None
+    def convert_datetime_to_iso(dt):
+        return Date().to_json(dt)
 
     def test_update_and_fetch(self):
         details = CourseDetails.fetch(self.course_location)
@@ -151,22 +165,12 @@ class CourseDetailsViewTest(CourseTestCase):
         self.assertEqual(details['intro_video'], encoded.get('intro_video', None), context + " intro_video not ==")
         self.assertEqual(details['effort'], encoded['effort'], context + " efforts not ==")
 
-    @staticmethod
-    def struct_to_datetime(struct_time):
-        return datetime.datetime(*struct_time[:6], tzinfo=UTC())
-
     def compare_date_fields(self, details, encoded, context, field):
         if details[field] is not None:
             date = Date()
             if field in encoded and encoded[field] is not None:
-                encoded_encoded = date.from_json(encoded[field])
-                dt1 = CourseDetailsViewTest.struct_to_datetime(encoded_encoded)
-
-                if isinstance(details[field], datetime.datetime):
-                    dt2 = details[field]
-                else:
-                    details_encoded = date.from_json(details[field])
-                    dt2 = CourseDetailsViewTest.struct_to_datetime(details_encoded)
+                dt1 = date.from_json(encoded[field])
+                dt2 = details[field]
 
                 expected_delta = datetime.timedelta(0)
                 self.assertEqual(dt1 - dt2, expected_delta, str(dt1) + "!=" + str(dt2) + " at " + context)

cms/djangoapps/contentstore/tests/tests.py

@@ -3,6 +3,10 @@ from django.core.urlresolvers import reverse
 
 from .utils import parse_json, user, registration
 from xmodule.modulestore.tests.django_utils import ModuleStoreTestCase
+from contentstore.tests.test_course_settings import CourseTestCase
+from xmodule.modulestore.tests.factories import CourseFactory
+import datetime
+from pytz import UTC
 
 
 class ContentStoreTestCase(ModuleStoreTestCase):
@@ -162,3 +166,21 @@ class AuthTestCase(ContentStoreTestCase):
         self.assertEqual(resp.status_code, 302)
 
         # Logged in should work.
+
+
+class ForumTestCase(CourseTestCase):
+    def setUp(self):
+        """ Creates the test course. """
+        super(ForumTestCase, self).setUp()
+        self.course = CourseFactory.create(org='testX', number='727', display_name='Forum Course')
+
+    def test_blackouts(self):
+        now = datetime.datetime.now(UTC)
+        self.course.discussion_blackouts = [(t.isoformat(), t2.isoformat()) for t, t2 in
+            [(now - datetime.timedelta(days=14), now - datetime.timedelta(days=11)),
+                (now + datetime.timedelta(days=24), now + datetime.timedelta(days=30))]]
+        self.assertTrue(self.course.forum_posts_allowed)
+        self.course.discussion_blackouts = [(t.isoformat(), t2.isoformat()) for t, t2 in
+            [(now - datetime.timedelta(days=14), now + datetime.timedelta(days=2)),
+                (now + datetime.timedelta(days=24), now + datetime.timedelta(days=30))]]
+        self.assertFalse(self.course.forum_posts_allowed)

cms/djangoapps/contentstore/utils.py

@@ -6,11 +6,10 @@ from django.core.urlresolvers import reverse
 import copy
 import logging
 import re
+from xmodule.modulestore.draft import DIRECT_ONLY_CATEGORIES
 
 log = logging.getLogger(__name__)
 
-DIRECT_ONLY_CATEGORIES = ['course', 'chapter', 'sequential', 'about', 'static_tab', 'course_info']
-
 #In order to instantiate an open ended tab automatically, need to have this data
 OPEN_ENDED_PANEL = {"name": "Open Ended Panel", "type": "open_ended"}
 NOTES_PANEL = {"name": "My Notes", "type": "notes"}
@@ -229,7 +228,7 @@ def add_extra_panel_tab(tab_type, course):
     course_tabs = copy.copy(course.tabs)
     changed = False
     #Check to see if open ended panel is defined in the course
-    
+
     tab_panel = EXTRA_TAB_PANELS.get(tab_type)
     if tab_panel not in course_tabs:
         #Add panel to the tabs if it is not defined

cms/djangoapps/contentstore/views/assets.py

@@ -25,6 +25,8 @@ from xmodule.modulestore.django import modulestore
 from xmodule.modulestore import Location
 from xmodule.contentstore.content import StaticContent
 from xmodule.util.date_utils import get_default_time_display
+from xmodule.modulestore import InvalidLocationError
+from xmodule.exceptions import NotFoundError
 
 from ..utils import get_url_reverse
 from .access import get_location_and_verify_access
@@ -62,7 +64,7 @@ def asset_index(request, org, course, name):
         asset_id = asset['_id']
         display_info = {}
         display_info['displayname'] = asset['displayname']
-        display_info['uploadDate'] = get_default_time_display(asset['uploadDate'].timetuple())
+        display_info['uploadDate'] = get_default_time_display(asset['uploadDate'])
 
         asset_location = StaticContent.compute_location(asset_id['org'], asset_id['course'], asset_id['name'])
         display_info['url'] = StaticContent.get_url_path_from_location(asset_location)
@@ -78,10 +80,17 @@ def asset_index(request, org, course, name):
         'active_tab': 'assets',
         'context_course': course_module,
         'assets': asset_display,
-        'upload_asset_callback_url': upload_asset_callback_url
+        'upload_asset_callback_url': upload_asset_callback_url,
+        'remove_asset_callback_url': reverse('remove_asset', kwargs={
+            'org': org,
+            'course': course,
+            'name': name
+        })
     })
 
 
+@login_required
+@ensure_csrf_cookie
 def upload_asset(request, org, course, coursename):
     '''
     cdodge: this method allows for POST uploading of files into the course asset library, which will
@@ -103,6 +112,9 @@ def upload_asset(request, org, course, coursename):
         logging.error('Could not find course' + location)
         return HttpResponseBadRequest()
 
+    if 'file' not in request.FILES:
+        return HttpResponseBadRequest()
+
     # compute a 'filename' which is similar to the location formatting, we're using the 'filename'
     # nomenclature since we're using a FileSystem paradigm here. We're just imposing
     # the Location string formatting expectations to keep things a bit more consistent
@@ -131,7 +143,7 @@ def upload_asset(request, org, course, coursename):
     readback = contentstore().find(content.location)
 
     response_payload = {'displayname': content.name,
-                        'uploadDate': get_default_time_display(readback.last_modified_at.timetuple()),
+                        'uploadDate': get_default_time_display(readback.last_modified_at),
                         'url': StaticContent.get_url_path_from_location(content.location),
                         'thumb_url': StaticContent.get_url_path_from_location(thumbnail_location) if thumbnail_content is not None else None,
                         'msg': 'Upload completed'
@@ -144,6 +156,57 @@ def upload_asset(request, org, course, coursename):
 
 @ensure_csrf_cookie
 @login_required
+def remove_asset(request, org, course, name):
+    '''
+    This method will perform a 'soft-delete' of an asset, which is basically to copy the asset from
+    the main GridFS collection and into a Trashcan
+    '''
+    get_location_and_verify_access(request, org, course, name)
+
+    location = request.POST['location']
+
+    # make sure the location is valid
+    try:
+        loc = StaticContent.get_location_from_path(location)
+    except InvalidLocationError:
+        # return a 'Bad Request' to browser as we have a malformed Location
+        response = HttpResponse()
+        response.status_code = 400
+        return response
+
+    # also make sure the item to delete actually exists
+    try:
+        content = contentstore().find(loc)
+    except NotFoundError:
+        response = HttpResponse()
+        response.status_code = 404
+        return response
+
+    # ok, save the content into the trashcan
+    contentstore('trashcan').save(content)
+
+    # see if there is a thumbnail as well, if so move that as well
+    if content.thumbnail_location is not None:
+        try:
+            thumbnail_content = contentstore().find(content.thumbnail_location)
+            contentstore('trashcan').save(thumbnail_content)
+            # hard delete thumbnail from origin
+            contentstore().delete(thumbnail_content.get_id())
+            # remove from any caching
+            del_cached_content(thumbnail_content.location)
+        except:
+            pass  # OK if this is left dangling
+
+    # delete the original
+    contentstore().delete(content.get_id())
+    # remove from cache
+    del_cached_content(content.location)
+
+    return HttpResponse()
+
+
+@ensure_csrf_cookie
+@login_required
 def import_course(request, org, course, name):
 
     location = get_location_and_verify_access(request, org, course, name)
@@ -227,11 +290,9 @@ def generate_export_course(request, org, course, name):
     root_dir = path(mkdtemp())
 
     # export out to a tempdir
-
     logging.debug('root = {0}'.format(root_dir))
 
     export_to_xml(modulestore('direct'), contentstore(), loc, root_dir, name, modulestore())
-    #filename = root_dir / name + '.tar.gz'
 
     logging.debug('tar file being generated at {0}'.format(export_file.name))
     tar_file = tarfile.open(name=export_file.name, mode='w:gz')

cms/djangoapps/contentstore/views/component.py

@@ -7,7 +7,7 @@ from django.contrib.auth.decorators import login_required
 from django.core.exceptions import PermissionDenied
 from django_future.csrf import ensure_csrf_cookie
 from django.conf import settings
-
+from xmodule.modulestore.exceptions import ItemNotFoundError, InvalidLocationError
 from mitxmako.shortcuts import render_to_response
 
 from xmodule.modulestore import Location
@@ -50,11 +50,18 @@ ADVANCED_COMPONENT_POLICY_KEY = 'advanced_modules'
 @login_required
 def edit_subsection(request, location):
     # check that we have permissions to edit this item
-    course = get_course_for_item(location)
+    try:
+        course = get_course_for_item(location)
+    except InvalidLocationError:
+        return HttpResponseBadRequest()
+
     if not has_access(request.user, course.location):
         raise PermissionDenied()
 
-    item = modulestore().get_item(location, depth=1)
+    try:
+        item = modulestore().get_item(location, depth=1)
+    except ItemNotFoundError:
+        return HttpResponseBadRequest()
 
     lms_link = get_lms_link_for_item(location, course_id=course.location.course_id)
     preview_link = get_lms_link_for_item(location, course_id=course.location.course_id, preview=True)
@@ -113,11 +120,18 @@ def edit_unit(request, location):
 
     id: A Location URL
     """
-    course = get_course_for_item(location)
+    try:
+        course = get_course_for_item(location)
+    except InvalidLocationError:
+        return HttpResponseBadRequest()
+
     if not has_access(request.user, course.location):
         raise PermissionDenied()
 
-    item = modulestore().get_item(location, depth=1)
+    try:
+        item = modulestore().get_item(location, depth=1)
+    except ItemNotFoundError:
+        return HttpResponseBadRequest()
 
     lms_link = get_lms_link_for_item(item.location, course_id=course.location.course_id)
 

cms/djangoapps/contentstore/views/course.py

@@ -2,7 +2,6 @@
 Views related to operations on course objects
 """
 import json
-import time
 
 from django.contrib.auth.decorators import login_required
 from django_future.csrf import ensure_csrf_cookie
@@ -32,6 +31,8 @@ from .component import OPEN_ENDED_COMPONENT_TYPES, \
      NOTE_COMPONENT_TYPES, ADVANCED_COMPONENT_POLICY_KEY
 
 from django_comment_common.utils import seed_permissions_roles
+import datetime
+from django.utils.timezone import UTC
 
 # TODO: should explicitly enumerate exports with __all__
 
@@ -130,7 +131,7 @@ def create_new_course(request):
         new_course.display_name = display_name
 
     # set a default start date to now
-    new_course.start = time.gmtime()
+    new_course.start = datetime.datetime.now(UTC())
 
     initialize_course_tabs(new_course)
 
@@ -357,52 +358,55 @@ def course_advanced_updates(request, org, course, name):
         # Whether or not to filter the tabs key out of the settings metadata
         filter_tabs = True
 
-        #Check to see if the user instantiated any advanced components. This is a hack
-        #that does the following : 
-        #   1) adds/removes the open ended panel tab to a course automatically if the user 
+        # Check to see if the user instantiated any advanced components. This is a hack
+        # that does the following :
+        #   1) adds/removes the open ended panel tab to a course automatically if the user
         #   has indicated that they want to edit the combinedopendended or peergrading module
         #   2) adds/removes the notes panel tab to a course automatically if the user has
         #   indicated that they want the notes module enabled in their course
         # TODO refactor the above into distinct advanced policy settings
         if ADVANCED_COMPONENT_POLICY_KEY in request_body:
-            #Get the course so that we can scrape current tabs
+            # Get the course so that we can scrape current tabs
             course_module = modulestore().get_item(location)
 
-            #Maps tab types to components 
+            # Maps tab types to components
             tab_component_map = {
-                'open_ended': OPEN_ENDED_COMPONENT_TYPES, 
+                'open_ended': OPEN_ENDED_COMPONENT_TYPES,
                 'notes': NOTE_COMPONENT_TYPES,
             }
 
-            #Check to see if the user instantiated any notes or open ended components
+            # Check to see if the user instantiated any notes or open ended components
             for tab_type in tab_component_map.keys():
                 component_types = tab_component_map.get(tab_type)
                 found_ac_type = False
                 for ac_type in component_types:
                     if ac_type in request_body[ADVANCED_COMPONENT_POLICY_KEY]:
-                        #Add tab to the course if needed
+                        # Add tab to the course if needed
                         changed, new_tabs = add_extra_panel_tab(tab_type, course_module)
-                        #If a tab has been added to the course, then send the metadata along to CourseMetadata.update_from_json
+                        # If a tab has been added to the course, then send the metadata along to CourseMetadata.update_from_json
                         if changed:
                             course_module.tabs = new_tabs
                             request_body.update({'tabs': new_tabs})
-                            #Indicate that tabs should not be filtered out of the metadata
+                            # Indicate that tabs should not be filtered out of the metadata
                             filter_tabs = False
-                        #Set this flag to avoid the tab removal code below.
+                        # Set this flag to avoid the tab removal code below.
                         found_ac_type = True
                         break
-                #If we did not find a module type in the advanced settings,
+                # If we did not find a module type in the advanced settings,
                 # we may need to remove the tab from the course.
                 if not found_ac_type:
-                    #Remove tab from the course if needed
+                    # Remove tab from the course if needed
                     changed, new_tabs = remove_extra_panel_tab(tab_type, course_module)
                     if changed:
                         course_module.tabs = new_tabs
                         request_body.update({'tabs': new_tabs})
-                        #Indicate that tabs should *not* be filtered out of the metadata
+                        # Indicate that tabs should *not* be filtered out of the metadata
                         filter_tabs = False
-
-        response_json = json.dumps(CourseMetadata.update_from_json(location,
+        try:
+            response_json = json.dumps(CourseMetadata.update_from_json(location,
                                                                    request_body,
                                                                    filter_tabs=filter_tabs))
+        except (TypeError, ValueError), e:
+            return HttpResponseBadRequest("Incorrect setting format. " + str(e), content_type="text/plain")
+
         return HttpResponse(response_json, mimetype="application/json")

cms/djangoapps/models/settings/course_details.py

@@ -3,26 +3,26 @@ from xmodule.modulestore.exceptions import ItemNotFoundError
 from xmodule.modulestore.inheritance import own_metadata
 import json
 from json.encoder import JSONEncoder
-import time
 from contentstore.utils import get_modulestore
 from models.settings import course_grading
 from contentstore.utils import update_item
 from xmodule.fields import Date
 import re
 import logging
+import datetime
 
 
 class CourseDetails(object):
     def __init__(self, location):
-        self.course_location = location    # a Location obj
+        self.course_location = location  # a Location obj
         self.start_date = None  # 'start'
-        self.end_date = None    # 'end'
+        self.end_date = None  # 'end'
         self.enrollment_start = None
         self.enrollment_end = None
-        self.syllabus = None    # a pdf file asset
-        self.overview = ""      # html to render as the overview
-        self.intro_video = None    # a video pointer
-        self.effort = None      # int hours/week
+        self.syllabus = None  # a pdf file asset
+        self.overview = ""  # html to render as the overview
+        self.intro_video = None  # a video pointer
+        self.effort = None  # int hours/week
 
     @classmethod
     def fetch(cls, course_location):
@@ -73,9 +73,9 @@ class CourseDetails(object):
         """
         Decode the json into CourseDetails and save any changed attrs to the db
         """
-        ## TODO make it an error for this to be undefined & for it to not be retrievable from modulestore
+        # TODO make it an error for this to be undefined & for it to not be retrievable from modulestore
         course_location = jsondict['course_location']
-        ## Will probably want to cache the inflight courses because every blur generates an update
+        # Will probably want to cache the inflight courses because every blur generates an update
         descriptor = get_modulestore(course_location).get_item(course_location)
 
         dirty = False
@@ -181,7 +181,7 @@ class CourseSettingsEncoder(json.JSONEncoder):
             return obj.__dict__
         elif isinstance(obj, Location):
             return obj.dict()
-        elif isinstance(obj, time.struct_time):
+        elif isinstance(obj, datetime.datetime):
             return Date().to_json(obj)
         else:
             return JSONEncoder.default(self, obj)

cms/envs/acceptance.py

@@ -23,7 +23,7 @@ MODULESTORE_OPTIONS = {
     'db': 'test_xmodule',
     'collection': 'acceptance_modulestore',
     'fs_root': TEST_ROOT / "data",
-    'render_template': 'mitxmako.shortcuts.render_to_string',
+    'render_template': 'mitxmako.shortcuts.render_to_string'
 }
 
 MODULESTORE = {

cms/envs/aws.py

@@ -112,9 +112,6 @@ TIME_ZONE = ENV_TOKENS.get('TIME_ZONE', TIME_ZONE)
 for feature, value in ENV_TOKENS.get('MITX_FEATURES', {}).items():
     MITX_FEATURES[feature] = value
 
-# load segment.io key, provide a dummy if it does not exist
-SEGMENT_IO_KEY = ENV_TOKENS.get('SEGMENT_IO_KEY', '***REMOVED***')
-
 LOGGING = get_logger_config(LOG_DIR,
                             logging_env=ENV_TOKENS['LOGGING_ENV'],
                             syslog_addr=(ENV_TOKENS['SYSLOG_SERVER'], 514),
@@ -126,6 +123,13 @@ LOGGING = get_logger_config(LOG_DIR,
 with open(ENV_ROOT / CONFIG_PREFIX + "auth.json") as auth_file:
     AUTH_TOKENS = json.load(auth_file)
 
+# If Segment.io key specified, load it and turn on Segment.io if the feature flag is set
+# Note that this is the Studio key. There is a separate key for the LMS.
+SEGMENT_IO_KEY = AUTH_TOKENS.get('SEGMENT_IO_KEY')
+if SEGMENT_IO_KEY:
+    MITX_FEATURES['SEGMENT_IO'] = ENV_TOKENS.get('SEGMENT_IO', False)
+
+
 AWS_ACCESS_KEY_ID = AUTH_TOKENS["AWS_ACCESS_KEY_ID"]
 AWS_SECRET_ACCESS_KEY = AUTH_TOKENS["AWS_SECRET_ACCESS_KEY"]
 DATABASES = AUTH_TOKENS['DATABASES']

cms/envs/common.py

@@ -25,19 +25,30 @@ Longer TODO:
 
 import sys
 import lms.envs.common
+from lms.envs.common import USE_TZ
 from path import path
 
 ############################ FEATURE CONFIGURATION #############################
 
 MITX_FEATURES = {
     'USE_DJANGO_PIPELINE': True,
+    
     'GITHUB_PUSH': False,
+    
     'ENABLE_DISCUSSION_SERVICE': False,
+    
     'AUTH_USE_MIT_CERTIFICATES': False,
-    'STUB_VIDEO_FOR_TESTING': False,   # do not display video when running automated acceptance tests
-    'STAFF_EMAIL': '',			# email address for staff (eg to request course creation)
+    
+    # do not display video when running automated acceptance tests
+    'STUB_VIDEO_FOR_TESTING': False,
+    
+    # email address for staff (eg to request course creation)
+    'STAFF_EMAIL': '',
+    
     'STUDIO_NPS_SURVEY': True,
-    'SEGMENT_IO': True,
+    
+    # Segment.io - must explicitly turn it on for production
+    'SEGMENT_IO': False,
 
     # Enable URL that shows information about the status of various services
     'ENABLE_SERVICE_STATUS': False,
@@ -183,7 +194,7 @@ STATICFILES_DIRS = [
 
 # Locale/Internationalization
 TIME_ZONE = 'America/New_York'  # http://en.wikipedia.org/wiki/List_of_tz_zones_by_name
-LANGUAGE_CODE = 'en'            # http://www.i18nguy.com/unicode/language-identifiers.html
+LANGUAGE_CODE = 'en'  # http://www.i18nguy.com/unicode/language-identifiers.html
 
 USE_I18N = True
 USE_L10N = True
@@ -227,7 +238,8 @@ PIPELINE_JS = {
         ) + ['js/hesitate.js', 'js/base.js',
              'js/models/feedback.js', 'js/views/feedback.js',
              'js/models/section.js', 'js/views/section.js',
-             'js/models/metadata_model.js', 'js/views/metadata_editor_view.js'],
+             'js/models/metadata_model.js', 'js/views/metadata_editor_view.js',
+             'js/views/assets.js'],
         'output_filename': 'js/cms-application.js',
         'test_order': 0
     },

cms/envs/dev.py

@@ -22,7 +22,7 @@ modulestore_options = {
     'db': 'xmodule',
     'collection': 'modulestore',
     'fs_root': GITHUB_REPO_ROOT,
-    'render_template': 'mitxmako.shortcuts.render_to_string',
+    'render_template': 'mitxmako.shortcuts.render_to_string'
 }
 
 MODULESTORE = {
@@ -43,10 +43,15 @@ CONTENTSTORE = {
     'OPTIONS': {
         'host': 'localhost',
         'db': 'xcontent',
+    },
+    # allow for additional options that can be keyed on a name, e.g. 'trashcan'
+    'ADDITIONAL_OPTIONS': {
+        'trashcan': {
+            'bucket': 'trash_fs'
+        }
     }
 }
 
-
 DATABASES = {
     'default': {
         'ENGINE': 'django.db.backends.sqlite3',
@@ -64,7 +69,7 @@ REPOS = {
     },
     'content-mit-6002x': {
         'branch': 'master',
-        #'origin': 'git@github.com:MITx/6002x-fall-2012.git',
+        # 'origin': 'git@github.com:MITx/6002x-fall-2012.git',
         'origin': 'git@github.com:MITx/content-mit-6002x.git',
     },
     '6.00x': {
@@ -163,8 +168,14 @@ MITX_FEATURES['STUDIO_NPS_SURVEY'] = False
 # Enable URL that shows information about the status of variuous services
 MITX_FEATURES['ENABLE_SERVICE_STATUS'] = True
 
-# segment-io key for dev
-SEGMENT_IO_KEY = 'mty8edrrsg'
+############################# SEGMENT-IO ##################################
+
+# If there's an environment variable set, grab it and turn on Segment.io
+# Note that this is the Studio key. There is a separate key for the LMS.
+import os
+SEGMENT_IO_KEY = os.environ.get('SEGMENT_IO_KEY')
+if SEGMENT_IO_KEY:
+    MITX_FEATURES['SEGMENT_IO'] = True
 
 
 #####################################################################

cms/envs/test.py

@@ -48,7 +48,7 @@ MODULESTORE_OPTIONS = {
     'db': 'test_xmodule',
     'collection': 'test_modulestore',
     'fs_root': TEST_ROOT / "data",
-    'render_template': 'mitxmako.shortcuts.render_to_string',
+    'render_template': 'mitxmako.shortcuts.render_to_string'
 }
 
 MODULESTORE = {
@@ -70,7 +70,13 @@ CONTENTSTORE = {
     'ENGINE': 'xmodule.contentstore.mongo.MongoContentStore',
     'OPTIONS': {
         'host': 'localhost',
-        'db': 'xcontent',
+        'db': 'test_xmodule',
+    },
+    # allow for additional options that can be keyed on a name, e.g. 'trashcan'
+    'ADDITIONAL_OPTIONS': {
+        'trashcan': {
+            'bucket': 'trash_fs'
+        }
     }
 }
 
@@ -121,7 +127,7 @@ CELERY_RESULT_BACKEND = 'cache'
 BROKER_TRANSPORT = 'memory'
 
 ################### Make tests faster
-#http://slacy.com/blog/2012/04/make-your-tests-faster-in-django-1-4/
+# http://slacy.com/blog/2012/04/make-your-tests-faster-in-django-1-4/
 PASSWORD_HASHERS = (
     'django.contrib.auth.hashers.SHA1PasswordHasher',
     'django.contrib.auth.hashers.MD5PasswordHasher',

cms/pydev_manage.py

+'''
+Used for pydev eclipse. Should be innocuous for everyone else.
+Created on May 8, 2013
+
+@author: dmitchell
+'''
+#!/home/<username>/mitx_all/python/bin/python
+from django.core import management
+
+if __name__ == '__main__':
+    management.execute_from_command_line()

cms/static/coffee/src/views/module_edit.coffee

@@ -44,8 +44,17 @@ class CMS.Views.ModuleEdit extends Backbone.View
         [@metadataEditor.getDisplayName()])
       @$el.find('.component-name').html(title)
 
+  customMetadata: ->
+      # Hack to support metadata fields that aren't part of the metadata editor (ie, LaTeX high level source).
+      # Walk through the set of elements which have the 'data-metadata_name' attribute and
+      # build up an object to pass back to the server on the subsequent POST.
+      # Note that these values will always be sent back on POST, even if they did not actually change.
+      _metadata = {}
+      _metadata[$(el).data("metadata-name")] = el.value for el in $('[data-metadata-name]',  @$component_editor())
+      return _metadata
+
   changedMetadata: ->
-    return @metadataEditor.getModifiedMetadataValues()
+    return _.extend(@metadataEditor.getModifiedMetadataValues(), @customMetadata())
 
   cloneTemplate: (parent, template) ->
     $.post("/clone_item", {

cms/static/js/base.js

@@ -32,8 +32,6 @@ $(document).ready(function() {
 
     $modal.bind('click', hideModal);
     $modalCover.bind('click', hideModal);
-    $('.uploads .upload-button').bind('click', showUploadModal);
-    $('.upload-modal .close-button').bind('click', hideModal);
 
     $body.on('click', '.embeddable-xml-input', function() {
         $(this).select();
@@ -145,8 +143,6 @@ $(document).ready(function() {
     $('.edit-section-start-cancel').bind('click', cancelSetSectionScheduleDate);
     $('.edit-section-start-save').bind('click', saveSetSectionScheduleDate);
 
-    $('.upload-modal .choose-file-button').bind('click', showFileSelectionMenu);
-
     $body.on('click', '.section-published-date .edit-button', editSectionPublishDate);
     $body.on('click', '.section-published-date .schedule-button', editSectionPublishDate);
     $body.on('click', '.edit-subsection-publish-settings .save-button', saveSetSectionScheduleDate);
@@ -398,69 +394,6 @@ function _deleteItem($el) {
     });
 }
 
-function showUploadModal(e) {
-    e.preventDefault();
-    $modal = $('.upload-modal').show();
-    $('.file-input').bind('change', startUpload);
-    $modalCover.show();
-}
-
-function showFileSelectionMenu(e) {
-    e.preventDefault();
-    $('.file-input').click();
-}
-
-function startUpload(e) {
-    $('.upload-modal h1').html(gettext('Uploading…'));
-    $('.upload-modal .file-name').html($('.file-input').val().replace('C:\\fakepath\\', ''));
-    $('.upload-modal .file-chooser').ajaxSubmit({
-        beforeSend: resetUploadBar,
-        uploadProgress: showUploadFeedback,
-        complete: displayFinishedUpload
-    });
-    $('.upload-modal .choose-file-button').hide();
-    $('.upload-modal .progress-bar').removeClass('loaded').show();
-}
-
-function resetUploadBar() {
-    var percentVal = '0%';
-    $('.upload-modal .progress-fill').width(percentVal);
-    $('.upload-modal .progress-fill').html(percentVal);
-}
-
-function showUploadFeedback(event, position, total, percentComplete) {
-    var percentVal = percentComplete + '%';
-    $('.upload-modal .progress-fill').width(percentVal);
-    $('.upload-modal .progress-fill').html(percentVal);
-}
-
-function displayFinishedUpload(xhr) {
-    if (xhr.status = 200) {
-        markAsLoaded();
-    }
-
-    var resp = JSON.parse(xhr.responseText);
-    $('.upload-modal .embeddable-xml-input').val(xhr.getResponseHeader('asset_url'));
-    $('.upload-modal .embeddable').show();
-    $('.upload-modal .file-name').hide();
-    $('.upload-modal .progress-fill').html(resp.msg);
-    $('.upload-modal .choose-file-button').html(gettext('Load Another File')).show();
-    $('.upload-modal .progress-fill').width('100%');
-
-    // see if this id already exists, if so, then user must have updated an existing piece of content
-    $("tr[data-id='" + resp.url + "']").remove();
-
-    var template = $('#new-asset-element').html();
-    var html = Mustache.to_html(template, resp);
-    $('table > tbody').prepend(html);
-
-    analytics.track('Uploaded a File', {
-        'course': course_location_analytics,
-        'asset_url': resp.url
-    });
-
-}
-
 function markAsLoaded() {
     $('.upload-modal .copy-button').css('display', 'inline-block');
     $('.upload-modal .progress-bar').addClass('loaded');

cms/static/js/models/feedback.js

@@ -42,6 +42,12 @@ CMS.Models.ErrorMessage = CMS.Models.SystemFeedback.extend({
     })
 });
 
+CMS.Models.ConfirmAssetDeleteMessage = CMS.Models.SystemFeedback.extend({
+    defaults: $.extend({}, CMS.Models.SystemFeedback.prototype.defaults, {
+        "intent": "warning"
+    })
+});
+
 CMS.Models.ConfirmationMessage = CMS.Models.SystemFeedback.extend({
     defaults: $.extend({}, CMS.Models.SystemFeedback.prototype.defaults, {
         "intent": "confirmation"

cms/static/js/views/assets.js

+$(document).ready(function() {
+    $('.uploads .upload-button').bind('click', showUploadModal);
+    $('.upload-modal .close-button').bind('click', hideModal);
+    $('.upload-modal .choose-file-button').bind('click', showFileSelectionMenu);
+    $('.remove-asset-button').bind('click', removeAsset);
+});
+
+function removeAsset(e){
+    e.preventDefault();
+
+    var that = this;
+    var msg = new CMS.Models.ConfirmAssetDeleteMessage({
+        title: gettext("Delete File Confirmation"),
+        message: gettext("Are you sure you wish to delete this item. It cannot be reversed!\n\nAlso any content that links/refers to this item will no longer work (e.g. broken images and/or links)"),
+        actions: {
+            primary: {
+                text: gettext("OK"),
+                click: function(view) {
+                    // call the back-end to actually remove the asset
+                    $.post(view.model.get('remove_asset_url'),
+                        { 'location': view.model.get('asset_location') },
+                        function() {
+                            // show the post-commit confirmation
+                            $(".wrapper-alert-confirmation").addClass("is-shown").attr('aria-hidden','false');
+                            view.model.get('row_to_remove').remove();
+                            analytics.track('Deleted Asset', {
+                                'course': course_location_analytics,
+                                'id': view.model.get('asset_location')
+                            });
+                        }
+                    );
+                    view.hide();
+                }
+            },
+            secondary: [{
+                text: gettext("Cancel"),
+                click: function(view) {
+                    view.hide();
+                }
+            }]
+        },
+        remove_asset_url: $('.asset-library').data('remove-asset-callback-url'),
+        asset_location: $(this).closest('tr').data('id'),
+        row_to_remove: $(this).closest('tr')
+    });
+
+    // workaround for now. We can't spawn multiple instances of the Prompt View
+    // so for now, a bit of hackery to just make sure we have a single instance
+    // note: confirm_delete_prompt is in asset_index.html
+    if (confirm_delete_prompt === null)
+        confirm_delete_prompt = new CMS.Views.Prompt({model: msg});
+    else
+    {
+        confirm_delete_prompt.model = msg;
+        confirm_delete_prompt.show();
+    }
+
+    return;
+}
+
+function showUploadModal(e) {
+    e.preventDefault();
+    $modal = $('.upload-modal').show();
+    $('.file-input').bind('change', startUpload);
+    $modalCover.show();
+}
+
+function showFileSelectionMenu(e) {
+    e.preventDefault();
+    $('.file-input').click();
+}
+
+function startUpload(e) {
+    var files = $('.file-input').get(0).files;
+    if (files.length === 0)
+        return;
+
+    $('.upload-modal h1').html(gettext('Uploading…'));
+    $('.upload-modal .file-name').html(files[0].name);
+    $('.upload-modal .file-chooser').ajaxSubmit({
+        beforeSend: resetUploadBar,
+        uploadProgress: showUploadFeedback,
+        complete: displayFinishedUpload
+    });
+    $('.upload-modal .choose-file-button').hide();
+    $('.upload-modal .progress-bar').removeClass('loaded').show();
+}
+
+function resetUploadBar() {
+    var percentVal = '0%';
+    $('.upload-modal .progress-fill').width(percentVal);
+    $('.upload-modal .progress-fill').html(percentVal);
+}
+
+function showUploadFeedback(event, position, total, percentComplete) {
+    var percentVal = percentComplete + '%';
+    $('.upload-modal .progress-fill').width(percentVal);
+    $('.upload-modal .progress-fill').html(percentVal);
+}
+
+function displayFinishedUpload(xhr) {
+    if (xhr.status == 200) {
+        markAsLoaded();
+    }
+
+    var resp = JSON.parse(xhr.responseText);
+    $('.upload-modal .embeddable-xml-input').val(xhr.getResponseHeader('asset_url'));
+    $('.upload-modal .embeddable').show();
+    $('.upload-modal .file-name').hide();
+    $('.upload-modal .progress-fill').html(resp.msg);
+    $('.upload-modal .choose-file-button').html(gettext('Load Another File')).show();
+    $('.upload-modal .progress-fill').width('100%');
+
+    // see if this id already exists, if so, then user must have updated an existing piece of content
+    $("tr[data-id='" + resp.url + "']").remove();
+
+    var template = $('#new-asset-element').html();
+    var html = Mustache.to_html(template, resp);
+    $('table > tbody').prepend(html);
+
+    // re-bind the listeners to delete it
+    $('.remove-asset-button').bind('click', removeAsset);
+
+    analytics.track('Uploaded a File', {
+        'course': course_location_analytics,
+        'asset_url': resp.url
+    });
+}
\ No newline at end of file

cms/static/sass/_base.scss

@@ -14,7 +14,7 @@ body {
   color: $gray-d2;
 }
 
-body, input {
+body, input, button {
   font-family: 'Open Sans', sans-serif;
 }
 

cms/static/sass/views/_assets.scss

@@ -76,6 +76,10 @@ body.course.uploads {
         width: 250px;
       }
 
+      .delete-col {
+        width: 20px;
+      }
+
       .embeddable-xml-input {
         @include box-shadow(none);
         width: 100%;

cms/templates/asset_index.html

 <%inherit file="base.html" />
 <%! from django.core.urlresolvers import reverse %>
+<%! from django.utils.translation import ugettext as _ %>
 <%block name="bodyclass">is-signedin course uploads</%block>
 <%block name="title">Files &amp; Uploads</%block>
 
@@ -7,6 +8,12 @@
 
 <%block name="jsextra">
 <script src="${static.url('js/vendor/mustache.js')}"></script>
+<script type="text/javascript" src="${static.url('js/views/assets.js')}"></script>
+
+<script type='text/javascript'>
+  // we just want a singleton
+  confirm_delete_prompt = null;
+</script>
 </%block>
 
 <%block name="content">
@@ -30,6 +37,9 @@
       <td class="embed-col">
         <input type="text" class="embeddable-xml-input" value='{{url}}' readonly>
       </td>
+      <td class="delete-col">
+        <a href="#" data-tooltip="${_('Delete this asset')}" class="remove-asset-button"><span class="delete-icon"></span></a>
+      </td>
     </tr>
   </script>
 
@@ -56,7 +66,7 @@
       <div class="page-actions">
         <input type="text" class="asset-search-input search wip-box" placeholder="search assets" style="display:none"/>
       </div>
-      <article class="asset-library">
+      <article class="asset-library" data-remove-asset-callback-url='${remove_asset_callback_url}'>
         <table>
           <thead>
             <tr>
@@ -64,6 +74,7 @@
               <th class="name-col">Name</th>
               <th class="date-col">Date Added</th>
               <th class="embed-col">URL</th>
+              <th class="delete-col"></th>
             </tr>
           </thead>
           <tbody id="asset_table_body">
@@ -86,6 +97,9 @@
               <td class="embed-col">
                 <input type="text" class="embeddable-xml-input" value="${asset['url']}" readonly>
               </td>
+              <td class="delete-col">
+                <a href="#" data-tooltip="${_('Delete this asset')}" class="remove-asset-button"><span class="delete-icon"></span></a>
+              </td>             
             </tr>
           % endfor
           </tbody>
@@ -129,3 +143,21 @@
 
 
 </%block>
+
+<%block name="view_alerts">
+<!-- alert: save confirmed with close -->
+<div class="wrapper wrapper-alert wrapper-alert-confirmation" role="status">
+  <div class="alert confirmation">
+    <i class="icon-ok"></i>
+
+    <div class="copy">
+      <h2 class="title title-3">${_('Your file has been deleted.')}</h2>
+    </div>
+
+    <a href="" rel="view" class="action action-alert-close">
+      <i class="icon-remove-sign"></i>
+      <span class="label">${_('close alert')}</span>
+    </a>
+  </div>
+</div>
+</%block>

cms/templates/edit_subsection.html

 <%inherit file="base.html" />
 <%!
    import logging
-   from xmodule.util.date_utils import get_time_struct_display
+   from xmodule.util.date_utils import get_default_time_display
 %>
 
 <%! from django.core.urlresolvers import reverse %>
@@ -36,11 +36,15 @@
         <div class="datepair" data-language="javascript">
           <div class="field field-start-date">
             <label for="start_date">Release Day</label>
-            <input type="text" id="start_date" name="start_date" value="${get_time_struct_display(subsection.lms.start, '%m/%d/%Y')}" placeholder="MM/DD/YYYY" class="date" size='15' autocomplete="off"/>
+            <input type="text" id="start_date" name="start_date" 
+                value="${subsection.lms.start.strftime('%m/%d/%Y') if subsection.lms.start else ''}" 
+                placeholder="MM/DD/YYYY" class="date" size='15' autocomplete="off"/>
           </div>
           <div class="field field-start-time">
             <label for="start_time">Release Time (<abbr title="Coordinated Universal Time">UTC</abbr>)</label>
-            <input type="text" id="start_time" name="start_time" value="${get_time_struct_display(subsection.lms.start, '%H:%M')}" placeholder="HH:MM" class="time" size='10' autocomplete="off"/>
+            <input type="text" id="start_time" name="start_time" 
+                value="${subsection.lms.start.strftime('%H:%M') if subsection.lms.start else ''}" 
+                placeholder="HH:MM" class="time" size='10' autocomplete="off"/>
           </div>
         </div>
         % if subsection.lms.start != parent_item.lms.start and subsection.lms.start:
@@ -48,7 +52,7 @@
             <p class="notice">The date above differs from the release date of ${parent_item.display_name_with_default}, which is unset.
             % else:
             <p class="notice">The date above differs from the release date of ${parent_item.display_name_with_default} –
-                ${get_time_struct_display(parent_item.lms.start, '%m/%d/%Y at %H:%M UTC')}.
+                ${get_default_time_display(parent_item.lms.start)}.
             % endif
             <a href="#" class="sync-date no-spinner">Sync to ${parent_item.display_name_with_default}.</a></p>
         % endif
@@ -65,11 +69,15 @@
       <div class="datepair date-setter">
         <div class="field field-start-date">
           <label for="due_date">Due Day</label>
-          <input type="text" id="due_date" name="due_date" value="${get_time_struct_display(subsection.lms.due, '%m/%d/%Y')}" placeholder="MM/DD/YYYY" class="date" size='15' autocomplete="off"/>
+          <input type="text" id="due_date" name="due_date" 
+          value="${subsection.lms.due.strftime('%m/%d/%Y') if subsection.lms.due else ''}" 
+          placeholder="MM/DD/YYYY" class="date" size='15' autocomplete="off"/>
         </div>
         <div class="field field-start-time">
           <label for="due_time">Due Time (<abbr title="Coordinated Universal Time">UTC</abbr>)</label>
-          <input type="text" id="due_time" name="due_time" value="${get_time_struct_display(subsection.lms.due, '%H:%M')}" placeholder="HH:MM" class="time" size='10' autocomplete="off"/>
+          <input type="text" id="due_time" name="due_time" 
+            value="${subsection.lms.due.strftime('%H:%M') if subsection.lms.due else ''}" 
+            placeholder="HH:MM" class="time" size='10' autocomplete="off"/>
         </div>
         <a href="#" class="remove-date">Remove due date</a>
       </div>

cms/templates/overview.html

 <%inherit file="base.html" />
 <%!
    import logging
-   from xmodule.util.date_utils import get_time_struct_display
+   from xmodule.util import date_utils
 %>
 <%! from django.core.urlresolvers import reverse %>
 <%block name="title">Course Outline</%block>
@@ -154,14 +154,19 @@
               <h3 class="section-name" data-name="${section.display_name_with_default | h}"></h3>
               <div class="section-published-date">
                 <%
-                  start_date_str = get_time_struct_display(section.lms.start, '%m/%d/%Y')
-                  start_time_str = get_time_struct_display(section.lms.start, '%H:%M')
+                  if section.lms.start is not None:
+                      start_date_str = section.lms.start.strftime('%m/%d/%Y')
+                      start_time_str = section.lms.start.strftime('%H:%M')
+                  else:
+                      start_date_str = ''
+                      start_time_str = ''
                 %>
                 %if section.lms.start is None:
                   <span class="published-status">This section has not been released.</span>
                   <a href="#" class="schedule-button" data-date="" data-time="" data-id="${section.location}">Schedule</a>
                 %else:
-                  <span class="published-status"><strong>Will Release:</strong> ${get_time_struct_display(section.lms.start, '%m/%d/%Y at %H:%M UTC')}</span>
+                  <span class="published-status"><strong>Will Release:</strong> 
+                  ${date_utils.get_default_time_display(section.lms.start)}</span>
                   <a href="#" class="edit-button" data-date="${start_date_str}" data-time="${start_time_str}" data-id="${section.location}">Edit</a>
                 %endif
               </div>

cms/urls.py

@@ -35,6 +35,8 @@ urlpatterns = ('',  # nopep8
         'contentstore.views.preview_dispatch', name='preview_dispatch'),
     url(r'^(?P<org>[^/]+)/(?P<course>[^/]+)/course/(?P<coursename>[^/]+)/upload_asset$',
         'contentstore.views.upload_asset', name='upload_asset'),
+
+
     url(r'^manage_users/(?P<location>.*?)$', 'contentstore.views.manage_users', name='manage_users'),
     url(r'^add_user/(?P<location>.*?)$',
         'contentstore.views.add_user', name='add_user'),
@@ -71,8 +73,11 @@ urlpatterns = ('',  # nopep8
         'contentstore.views.edit_static', name='edit_static'),
     url(r'^edit_tabs/(?P<org>[^/]+)/(?P<course>[^/]+)/course/(?P<coursename>[^/]+)$',
         'contentstore.views.edit_tabs', name='edit_tabs'),
+
     url(r'^(?P<org>[^/]+)/(?P<course>[^/]+)/assets/(?P<name>[^/]+)$',
         'contentstore.views.asset_index', name='asset_index'),
+    url(r'^(?P<org>[^/]+)/(?P<course>[^/]+)/assets/(?P<name>[^/]+)/remove$',
+        'contentstore.views.assets.remove_asset', name='remove_asset'),
 
     # this is a generic method to return the data/metadata associated with a xmodule
     url(r'^module_info/(?P<module_location>.*)$',

cms/xmodule_namespace.py

@@ -5,7 +5,6 @@ Namespace defining common fields used by Studio for all blocks
 import datetime
 
 from xblock.core import Namespace, Scope, ModelType, String
-from xmodule.fields import StringyBoolean
 
 
 class DateTuple(ModelType):
@@ -28,4 +27,3 @@ class CmsNamespace(Namespace):
     """
     published_date = DateTuple(help="Date when the module was published", scope=Scope.settings)
     published_by = String(help="Id of the user who published this module", scope=Scope.settings)
-

common/djangoapps/contentserver/middleware.py

-import logging
-import time
-
-from django.http import HttpResponse, Http404, HttpResponseNotModified
+from django.http import HttpResponse, HttpResponseNotModified
 
 from xmodule.contentstore.django import contentstore
 from xmodule.contentstore.content import StaticContent, XASSET_LOCATION_TAG
@@ -20,7 +17,7 @@ class StaticContentServer(object):
                 # return a 'Bad Request' to browser as we have a malformed Location
                 response = HttpResponse()
                 response.status_code = 400
-                return response     
+                return response
 
             # first look in our cache so we don't have to round-trip to the DB
             content = get_cached_content(loc)

common/djangoapps/mitxmako/tests.py

 from django.test import TestCase
 from django.test.utils import override_settings
 from django.core.urlresolvers import reverse
-from django.conf import settings
 from mitxmako.shortcuts import marketing_link
 from mock import patch
-from nose.plugins.skip import SkipTest
+from util.testing import UrlResetMixin
 
-class ShortcutsTests(TestCase):
+
+class ShortcutsTests(UrlResetMixin, TestCase):
     """
     Test the mitxmako shortcuts file
     """
-    # TODO: fix this test. It is causing intermittent test failures on
-    # subsequent tests due to the way urls are loaded
-    raise SkipTest()
     @override_settings(MKTG_URLS={'ROOT': 'dummy-root', 'ABOUT': '/about-us'})
     @override_settings(MKTG_URL_LINK_MAP={'ABOUT': 'login'})
     def test_marketing_link(self):

common/djangoapps/student/management/commands/assigngroups.py

@@ -14,6 +14,7 @@ import sys
 import datetime
 
 import json
+from pytz import UTC
 
 middleware.MakoMiddleware()
 
@@ -32,7 +33,7 @@ def group_from_value(groups, v):
 
 
 class Command(BaseCommand):
-    help =  \
+    help = \
 ''' Assign users to test groups. Takes a list
 of groups:
 a:0.3,b:0.4,c:0.3 file.txt "Testing something"
@@ -75,7 +76,7 @@ Will log what happened to file.txt.
             utg = UserTestGroup()
             utg.name = group
             utg.description = json.dumps({"description": args[2]},
-                                         {"time": datetime.datetime.utcnow().isoformat()})
+                                         {"time": datetime.datetime.now(UTC).isoformat()})
             group_objects[group] = utg
             group_objects[group].save()
 

common/djangoapps/student/management/commands/pearson_export_cdd.py

@@ -8,6 +8,7 @@ from django.conf import settings
 from django.core.management.base import BaseCommand, CommandError
 
 from student.models import TestCenterUser
+from pytz import UTC
 
 
 class Command(BaseCommand):
@@ -58,7 +59,7 @@ class Command(BaseCommand):
     def handle(self, **options):
         # update time should use UTC in order to be comparable to the user_updated_at
         # field
-        uploaded_at = datetime.utcnow()
+        uploaded_at = datetime.now(UTC)
 
         # if specified destination is an existing directory, then
         # create a filename for it automatically.  If it doesn't exist,
@@ -100,7 +101,7 @@ class Command(BaseCommand):
                                     extrasaction='ignore')
             writer.writeheader()
             for tcu in TestCenterUser.objects.order_by('id'):
-                if tcu.needs_uploading:   # or dump_all
+                if tcu.needs_uploading:  # or dump_all
                     record = dict((csv_field, ensure_encoding(getattr(tcu, model_field)))
                                   for csv_field, model_field
                                   in Command.CSV_TO_MODEL_FIELDS.items())

common/djangoapps/student/management/commands/pearson_export_ead.py

@@ -8,6 +8,7 @@ from django.conf import settings
 from django.core.management.base import BaseCommand, CommandError
 
 from student.models import TestCenterRegistration, ACCOMMODATION_REJECTED_CODE
+from pytz import UTC
 
 
 class Command(BaseCommand):
@@ -51,7 +52,7 @@ class Command(BaseCommand):
     def handle(self, **options):
         # update time should use UTC in order to be comparable to the user_updated_at
         # field
-        uploaded_at = datetime.utcnow()
+        uploaded_at = datetime.now(UTC)
 
         # if specified destination is an existing directory, then
         # create a filename for it automatically.  If it doesn't exist,

common/djangoapps/student/management/commands/pearson_import_conf_zip.py

@@ -13,6 +13,7 @@ from django.core.management.base import BaseCommand, CommandError
 from django.conf import settings
 
 from student.models import TestCenterUser, TestCenterRegistration
+from pytz import UTC
 
 
 class Command(BaseCommand):
@@ -68,7 +69,7 @@ class Command(BaseCommand):
                     Command.datadog_error("Found authorization record for user {}".format(registration.testcenter_user.user.username), eacfile.name)
                     # now update the record:
                     registration.upload_status = row['Status']
-                    registration.upload_error_message =  row['Message']
+                    registration.upload_error_message = row['Message']
                     try:
                         registration.processed_at = strftime('%Y-%m-%d %H:%M:%S', strptime(row['Date'], '%Y/%m/%d %H:%M:%S'))
                     except ValueError as ve:
@@ -80,7 +81,7 @@ class Command(BaseCommand):
                         except ValueError as ve:
                             Command.datadog_error("Bad AuthorizationID value found for {}: message {}".format(client_authorization_id, ve), eacfile.name)
 
-                    registration.confirmed_at = datetime.utcnow()
+                    registration.confirmed_at = datetime.now(UTC)
                     registration.save()
                 except TestCenterRegistration.DoesNotExist:
                     Command.datadog_error("Failed to find record for client_auth_id {}".format(client_authorization_id), eacfile.name)

common/djangoapps/student/management/commands/pearson_make_tc_registration.py

 from optparse import make_option
-from time import strftime
 
 from django.contrib.auth.models import User
 from django.core.management.base import BaseCommand, CommandError
@@ -128,8 +127,8 @@ class Command(BaseCommand):
             exam = CourseDescriptor.TestCenterExam(course_id, exam_name, exam_info)
             # update option values for date_first and date_last to use YYYY-MM-DD format
             # instead of YYYY-MM-DDTHH:MM
-            our_options['eligibility_appointment_date_first'] = strftime("%Y-%m-%d", exam.first_eligible_appointment_date)
-            our_options['eligibility_appointment_date_last'] = strftime("%Y-%m-%d", exam.last_eligible_appointment_date)
+            our_options['eligibility_appointment_date_first'] = exam.first_eligible_appointment_date.strftime("%Y-%m-%d")
+            our_options['eligibility_appointment_date_last'] = exam.last_eligible_appointment_date.strftime("%Y-%m-%d")
 
         if exam is None:
             raise CommandError("Exam for course_id {} does not exist".format(course_id))

common/djangoapps/student/models.py

@@ -16,7 +16,6 @@ import json
 import logging
 import uuid
 from random import randint
-from time import strftime
 
 
 from django.conf import settings
@@ -27,6 +26,7 @@ from django.dispatch import receiver
 from django.forms import ModelForm, forms
 
 import comment_client as cc
+from pytz import UTC
 
 
 log = logging.getLogger(__name__)
@@ -54,7 +54,7 @@ class UserProfile(models.Model):
     class Meta:
         db_table = "auth_userprofile"
 
-    ## CRITICAL TODO/SECURITY
+    # CRITICAL TODO/SECURITY
     # Sanitize all fields.
     # This is not visible to other users, but could introduce holes later
     user = models.OneToOneField(User, unique=True, db_index=True, related_name='profile')
@@ -254,7 +254,7 @@ class TestCenterUserForm(ModelForm):
     def update_and_save(self):
         new_user = self.save(commit=False)
         # create additional values here:
-        new_user.user_updated_at = datetime.utcnow()
+        new_user.user_updated_at = datetime.now(UTC)
         new_user.upload_status = ''
         new_user.save()
         log.info("Updated demographic information for user's test center exam registration: username \"{}\" ".format(new_user.user.username))
@@ -429,8 +429,8 @@ class TestCenterRegistration(models.Model):
         registration.course_id = exam.course_id
         registration.accommodation_request = accommodation_request.strip()
         registration.exam_series_code = exam.exam_series_code
-        registration.eligibility_appointment_date_first = strftime("%Y-%m-%d", exam.first_eligible_appointment_date)
-        registration.eligibility_appointment_date_last = strftime("%Y-%m-%d", exam.last_eligible_appointment_date)
+        registration.eligibility_appointment_date_first = exam.first_eligible_appointment_date.strftime("%Y-%m-%d")
+        registration.eligibility_appointment_date_last = exam.last_eligible_appointment_date.strftime("%Y-%m-%d")
         registration.client_authorization_id = cls._create_client_authorization_id()
         # accommodation_code remains blank for now, along with Pearson confirmation information
         return registration
@@ -556,7 +556,7 @@ class TestCenterRegistrationForm(ModelForm):
     def update_and_save(self):
         registration = self.save(commit=False)
         # create additional values here:
-        registration.user_updated_at = datetime.utcnow()
+        registration.user_updated_at = datetime.now(UTC)
         registration.upload_status = ''
         registration.save()
         log.info("Updated registration information for user's test center exam registration: username \"{}\" course \"{}\", examcode \"{}\"".format(registration.testcenter_user.user.username, registration.course_id, registration.exam_series_code))
@@ -598,7 +598,7 @@ def unique_id_for_user(user):
     return h.hexdigest()
 
 
-## TODO: Should be renamed to generic UserGroup, and possibly
+# TODO: Should be renamed to generic UserGroup, and possibly
 # Given an optional field for type of group
 class UserTestGroup(models.Model):
     users = models.ManyToManyField(User, db_index=True)
@@ -626,7 +626,6 @@ class Registration(models.Model):
     def activate(self):
         self.user.is_active = True
         self.user.save()
-        #self.delete()
 
 
 class PendingNameChange(models.Model):
@@ -648,7 +647,7 @@ class CourseEnrollment(models.Model):
     created = models.DateTimeField(auto_now_add=True, null=True, db_index=True)
 
     class Meta:
-        unique_together = (('user', 'course_id'), )
+        unique_together = (('user', 'course_id'),)
 
     def __unicode__(self):
         return "[CourseEnrollment] %s: %s (%s)" % (self.user, self.course_id, self.created)
@@ -662,16 +661,17 @@ class CourseEnrollmentAllowed(models.Model):
     """
     email = models.CharField(max_length=255, db_index=True)
     course_id = models.CharField(max_length=255, db_index=True)
+    auto_enroll = models.BooleanField(default=0)
 
     created = models.DateTimeField(auto_now_add=True, null=True, db_index=True)
 
     class Meta:
-        unique_together = (('email', 'course_id'), )
+        unique_together = (('email', 'course_id'),)
 
     def __unicode__(self):
         return "[CourseEnrollmentAllowed] %s: %s (%s)" % (self.email, self.course_id, self.created)
 
-#cache_relation(User.profile)
+# cache_relation(User.profile)
 
 #### Helper methods for use from python manage.py shell and other classes.
 

common/djangoapps/student/tests/factories.py

@@ -5,6 +5,7 @@ from django.contrib.auth.models import Group
 from datetime import datetime
 from factory import DjangoModelFactory, SubFactory, PostGenerationMethodCall, post_generation, Sequence
 from uuid import uuid4
+from pytz import UTC
 
 # Factories don't have __init__ methods, and are self documenting
 # pylint: disable=W0232
@@ -46,8 +47,8 @@ class UserFactory(DjangoModelFactory):
     is_staff = False
     is_active = True
     is_superuser = False
-    last_login = datetime(2012, 1, 1)
-    date_joined = datetime(2011, 1, 1)
+    last_login = datetime(2012, 1, 1, tzinfo=UTC)
+    date_joined = datetime(2011, 1, 1, tzinfo=UTC)
 
     @post_generation
     def profile(obj, create, extracted, **kwargs):

common/djangoapps/student/views.py

@@ -32,7 +32,7 @@ from student.models import (Registration, UserProfile, TestCenterUser, TestCente
                             TestCenterRegistration, TestCenterRegistrationForm,
                             PendingNameChange, PendingEmailChange,
                             CourseEnrollment, unique_id_for_user,
-                            get_testcenter_registration)
+                            get_testcenter_registration, CourseEnrollmentAllowed)
 
 from certificates.models import CertificateStatuses, certificate_status_for_student
 
@@ -49,6 +49,7 @@ from courseware.views import get_module_for_descriptor, jump_to
 from courseware.model_data import ModelDataCache
 
 from statsd import statsd
+from pytz import UTC
 
 log = logging.getLogger("mitx.student")
 Article = namedtuple('Article', 'title url author image deck publication publish_date')
@@ -77,7 +78,7 @@ def index(request, extra_context={}, user=None):
     '''
 
     # The course selection work is done in courseware.courses.
-    domain = settings.MITX_FEATURES.get('FORCE_UNIVERSITY_DOMAIN')     # normally False
+    domain = settings.MITX_FEATURES.get('FORCE_UNIVERSITY_DOMAIN')  # normally False
     # do explicit check, because domain=None is valid
     if domain == False:
         domain = request.META.get('HTTP_HOST')
@@ -264,7 +265,6 @@ def dashboard(request):
     if not user.is_active:
         message = render_to_string('registration/activate_account_notice.html', {'email': user.email})
 
-
     # Global staff can see what courses errored on their dashboard
     staff_access = False
     errored_courses = {}
@@ -355,7 +355,7 @@ def change_enrollment(request):
             course = course_from_id(course_id)
         except ItemNotFoundError:
             log.warning("User {0} tried to enroll in non-existent course {1}"
-                      .format(user.username, course_id))
+                        .format(user.username, course_id))
             return HttpResponseBadRequest("Course id is invalid")
 
         if not has_access(user, course, 'enroll'):
@@ -363,9 +363,9 @@ def change_enrollment(request):
 
         org, course_num, run = course_id.split("/")
         statsd.increment("common.student.enrollment",
-                        tags=["org:{0}".format(org),
-                              "course:{0}".format(course_num),
-                              "run:{0}".format(run)])
+                         tags=["org:{0}".format(org),
+                               "course:{0}".format(course_num),
+                               "run:{0}".format(run)])
 
         try:
             enrollment, created = CourseEnrollment.objects.get_or_create(user=user, course_id=course.id)
@@ -382,9 +382,9 @@ def change_enrollment(request):
 
             org, course_num, run = course_id.split("/")
             statsd.increment("common.student.unenrollment",
-                            tags=["org:{0}".format(org),
-                                  "course:{0}".format(course_num),
-                                  "run:{0}".format(run)])
+                             tags=["org:{0}".format(org),
+                                   "course:{0}".format(course_num),
+                                   "run:{0}".format(run)])
 
             return HttpResponse()
         except CourseEnrollment.DoesNotExist:
@@ -454,7 +454,6 @@ def login_user(request, error=""):
             expires_time = time.time() + max_age
             expires = cookie_date(expires_time)
 
-
         response.set_cookie(settings.EDXMKTG_COOKIE_NAME,
                             'true', max_age=max_age,
                             expires=expires, domain=settings.SESSION_COOKIE_DOMAIN,
@@ -515,8 +514,8 @@ def _do_create_account(post_vars):
     Note: this function is also used for creating test users.
     """
     user = User(username=post_vars['username'],
-             email=post_vars['email'],
-             is_active=False)
+                email=post_vars['email'],
+                is_active=False)
     user.set_password(post_vars['password'])
     registration = Registration()
     # TODO: Rearrange so that if part of the process fails, the whole process fails.
@@ -632,7 +631,7 @@ def create_account(request, post_override=None):
 
     # Ok, looks like everything is legit.  Create the account.
     ret = _do_create_account(post_vars)
-    if isinstance(ret, HttpResponse):		# if there was an error then return that
+    if isinstance(ret, HttpResponse):  # if there was an error then return that
         return ret
     (user, profile, registration) = ret
 
@@ -670,7 +669,7 @@ def create_account(request, post_override=None):
 
     if DoExternalAuth:
         eamap.user = login_user
-        eamap.dtsignup = datetime.datetime.now()
+        eamap.dtsignup = datetime.datetime.now(UTC)
         eamap.save()
         log.debug('Updated ExternalAuthMap for %s to be %s' % (post_vars['username'], eamap))
 
@@ -698,7 +697,6 @@ def create_account(request, post_override=None):
         expires_time = time.time() + max_age
         expires = cookie_date(expires_time)
 
-
     response.set_cookie(settings.EDXMKTG_COOKIE_NAME,
                         'true', max_age=max_age,
                         expires=expires, domain=settings.SESSION_COOKIE_DOMAIN,
@@ -708,7 +706,6 @@ def create_account(request, post_override=None):
     return response
 
 
-
 def exam_registration_info(user, course):
     """ Returns a Registration object if the user is currently registered for a current
     exam of the course.  Returns None if the user is not registered, or if there is no
@@ -849,7 +846,6 @@ def create_exam_registration(request, post_override=None):
             response_data['non_field_errors'] = form.non_field_errors()
             return HttpResponse(json.dumps(response_data), mimetype="application/json")
 
-
     # only do the following if there is accommodation text to send,
     # and a destination to which to send it.
     # TODO: still need to create the accommodation email templates
@@ -872,7 +868,6 @@ def create_exam_registration(request, post_override=None):
 #            response_data['non_field_errors'] =  [ 'Could not send accommodation e-mail.', ]
 #            return HttpResponse(json.dumps(response_data), mimetype="application/json")
 
-
     js = {'success': True}
     return HttpResponse(json.dumps(js), mimetype="application/json")
 
@@ -916,6 +911,16 @@ def activate_account(request, key):
         if not r[0].user.is_active:
             r[0].activate()
             already_active = False
+
+        #Enroll student in any pending courses he/she may have if auto_enroll flag is set
+        student = User.objects.filter(id=r[0].user_id)
+        if student:
+            ceas = CourseEnrollmentAllowed.objects.filter(email=student[0].email)
+            for cea in ceas:
+                if cea.auto_enroll:
+                    course_id = cea.course_id
+                    enrollment, created = CourseEnrollment.objects.get_or_create(user_id=student[0].id, course_id=course_id)
+
         resp = render_to_response("registration/activation_complete.html", {'user_logged_in': user_logged_in, 'already_active': already_active})
         return resp
     if len(r) == 0:

common/djangoapps/terrain/ui_helpers.py

@@ -3,6 +3,7 @@
 
 from lettuce import world
 import time
+import platform
 from urllib import quote_plus
 from selenium.common.exceptions import WebDriverException, StaleElementReferenceException
 from selenium.webdriver.support import expected_conditions as EC
@@ -57,20 +58,28 @@ def css_find(css, wait_time=5):
 
 
 @world.absorb
-def css_click(css_selector):
+def css_click(css_selector, index=0, attempts=5):
     """
     Perform a click on a CSS selector, retrying if it initially fails
+    This function will return if the click worked (since it is try/excepting all errors)
     """
     assert is_css_present(css_selector)
-    try:
-        world.browser.find_by_css(css_selector).click()
-
-    except WebDriverException:
-        # Occassionally, MathJax or other JavaScript can cover up
-        # an element  temporarily.
-        # If this happens, wait a second, then try again
-        world.wait(1)
-        world.browser.find_by_css(css_selector).click()
+    attempt = 0
+    result = False
+    while attempt < attempts:
+        try:
+            world.css_find(css_selector)[index].click()
+            result = True
+            break
+        except WebDriverException:
+            # Occasionally, MathJax or other JavaScript can cover up
+            # an element temporarily.
+            # If this happens, wait a second, then try again
+            world.wait(1)
+            attempt += 1
+        except:
+            attempt += 1
+    return result
 
 
 @world.absorb
@@ -158,3 +167,8 @@ def click_tools():
     tools_css = 'li.nav-course-tools'
     if world.browser.is_element_present_by_css(tools_css):
         world.css_click(tools_css)
+
+
+@world.absorb
+def is_mac():
+    return platform.mac_ver()[0] is not ''

common/djangoapps/tests.py

+'''
+Created on Jun 6, 2013
+
+@author: dmitchell
+'''
+from xmodule.modulestore.tests.factories import CourseFactory, ItemFactory
+from student.tests.factories import AdminFactory
+from xmodule.modulestore.tests.django_utils import ModuleStoreTestCase
+import xmodule_modifiers
+import datetime
+from pytz import UTC
+from xmodule.modulestore.tests import factories
+
+class TestXmoduleModfiers(ModuleStoreTestCase):
+
+    # FIXME disabled b/c start date inheritance is not occuring and render_... in get_html is failing due
+    # to middleware.lookup['main'] not being defined
+    def _test_add_histogram(self):
+        instructor = AdminFactory.create()
+        self.client.login(username=instructor.username, password='test')
+
+        course = CourseFactory.create(org='test',
+            number='313', display_name='histogram test')
+        section = ItemFactory.create(
+            parent_location=course.location, display_name='chapter hist',
+            template='i4x://edx/templates/chapter/Empty')
+        problem = ItemFactory.create(
+            parent_location=section.location, display_name='problem hist 1',
+            template='i4x://edx/templates/problem/Blank_Common_Problem')
+        problem.has_score = False  # don't trip trying to retrieve db data
+
+        late_problem = ItemFactory.create(
+            parent_location=section.location, display_name='problem hist 2',
+            template='i4x://edx/templates/problem/Blank_Common_Problem')
+        late_problem.lms.start = datetime.datetime.now(UTC) + datetime.timedelta(days=32)
+        late_problem.has_score = False
+
+
+        problem_module = factories.get_test_xmodule_for_descriptor(problem)
+        problem_module.get_html = xmodule_modifiers.add_histogram(lambda:'', problem_module, instructor)
+
+        self.assertRegexpMatches(
+            problem_module.get_html(), r'.*<font color=\'green\'>Not yet</font>.*')
+
+        problem_module = factories.get_test_xmodule_for_descriptor(late_problem)
+        problem_module.get_html = xmodule_modifiers.add_histogram(lambda: '', problem_module, instructor)
+
+        self.assertRegexpMatches(
+            problem_module.get_html(), r'.*<font color=\'red\'>Yes!</font>.*')

common/djangoapps/track/views.py

@@ -14,6 +14,7 @@ from mitxmako.shortcuts import render_to_response
 
 from django_future.csrf import ensure_csrf_cookie
 from track.models import TrackingLog
+from pytz import UTC
 
 log = logging.getLogger("tracking")
 
@@ -59,7 +60,7 @@ def user_track(request):
         "event": request.GET['event'],
         "agent": agent,
         "page": request.GET['page'],
-        "time": datetime.datetime.utcnow().isoformat(),
+        "time": datetime.datetime.now(UTC).isoformat(),
         "host": request.META['SERVER_NAME'],
         }
     log_event(event)
@@ -85,11 +86,11 @@ def server_track(request, event_type, event, page=None):
         "event": event,
         "agent": agent,
         "page": page,
-        "time": datetime.datetime.utcnow().isoformat(),
+        "time": datetime.datetime.now(UTC).isoformat(),
         "host": request.META['SERVER_NAME'],
         }
 
-    if event_type.startswith("/event_logs") and request.user.is_staff:  	# don't log
+    if event_type.startswith("/event_logs") and request.user.is_staff:  # don't log
         return
     log_event(event)
 

common/djangoapps/util/testing.py

+import sys
+
+from django.conf import settings
+from django.core.urlresolvers import clear_url_caches
+
+
+class UrlResetMixin(object):
+    """Mixin to reset urls.py before and after a test
+
+    Django memoizes the function that reads the urls module (whatever module
+    urlconf names). The module itself is also stored by python in sys.modules.
+    To fully reload it, we need to reload the python module, and also clear django's
+    cache of the parsed urls.
+
+    However, the order in which we do this doesn't matter, because neither one will
+    get reloaded until the next request
+
+    Doing this is expensive, so it should only be added to tests that modify settings
+    that affect the contents of urls.py
+    """
+
+    def _reset_urls(self, urlconf=None):
+        if urlconf is None:
+            urlconf = settings.ROOT_URLCONF
+
+        if urlconf in sys.modules:
+            reload(sys.modules[urlconf])
+        clear_url_caches()
+
+    def setUp(self):
+        """Reset django default urlconf before tests and after tests"""
+        super(UrlResetMixin, self).setUp()
+        self._reset_urls()
+        self.addCleanup(self._reset_urls)

common/djangoapps/util/views.py

@@ -12,6 +12,7 @@ from django.core.validators import ValidationError, validate_email
 from django.http import Http404, HttpResponse, HttpResponseBadRequest, HttpResponseNotAllowed, HttpResponseServerError
 from django.shortcuts import redirect
 from django_future.csrf import ensure_csrf_cookie
+from dogapi import dog_stats_api
 from mitxmako.shortcuts import render_to_response, render_to_string
 from urllib import urlencode
 import zendesk
@@ -73,11 +74,64 @@ class _ZendeskApi(object):
         self._zendesk_instance.update_ticket(ticket_id=ticket_id, data=update)
 
 
-def submit_feedback_via_zendesk(request):
+def _record_feedback_in_zendesk(realname, email, subject, details, tags, additional_info):
     """
     Create a new user-requested Zendesk ticket.
 
-    If Zendesk submission is not enabled, any request will raise `Http404`.
+    Once created, the ticket will be updated with a private comment containing
+    additional information from the browser and server, such as HTTP headers
+    and user state. Returns a boolean value indicating whether ticket creation
+    was successful, regardless of whether the private comment update succeeded.
+    """
+    zendesk_api = _ZendeskApi()
+
+    additional_info_string = (
+        "Additional information:\n\n" +
+        "\n".join("%s: %s" % (key, value) for (key, value) in additional_info.items() if value is not None)
+    )
+
+    # Tag all issues with LMS to distinguish channel in Zendesk; requested by student support team
+    zendesk_tags = list(tags.values()) + ["LMS"]
+    new_ticket = {
+        "ticket": {
+            "requester": {"name": realname, "email": email},
+            "subject": subject,
+            "comment": {"body": details},
+            "tags": zendesk_tags
+        }
+    }
+    try:
+        ticket_id = zendesk_api.create_ticket(new_ticket)
+    except zendesk.ZendeskError as err:
+        log.error("Error creating Zendesk ticket: %s", str(err))
+        return False
+
+    # Additional information is provided as a private update so the information
+    # is not visible to the user.
+    ticket_update = {"ticket": {"comment": {"public": False, "body": additional_info_string}}}
+    try:
+        zendesk_api.update_ticket(ticket_id, ticket_update)
+    except zendesk.ZendeskError as err:
+        log.error("Error updating Zendesk ticket: %s", str(err))
+        # The update is not strictly necessary, so do not indicate failure to the user
+        pass
+
+    return True
+
+
+DATADOG_FEEDBACK_METRIC = "lms_feedback_submissions"
+
+
+def _record_feedback_in_datadog(tags):
+    datadog_tags = ["{k}:{v}".format(k=k, v=v) for k, v in tags.items()]
+    dog_stats_api.increment(DATADOG_FEEDBACK_METRIC, tags=datadog_tags)
+
+
+def submit_feedback(request):
+    """
+    Create a new user-requested ticket, currently implemented with Zendesk.
+
+    If feedback submission is not enabled, any request will raise `Http404`.
     If any configuration parameter (`ZENDESK_URL`, `ZENDESK_USER`, or
     `ZENDESK_API_KEY`) is missing, any request will raise an `Exception`.
     The request must be a POST request specifying `subject` and `details`.
@@ -85,12 +139,9 @@ def submit_feedback_via_zendesk(request):
     `email`. If the user is authenticated, the `name` and `email` will be
     populated from the user's information. If any required parameter is
     missing, a 400 error will be returned indicating which field is missing and
-    providing an error message. If Zendesk returns any error on ticket
-    creation, a 500 error will be returned with no body. Once created, the
-    ticket will be updated with a private comment containing additional
-    information from the browser and server, such as HTTP headers and user
-    state. Whether or not the update succeeds, if the user's ticket is
-    successfully created, an empty successful response (200) will be returned.
+    providing an error message. If Zendesk ticket creation fails, 500 error
+    will be returned with no body; if ticket creation succeeds, an empty
+    successful response (200) will be returned.
     """
     if not settings.MITX_FEATURES.get('ENABLE_FEEDBACK_SUBMISSION', False):
         raise Http404()
@@ -124,9 +175,9 @@ def submit_feedback_via_zendesk(request):
 
     subject = request.POST["subject"]
     details = request.POST["details"]
-    tags = []
-    if "tag" in request.POST:
-        tags = [request.POST["tag"]]
+    tags = dict(
+        [(tag, request.POST[tag]) for tag in ["issue_type", "course_id"] if tag in request.POST]
+    )
 
     if request.user.is_authenticated():
         realname = request.user.profile.name
@@ -140,41 +191,18 @@ def submit_feedback_via_zendesk(request):
         except ValidationError:
             return build_error_response(400, "email", required_field_errs["email"])
 
-    for header in ["HTTP_REFERER", "HTTP_USER_AGENT"]:
-        additional_info[header] = request.META.get(header)
+    for header, pretty in [
+        ("HTTP_REFERER", "Page"),
+        ("HTTP_USER_AGENT", "Browser"),
+        ("REMOTE_ADDR", "Client IP"),
+        ("SERVER_NAME", "Host")
+    ]:
+        additional_info[pretty] = request.META.get(header)
 
-    zendesk_api = _ZendeskApi()
-
-    additional_info_string = (
-        "Additional information:\n\n" +
-        "\n".join("%s: %s" % (key, value) for (key, value) in additional_info.items() if value is not None)
-    )
-
-    new_ticket = {
-        "ticket": {
-            "requester": {"name": realname, "email": email},
-            "subject": subject,
-            "comment": {"body": details},
-            "tags": tags
-        }
-    }
-    try:
-        ticket_id = zendesk_api.create_ticket(new_ticket)
-    except zendesk.ZendeskError as err:
-        log.error("Error creating Zendesk ticket: %s", str(err))
-        return HttpResponse(status=500)
-
-    # Additional information is provided as a private update so the information
-    # is not visible to the user.
-    ticket_update = {"ticket": {"comment": {"public": False, "body": additional_info_string}}}
-    try:
-        zendesk_api.update_ticket(ticket_id, ticket_update)
-    except zendesk.ZendeskError as err:
-        log.error("Error updating Zendesk ticket: %s", str(err))
-        # The update is not strictly necessary, so do not indicate failure to the user
-        pass
+    success = _record_feedback_in_zendesk(realname, email, subject, details, tags, additional_info)
+    _record_feedback_in_datadog(tags)
 
-    return HttpResponse()
+    return HttpResponse(status=(200 if success else 500))
 
 
 def info(request):

common/djangoapps/xmodule_modifiers.py

 import re
 import json
 import logging
-import time
 import static_replace
 
 from django.conf import settings
@@ -9,6 +8,8 @@ from functools import wraps
 from mitxmako.shortcuts import render_to_string
 from xmodule.seq_module import SequenceModule
 from xmodule.vertical_module import VerticalModule
+import datetime
+from django.utils.timezone import UTC
 
 log = logging.getLogger("mitx.xmodule_modifiers")
 
@@ -83,7 +84,7 @@ def grade_histogram(module_id):
     cursor.execute(q, [module_id])
 
     grades = list(cursor.fetchall())
-    grades.sort(key=lambda x: x[0])          # Add ORDER BY to sql query?
+    grades.sort(key=lambda x: x[0])  # Add ORDER BY to sql query?
     if len(grades) >= 1 and grades[0][0] is None:
         return []
     return grades
@@ -101,7 +102,7 @@ def add_histogram(get_html, module, user):
     @wraps(get_html)
     def _get_html():
 
-        if type(module) in [SequenceModule, VerticalModule]:  	# TODO: make this more general, eg use an XModule attribute instead
+        if type(module) in [SequenceModule, VerticalModule]:  # TODO: make this more general, eg use an XModule attribute instead
             return get_html()
 
         module_id = module.id
@@ -132,7 +133,7 @@ def add_histogram(get_html, module, user):
 
         # useful to indicate to staff if problem has been released or not
         # TODO (ichuang): use _has_access_descriptor.can_load in lms.courseware.access, instead of now>mstart comparison here
-        now = time.gmtime()
+        now = datetime.datetime.now(UTC())
         is_released = "unknown"
         mstart = module.descriptor.lms.start
 

common/lib/calc/calcfunctions.py

+"""
+Provide the mathematical functions that numpy doesn't.
+
+Specifically, the secant/cosecant/cotangents and their inverses and
+hyperbolic counterparts
+"""
+import numpy
+
+
+# Normal Trig
+def sec(arg):
+    """
+    Secant
+    """
+    return 1 / numpy.cos(arg)
+
+
+def csc(arg):
+    """
+    Cosecant
+    """
+    return 1 / numpy.sin(arg)
+
+
+def cot(arg):
+    """
+    Cotangent
+    """
+    return 1 / numpy.tan(arg)
+
+
+# Inverse Trig
+# http://en.wikipedia.org/wiki/Inverse_trigonometric_functions#Relationships_among_the_inverse_trigonometric_functions
+def arcsec(val):
+    """
+    Inverse secant
+    """
+    return numpy.arccos(1. / val)
+
+
+def arccsc(val):
+    """
+    Inverse cosecant
+    """
+    return numpy.arcsin(1. / val)
+
+
+def arccot(val):
+    """
+    Inverse cotangent
+    """
+    if numpy.real(val) < 0:
+        return -numpy.pi / 2 - numpy.arctan(val)
+    else:
+        return numpy.pi / 2 - numpy.arctan(val)
+
+
+# Hyperbolic Trig
+def sech(arg):
+    """
+    Hyperbolic secant
+    """
+    return 1 / numpy.cosh(arg)
+
+
+def csch(arg):
+    """
+    Hyperbolic cosecant
+    """
+    return 1 / numpy.sinh(arg)
+
+
+def coth(arg):
+    """
+    Hyperbolic cotangent
+    """
+    return 1 / numpy.tanh(arg)
+
+
+# And their inverses
+def arcsech(val):
+    """
+    Inverse hyperbolic secant
+    """
+    return numpy.arccosh(1. / val)
+
+
+def arccsch(val):
+    """
+    Inverse hyperbolic cosecant
+    """
+    return numpy.arcsinh(1. / val)
+
+
+def arccoth(val):
+    """
+    Inverse hyperbolic cotangent
+    """
+    return numpy.arctanh(1. / val)

common/lib/calc/tests/test_calc.py

@@ -194,6 +194,105 @@ class EvaluatorTest(unittest.TestCase):
         arctan_angles = arcsin_angles
         self.assert_function_values('arctan', arctan_inputs, arctan_angles)
 
+    def test_reciprocal_trig_functions(self):
+        """
+        Test the reciprocal trig functions provided in calc.py
+
+        which are: sec, csc, cot, arcsec, arccsc, arccot
+        """
+        angles = ['-pi/4', 'pi/6', 'pi/5', '5*pi/4', '9*pi/4', '1 + j']
+        sec_values = [1.414, 1.155, 1.236, -1.414, 1.414, 0.498 + 0.591j]
+        csc_values = [-1.414, 2, 1.701, -1.414, 1.414, 0.622 - 0.304j]
+        cot_values = [-1, 1.732, 1.376, 1, 1, 0.218 - 0.868j]
+
+        self.assert_function_values('sec', angles, sec_values)
+        self.assert_function_values('csc', angles, csc_values)
+        self.assert_function_values('cot', angles, cot_values)
+
+        arcsec_inputs = ['1.1547', '1.2361', '2', '-2', '-1.4142', '0.4983+0.5911*j']
+        arcsec_angles = [0.524, 0.628, 1.047, 2.094, 2.356, 1 + 1j]
+        self.assert_function_values('arcsec', arcsec_inputs, arcsec_angles)
+
+        arccsc_inputs = ['-1.1547', '-1.4142', '2', '1.7013', '1.1547', '0.6215-0.3039*j']
+        arccsc_angles = [-1.047, -0.785, 0.524, 0.628, 1.047, 1 + 1j]
+        self.assert_function_values('arccsc', arccsc_inputs, arccsc_angles)
+
+        # Has the same range as arccsc
+        arccot_inputs = ['-0.5774', '-1', '1.7321', '1.3764', '0.5774', '(0.2176-0.868*j)']
+        arccot_angles = arccsc_angles
+        self.assert_function_values('arccot', arccot_inputs, arccot_angles)
+
+    def test_hyperbolic_functions(self):
+        """
+        Test the hyperbolic functions
+
+        which are: sinh, cosh, tanh, sech, csch, coth
+        """
+        inputs = ['0', '0.5', '1', '2', '1+j']
+        neg_inputs = ['0', '-0.5', '-1', '-2', '-1-j']
+        negate = lambda x: [-k for k in x]
+
+        # sinh is odd
+        sinh_vals = [0, 0.521, 1.175, 3.627, 0.635 + 1.298j]
+        self.assert_function_values('sinh', inputs, sinh_vals)
+        self.assert_function_values('sinh', neg_inputs, negate(sinh_vals))
+
+        # cosh is even - do not negate
+        cosh_vals = [1, 1.128, 1.543, 3.762, 0.834 + 0.989j]
+        self.assert_function_values('cosh', inputs, cosh_vals)
+        self.assert_function_values('cosh', neg_inputs, cosh_vals)
+
+        # tanh is odd
+        tanh_vals = [0, 0.462, 0.762, 0.964, 1.084 + 0.272j]
+        self.assert_function_values('tanh', inputs, tanh_vals)
+        self.assert_function_values('tanh', neg_inputs, negate(tanh_vals))
+
+        # sech is even - do not negate
+        sech_vals = [1, 0.887, 0.648, 0.266, 0.498 - 0.591j]
+        self.assert_function_values('sech', inputs, sech_vals)
+        self.assert_function_values('sech', neg_inputs, sech_vals)
+
+        # the following functions do not have 0 in their domain
+        inputs = inputs[1:]
+        neg_inputs = neg_inputs[1:]
+
+        # csch is odd
+        csch_vals = [1.919, 0.851, 0.276, 0.304 - 0.622j]
+        self.assert_function_values('csch', inputs, csch_vals)
+        self.assert_function_values('csch', neg_inputs, negate(csch_vals))
+
+        # coth is odd
+        coth_vals = [2.164, 1.313, 1.037, 0.868 - 0.218j]
+        self.assert_function_values('coth', inputs, coth_vals)
+        self.assert_function_values('coth', neg_inputs, negate(coth_vals))
+
+    def test_hyperbolic_inverses(self):
+        """
+        Test the inverse hyperbolic functions
+
+        which are of the form arc[X]h
+        """
+        results = [0, 0.5, 1, 2, 1 + 1j]
+
+        sinh_vals = ['0', '0.5211', '1.1752', '3.6269', '0.635+1.2985*j']
+        self.assert_function_values('arcsinh', sinh_vals, results)
+
+        cosh_vals = ['1', '1.1276', '1.5431', '3.7622', '0.8337+0.9889*j']
+        self.assert_function_values('arccosh', cosh_vals, results)
+
+        tanh_vals = ['0', '0.4621', '0.7616', '0.964', '1.0839+0.2718*j']
+        self.assert_function_values('arctanh', tanh_vals, results)
+
+        sech_vals = ['1.0', '0.8868', '0.6481', '0.2658', '0.4983-0.5911*j']
+        self.assert_function_values('arcsech', sech_vals, results)
+
+        results = results[1:]
+        csch_vals = ['1.919', '0.8509', '0.2757', '0.3039-0.6215*j']
+        self.assert_function_values('arccsch', csch_vals, results)
+
+        coth_vals = ['2.164', '1.313', '1.0373', '0.868-0.2176*j']
+        self.assert_function_values('arccoth', coth_vals, results)
+
     def test_other_functions(self):
         """
         Test the non-trig functions provided in calc.py

common/lib/capa/capa/capa_problem.py

@@ -470,6 +470,7 @@ class LoncapaProblem(object):
                     python_path=python_path,
                     cache=self.system.cache,
                     slug=self.problem_id,
+                    unsafely=self.system.can_execute_unsafe_code(),
                 )
             except Exception as err:
                 log.exception("Error while execing script code: " + all_code)

common/lib/capa/capa/inputtypes.py

@@ -144,11 +144,11 @@ class InputTypeBase(object):
         self.tag = xml.tag
         self.system = system
 
-        ## NOTE: ID should only come from one place.  If it comes from multiple,
-        ## we use state first, XML second (in case the xml changed, but we have
-        ## existing state with an old id). Since we don't make this guarantee,
-        ## we can swap this around in the future if there's a more logical
-        ## order.
+        # NOTE: ID should only come from one place.  If it comes from multiple,
+        # we use state first, XML second (in case the xml changed, but we have
+        # existing state with an old id). Since we don't make this guarantee,
+        # we can swap this around in the future if there's a more logical
+        # order.
 
         self.input_id = state.get('id', xml.get('id'))
         if self.input_id is None:
@@ -769,7 +769,7 @@ class MatlabInput(CodeInput):
 
         # construct xqueue headers
         qinterface = self.system.xqueue['interface']
-        qtime = datetime.strftime(datetime.utcnow(), xqueue_interface.dateformat)
+        qtime = datetime.utcnow().strftime(xqueue_interface.dateformat)
         callback_url = self.system.xqueue['construct_callback']('ungraded_response')
         anonymous_student_id = self.system.anonymous_student_id
         queuekey = xqueue_interface.make_hashkey(str(self.system.seed) + qtime +

common/lib/capa/capa/responsetypes.py

@@ -288,7 +288,14 @@ class LoncapaResponse(object):
             }
 
             try:
-                safe_exec.safe_exec(code, globals_dict, python_path=self.context['python_path'], slug=self.id)
+                safe_exec.safe_exec(
+                    code,
+                    globals_dict,
+                    python_path=self.context['python_path'],
+                    slug=self.id,
+                    random_seed=self.context['seed'],
+                    unsafely=self.system.can_execute_unsafe_code(),
+                )
             except Exception as err:
                 msg = 'Error %s in evaluating hint function %s' % (err, hintfn)
                 msg += "\nSee XML source line %s" % getattr(
@@ -973,7 +980,14 @@ class CustomResponse(LoncapaResponse):
                             'ans': ans,
                         }
                         globals_dict.update(kwargs)
-                        safe_exec.safe_exec(code, globals_dict, python_path=self.context['python_path'], slug=self.id)
+                        safe_exec.safe_exec(
+                            code,
+                            globals_dict,
+                            python_path=self.context['python_path'],
+                            slug=self.id,
+                            random_seed=self.context['seed'],
+                            unsafely=self.system.can_execute_unsafe_code(),
+                        )
                         return globals_dict['cfn_return']
                     return check_function
 
@@ -1090,7 +1104,14 @@ class CustomResponse(LoncapaResponse):
         # exec the check function
         if isinstance(self.code, basestring):
             try:
-                safe_exec.safe_exec(self.code, self.context, cache=self.system.cache, slug=self.id)
+                safe_exec.safe_exec(
+                    self.code,
+                    self.context,
+                    cache=self.system.cache,
+                    slug=self.id,
+                    random_seed=self.context['seed'],
+                    unsafely=self.system.can_execute_unsafe_code(),
+                )
             except Exception as err:
                 self._handle_exec_exception(err)
 
@@ -1717,6 +1738,7 @@ class FormulaResponse(LoncapaResponse):
             student_variables = dict()
             # ranges give numerical ranges for testing
             for var in ranges:
+                # TODO: allow specified ranges (i.e. integers and complex numbers) for random variables
                 value = random.uniform(*ranges[var])
                 instructor_variables[str(var)] = value
                 student_variables[str(var)] = value
@@ -1814,7 +1836,14 @@ class SchematicResponse(LoncapaResponse):
         ]
         self.context.update({'submission': submission})
         try:
-            safe_exec.safe_exec(self.code, self.context, cache=self.system.cache, slug=self.id)
+            safe_exec.safe_exec(
+                self.code,
+                self.context,
+                cache=self.system.cache,
+                slug=self.id,
+                random_seed=self.context['seed'],
+                unsafely=self.system.can_execute_unsafe_code(),
+            )
         except Exception as err:
             msg = 'Error %s in evaluating SchematicResponse' % err
             raise ResponseError(msg)

common/lib/capa/capa/safe_exec/safe_exec.py

 """Capa's specialized use of codejail.safe_exec."""
 
 from codejail.safe_exec import safe_exec as codejail_safe_exec
+from codejail.safe_exec import not_safe_exec as codejail_not_safe_exec
 from codejail.safe_exec import json_safe, SafeExecException
 from . import lazymod
 from statsd import statsd
@@ -71,7 +72,7 @@ def update_hash(hasher, obj):
 
 
 @statsd.timed('capa.safe_exec.time')
-def safe_exec(code, globals_dict, random_seed=None, python_path=None, cache=None, slug=None):
+def safe_exec(code, globals_dict, random_seed=None, python_path=None, cache=None, slug=None, unsafely=False):
     """
     Execute python code safely.
 
@@ -90,6 +91,8 @@ def safe_exec(code, globals_dict, random_seed=None, python_path=None, cache=None
     `slug` is an arbitrary string, a description that's meaningful to the
     caller, that will be used in log messages.
 
+    If `unsafely` is true, then the code will actually be executed without sandboxing.
+
     """
     # Check the cache for a previous result.
     if cache:
@@ -111,9 +114,15 @@ def safe_exec(code, globals_dict, random_seed=None, python_path=None, cache=None
     # Create the complete code we'll run.
     code_prolog = CODE_PROLOG % random_seed
 
+    # Decide which code executor to use.
+    if unsafely:
+        exec_fn = codejail_not_safe_exec
+    else:
+        exec_fn = codejail_safe_exec
+
     # Run the code!  Results are side effects in globals_dict.
     try:
-        codejail_safe_exec(
+        exec_fn(
             code_prolog + LAZY_IMPORTS + code, globals_dict,
             python_path=python_path, slug=slug,
         )

common/lib/capa/capa/safe_exec/tests/test_safe_exec.py

 """Test safe_exec.py"""
 
 import hashlib
+import os
 import os.path
 import random
 import textwrap
 import unittest
 
+from nose.plugins.skip import SkipTest
+
 from capa.safe_exec import safe_exec, update_hash
 from codejail.safe_exec import SafeExecException
+from codejail.jail_code import is_configured
 
 
 class TestSafeExec(unittest.TestCase):
@@ -68,6 +72,24 @@ class TestSafeExec(unittest.TestCase):
         self.assertIn("ZeroDivisionError", cm.exception.message)
 
 
+class TestSafeOrNot(unittest.TestCase):
+    def test_cant_do_something_forbidden(self):
+        # Can't test for forbiddenness if CodeJail isn't configured for python.
+        if not is_configured("python"):
+            raise SkipTest
+
+        g = {}
+        with self.assertRaises(SafeExecException) as cm:
+            safe_exec("import os; files = os.listdir('/')", g)
+        self.assertIn("OSError", cm.exception.message)
+        self.assertIn("Permission denied", cm.exception.message)
+
+    def test_can_do_something_forbidden_if_run_unsafely(self):
+        g = {}
+        safe_exec("import os; files = os.listdir('/')", g, unsafely=True)
+        self.assertEqual(g['files'], os.listdir('/'))
+
+
 class DictCache(object):
     """A cache implementation over a simple dict, for testing."""
 

common/lib/capa/capa/templates/annotationinput.html

@@ -14,7 +14,7 @@
     <div class="block block-comment">${comment}</div>
 
     <div class="block">${comment_prompt}</div>
-    <textarea class="comment" id="input_${id}_comment" name="input_${id}_comment">${comment_value|h}</textarea>
+    <textarea class="comment" id="input_${id}_comment" name="input_${id}_comment" aria-describedby="answer_${id}">${comment_value|h}</textarea>
 
     <div class="block">${tag_prompt}</div>
     <ul class="tags">
@@ -22,11 +22,11 @@
         <li>
             % if has_options_value:
                 % if all([c == 'correct' for c in option['choice'], status]):
-                <span class="tag-status correct" id="status_${id}"></span>
+                <span class="tag-status correct" id="status_${id}" aria-describedby="input_${id}_comment"><span class="sr">Status: Correct</span></span>
                 % elif all([c == 'partially-correct' for c in option['choice'], status]):
-                <span class="tag-status partially-correct" id="status_${id}"></span>
+                <span class="tag-status partially-correct" id="status_${id}" aria-describedby="input_${id}_comment"><span class="sr">Status: Partially Correct</span></span>
                 % elif all([c == 'incorrect' for c in option['choice'], status]):
-                <span class="tag-status incorrect" id="status_${id}"></span>
+                <span class="tag-status incorrect" id="status_${id}" aria-describedby="input_${id}_comment"><span class="sr">Status: Incorrect</span></span>
                 % endif
             % endif
 
@@ -53,11 +53,11 @@
     % endif
 
     % if status == 'unsubmitted':
-    <span class="unanswered" style="display:inline-block;" id="status_${id}"></span>
+    <span class="unanswered" style="display:inline-block;" id="status_${id}" aria-describedby="input_${id}"><span class="sr">Status: Unanswered</span></span>
     % elif status == 'incomplete':
-    <span class="incorrect" id="status_${id}"></span>
+    <span class="incorrect" id="status_${id}" aria-describedby="input_${id}"><span class="sr">Status: Incorrect</span></span>
     % elif status == 'incorrect' and not has_options_value:
-    <span class="incorrect" id="status_${id}"></span>
+    <span class="incorrect" id="status_${id}" aria-describedby="input_${id}"><span class="sr">Status: Incorrect</span></span>
     % endif
 
     <p id="answer_${id}" class="answer answer-annotation"></p>

common/lib/capa/capa/templates/chemicalequationinput.html

@@ -11,13 +11,13 @@
     <div class="incorrect" id="status_${id}">
     % endif
 
-    <input type="text" name="input_${id}" id="input_${id}" data-input-id="${id}" value="${value|h}"
+    <input type="text" name="input_${id}" id="input_${id}" aria-describedby="answer_${id}" data-input-id="${id}" value="${value|h}"
    % if size:
    size="${size}"
    % endif
    />
 
-      <p class="status">
+      <p class="status" aria-describedby="input_${id}">
         % if status == 'unsubmitted':
         unanswered
         % elif status == 'correct':

common/lib/capa/capa/templates/choicegroup.html

@@ -3,12 +3,12 @@
     % if input_type == 'checkbox' or not value:
         % if status == 'unsubmitted' or show_correctness == 'never':
         <span class="unanswered" style="display:inline-block;" id="status_${id}"></span>
-        % elif status == 'correct': 
-        <span class="correct" id="status_${id}"></span>
+        % elif status == 'correct':
+        <span class="correct" id="status_${id}"><span class="sr">Status: correct</span></span>
         % elif status == 'incorrect':
-        <span class="incorrect" id="status_${id}"></span>
+        <span class="incorrect" id="status_${id}"><span class="sr">Status: incorrect</span></span>
         % elif status == 'incomplete':
-        <span class="incorrect" id="status_${id}"></span>
+        <span class="incorrect" id="status_${id}"><span class="sr">Status: incomplete</span></span>
         % endif
     % endif
     </div>
@@ -18,7 +18,7 @@
         % for choice_id, choice_description in choices:
         <label for="input_${id}_${choice_id}"
             % if input_type == 'radio' and ( (isinstance(value, basestring) and (choice_id == value))  or  (not isinstance(value, basestring) and choice_id in value) ):
-            <% 
+            <%
                 if status == 'correct':
                     correctness = 'correct'
                 elif status == 'incorrect':
@@ -31,14 +31,29 @@
             % endif
             % endif
             >
-            <input type="${input_type}" name="input_${id}${name_array_suffix}" id="input_${id}_${choice_id}" value="${choice_id}"
+            <input type="${input_type}" name="input_${id}${name_array_suffix}" id="input_${id}_${choice_id}" aria-describedby="answer_${id}" value="${choice_id}"
             % if input_type == 'radio' and ( (isinstance(value, basestring) and (choice_id == value))  or  (not isinstance(value, basestring) and choice_id in value) ):
-            checked="true" 
+            checked="true"
             % elif input_type != 'radio' and choice_id in value:
             checked="true"
             % endif
 
-            /> ${choice_description} </label>
+            /> ${choice_description}
+
+            % if input_type == 'radio' and ( (isinstance(value, basestring) and (choice_id == value))  or  (not isinstance(value, basestring) and choice_id in value) ):
+            <%
+                if status == 'correct':
+                    correctness = 'correct'
+                elif status == 'incorrect':
+                    correctness = 'incorrect'
+                else:
+                    correctness = None
+            %>
+            % if correctness and not show_correctness=='never':
+            <span class="sr" aria-describedby="input_${id}_${choice_id}">Status: ${correctness}</span>
+            % endif
+            % endif
+        </label>
         % endfor
         <span id="answer_${id}"></span>
     </fieldset>

common/lib/capa/capa/templates/codeinput.html

 <section id="textbox_${id}" class="textbox">
-  <textarea rows="${rows}" cols="${cols}" name="input_${id}" id="input_${id}"
+  <textarea rows="${rows}" cols="${cols}" name="input_${id}" aria-describedby="answer_${id}" id="input_${id}"
        % if hidden:
       	    style="display:none;"
        % endif
@@ -7,13 +7,13 @@
 
   <div class="grader-status">
     % if status == 'unsubmitted':
-      <span class="unanswered" style="display:inline-block;" id="status_${id}">Unanswered</span>
+      <span class="unanswered" style="display:inline-block;" id="status_${id}" aria-describedby="input_${id}"><span class="sr">Status: </span>Unanswered</span>
     % elif status == 'correct':
-      <span class="correct" id="status_${id}">Correct</span>
+      <span class="correct" id="status_${id}" aria-describedby="input_${id}"><span class="sr">Status: </span>Correct</span>
     % elif status == 'incorrect':
-      <span class="incorrect" id="status_${id}">Incorrect</span>
+      <span class="incorrect" id="status_${id}" aria-describedby="input_${id}"><span class="sr">Status: </span>Incorrect</span>
     % elif status == 'queued':
-      <span class="processing" id="status_${id}">Queued</span>
+      <span class="processing" id="status_${id}" aria-describedby="input_${id}"><span class="sr">Status: </span>Queued</span>
       <span style="display:none;" class="xqueue" id="${id}" >${queue_len}</span>
     % endif
 

common/lib/capa/capa/templates/crystallography.html

@@ -20,9 +20,9 @@
     % endif
 
 
-    <input type="text" name="input_${id}" id="input_${id}" value="${value|h}" style="display:none;"/>
+    <input type="text" name="input_${id}" aria-describedby="answer_${id}" id="input_${id}" value="${value|h}" style="display:none;"/>
 
-    <p class="status">
+    <p class="status" aria-describedby="input_${id}">
         % if status == 'unsubmitted':
         unanswered
         % elif status == 'correct':

common/lib/capa/capa/templates/designprotein2dinput.html

@@ -11,12 +11,12 @@
     % elif status == 'incomplete':
     <div class="incomplete" id="status_${id}">
     % endif
-    
+
     <div id="protex_container"></div>
     <input type="hidden" name="target_shape" id="target_shape" value ="${target_shape}"></input>
-    <input type="hidden" name="input_${id}" id="input_${id}" value="${value|h}"/>
+    <input type="hidden" name="input_${id}" id="input_${id}" aria-describedby="answer_${id}" value="${value|h}"/>
 
-    <p class="status">
+    <p class="status" aria-describedby="input_${id}">
       % if status == 'unsubmitted':
       unanswered
       % elif status == 'correct':

common/lib/capa/capa/templates/drag_and_drop_input.html

@@ -19,10 +19,10 @@
     % endif
 
 
-    <input type="text" name="input_${id}" id="input_${id}" value="${value|h}"
+    <input type="text" name="input_${id}" id="input_${id}" aria-describedby="answer_${id}" value="${value|h}"
     style="display:none;"/>
 
-    <p class="status">
+    <p class="status" aria-describedby="input_${id}">
         % if status == 'unsubmitted':
         unanswered
         % elif status == 'correct':

common/lib/capa/capa/templates/editageneinput.html

@@ -11,13 +11,13 @@
     % elif status == 'incomplete':
     <div class="incomplete" id="status_${id}">
     % endif
-    
+
     <div id="genex_container"></div>
     <input type="hidden" name="genex_dna_sequence" id="genex_dna_sequence" value ="${genex_dna_sequence}"></input>
     <input type="hidden" name="genex_problem_number" id="genex_problem_number" value ="${genex_problem_number}"></input>
-    <input type="hidden" name="input_${id}" id="input_${id}" value="${value|h}"/>
+    <input type="hidden" name="input_${id}" aria-describedby="answer_${id}" id="input_${id}" value="${value|h}"/>
 
-    <p class="status">
+    <p class="status" aria-describedby="input_${id}">
       % if status == 'unsubmitted':
       unanswered
       % elif status == 'correct':

common/lib/capa/capa/templates/editamolecule.html

@@ -16,13 +16,13 @@
 
     <br/>
 
-    <input type="hidden" name="input_${id}" id="input_${id}" value="${value|h}"/>
+    <input type="hidden" name="input_${id}" id="input_${id}" aria-describedby="answer_${id}" value="${value|h}"/>
 
     <button id="reset_${id}" class="reset">Reset</button>
 
     <p id="answer_${id}" class="answer"></p>
 
-    <p class="status">
+    <p class="status" aria-describedby="input_${id}">
       % if status == 'unsubmitted':
       unanswered
       % elif status == 'correct':

common/lib/capa/capa/templates/imageinput.html

@@ -5,12 +5,20 @@
   </div>
 
   % if status == 'unsubmitted':
-  <span class="unanswered" style="display:inline-block;" id="status_${id}"></span>
-  % elif status == 'correct': 
-  <span class="correct" id="status_${id}"></span>
+  <span class="unanswered" style="display:inline-block;" id="status_${id}" aria-describedby="input_${id}">
+    <span class="sr">Status: unanswered</span>
+  </span>
+  % elif status == 'correct':
+  <span class="correct" id="status_${id}" aria-describedby="input_${id}">
+    <span class="sr">Status: correct</span>
+  </span>
   % elif status == 'incorrect':
-  <span class="incorrect" id="status_${id}"></span>
+  <span class="incorrect" id="status_${id}" aria-describedby="input_${id}">
+    <span class="sr">Status: incorrect</span>
+  </span>
   % elif status == 'incomplete':
-  <span class="incorrect" id="status_${id}"></span>
+  <span class="incorrect" id="status_${id}" aria-describedby="input_${id}">
+    <span class="sr">Status: incorrect</span>
+  </span>
   % endif
 </span>

common/lib/capa/capa/templates/jstextline.html

@@ -19,13 +19,21 @@
   % endif
 
   % if status == 'unsubmitted':
-  <span class="unanswered" style="display:inline-block;" id="status_${id}"></span>
+  <span class="unanswered" style="display:inline-block;" id="status_${id}" aria-describedby="input_${id}">
+    <span class="sr">Status: unanswered</span>
+  </span>
   % elif status == 'correct':
-  <span class="correct" id="status_${id}"></span>
+  <span class="correct" id="status_${id}" aria-describedby="input_${id}">
+    <span class="sr">Status: correct</span>
+  </span>
   % elif status == 'incorrect':
-  <span class="incorrect" id="status_${id}"></span>
+  <span class="incorrect" id="status_${id}" aria-describedby="input_${id}">
+    <span class="sr">Status: incorrect</span>
+  </span>
   % elif status == 'incomplete':
-  <span class="incorrect" id="status_${id}"></span>
+  <span class="incorrect" id="status_${id}" aria-describedby="input_${id}">
+    <span class="sr">Status: incorrect</span>
+  </span>
   % endif
   % if msg:
     <br/>

common/lib/capa/capa/templates/matlabinput.html

 <section id="textbox_${id}" class="textbox">
-  <textarea rows="${rows}" cols="${cols}" name="input_${id}" id="input_${id}"
+  <textarea rows="${rows}" cols="${cols}" name="input_${id}" aria-describedby="answer_${id}" id="input_${id}"
        % if hidden:
       	    style="display:none;"
        % endif
@@ -7,13 +7,13 @@
 
   <div class="grader-status">
     % if status == 'unsubmitted':
-      <span class="unanswered" style="display:inline-block;" id="status_${id}">Unanswered</span>
+      <span class="unanswered" style="display:inline-block;" id="status_${id}"><span class="sr">Status: </span>Unanswered</span>
     % elif status == 'correct':
-      <span class="correct" id="status_${id}">Correct</span>
+      <span class="correct" id="status_${id}" aria-describedby="input_${id}"><span class="sr">Status: </span>Correct</span>
     % elif status == 'incorrect':
-      <span class="incorrect" id="status_${id}">Incorrect</span>
+      <span class="incorrect" id="status_${id}" aria-describedby="input_${id}"><span class="sr">Status: </span>Incorrect</span>
     % elif status == 'queued':
-      <span class="processing" id="status_${id}">Queued</span>
+      <span class="processing" id="status_${id}" aria-describedby="input_${id}"><span class="sr">Status: </span>Queued</span>
       <span style="display:none;" class="xqueue" id="${id}" >${queue_len}</span>
     % endif
 
@@ -71,7 +71,7 @@
           $(parent_elt).find('.action').after(alert_elem);
           $(parent_elt).find('.capa_alert').css({opacity: 0}).animate({opacity: 1}, 700);
       }
-      
+
 
       // hook up the plot button
       var plot = function(event) {
@@ -97,10 +97,10 @@
               }
           }
 
-          var save_callback = function(response) { 
+          var save_callback = function(response) {
                   if(response.success) {
                       // send information to the problem's plot functionality
-                      Problem.inputAjax(url, input_id, 'plot', 
+                      Problem.inputAjax(url, input_id, 'plot',
                         {'submission': submission}, plot_callback);
                   }
                   else {

common/lib/capa/capa/templates/optioninput.html

 <form class="option-input">
-   <select name="input_${id}" id="input_${id}" >
+   <select name="input_${id}" id="input_${id}" aria-describedby="answer_${id}">
       <option value="option_${id}_dummy_default">  </option>
       % for option_id, option_description in options:
           <option value="${option_id}"
@@ -13,12 +13,20 @@
   <span id="answer_${id}"></span>
 
   % if status == 'unsubmitted':
-  <span class="unanswered" style="display:inline-block;" id="status_${id}"></span>
-  % elif status == 'correct': 
-  <span class="correct" id="status_${id}"></span>
+  <span class="unanswered" style="display:inline-block;" id="status_${id}" aria-describedby="input_${id}">
+    <span class="sr">Status: unsubmitted</span>
+  </span>
+  % elif status == 'correct':
+  <span class="correct" id="status_${id}" aria-describedby="input_${id}">
+    <span class="sr">Status: correct</span>
+  </span>
   % elif status == 'incorrect':
-  <span class="incorrect" id="status_${id}"></span>
+  <span class="incorrect" id="status_${id}" aria-describedby="input_${id}">
+    <span class="sr">Status: incorrect</span>
+  </span>
   % elif status == 'incomplete':
-  <span class="incorrect" id="status_${id}"></span>
+  <span class="incorrect" id="status_${id}" aria-describedby="input_${id}">
+    <span class="sr">Status: incomplete</span>
+  </span>
   % endif
 </form>

common/lib/capa/capa/templates/schematicinput.html

 <span>
-  <input type="hidden" class="schematic" height="${height}" width="${width}" parts="${parts}" analyses="${analyses}" name="input_${id}" id="input_${id}" value="" initial_value=""/>
+  <input type="hidden" class="schematic" height="${height}" width="${width}" parts="${parts}" analyses="${analyses}" name="input_${id}" id="input_${id}" aria-describedby="answer_${id}" value="" initial_value=""/>
 
   <div id="value_${id}" style="display:none">${value}</div>
   <div id="initial_value_${id}" style="display:none">${initial_value}</div>
@@ -13,13 +13,21 @@
 
   <span id="answer_${id}"></span>
   % if status == 'unsubmitted':
-  <span class="ui-icon ui-icon-bullet" style="display:inline-block;" id="status_${id}"></span>
-  % elif status == 'correct': 
-  <span class="ui-icon ui-icon-check" style="display:inline-block;" id="status_${id}"></span>
+  <span class="ui-icon ui-icon-bullet" style="display:inline-block;" id="status_${id}" aria-describedby="input_${id}">
+    <span class="sr">Status: unsubmitted</span>
+  </span>
+  % elif status == 'correct':
+  <span class="ui-icon ui-icon-check" style="display:inline-block;" id="status_${id}" aria-describedby="input_${id}">
+    <span class="sr">Status: correct</span>
+  </span>
   % elif status == 'incorrect':
-  <span class="ui-icon ui-icon-close" style="display:inline-block;" id="status_${id}"></span>
+  <span class="ui-icon ui-icon-close" style="display:inline-block;" id="status_${id}" aria-describedby="input_${id}">
+    <span class="sr">Status: incorrect</span>
+  </span>
   % elif status == 'incomplete':
-  <span class="ui-icon ui-icon-close" style="display:inline-block;" id="status_${id}"></span>
+  <span class="ui-icon ui-icon-close" style="display:inline-block;" id="status_${id}" aria-describedby="input_${id}">
+    <span class="sr">Status: incomplete</span>
+  </span>
   % endif
 </span>
 

common/lib/capa/capa/templates/textline.html

@@ -20,7 +20,7 @@
       <div style="display:none;" name="${hidden}" inputid="input_${id}" />
     % endif
 
-  <input type="text" name="input_${id}" id="input_${id}" value="${value|h}"
+  <input type="text" name="input_${id}" id="input_${id}" aria-describedby="answer_${id}" value="${value|h}"
         % if do_math:
             class="math"
         % endif
@@ -33,7 +33,7 @@
    />
    ${trailing_text | h}
 
-      <p class="status">
+      <p class="status" aria-describedby="input_${id}">
         % if status == 'unsubmitted':
         unanswered
         % elif status == 'correct':

common/lib/capa/capa/templates/vsepr_input.html

@@ -21,11 +21,11 @@
     <div class="incorrect" id="status_${id}">
     % endif
 
-  <input type="text" name="input_${id}" id="input_${id}" value="${value|h}"
+  <input type="text" name="input_${id}" id="input_${id}" aria-describedby="answer_${id}" value="${value|h}"
       style="display:none;"
    />
 
-      <p class="status">
+      <p class="status" aria-describedby="input_${id}">
         % if status == 'unsubmitted':
         unanswered
         % elif status == 'correct':

common/lib/capa/capa/tests/test_responsetypes.py

@@ -640,6 +640,23 @@ class StringResponseTest(ResponseTest):
         correct_map = problem.grade_answers(input_dict)
         self.assertEquals(correct_map.get_hint('1_2_1'), "Hello??")
 
+    def test_hint_function_randomization(self):
+        # The hint function should get the seed from the problem.
+        problem = self.build_problem(
+            answer="1",
+            hintfn="gimme_a_random_hint",
+            script=textwrap.dedent("""
+                def gimme_a_random_hint(answer_ids, student_answers, new_cmap, old_cmap):
+                    answer = str(random.randint(0, 1e9))
+                    new_cmap.set_hint_and_mode(answer_ids[0], answer, "always")
+
+            """)
+        )
+        correct_map = problem.grade_answers({'1_2_1': '2'})
+        hint = correct_map.get_hint('1_2_1')
+        r = random.Random(problem.seed)
+        self.assertEqual(hint, str(r.randint(0, 1e9)))
+
 
 class CodeResponseTest(ResponseTest):
     from response_xml_factory import CodeResponseXMLFactory
@@ -948,7 +965,6 @@ class CustomResponseTest(ResponseTest):
     xml_factory_class = CustomResponseXMLFactory
 
     def test_inline_code(self):
-
         # For inline code, we directly modify global context variables
         # 'answers' is a list of answers provided to us
         # 'correct' is a list we fill in with True/False
@@ -961,15 +977,14 @@ class CustomResponseTest(ResponseTest):
         self.assert_grade(problem, '0', 'incorrect')
 
     def test_inline_message(self):
-
         # Inline code can update the global messages list
         # to pass messages to the CorrectMap for a particular input
         # The code can also set the global overall_message (str)
         # to pass a message that applies to the whole response
         inline_script = textwrap.dedent("""
-        messages[0] = "Test Message"
-        overall_message = "Overall message"
-        """)
+            messages[0] = "Test Message"
+            overall_message = "Overall message"
+            """)
         problem = self.build_problem(answer=inline_script)
 
         input_dict = {'1_2_1': '0'}
@@ -983,8 +998,19 @@ class CustomResponseTest(ResponseTest):
         overall_msg = correctmap.get_overall_message()
         self.assertEqual(overall_msg, "Overall message")
 
-    def test_function_code_single_input(self):
+    def test_inline_randomization(self):
+        # Make sure the seed from the problem gets fed into the script execution.
+        inline_script = """messages[0] = str(random.randint(0, 1e9))"""
+        problem = self.build_problem(answer=inline_script)
+
+        input_dict = {'1_2_1': '0'}
+        correctmap = problem.grade_answers(input_dict)
 
+        input_msg = correctmap.get_msg('1_2_1')
+        r = random.Random(problem.seed)
+        self.assertEqual(input_msg, str(r.randint(0, 1e9)))
+
+    def test_function_code_single_input(self):
         # For function code, we pass in these arguments:
         #
         #   'expect' is the expect attribute of the <customresponse>
@@ -1212,6 +1238,29 @@ class CustomResponseTest(ResponseTest):
         with self.assertRaises(ResponseError):
             problem.grade_answers({'1_2_1': '42'})
 
+    def test_setup_randomization(self):
+        # Ensure that the problem setup script gets the random seed from the problem.
+        script = textwrap.dedent("""
+            num = random.randint(0, 1e9)
+            """)
+        problem = self.build_problem(script=script)
+        r = random.Random(problem.seed)
+        self.assertEqual(r.randint(0, 1e9), problem.context['num'])
+
+    def test_check_function_randomization(self):
+        # The check function should get random-seeded from the problem.
+        script = textwrap.dedent("""
+            def check_func(expect, answer_given):
+                return {'ok': True, 'msg': str(random.randint(0, 1e9))}
+        """)
+
+        problem = self.build_problem(script=script, cfn="check_func", expect="42")
+        input_dict = {'1_2_1': '42'}
+        correct_map = problem.grade_answers(input_dict)
+        msg = correct_map.get_msg('1_2_1')
+        r = random.Random(problem.seed)
+        self.assertEqual(msg, str(r.randint(0, 1e9)))
+
     def test_module_imports_inline(self):
         '''
         Check that the correct modules are available to custom
@@ -1275,7 +1324,6 @@ class SchematicResponseTest(ResponseTest):
     xml_factory_class = SchematicResponseXMLFactory
 
     def test_grade(self):
-
         # Most of the schematic-specific work is handled elsewhere
         # (in client-side JavaScript)
         # The <schematicresponse> is responsible only for executing the
@@ -1290,7 +1338,7 @@ class SchematicResponseTest(ResponseTest):
 
         # The actual dictionary would contain schematic information
         # sent from the JavaScript simulation
-        submission_dict = {'test': 'test'}
+        submission_dict = {'test': 'the_answer'}
         input_dict = {'1_2_1': json.dumps(submission_dict)}
         correct_map = problem.grade_answers(input_dict)
 
@@ -1299,8 +1347,19 @@ class SchematicResponseTest(ResponseTest):
         # is what we expect)
         self.assertEqual(correct_map.get_correctness('1_2_1'), 'correct')
 
-    def test_script_exception(self):
+    def test_check_function_randomization(self):
+        # The check function should get a random seed from the problem.
+        script = "correct = ['correct' if (submission[0]['num'] == random.randint(0, 1e9)) else 'incorrect']"
+        problem = self.build_problem(answer=script)
 
+        r = random.Random(problem.seed)
+        submission_dict = {'num': r.randint(0, 1e9)}
+        input_dict = {'1_2_1': json.dumps(submission_dict)}
+        correct_map = problem.grade_answers(input_dict)
+
+        self.assertEqual(correct_map.get_correctness('1_2_1'), 'correct')
+
+    def test_script_exception(self):
         # Construct a script that will raise an exception
         script = "raise Exception('test')"
         problem = self.build_problem(answer=script)

common/lib/xmodule/xmodule/abtest_module.py

@@ -6,7 +6,7 @@ from xmodule.x_module import XModule
 from xmodule.raw_module import RawDescriptor
 from xmodule.xml_module import XmlDescriptor
 from xmodule.exceptions import InvalidDefinitionError
-from xblock.core import String, Scope, Object
+from xblock.core import String, Scope, Dict
 
 DEFAULT = "_DEFAULT_GROUP"
 
@@ -32,9 +32,9 @@ def group_from_value(groups, v):
 
 
 class ABTestFields(object):
-    group_portions = Object(help="What proportions of students should go in each group", default={DEFAULT: 1}, scope=Scope.content)
-    group_assignments = Object(help="What group this user belongs to", scope=Scope.preferences, default={})
-    group_content = Object(help="What content to display to each group", scope=Scope.content, default={DEFAULT: []})
+    group_portions = Dict(help="What proportions of students should go in each group", default={DEFAULT: 1}, scope=Scope.content)
+    group_assignments = Dict(help="What group this user belongs to", scope=Scope.preferences, default={})
+    group_content = Dict(help="What content to display to each group", scope=Scope.content, default={DEFAULT: []})
     experiment = String(help="Experiment that this A/B test belongs to", scope=Scope.content)
     has_children = True
 

common/lib/xmodule/xmodule/annotatable_module.py

@@ -125,6 +125,5 @@ class AnnotatableModule(AnnotatableFields, XModule):
 
 class AnnotatableDescriptor(AnnotatableFields, RawDescriptor):
     module_class = AnnotatableModule
-    stores_state = True
     template_dir_name = "annotatable"
     mako_template = "widgets/raw-edit.html"

common/lib/xmodule/xmodule/capa_module.py

@@ -11,16 +11,16 @@ import sys
 from pkg_resources import resource_string
 
 from capa.capa_problem import LoncapaProblem
-from capa.responsetypes import StudentInputError,\
+from capa.responsetypes import StudentInputError, \
     ResponseError, LoncapaProblemError
 from capa.util import convert_files_to_filenames
 from .progress import Progress
 from xmodule.x_module import XModule
 from xmodule.raw_module import RawDescriptor
 from xmodule.exceptions import NotFoundError, ProcessingError
-from xblock.core import Scope, String, Boolean, Object
-from .fields import Timedelta, Date, StringyInteger, StringyFloat
-from xmodule.util.date_utils import time_to_datetime
+from xblock.core import Scope, String, Boolean, Dict, Integer, Float
+from .fields import Timedelta, Date
+from django.utils.timezone import UTC
 
 log = logging.getLogger("mitx.courseware")
 
@@ -65,11 +65,11 @@ class ComplexEncoder(json.JSONEncoder):
 
 
 class CapaFields(object):
-    attempts = StringyInteger(help="Number of attempts taken by the student on this problem", default=0, scope=Scope.user_state)
-    max_attempts = StringyInteger(
+    attempts = Integer(help="Number of attempts taken by the student on this problem", default=0, scope=Scope.user_state)
+    max_attempts = Integer(
         display_name="Maximum Attempts",
         help="Defines the number of times a student can try to answer this problem. If the value is not set, infinite attempts are allowed.",
-        values={"min": 1}, scope=Scope.settings
+        values={"min": 0}, scope=Scope.settings
     )
     due = Date(help="Date that this problem is due by", scope=Scope.settings)
     graceperiod = Timedelta(help="Amount of time after the due date that submissions will be accepted", scope=Scope.settings)
@@ -95,12 +95,12 @@ class CapaFields(object):
                                                         {"display_name": "Per Student", "value": "per_student"}]
     )
     data = String(help="XML data for the problem", scope=Scope.content)
-    correct_map = Object(help="Dictionary with the correctness of current student answers", scope=Scope.user_state, default={})
-    input_state = Object(help="Dictionary for maintaining the state of inputtypes", scope=Scope.user_state)
-    student_answers = Object(help="Dictionary with the current student responses", scope=Scope.user_state)
+    correct_map = Dict(help="Dictionary with the correctness of current student answers", scope=Scope.user_state, default={})
+    input_state = Dict(help="Dictionary for maintaining the state of inputtypes", scope=Scope.user_state)
+    student_answers = Dict(help="Dictionary with the current student responses", scope=Scope.user_state)
     done = Boolean(help="Whether the student has answered the problem", scope=Scope.user_state)
-    seed = StringyInteger(help="Random seed for this student", scope=Scope.user_state)
-    weight = StringyFloat(
+    seed = Integer(help="Random seed for this student", scope=Scope.user_state)
+    weight = Float(
         display_name="Problem Weight",
         help="Defines the number of points each problem is worth. If the value is not set, each response field in the problem is worth one point.",
         values={"min": 0, "step": .1},
@@ -117,6 +117,8 @@ class CapaModule(CapaFields, XModule):
     '''
     An XModule implementing LonCapa format problems, implemented by way of
     capa.capa_problem.LoncapaProblem
+
+    CapaModule.__init__ takes the same arguments as xmodule.x_module:XModule.__init__
     '''
     icon_class = 'problem'
 
@@ -131,10 +133,11 @@ class CapaModule(CapaFields, XModule):
     js_module_name = "Problem"
     css = {'scss': [resource_string(__name__, 'css/capa/display.scss')]}
 
-    def __init__(self, system, location, descriptor, model_data):
-        XModule.__init__(self, system, location, descriptor, model_data)
+    def __init__(self, *args, **kwargs):
+        """ Accepts the same arguments as xmodule.x_module:XModule.__init__ """
+        XModule.__init__(self, *args, **kwargs)
 
-        due_date = time_to_datetime(self.due)
+        due_date = self.due
 
         if self.graceperiod is not None and due_date:
             self.close_date = due_date + self.graceperiod
@@ -315,7 +318,7 @@ class CapaModule(CapaFields, XModule):
         # If the user has forced the save button to display,
         # then show it as long as the problem is not closed
         # (past due / too many attempts)
-        if self.force_save_button == "true":
+        if self.force_save_button:
             return not self.closed()
         else:
             is_survey_question = (self.max_attempts == 0)
@@ -502,7 +505,7 @@ class CapaModule(CapaFields, XModule):
         Is it now past this problem's due date, including grace period?
         """
         return (self.close_date is not None and
-                datetime.datetime.utcnow() > self.close_date)
+                datetime.datetime.now(UTC()) > self.close_date)
 
     def closed(self):
         ''' Is the student still allowed to submit answers? '''
@@ -747,7 +750,7 @@ class CapaModule(CapaFields, XModule):
 
         # Problem queued. Students must wait a specified waittime before they are allowed to submit
         if self.lcp.is_queued():
-            current_time = datetime.datetime.now()
+            current_time = datetime.datetime.now(UTC())
             prev_submit_time = self.lcp.get_recentmost_queuetime()
             waittime_between_requests = self.system.xqueue['waittime']
             if (current_time - prev_submit_time).total_seconds() < waittime_between_requests:
@@ -902,7 +905,6 @@ class CapaDescriptor(CapaFields, RawDescriptor):
 
     module_class = CapaModule
 
-    stores_state = True
     has_score = True
     template_dir_name = 'problem'
     mako_template = "widgets/problem-edit.html"

common/lib/xmodule/xmodule/combined_open_ended_module.py

@@ -5,10 +5,10 @@ from pkg_resources import resource_string
 
 from xmodule.raw_module import RawDescriptor
 from .x_module import XModule
-from xblock.core import Integer, Scope, String, List
+from xblock.core import Integer, Scope, String, List, Float, Boolean
 from xmodule.open_ended_grading_classes.combined_open_ended_modulev1 import CombinedOpenEndedV1Module, CombinedOpenEndedV1Descriptor
 from collections import namedtuple
-from .fields import Date, StringyFloat, StringyInteger, StringyBoolean
+from .fields import Date
 
 log = logging.getLogger("mitx.courseware")
 
@@ -53,27 +53,27 @@ class CombinedOpenEndedFields(object):
         help="This name appears in the horizontal navigation at the top of the page.",
         default="Open Ended Grading", scope=Scope.settings
     )
-    current_task_number = StringyInteger(help="Current task that the student is on.", default=0, scope=Scope.user_state)
+    current_task_number = Integer(help="Current task that the student is on.", default=0, scope=Scope.user_state)
     task_states = List(help="List of state dictionaries of each task within this module.", scope=Scope.user_state)
     state = String(help="Which step within the current task that the student is on.", default="initial",
                    scope=Scope.user_state)
-    student_attempts = StringyInteger(help="Number of attempts taken by the student on this problem", default=0,
+    student_attempts = Integer(help="Number of attempts taken by the student on this problem", default=0,
                                       scope=Scope.user_state)
-    ready_to_reset = StringyBoolean(
+    ready_to_reset = Boolean(
         help="If the problem is ready to be reset or not.", default=False,
         scope=Scope.user_state
     )
-    attempts = StringyInteger(
+    attempts = Integer(
         display_name="Maximum Attempts",
         help="The number of times the student can try to answer this problem.", default=1,
         scope=Scope.settings, values = {"min" : 1 }
     )
-    is_graded = StringyBoolean(display_name="Graded", help="Whether or not the problem is graded.", default=False, scope=Scope.settings)
-    accept_file_upload = StringyBoolean(
+    is_graded = Boolean(display_name="Graded", help="Whether or not the problem is graded.", default=False, scope=Scope.settings)
+    accept_file_upload = Boolean(
         display_name="Allow File Uploads",
         help="Whether or not the student can submit files as a response.", default=False, scope=Scope.settings
     )
-    skip_spelling_checks = StringyBoolean(
+    skip_spelling_checks = Boolean(
         display_name="Disable Quality Filter",
         help="If False, the Quality Filter is enabled and submissions with poor spelling, short length, or poor grammar will not be peer reviewed.",
         default=False, scope=Scope.settings
@@ -86,7 +86,7 @@ class CombinedOpenEndedFields(object):
     )
     version = VersionInteger(help="Current version number", default=DEFAULT_VERSION, scope=Scope.settings)
     data = String(help="XML data for the problem", scope=Scope.content)
-    weight = StringyFloat(
+    weight = Float(
         display_name="Problem Weight",
         help="Defines the number of points each problem is worth. If the value is not set, each problem is worth one point.",
         scope=Scope.settings, values = {"min" : 0 , "step": ".1"}
@@ -116,6 +116,8 @@ class CombinedOpenEndedModule(CombinedOpenEndedFields, XModule):
     incorporates multiple children (tasks):
         openendedmodule
         selfassessmentmodule
+
+    CombinedOpenEndedModule.__init__ takes the same arguments as xmodule.x_module:XModule.__init__
     """
     STATE_VERSION = 1
 
@@ -139,8 +141,7 @@ class CombinedOpenEndedModule(CombinedOpenEndedFields, XModule):
 
     css = {'scss': [resource_string(__name__, 'css/combinedopenended/display.scss')]}
 
-    def __init__(self, system, location, descriptor, model_data):
-        XModule.__init__(self, system, location, descriptor, model_data)
+    def __init__(self, *args, **kwargs):
         """
         Definition file should have one or many task blocks, a rubric block, and a prompt block:
 
@@ -175,9 +176,9 @@ class CombinedOpenEndedModule(CombinedOpenEndedFields, XModule):
         </combinedopenended>
 
         """
+        XModule.__init__(self, *args, **kwargs)
 
-        self.system = system
-        self.system.set('location', location)
+        self.system.set('location', self.location)
 
         if self.task_states is None:
             self.task_states = []
@@ -189,13 +190,11 @@ class CombinedOpenEndedModule(CombinedOpenEndedFields, XModule):
 
         attributes = self.student_attributes + self.settings_attributes
 
-        static_data = {
-            'rewrite_content_links': self.rewrite_content_links,
-        }
+        static_data = {}
         instance_state = {k: getattr(self, k) for k in attributes}
         self.child_descriptor = version_tuple.descriptor(self.system)
         self.child_definition = version_tuple.descriptor.definition_from_xml(etree.fromstring(self.data), self.system)
-        self.child_module = version_tuple.module(self.system, location, self.child_definition, self.child_descriptor,
+        self.child_module = version_tuple.module(self.system, self.location, self.child_definition, self.child_descriptor,
                                                  instance_state=instance_state, static_data=static_data,
                                                  attributes=attributes)
         self.save_instance_data()
@@ -239,7 +238,6 @@ class CombinedOpenEndedDescriptor(CombinedOpenEndedFields, RawDescriptor):
     mako_template = "widgets/open-ended-edit.html"
     module_class = CombinedOpenEndedModule
 
-    stores_state = True
     has_score = True
     always_recalculate_grades = True
     template_dir_name = "combinedopenended"

common/lib/xmodule/xmodule/conditional_module.py

@@ -92,7 +92,7 @@ class ConditionalModule(ConditionalFields, XModule):
         if xml_value and self.required_modules:
             for module in self.required_modules:
                 if not hasattr(module, attr_name):
-                    # We don't throw an exception here because it is possible for 
+                    # We don't throw an exception here because it is possible for
                     # the descriptor of a required module to have a property but
                     # for the resulting module to be a (flavor of) ErrorModule.
                     # So just log and return false.
@@ -161,7 +161,6 @@ class ConditionalDescriptor(ConditionalFields, SequenceDescriptor):
 
     filename_extension = "xml"
 
-    stores_state = True
     has_score = False
 
     @staticmethod

common/lib/xmodule/xmodule/contentstore/django.py

@@ -3,7 +3,7 @@ from importlib import import_module
 
 from django.conf import settings
 
-_CONTENTSTORE = None
+_CONTENTSTORE = {}
 
 
 def load_function(path):
@@ -17,13 +17,16 @@ def load_function(path):
     return getattr(import_module(module_path), name)
 
 
-def contentstore():
+def contentstore(name='default'):
     global _CONTENTSTORE
 
-    if _CONTENTSTORE is None:
+    if name not in _CONTENTSTORE:
         class_ = load_function(settings.CONTENTSTORE['ENGINE'])
         options = {}
         options.update(settings.CONTENTSTORE['OPTIONS'])
-        _CONTENTSTORE = class_(**options)
+        if 'ADDITIONAL_OPTIONS' in settings.CONTENTSTORE:
+            if name in settings.CONTENTSTORE['ADDITIONAL_OPTIONS']:
+                options.update(settings.CONTENTSTORE['ADDITIONAL_OPTIONS'][name])
+        _CONTENTSTORE[name] = class_(**options)
 
-    return _CONTENTSTORE
+    return _CONTENTSTORE[name]

common/lib/xmodule/xmodule/contentstore/mongo.py

-from bson.son import SON
 from pymongo import Connection
 import gridfs
 from gridfs.errors import NoFile
@@ -15,15 +14,16 @@ import os
 
 
 class MongoContentStore(ContentStore):
-    def __init__(self, host, db, port=27017, user=None, password=None, **kwargs):
+    def __init__(self, host, db, port=27017, user=None, password=None, bucket='fs', **kwargs):
         logging.debug('Using MongoDB for static content serving at host={0} db={1}'.format(host, db))
         _db = Connection(host=host, port=port, **kwargs)[db]
 
         if user is not None and password is not None:
             _db.authenticate(user, password)
 
-        self.fs = gridfs.GridFS(_db)
-        self.fs_files = _db["fs.files"]   # the underlying collection GridFS uses
+        self.fs = gridfs.GridFS(_db, bucket)
+
+        self.fs_files = _db[bucket + ".files"]   # the underlying collection GridFS uses
 
     def save(self, content):
         id = content.get_id()
@@ -43,7 +43,7 @@ class MongoContentStore(ContentStore):
         if self.fs.exists({"_id": id}):
             self.fs.delete(id)
 
-    def find(self, location):
+    def find(self, location, throw_on_not_found=True):
         id = StaticContent.get_id_from_location(location)
         try:
             with self.fs.get(id) as fp:
@@ -52,7 +52,10 @@ class MongoContentStore(ContentStore):
                                      thumbnail_location=fp.thumbnail_location if hasattr(fp, 'thumbnail_location') else None,
                                      import_path=fp.import_path if hasattr(fp, 'import_path') else None)
         except NoFile:
-            raise NotFoundError()
+            if throw_on_not_found:
+                raise NotFoundError()
+            else:
+                return None
 
     def export(self, location, output_directory):
         content = self.find(location)

common/lib/xmodule/xmodule/contentstore/utils.py

+from xmodule.modulestore import Location
+from xmodule.contentstore.content import StaticContent
+from .django import contentstore
+
+
+def empty_asset_trashcan(course_locs):
+    '''
+    This method will hard delete all assets (optionally within a course_id) from the trashcan
+    '''
+    store = contentstore('trashcan')
+
+    for course_loc in course_locs:
+        # first delete all of the thumbnails
+        thumbs = store.get_all_content_thumbnails_for_course(course_loc)
+        for thumb in thumbs:
+            thumb_loc = Location(thumb["_id"])
+            id = StaticContent.get_id_from_location(thumb_loc)
+            print "Deleting {0}...".format(id)
+            store.delete(id)
+
+        # then delete all of the assets
+        assets = store.get_all_content_for_course(course_loc)
+        for asset in assets:
+            asset_loc = Location(asset["_id"])
+            id = StaticContent.get_id_from_location(asset_loc)
+            print "Deleting {0}...".format(id)
+            store.delete(id)
+
+
+def restore_asset_from_trashcan(location):
+    '''
+    This method will restore an asset which got soft deleted and put back in the original course
+    '''
+    trash = contentstore('trashcan')
+    store = contentstore()
+
+    loc = StaticContent.get_location_from_path(location)
+    content = trash.find(loc)
+
+    # ok, save the content into the courseware
+    store.save(content)
+
+    # see if there is a thumbnail as well, if so move that as well
+    if content.thumbnail_location is not None:
+        try:
+            thumbnail_content = trash.find(content.thumbnail_location)
+            store.save(thumbnail_content)
+        except:
+            pass  # OK if this is left dangling

common/lib/xmodule/xmodule/css/capa/display.scss

@@ -551,10 +551,24 @@ section.problem {
   section.action {
     margin-top: 20px;
 
-    input.save {
+    .save, .check, .show {
+      height: ($baseline*2);
+      font-weight: 600;
+      vertical-align: middle;
+    }
+
+    .save {
       @extend .blue-button;
     }
 
+    .show {
+
+      .show-label {
+        font-size: 1.0em;
+        font-weight: 600;
+      }
+    }
+
     .submission_feedback {
       // background: #F3F3F3;
       // border: 1px solid #ddd;
@@ -811,13 +825,13 @@ section.problem {
       }
       .selected-grade {
         background: #666;
-        color: white; 
+        color: white;
       }
       input[type=radio]:checked + label {
         background: #666;
         color: white; }
       input[class='score-selection'] {
-        display: none; 
+        display: none;
       }
   }
 
@@ -878,11 +892,11 @@ section.problem {
         .tag-status, .tag { padding: .25em .5em; }
       }
     }
-    textarea.comment { 
+    textarea.comment {
       $num-lines-to-show: 5;
       $line-height: 1.4em;
       $padding: .2em;
-      width: 100%; 
+      width: 100%;
       padding: $padding (2 * $padding);
       line-height: $line-height;
       height: ($num-lines-to-show * $line-height) + (2*$padding) - (($line-height - 1)/2);

common/lib/xmodule/xmodule/error_module.py

@@ -87,18 +87,18 @@ class ErrorDescriptor(ErrorFields, JSONEditingDescriptor):
                 # but url_names aren't guaranteed to be unique between descriptor types,
                 # and ErrorDescriptor can wrap any type.  When the wrapped module is fixed,
                 # it will be written out with the original url_name.
-                name=hashlib.sha1(contents).hexdigest()
+                name=hashlib.sha1(contents.encode('utf8')).hexdigest()
             )
 
         # real metadata stays in the content, but add a display name
         model_data = {
             'error_msg': str(error_msg),
             'contents': contents,
-            'display_name': 'Error: ' + location.name
+            'display_name': 'Error: ' + location.name,
+            'location': location,
         }
         return cls(
             system,
-            location,
             model_data,
         )
 

common/lib/xmodule/xmodule/exceptions.py

@@ -12,3 +12,12 @@ class ProcessingError(Exception):
     For example: if an exception occurs while checking a capa problem.
     '''
     pass
+
+class InvalidVersionError(Exception):
+    """
+    Tried to save an item with a location that a store cannot support (e.g., draft version
+    for a non-leaf node)
+    """
+    def __init__(self, location):
+        super(InvalidVersionError, self).__init__()
+        self.location = location

common/lib/xmodule/xmodule/fields.py

@@ -2,20 +2,41 @@ import time
 import logging
 import re
 
-from datetime import timedelta
 from xblock.core import ModelType
 import datetime
 import dateutil.parser
 
-from xblock.core import Integer, Float, Boolean
+from pytz import UTC
 
 log = logging.getLogger(__name__)
 
 
 class Date(ModelType):
     '''
-    Date fields know how to parse and produce json (iso) compatible formats.
+    Date fields know how to parse and produce json (iso) compatible formats. Converts to tz aware datetimes.
     '''
+    # See note below about not defaulting these
+    CURRENT_YEAR = datetime.datetime.now(UTC).year
+    PREVENT_DEFAULT_DAY_MON_SEED1 = datetime.datetime(CURRENT_YEAR, 1, 1, tzinfo=UTC)
+    PREVENT_DEFAULT_DAY_MON_SEED2 = datetime.datetime(CURRENT_YEAR, 2, 2, tzinfo=UTC)
+
+    def _parse_date_wo_default_month_day(self, field):
+        """
+        Parse the field as an iso string but prevent dateutils from defaulting the day or month while
+        allowing it to default the other fields.
+        """
+        # It's not trivial to replace dateutil b/c parsing timezones as Z, +03:30, -400 is hard in python
+        # however, we don't want dateutil to default the month or day (but some tests at least expect
+        # us to default year); so, we'll see if dateutil uses the defaults for these the hard way
+        result = dateutil.parser.parse(field, default=self.PREVENT_DEFAULT_DAY_MON_SEED1)
+        result_other = dateutil.parser.parse(field, default=self.PREVENT_DEFAULT_DAY_MON_SEED2)
+        if result != result_other:
+            log.warning("Field {0} is missing month or day".format(self._name, field))
+            return None
+        if result.tzinfo is None:
+            result = result.replace(tzinfo=UTC)
+        return result
+
     def from_json(self, field):
         """
         Parse an optional metadata key containing a time: if present, complain
@@ -27,11 +48,12 @@ class Date(ModelType):
         elif field is "":
             return None
         elif isinstance(field, basestring):
-            d = dateutil.parser.parse(field)
-            return d.utctimetuple()
+            return self._parse_date_wo_default_month_day(field)
         elif isinstance(field, (int, long, float)):
-            return time.gmtime(field / 1000)
+            return datetime.datetime.fromtimestamp(field / 1000, UTC)
         elif isinstance(field, time.struct_time):
+            return datetime.datetime.fromtimestamp(time.mktime(field), UTC)
+        elif isinstance(field, datetime.datetime):
             return field
         else:
             msg = "Field {0} has bad value '{1}'".format(
@@ -49,7 +71,11 @@ class Date(ModelType):
             # struct_times are always utc
             return time.strftime('%Y-%m-%dT%H:%M:%SZ', value)
         elif isinstance(value, datetime.datetime):
-            return value.isoformat() + 'Z'
+            if value.tzinfo is None or value.utcoffset().total_seconds() == 0:
+                # isoformat adds +00:00 rather than Z
+                return value.strftime('%Y-%m-%dT%H:%M:%SZ')
+            else:
+                return value.isoformat()
 
 
 TIMEDELTA_REGEX = re.compile(r'^((?P<days>\d+?) day(?:s?))?(\s)?((?P<hours>\d+?) hour(?:s?))?(\s)?((?P<minutes>\d+?) minute(?:s)?)?(\s)?((?P<seconds>\d+?) second(?:s)?)?$')
@@ -66,6 +92,8 @@ class Timedelta(ModelType):
 
         Returns a datetime.timedelta parsed from the string
         """
+        if time_str is None:
+            return None
         parts = TIMEDELTA_REGEX.match(time_str)
         if not parts:
             return
@@ -74,7 +102,7 @@ class Timedelta(ModelType):
         for (name, param) in parts.iteritems():
             if param:
                 time_params[name] = int(param)
-        return timedelta(**time_params)
+        return datetime.timedelta(**time_params)
 
     def to_json(self, value):
         values = []
@@ -83,42 +111,3 @@ class Timedelta(ModelType):
             if cur_value > 0:
                 values.append("%d %s" % (cur_value, attr))
         return ' '.join(values)
-
-
-class StringyInteger(Integer):
-    """
-    A model type that converts from strings to integers when reading from json.
-    If value does not parse as an int, returns None.
-    """
-    def from_json(self, value):
-        try:
-            return int(value)
-        except:
-            return None
-
-
-class StringyFloat(Float):
-    """
-    A model type that converts from string to floats when reading from json.
-    If value does not parse as a float, returns None.
-    """
-    def from_json(self, value):
-        try:
-            return float(value)
-        except:
-            return None
-
-
-class StringyBoolean(Boolean):
-    """
-    Reads strings from JSON as booleans.
-
-    If the string is 'true' (case insensitive), then return True,
-    otherwise False.
-
-    JSON values that aren't strings are returned as-is.
-    """
-    def from_json(self, value):
-        if isinstance(value, basestring):
-            return value.lower() == 'true'
-        return value

common/lib/xmodule/xmodule/foldit_module.py

@@ -8,7 +8,6 @@ from xmodule.x_module import XModule
 from xmodule.xml_module import XmlDescriptor
 from xblock.core import Scope, Integer, String
 from .fields import Date
-from xmodule.util.date_utils import time_to_datetime
 
 
 log = logging.getLogger(__name__)
@@ -31,9 +30,7 @@ class FolditModule(FolditFields, XModule):
     css = {'scss': [resource_string(__name__, 'css/foldit/leaderboard.scss')]}
 
     def __init__(self, *args, **kwargs):
-        XModule.__init__(self, *args, **kwargs)
         """
-
         Example:
          <foldit show_basic_score="true"
             required_level="4"
@@ -42,8 +39,8 @@ class FolditModule(FolditFields, XModule):
             required_sublevel_half_credit="3"
             show_leaderboard="false"/>
         """
-
-        self.due_time = time_to_datetime(self.due)
+        XModule.__init__(self, *args, **kwargs)
+        self.due_time = self.due
 
     def is_complete(self):
         """
@@ -102,7 +99,7 @@ class FolditModule(FolditFields, XModule):
         from foldit.models import Score
 
         leaders = [(e['username'], e['score']) for e in Score.get_tops_n(10)]
-        leaders.sort(key=lambda x: -x[1])
+        leaders.sort(key=lambda x:-x[1])
 
         return leaders
 
@@ -186,7 +183,6 @@ class FolditDescriptor(FolditFields, XmlDescriptor, EditingDescriptor):
     module_class = FolditModule
     filename_extension = "xml"
 
-    stores_state = True
     has_score = True
     template_dir_name = "foldit"
 

common/lib/xmodule/xmodule/js/fixtures/problem_content.html

@@ -13,7 +13,7 @@
     <input class="check" type="button" value="Check">
     <input class="reset" type="button" value="Reset">
     <input class="save" type="button" value="Save">
-    <input class="show" type="button" value="Show Answer">
+    <button class="show"><span class="show-label">Show Answer(s)</span> <span class="sr">(for question(s) above - adjacent to each field)</span></button>
     <a href="/courseware/6.002_Spring_2012/${ explain }" class="new-page">Explanation</a>
     <section class="submission_feedback"></section>
   </section>

common/lib/xmodule/xmodule/js/fixtures/videoalpha.html

+<div class="course-content">
+  <div id="video_example">
+    <div id="example">
+      <div
+        id="video_id"
+        class="video"
+        data-streams="0.75:slowerSpeedYoutubeId,1.0:normalSpeedYoutubeId"
+        data-show-captions="true"
+        data-start=""
+        data-end=""
+        data-caption-asset-path="/static/subs/">
+        <div class="tc-wrapper">
+          <article class="video-wrapper">
+            <section class="video-player">
+              <div id="id"></div>
+            </section>
+            <section class="video-controls"></section>
+          </article>
+        </div>
+      </div>
+    </div>
+  </div>
+</div>
\ No newline at end of file

common/lib/xmodule/xmodule/js/fixtures/videoalpha_html5.html

+<div class="course-content">
+  <div id="video_example">
+    <div id="example">
+      <div
+        id="video_id"
+        class="video"
+        data-show-captions="true"
+        data-start=""
+        data-end=""
+        data-caption-asset-path="/static/subs/"
+        data-sub="test_name_of_the_subtitles"
+        data-mp4-source="test.mp4"
+        data-webm-source="test.webm"
+        data-ogg-source="test.ogv"
+        >
+        <div class="tc-wrapper">
+          <article class="video-wrapper">
+            <section class="video-player">
+              <div id="id"></div>
+            </section>
+            <section class="video-controls"></section>
+          </article>
+        </div>
+      </div>
+    </div>
+  </div>
+</div>
\ No newline at end of file