Commit df0e1c49 by Ned Batchelder

Merge pull request #1768 from edx/ned/merge-master-to-rc-2013-11-21

Merge rc/2013-11-21 to master
parents ceb15209 368c5758
...@@ -5,6 +5,17 @@ These are notable changes in edx-platform. This is a rolling list of changes, ...@@ -5,6 +5,17 @@ These are notable changes in edx-platform. This is a rolling list of changes,
in roughly chronological order, most recent first. Add your entries at or near in roughly chronological order, most recent first. Add your entries at or near
the top. Include a label indicating the component affected. the top. Include a label indicating the component affected.
LMS: Add feature for providing background grade report generation via Celery
instructor task, with reports uploaded to S3. Feature is visible on the beta
instructor dashboard. LMS-58
LMS: Beta-tester status is now set on a per-course-run basis, rather than being
valid across all runs with the same course name. Old group membership will
still work across runs, but new beta-testers will only be added to a single
course run.
Blades: Enabled several Video Jasmine tests. BLD-463.
Studio: Continued modification of Studio pages to follow a RESTful framework. Studio: Continued modification of Studio pages to follow a RESTful framework.
includes Settings pages, edit page for Subsection and Unit, and interfaces includes Settings pages, edit page for Subsection and Unit, and interfaces
for updating xblocks (xmodules) and getting their editing HTML. for updating xblocks (xmodules) and getting their editing HTML.
...@@ -19,7 +30,8 @@ Blades: Fix transcripts 500 error in studio (BLD-530) ...@@ -19,7 +30,8 @@ Blades: Fix transcripts 500 error in studio (BLD-530)
LMS: Add error recovery when a user loads or switches pages in an LMS: Add error recovery when a user loads or switches pages in an
inline discussion. inline discussion.
Blades: Allow multiple strings as the correct answer to a string response question. BLD-474. Blades: Allow multiple strings as the correct answer to a string response
question. BLD-474.
Blades: a11y - Videos will alert screenreaders when the video is over. Blades: a11y - Videos will alert screenreaders when the video is over.
...@@ -27,6 +39,7 @@ LMS: Trap focus on the loading element when a user loads more threads ...@@ -27,6 +39,7 @@ LMS: Trap focus on the loading element when a user loads more threads
in the forum sidebar to improve accessibility. in the forum sidebar to improve accessibility.
LMS: Add error recovery when a user loads more threads in the forum sidebar. LMS: Add error recovery when a user loads more threads in the forum sidebar.
>>>>>>> origin/master
LMS: Add a user-visible alert modal when a forums AJAX request fails. LMS: Add a user-visible alert modal when a forums AJAX request fails.
...@@ -47,7 +60,8 @@ text like with bold or italics. (BLD-449) ...@@ -47,7 +60,8 @@ text like with bold or italics. (BLD-449)
LMS: Beta instructor dashboard will only count actively enrolled students for LMS: Beta instructor dashboard will only count actively enrolled students for
course enrollment numbers. course enrollment numbers.
Blades: Fix speed menu that is not rendered correctly when YouTube is unavailable. (BLD-457). Blades: Fix speed menu that is not rendered correctly when YouTube is
unavailable. (BLD-457).
LMS: Users with is_staff=True no longer have the STAFF label appear on LMS: Users with is_staff=True no longer have the STAFF label appear on
their forum posts. their forum posts.
......
...@@ -769,7 +769,7 @@ class CourseEnrollment(models.Model): ...@@ -769,7 +769,7 @@ class CourseEnrollment(models.Model):
activation_changed = True activation_changed = True
mode_changed = False mode_changed = False
# if mode is None, the call to update_enrollment didn't specify a new # if mode is None, the call to update_enrollment didn't specify a new
# mode, so leave as-is # mode, so leave as-is
if self.mode != mode and mode is not None: if self.mode != mode and mode is not None:
self.mode = mode self.mode = mode
...@@ -967,6 +967,14 @@ class CourseEnrollment(models.Model): ...@@ -967,6 +967,14 @@ class CourseEnrollment(models.Model):
def enrollments_for_user(cls, user): def enrollments_for_user(cls, user):
return CourseEnrollment.objects.filter(user=user, is_active=1) return CourseEnrollment.objects.filter(user=user, is_active=1)
@classmethod
def users_enrolled_in(cls, course_id):
"""Return a queryset of User for every user enrolled in the course."""
return User.objects.filter(
courseenrollment__course_id=course_id,
courseenrollment__is_active=True
)
def activate(self): def activate(self):
"""Makes this `CourseEnrollment` record active. Saves immediately.""" """Makes this `CourseEnrollment` record active. Saves immediately."""
self.update_enrollment(is_active=True) self.update_enrollment(is_active=True)
......
...@@ -45,7 +45,7 @@ class ABTestModule(ABTestFields, XModule): ...@@ -45,7 +45,7 @@ class ABTestModule(ABTestFields, XModule):
""" """
def __init__(self, *args, **kwargs): def __init__(self, *args, **kwargs):
XModule.__init__(self, *args, **kwargs) super(ABTestModule, self).__init__(*args, **kwargs)
if self.group is None: if self.group is None:
self.group = group_from_value( self.group = group_from_value(
......
...@@ -50,7 +50,7 @@ class AnnotatableModule(AnnotatableFields, XModule): ...@@ -50,7 +50,7 @@ class AnnotatableModule(AnnotatableFields, XModule):
icon_class = 'annotatable' icon_class = 'annotatable'
def __init__(self, *args, **kwargs): def __init__(self, *args, **kwargs):
XModule.__init__(self, *args, **kwargs) super(AnnotatableModule, self).__init__(*args, **kwargs)
xmltree = etree.fromstring(self.data) xmltree = etree.fromstring(self.data)
......
...@@ -190,7 +190,7 @@ class CapaModule(CapaFields, XModule): ...@@ -190,7 +190,7 @@ class CapaModule(CapaFields, XModule):
""" """
Accepts the same arguments as xmodule.x_module:XModule.__init__ Accepts the same arguments as xmodule.x_module:XModule.__init__
""" """
XModule.__init__(self, *args, **kwargs) super(CapaModule, self).__init__(*args, **kwargs)
due_date = self.due due_date = self.due
......
...@@ -412,7 +412,7 @@ class CombinedOpenEndedModule(CombinedOpenEndedFields, XModule): ...@@ -412,7 +412,7 @@ class CombinedOpenEndedModule(CombinedOpenEndedFields, XModule):
See DEFAULT_DATA for a sample. See DEFAULT_DATA for a sample.
""" """
XModule.__init__(self, *args, **kwargs) super(CombinedOpenEndedModule, self).__init__(*args, **kwargs)
self.system.set('location', self.location) self.system.set('location', self.location)
......
...@@ -75,7 +75,7 @@ class CrowdsourceHinterModule(CrowdsourceHinterFields, XModule): ...@@ -75,7 +75,7 @@ class CrowdsourceHinterModule(CrowdsourceHinterFields, XModule):
js_module_name = "Hinter" js_module_name = "Hinter"
def __init__(self, *args, **kwargs): def __init__(self, *args, **kwargs):
XModule.__init__(self, *args, **kwargs) super(CrowdsourceHinterModule, self).__init__(*args, **kwargs)
# We need to know whether we are working with a FormulaResponse problem. # We need to know whether we are working with a FormulaResponse problem.
try: try:
responder = self.get_display_items()[0].lcp.responders.values()[0] responder = self.get_display_items()[0].lcp.responders.values()[0]
......
...@@ -39,7 +39,7 @@ class FolditModule(FolditFields, XModule): ...@@ -39,7 +39,7 @@ class FolditModule(FolditFields, XModule):
required_sublevel_half_credit="3" required_sublevel_half_credit="3"
show_leaderboard="false"/> show_leaderboard="false"/>
""" """
XModule.__init__(self, *args, **kwargs) super(FolditModule, self).__init__(*args, **kwargs)
self.due_time = self.due self.due_time = self.due
def is_complete(self): def is_complete(self):
......
...@@ -93,7 +93,6 @@ class CombinedOpenEndedV1Module(): ...@@ -93,7 +93,6 @@ class CombinedOpenEndedV1Module():
Definition file should have one or many task blocks, a rubric block, and a prompt block. See DEFAULT_DATA in combined_open_ended_module for a sample. Definition file should have one or many task blocks, a rubric block, and a prompt block. See DEFAULT_DATA in combined_open_ended_module for a sample.
""" """
self.instance_state = instance_state self.instance_state = instance_state
self.display_name = instance_state.get('display_name', "Open Ended") self.display_name = instance_state.get('display_name', "Open Ended")
......
...@@ -39,7 +39,7 @@ class RandomizeModule(RandomizeFields, XModule): ...@@ -39,7 +39,7 @@ class RandomizeModule(RandomizeFields, XModule):
modules. modules.
""" """
def __init__(self, *args, **kwargs): def __init__(self, *args, **kwargs):
XModule.__init__(self, *args, **kwargs) super(RandomizeModule, self).__init__(*args, **kwargs)
# NOTE: calling self.get_children() creates a circular reference-- # NOTE: calling self.get_children() creates a circular reference--
# it calls get_child_descriptors() internally, but that doesn't work until # it calls get_child_descriptors() internally, but that doesn't work until
......
...@@ -38,7 +38,7 @@ class SequenceModule(SequenceFields, XModule): ...@@ -38,7 +38,7 @@ class SequenceModule(SequenceFields, XModule):
def __init__(self, *args, **kwargs): def __init__(self, *args, **kwargs):
XModule.__init__(self, *args, **kwargs) super(SequenceModule, self).__init__(*args, **kwargs)
# if position is specified in system, then use that instead # if position is specified in system, then use that instead
if getattr(self.system, 'position', None) is not None: if getattr(self.system, 'position', None) is not None:
......
...@@ -143,7 +143,6 @@ class CapaFactory(object): ...@@ -143,7 +143,6 @@ class CapaFactory(object):
DictFieldData(field_data), DictFieldData(field_data),
ScopeIds(None, None, location, location), ScopeIds(None, None, location, location),
) )
system.xmodule_instance = module
if correct: if correct:
# TODO: probably better to actually set the internal state properly, but... # TODO: probably better to actually set the internal state properly, but...
......
...@@ -143,7 +143,6 @@ class CHModuleFactory(object): ...@@ -143,7 +143,6 @@ class CHModuleFactory(object):
return capa_module return capa_module
system.get_module = fake_get_module system.get_module = fake_get_module
module = CrowdsourceHinterModule(descriptor, system, DictFieldData(field_data), Mock()) module = CrowdsourceHinterModule(descriptor, system, DictFieldData(field_data), Mock())
system.xmodule_instance = module
return module return module
......
...@@ -3,10 +3,13 @@ Tests for ErrorModule and NonStaffErrorModule ...@@ -3,10 +3,13 @@ Tests for ErrorModule and NonStaffErrorModule
""" """
import unittest import unittest
from xmodule.tests import get_test_system from xmodule.tests import get_test_system
import xmodule.error_module as error_module from xmodule.error_module import ErrorDescriptor, ErrorModule, NonStaffErrorDescriptor
from xmodule.modulestore import Location from xmodule.modulestore import Location
from xmodule.x_module import XModuleDescriptor from xmodule.x_module import XModuleDescriptor, XModule
from mock import MagicMock from mock import MagicMock, Mock, patch
from xblock.runtime import Runtime, UsageStore
from xblock.field_data import FieldData
from xblock.fields import ScopeIds
class SetupTestErrorModules(): class SetupTestErrorModules():
...@@ -27,9 +30,9 @@ class TestErrorModule(unittest.TestCase, SetupTestErrorModules): ...@@ -27,9 +30,9 @@ class TestErrorModule(unittest.TestCase, SetupTestErrorModules):
SetupTestErrorModules.setUp(self) SetupTestErrorModules.setUp(self)
def test_error_module_xml_rendering(self): def test_error_module_xml_rendering(self):
descriptor = error_module.ErrorDescriptor.from_xml( descriptor = ErrorDescriptor.from_xml(
self.valid_xml, self.system, self.org, self.course, self.error_msg) self.valid_xml, self.system, self.org, self.course, self.error_msg)
self.assertIsInstance(descriptor, error_module.ErrorDescriptor) self.assertIsInstance(descriptor, ErrorDescriptor)
descriptor.xmodule_runtime = self.system descriptor.xmodule_runtime = self.system
context_repr = self.system.render(descriptor, 'student_view').content context_repr = self.system.render(descriptor, 'student_view').content
self.assertIn(self.error_msg, context_repr) self.assertIn(self.error_msg, context_repr)
...@@ -41,9 +44,9 @@ class TestErrorModule(unittest.TestCase, SetupTestErrorModules): ...@@ -41,9 +44,9 @@ class TestErrorModule(unittest.TestCase, SetupTestErrorModules):
location=self.location, location=self.location,
_field_data=self.valid_xml) _field_data=self.valid_xml)
error_descriptor = error_module.ErrorDescriptor.from_descriptor( error_descriptor = ErrorDescriptor.from_descriptor(
descriptor, self.error_msg) descriptor, self.error_msg)
self.assertIsInstance(error_descriptor, error_module.ErrorDescriptor) self.assertIsInstance(error_descriptor, ErrorDescriptor)
error_descriptor.xmodule_runtime = self.system error_descriptor.xmodule_runtime = self.system
context_repr = self.system.render(error_descriptor, 'student_view').content context_repr = self.system.render(error_descriptor, 'student_view').content
self.assertIn(self.error_msg, context_repr) self.assertIn(self.error_msg, context_repr)
...@@ -58,12 +61,12 @@ class TestNonStaffErrorModule(unittest.TestCase, SetupTestErrorModules): ...@@ -58,12 +61,12 @@ class TestNonStaffErrorModule(unittest.TestCase, SetupTestErrorModules):
SetupTestErrorModules.setUp(self) SetupTestErrorModules.setUp(self)
def test_non_staff_error_module_create(self): def test_non_staff_error_module_create(self):
descriptor = error_module.NonStaffErrorDescriptor.from_xml( descriptor = NonStaffErrorDescriptor.from_xml(
self.valid_xml, self.system, self.org, self.course) self.valid_xml, self.system, self.org, self.course)
self.assertIsInstance(descriptor, error_module.NonStaffErrorDescriptor) self.assertIsInstance(descriptor, NonStaffErrorDescriptor)
def test_from_xml_render(self): def test_from_xml_render(self):
descriptor = error_module.NonStaffErrorDescriptor.from_xml( descriptor = NonStaffErrorDescriptor.from_xml(
self.valid_xml, self.system, self.org, self.course) self.valid_xml, self.system, self.org, self.course)
descriptor.xmodule_runtime = self.system descriptor.xmodule_runtime = self.system
context_repr = self.system.render(descriptor, 'student_view').content context_repr = self.system.render(descriptor, 'student_view').content
...@@ -76,10 +79,66 @@ class TestNonStaffErrorModule(unittest.TestCase, SetupTestErrorModules): ...@@ -76,10 +79,66 @@ class TestNonStaffErrorModule(unittest.TestCase, SetupTestErrorModules):
location=self.location, location=self.location,
_field_data=self.valid_xml) _field_data=self.valid_xml)
error_descriptor = error_module.NonStaffErrorDescriptor.from_descriptor( error_descriptor = NonStaffErrorDescriptor.from_descriptor(
descriptor, self.error_msg) descriptor, self.error_msg)
self.assertIsInstance(error_descriptor, error_module.ErrorDescriptor) self.assertIsInstance(error_descriptor, ErrorDescriptor)
error_descriptor.xmodule_runtime = self.system error_descriptor.xmodule_runtime = self.system
context_repr = self.system.render(error_descriptor, 'student_view').content context_repr = self.system.render(error_descriptor, 'student_view').content
self.assertNotIn(self.error_msg, context_repr) self.assertNotIn(self.error_msg, context_repr)
self.assertNotIn(str(descriptor), context_repr) self.assertNotIn(str(descriptor), context_repr)
class BrokenModule(XModule):
def __init__(self, *args, **kwargs):
super(BrokenModule, self).__init__(*args, **kwargs)
raise Exception("This is a broken xmodule")
class BrokenDescriptor(XModuleDescriptor):
module_class = BrokenModule
class TestException(Exception):
"""An exception type to use to verify raises in tests"""
pass
class TestErrorModuleConstruction(unittest.TestCase):
"""
Test that error module construction happens correctly
"""
def setUp(self):
field_data = Mock(spec=FieldData)
self.descriptor = BrokenDescriptor(
Runtime(Mock(spec=UsageStore), field_data),
field_data,
ScopeIds(None, None, None, 'i4x://org/course/broken/name')
)
self.descriptor.xmodule_runtime = Runtime(Mock(spec=UsageStore), field_data)
self.descriptor.xmodule_runtime.error_descriptor_class = ErrorDescriptor
self.descriptor.xmodule_runtime.xmodule_instance = None
def test_broken_module(self):
"""
Test that when an XModule throws an error during __init__, we
get an ErrorModule back from XModuleDescriptor._xmodule
"""
module = self.descriptor._xmodule
self.assertIsInstance(module, ErrorModule)
@patch.object(ErrorDescriptor, '__init__', Mock(side_effect=TestException))
def test_broken_error_descriptor(self):
"""
Test that a broken error descriptor doesn't cause an infinite loop
"""
with self.assertRaises(TestException):
module = self.descriptor._xmodule
@patch.object(ErrorModule, '__init__', Mock(side_effect=TestException))
def test_broken_error_module(self):
"""
Test that a broken error module doesn't cause an infinite loop
"""
with self.assertRaises(TestException):
module = self.descriptor._xmodule
...@@ -41,13 +41,16 @@ class PeerGradingModuleTest(unittest.TestCase, DummyModulestore): ...@@ -41,13 +41,16 @@ class PeerGradingModuleTest(unittest.TestCase, DummyModulestore):
}) })
save_dict.extend(('rubric_scores[]', val) for val in (0, 1)) save_dict.extend(('rubric_scores[]', val) for val in (0, 1))
def get_module_system(self, descriptor):
test_system = get_test_system()
test_system.open_ended_grading_interface = None
return test_system
def setUp(self): def setUp(self):
""" """
Create a peer grading module from a test system Create a peer grading module from a test system
@return: @return:
""" """
self.test_system = get_test_system()
self.test_system.open_ended_grading_interface = None
self.setup_modulestore(COURSE) self.setup_modulestore(COURSE)
self.peer_grading = self.get_module_from_location(self.problem_location, COURSE) self.peer_grading = self.get_module_from_location(self.problem_location, COURSE)
self.coe = self.get_module_from_location(self.coe_location, COURSE) self.coe = self.get_module_from_location(self.coe_location, COURSE)
...@@ -173,13 +176,16 @@ class PeerGradingModuleScoredTest(unittest.TestCase, DummyModulestore): ...@@ -173,13 +176,16 @@ class PeerGradingModuleScoredTest(unittest.TestCase, DummyModulestore):
["i4x", "edX", "open_ended", "peergrading", "PeerGradingScored"] ["i4x", "edX", "open_ended", "peergrading", "PeerGradingScored"]
) )
def get_module_system(self, descriptor):
test_system = get_test_system()
test_system.open_ended_grading_interface = None
return test_system
def setUp(self): def setUp(self):
""" """
Create a peer grading module from a test system Create a peer grading module from a test system
@return: @return:
""" """
self.test_system = get_test_system()
self.test_system.open_ended_grading_interface = None
self.setup_modulestore(COURSE) self.setup_modulestore(COURSE)
def test_metadata_load(self): def test_metadata_load(self):
...@@ -213,12 +219,15 @@ class PeerGradingModuleLinkedTest(unittest.TestCase, DummyModulestore): ...@@ -213,12 +219,15 @@ class PeerGradingModuleLinkedTest(unittest.TestCase, DummyModulestore):
coe_location = Location(["i4x", "edX", "open_ended", "combinedopenended", coe_location = Location(["i4x", "edX", "open_ended", "combinedopenended",
"SampleQuestion"]) "SampleQuestion"])
def get_module_system(self, descriptor):
test_system = get_test_system()
test_system.open_ended_grading_interface = None
return test_system
def setUp(self): def setUp(self):
""" """
Create a peer grading module from a test system. Create a peer grading module from a test system.
""" """
self.test_system = get_test_system()
self.test_system.open_ended_grading_interface = None
self.setup_modulestore(COURSE) self.setup_modulestore(COURSE)
@property @property
...@@ -270,14 +279,15 @@ class PeerGradingModuleLinkedTest(unittest.TestCase, DummyModulestore): ...@@ -270,14 +279,15 @@ class PeerGradingModuleLinkedTest(unittest.TestCase, DummyModulestore):
else: else:
pg_descriptor.get_required_module_descriptors = lambda: [] pg_descriptor.get_required_module_descriptors = lambda: []
test_system = self.get_module_system(pg_descriptor)
# Initialize the peer grading module. # Initialize the peer grading module.
peer_grading = PeerGradingModule( peer_grading = PeerGradingModule(
pg_descriptor, pg_descriptor,
self.test_system, test_system,
self.field_data, self.field_data,
self.scope_ids, self.scope_ids,
) )
self.test_system.xmodule_instance = peer_grading
return peer_grading return peer_grading
...@@ -384,13 +394,16 @@ class PeerGradingModuleTrackChangesTest(unittest.TestCase, DummyModulestore): ...@@ -384,13 +394,16 @@ class PeerGradingModuleTrackChangesTest(unittest.TestCase, DummyModulestore):
mock_track_changes_problem = Mock(side_effect=[MockedTrackChangesProblem()]) mock_track_changes_problem = Mock(side_effect=[MockedTrackChangesProblem()])
pgm_location = Location(["i4x", "edX", "open_ended", "peergrading", "PeerGradingSample"]) pgm_location = Location(["i4x", "edX", "open_ended", "peergrading", "PeerGradingSample"])
def get_module_system(self, descriptor):
test_system = get_test_system()
test_system.open_ended_grading_interface = None
return test_system
def setUp(self): def setUp(self):
""" """
Create a peer grading module from a test system Create a peer grading module from a test system
@return: @return:
""" """
self.test_system = get_test_system()
self.test_system.open_ended_grading_interface = None
self.setup_modulestore(COURSE) self.setup_modulestore(COURSE)
self.peer_grading = self.get_module_from_location(self.pgm_location, COURSE) self.peer_grading = self.get_module_from_location(self.pgm_location, COURSE)
......
...@@ -78,7 +78,9 @@ class DummyModulestore(object): ...@@ -78,7 +78,9 @@ class DummyModulestore(object):
""" """
A mixin that allows test classes to have convenience functions to get a module given a location A mixin that allows test classes to have convenience functions to get a module given a location
""" """
get_test_system = get_test_system()
def get_module_system(self, descriptor):
raise NotImplementedError("Sub-tests must specify how to generate a module-system")
def setup_modulestore(self, name): def setup_modulestore(self, name):
self.modulestore = XMLModuleStore(DATA_DIR, course_dirs=[name]) self.modulestore = XMLModuleStore(DATA_DIR, course_dirs=[name])
...@@ -93,7 +95,7 @@ class DummyModulestore(object): ...@@ -93,7 +95,7 @@ class DummyModulestore(object):
if not isinstance(location, Location): if not isinstance(location, Location):
location = Location(location) location = Location(location)
descriptor = self.modulestore.get_instance(course.id, location, depth=None) descriptor = self.modulestore.get_instance(course.id, location, depth=None)
descriptor.xmodule_runtime = self.test_system descriptor.xmodule_runtime = self.get_module_system(descriptor)
return descriptor return descriptor
# Task state for a module with self assessment then instructor assessment. # Task state for a module with self assessment then instructor assessment.
......
...@@ -31,9 +31,6 @@ class TimeLimitModule(TimeLimitFields, XModule): ...@@ -31,9 +31,6 @@ class TimeLimitModule(TimeLimitFields, XModule):
Wrapper module which imposes a time constraint for the completion of its child. Wrapper module which imposes a time constraint for the completion of its child.
''' '''
def __init__(self, *args, **kwargs):
XModule.__init__(self, *args, **kwargs)
# For a timed activity, we are only interested here # For a timed activity, we are only interested here
# in time-related accommodations, and these should be disjoint. # in time-related accommodations, and these should be disjoint.
# (For proctored exams, it is possible to have multiple accommodations # (For proctored exams, it is possible to have multiple accommodations
......
...@@ -16,9 +16,6 @@ class VerticalFields(object): ...@@ -16,9 +16,6 @@ class VerticalFields(object):
class VerticalModule(VerticalFields, XModule): class VerticalModule(VerticalFields, XModule):
''' Layout module for laying out submodules vertically.''' ''' Layout module for laying out submodules vertically.'''
def __init__(self, *args, **kwargs):
XModule.__init__(self, *args, **kwargs)
def student_view(self, context): def student_view(self, context):
fragment = Fragment() fragment = Fragment()
contents = [] contents = []
......
...@@ -393,6 +393,7 @@ class XModule(XModuleMixin, HTMLSnippet, XBlock): # pylint: disable=abstract-me ...@@ -393,6 +393,7 @@ class XModule(XModuleMixin, HTMLSnippet, XBlock): # pylint: disable=abstract-me
super(XModule, self).__init__(*args, **kwargs) super(XModule, self).__init__(*args, **kwargs)
self._loaded_children = None self._loaded_children = None
self.system = self.runtime self.system = self.runtime
self.runtime.xmodule_instance = self
def __unicode__(self): def __unicode__(self):
return u'<x_module(id={0})>'.format(self.id) return u'<x_module(id={0})>'.format(self.id)
...@@ -762,7 +763,7 @@ class XModuleDescriptor(XModuleMixin, HTMLSnippet, ResourceTemplates, XBlock): ...@@ -762,7 +763,7 @@ class XModuleDescriptor(XModuleMixin, HTMLSnippet, ResourceTemplates, XBlock):
assert self.xmodule_runtime.error_descriptor_class is not None assert self.xmodule_runtime.error_descriptor_class is not None
if self.xmodule_runtime.xmodule_instance is None: if self.xmodule_runtime.xmodule_instance is None:
try: try:
self.xmodule_runtime.xmodule_instance = self.xmodule_runtime.construct_xblock_from_class( self.xmodule_runtime.construct_xblock_from_class(
self.module_class, self.module_class,
descriptor=self, descriptor=self,
scope_ids=self.scope_ids, scope_ids=self.scope_ids,
...@@ -770,6 +771,10 @@ class XModuleDescriptor(XModuleMixin, HTMLSnippet, ResourceTemplates, XBlock): ...@@ -770,6 +771,10 @@ class XModuleDescriptor(XModuleMixin, HTMLSnippet, ResourceTemplates, XBlock):
) )
self.xmodule_runtime.xmodule_instance.save() self.xmodule_runtime.xmodule_instance.save()
except Exception: # pylint: disable=broad-except except Exception: # pylint: disable=broad-except
# xmodule_instance is set by the XModule.__init__. If we had an error after that,
# we need to clean it out so that we can set up the ErrorModule instead
self.xmodule_runtime.xmodule_instance = None
if isinstance(self, self.xmodule_runtime.error_descriptor_class): if isinstance(self, self.xmodule_runtime.error_descriptor_class):
log.exception('Error creating an ErrorModule from an ErrorDescriptor') log.exception('Error creating an ErrorModule from an ErrorDescriptor')
raise raise
...@@ -1066,6 +1071,7 @@ class ModuleSystem(ConfigurableFragmentWrapper, Runtime): # pylint: disable=abs ...@@ -1066,6 +1071,7 @@ class ModuleSystem(ConfigurableFragmentWrapper, Runtime): # pylint: disable=abs
""" """
The url prefix to be used by XModules to call into handle_ajax The url prefix to be used by XModules to call into handle_ajax
""" """
assert self.xmodule_instance is not None
return self.handler_url(self.xmodule_instance, 'xmodule_handler', '', '').rstrip('/?') return self.handler_url(self.xmodule_instance, 'xmodule_handler', '', '').rstrip('/?')
......
...@@ -241,7 +241,11 @@ def _has_access_descriptor(user, descriptor, action, course_context=None): ...@@ -241,7 +241,11 @@ def _has_access_descriptor(user, descriptor, action, course_context=None):
# Check start date # Check start date
if descriptor.start is not None: if descriptor.start is not None:
now = datetime.now(UTC()) now = datetime.now(UTC())
effective_start = _adjust_start_date_for_beta_testers(user, descriptor) effective_start = _adjust_start_date_for_beta_testers(
user,
descriptor,
course_context=course_context
)
if now > effective_start: if now > effective_start:
# after start date, everyone can see it # after start date, everyone can see it
debug("Allow: now > effective start date") debug("Allow: now > effective start date")
...@@ -337,7 +341,7 @@ def _dispatch(table, action, user, obj): ...@@ -337,7 +341,7 @@ def _dispatch(table, action, user, obj):
type(obj), action)) type(obj), action))
def _adjust_start_date_for_beta_testers(user, descriptor): def _adjust_start_date_for_beta_testers(user, descriptor, course_context=None):
""" """
If user is in a beta test group, adjust the start date by the appropriate number of If user is in a beta test group, adjust the start date by the appropriate number of
days. days.
...@@ -364,7 +368,7 @@ def _adjust_start_date_for_beta_testers(user, descriptor): ...@@ -364,7 +368,7 @@ def _adjust_start_date_for_beta_testers(user, descriptor):
# bail early if no beta testing is set up # bail early if no beta testing is set up
return descriptor.start return descriptor.start
if CourseBetaTesterRole(descriptor.location).has_user(user): if CourseBetaTesterRole(descriptor.location, course_context=course_context).has_user(user):
debug("Adjust start time: user in beta role for %s", descriptor) debug("Adjust start time: user in beta role for %s", descriptor)
delta = timedelta(descriptor.days_early_for_beta) delta = timedelta(descriptor.days_early_for_beta)
effective = descriptor.start - delta effective = descriptor.start - delta
......
# Compute grades using real division, with no integer truncation # Compute grades using real division, with no integer truncation
from __future__ import division from __future__ import division
from collections import defaultdict
import random import random
import logging import logging
...@@ -9,14 +9,18 @@ from collections import defaultdict ...@@ -9,14 +9,18 @@ from collections import defaultdict
from django.conf import settings from django.conf import settings
from django.contrib.auth.models import User from django.contrib.auth.models import User
from django.db import transaction from django.db import transaction
from django.test.client import RequestFactory
from dogapi import dog_stats_api
from courseware.model_data import FieldDataCache, DjangoKeyValueStore from courseware import courses
from courseware.model_data import FieldDataCache
from xblock.fields import Scope from xblock.fields import Scope
from .module_render import get_module, get_module_for_descriptor
from xmodule import graders from xmodule import graders
from xmodule.capa_module import CapaModule from xmodule.capa_module import CapaModule
from xmodule.graders import Score from xmodule.graders import Score
from .models import StudentModule from .models import StudentModule
from .module_render import get_module, get_module_for_descriptor
log = logging.getLogger("mitx.courseware") log = logging.getLogger("mitx.courseware")
...@@ -331,7 +335,6 @@ def _progress_summary(student, request, course): ...@@ -331,7 +335,6 @@ def _progress_summary(student, request, course):
module_creator = section_module.xmodule_runtime.get_module module_creator = section_module.xmodule_runtime.get_module
for module_descriptor in yield_dynamic_descriptor_descendents(section_module, module_creator): for module_descriptor in yield_dynamic_descriptor_descendents(section_module, module_creator):
course_id = course.id course_id = course.id
(correct, total) = get_score(course_id, student, module_descriptor, module_creator) (correct, total) = get_score(course_id, student, module_descriptor, module_creator)
if correct is None and total is None: if correct is None and total is None:
...@@ -447,3 +450,52 @@ def manual_transaction(): ...@@ -447,3 +450,52 @@ def manual_transaction():
raise raise
else: else:
transaction.commit() transaction.commit()
def iterate_grades_for(course_id, students):
"""Given a course_id and an iterable of students (User), yield a tuple of:
(student, gradeset, err_msg) for every student enrolled in the course.
If an error occured, gradeset will be an empty dict and err_msg will be an
exception message. If there was no error, err_msg is an empty string.
The gradeset is a dictionary with the following fields:
- grade : A final letter grade.
- percent : The final percent for the class (rounded up).
- section_breakdown : A breakdown of each section that makes
up the grade. (For display)
- grade_breakdown : A breakdown of the major components that
make up the final grade. (For display)
- raw_scores: contains scores for every graded module
"""
course = courses.get_course_by_id(course_id)
# We make a fake request because grading code expects to be able to look at
# the request. We have to attach the correct user to the request before
# grading that student.
request = RequestFactory().get('/')
for student in students:
with dog_stats_api.timer('lms.grades.iterate_grades_for', tags=['action:{}'.format(course_id)]):
try:
request.user = student
# Grading calls problem rendering, which calls masquerading,
# which checks session vars -- thus the empty session dict below.
# It's not pretty, but untangling that is currently beyond the
# scope of this feature.
request.session = {}
gradeset = grade(student, request, course)
yield student, gradeset, ""
except Exception as exc: # pylint: disable=broad-except
# Keep marching on even if this student couldn't be graded for
# some reason, but log it for future reference.
log.exception(
'Cannot grade student %s (%s) in course %s because of exception: %s',
student.username,
student.id,
course_id,
exc.message
)
yield student, {}, exc.message
...@@ -187,6 +187,6 @@ class OrgStaffRole(OrgRole): ...@@ -187,6 +187,6 @@ class OrgStaffRole(OrgRole):
class OrgInstructorRole(OrgRole): class OrgInstructorRole(OrgRole):
"""An organization staff member""" """An organization instructor"""
def __init__(self, *args, **kwargs): def __init__(self, *args, **kwargs):
super(OrgInstructorRole, self).__init__('staff', *args, **kwargs) super(OrgInstructorRole, self).__init__('instructor', *args, **kwargs)
...@@ -14,7 +14,14 @@ from student.tests.factories import RegistrationFactory # Imported to re-export ...@@ -14,7 +14,14 @@ from student.tests.factories import RegistrationFactory # Imported to re-export
from student.tests.factories import UserProfileFactory as StudentUserProfileFactory from student.tests.factories import UserProfileFactory as StudentUserProfileFactory
from courseware.models import StudentModule, XModuleUserStateSummaryField from courseware.models import StudentModule, XModuleUserStateSummaryField
from courseware.models import XModuleStudentInfoField, XModuleStudentPrefsField from courseware.models import XModuleStudentInfoField, XModuleStudentPrefsField
from courseware.roles import CourseInstructorRole, CourseStaffRole from courseware.roles import (
CourseInstructorRole,
CourseStaffRole,
CourseBetaTesterRole,
GlobalStaff,
OrgStaffRole,
OrgInstructorRole,
)
from xmodule.modulestore import Location from xmodule.modulestore import Location
...@@ -54,6 +61,59 @@ class StaffFactory(UserFactory): ...@@ -54,6 +61,59 @@ class StaffFactory(UserFactory):
CourseStaffRole(extracted).add_users(self) CourseStaffRole(extracted).add_users(self)
class BetaTesterFactory(UserFactory):
"""
Given a course Location, returns a User object with beta-tester
permissions for `course`.
"""
last_name = "Beta-Tester"
@post_generation
def course(self, create, extracted, **kwargs):
if extracted is None:
raise ValueError("Must specify a course location for a beta-tester user")
CourseBetaTesterRole(extracted).add_users(self)
class OrgStaffFactory(UserFactory):
"""
Given a course Location, returns a User object with org-staff
permissions for `course`.
"""
last_name = "Org-Staff"
@post_generation
def course(self, create, extracted, **kwargs):
if extracted is None:
raise ValueError("Must specify a course location for an org-staff user")
OrgStaffRole(extracted).add_users(self)
class OrgInstructorFactory(UserFactory):
"""
Given a course Location, returns a User object with org-instructor
permissions for `course`.
"""
last_name = "Org-Instructor"
@post_generation
def course(self, create, extracted, **kwargs):
if extracted is None:
raise ValueError("Must specify a course location for an org-instructor user")
OrgInstructorRole(extracted).add_users(self)
class GlobalStaffFactory(UserFactory):
"""
Returns a User object with global staff access
"""
last_name = "GlobalStaff"
@post_generation
def set_staff(self, create, extracted, **kwargs):
GlobalStaff().add_users(self)
class StudentModuleFactory(DjangoModelFactory): class StudentModuleFactory(DjangoModelFactory):
FACTORY_FOR = StudentModule FACTORY_FOR = StudentModule
......
...@@ -29,7 +29,6 @@ from courseware.models import StudentModule ...@@ -29,7 +29,6 @@ from courseware.models import StudentModule
# modules which are mocked in test cases. # modules which are mocked in test cases.
import instructor_task.api import instructor_task.api
from instructor.access import allow_access from instructor.access import allow_access
import instructor.views.api import instructor.views.api
from instructor.views.api import _split_input_list, _msk_from_problem_urlname, common_exceptions_400 from instructor.views.api import _split_input_list, _msk_from_problem_urlname, common_exceptions_400
...@@ -128,6 +127,8 @@ class TestInstructorAPIDenyLevels(ModuleStoreTestCase, LoginEnrollmentTestCase): ...@@ -128,6 +127,8 @@ class TestInstructorAPIDenyLevels(ModuleStoreTestCase, LoginEnrollmentTestCase):
('send_email', {'send_to': 'staff', 'subject': 'test', 'message': 'asdf'}), ('send_email', {'send_to': 'staff', 'subject': 'test', 'message': 'asdf'}),
('list_instructor_tasks', {}), ('list_instructor_tasks', {}),
('list_background_email_tasks', {}), ('list_background_email_tasks', {}),
('list_grade_downloads', {}),
('calculate_grades_csv', {}),
] ]
# Endpoints that only Instructors can access # Endpoints that only Instructors can access
self.instructor_level_endpoints = [ self.instructor_level_endpoints = [
...@@ -790,6 +791,50 @@ class TestInstructorAPILevelsDataDump(ModuleStoreTestCase, LoginEnrollmentTestCa ...@@ -790,6 +791,50 @@ class TestInstructorAPILevelsDataDump(ModuleStoreTestCase, LoginEnrollmentTestCa
self.assertTrue(body.startswith('"User ID","Anonymized user ID"\n"2","42"\n')) self.assertTrue(body.startswith('"User ID","Anonymized user ID"\n"2","42"\n'))
self.assertTrue(body.endswith('"7","42"\n')) self.assertTrue(body.endswith('"7","42"\n'))
def test_list_grade_downloads(self):
url = reverse('list_grade_downloads', kwargs={'course_id': self.course.id})
with patch('instructor_task.models.LocalFSGradesStore.links_for') as mock_links_for:
mock_links_for.return_value = [
('mock_file_name_1', 'https://1.mock.url'),
('mock_file_name_2', 'https://2.mock.url'),
]
response = self.client.get(url, {})
expected_response = {
"downloads": [
{
"url": "https://1.mock.url",
"link": "<a href=\"https://1.mock.url\">mock_file_name_1</a>",
"name": "mock_file_name_1"
},
{
"url": "https://2.mock.url",
"link": "<a href=\"https://2.mock.url\">mock_file_name_2</a>",
"name": "mock_file_name_2"
}
]
}
res_json = json.loads(response.content)
self.assertEqual(res_json, expected_response)
def test_calculate_grades_csv_success(self):
url = reverse('calculate_grades_csv', kwargs={'course_id': self.course.id})
with patch('instructor_task.api.submit_calculate_grades_csv') as mock_cal_grades:
mock_cal_grades.return_value = True
response = self.client.get(url, {})
success_status = "Your grade report is being generated! You can view the status of the generation task in the 'Pending Instructor Tasks' section."
self.assertIn(success_status, response.content)
def test_calculate_grades_csv_already_running(self):
url = reverse('calculate_grades_csv', kwargs={'course_id': self.course.id})
with patch('instructor_task.api.submit_calculate_grades_csv') as mock_cal_grades:
mock_cal_grades.side_effect = AlreadyRunningError()
response = self.client.get(url, {})
already_running_status = "A grade report generation task is already in progress. Check the 'Pending Instructor Tasks' table for the status of the task. When completed, the report will be available for download in the table below."
self.assertIn(already_running_status, response.content)
def test_get_students_features_csv(self): def test_get_students_features_csv(self):
""" """
Test that some minimum of information is formatted Test that some minimum of information is formatted
...@@ -802,7 +847,7 @@ class TestInstructorAPILevelsDataDump(ModuleStoreTestCase, LoginEnrollmentTestCa ...@@ -802,7 +847,7 @@ class TestInstructorAPILevelsDataDump(ModuleStoreTestCase, LoginEnrollmentTestCa
def test_get_distribution_no_feature(self): def test_get_distribution_no_feature(self):
""" """
Test that get_distribution lists available features Test that get_distribution lists available features
when supplied no feature quparameter. when supplied no feature parameter.
""" """
url = reverse('get_distribution', kwargs={'course_id': self.course.id}) url = reverse('get_distribution', kwargs={'course_id': self.course.id})
response = self.client.get(url) response = self.client.get(url)
......
...@@ -32,6 +32,7 @@ from student.models import unique_id_for_user ...@@ -32,6 +32,7 @@ from student.models import unique_id_for_user
import instructor_task.api import instructor_task.api
from instructor_task.api_helper import AlreadyRunningError from instructor_task.api_helper import AlreadyRunningError
from instructor_task.views import get_task_completion_info from instructor_task.views import get_task_completion_info
from instructor_task.models import GradesStore
import instructor.enrollment as enrollment import instructor.enrollment as enrollment
from instructor.enrollment import enroll_email, unenroll_email, get_email_params from instructor.enrollment import enroll_email, unenroll_email, get_email_params
from instructor.views.tools import strip_if_string, get_student_from_identifier from instructor.views.tools import strip_if_string, get_student_from_identifier
...@@ -753,6 +754,42 @@ def list_instructor_tasks(request, course_id): ...@@ -753,6 +754,42 @@ def list_instructor_tasks(request, course_id):
@ensure_csrf_cookie @ensure_csrf_cookie
@cache_control(no_cache=True, no_store=True, must_revalidate=True) @cache_control(no_cache=True, no_store=True, must_revalidate=True)
@require_level('staff') @require_level('staff')
def list_grade_downloads(_request, course_id):
"""
List grade CSV files that are available for download for this course.
"""
grades_store = GradesStore.from_config()
response_payload = {
'downloads': [
dict(name=name, url=url, link='<a href="{}">{}</a>'.format(url, name))
for name, url in grades_store.links_for(course_id)
]
}
return JsonResponse(response_payload)
@ensure_csrf_cookie
@cache_control(no_cache=True, no_store=True, must_revalidate=True)
@require_level('staff')
def calculate_grades_csv(request, course_id):
"""
AlreadyRunningError is raised if the course's grades are already being updated.
"""
try:
instructor_task.api.submit_calculate_grades_csv(request, course_id)
success_status = _("Your grade report is being generated! You can view the status of the generation task in the 'Pending Instructor Tasks' section.")
return JsonResponse({"status": success_status})
except AlreadyRunningError:
already_running_status = _("A grade report generation task is already in progress. Check the 'Pending Instructor Tasks' table for the status of the task. When completed, the report will be available for download in the table below.")
return JsonResponse({
"status": already_running_status
})
@ensure_csrf_cookie
@cache_control(no_cache=True, no_store=True, must_revalidate=True)
@require_level('staff')
@require_query_params('rolename') @require_query_params('rolename')
def list_forum_members(request, course_id): def list_forum_members(request, course_id):
""" """
......
...@@ -37,4 +37,10 @@ urlpatterns = patterns('', # nopep8 ...@@ -37,4 +37,10 @@ urlpatterns = patterns('', # nopep8
'instructor.views.api.proxy_legacy_analytics', name="proxy_legacy_analytics"), 'instructor.views.api.proxy_legacy_analytics', name="proxy_legacy_analytics"),
url(r'^send_email$', url(r'^send_email$',
'instructor.views.api.send_email', name="send_email"), 'instructor.views.api.send_email', name="send_email"),
# Grade downloads...
url(r'^list_grade_downloads$',
'instructor.views.api.list_grade_downloads', name="list_grade_downloads"),
url(r'calculate_grades_csv$',
'instructor.views.api.calculate_grades_csv', name="calculate_grades_csv"),
) )
...@@ -171,6 +171,8 @@ def _section_data_download(course_id, access): ...@@ -171,6 +171,8 @@ def _section_data_download(course_id, access):
'get_students_features_url': reverse('get_students_features', kwargs={'course_id': course_id}), 'get_students_features_url': reverse('get_students_features', kwargs={'course_id': course_id}),
'get_anon_ids_url': reverse('get_anon_ids', kwargs={'course_id': course_id}), 'get_anon_ids_url': reverse('get_anon_ids', kwargs={'course_id': course_id}),
'list_instructor_tasks_url': reverse('list_instructor_tasks', kwargs={'course_id': course_id}), 'list_instructor_tasks_url': reverse('list_instructor_tasks', kwargs={'course_id': course_id}),
'list_grade_downloads_url': reverse('list_grade_downloads', kwargs={'course_id': course_id}),
'calculate_grades_csv_url': reverse('calculate_grades_csv', kwargs={'course_id': course_id}),
} }
return section_data return section_data
......
...@@ -16,7 +16,8 @@ from instructor_task.models import InstructorTask ...@@ -16,7 +16,8 @@ from instructor_task.models import InstructorTask
from instructor_task.tasks import (rescore_problem, from instructor_task.tasks import (rescore_problem,
reset_problem_attempts, reset_problem_attempts,
delete_problem_state, delete_problem_state,
send_bulk_course_email) send_bulk_course_email,
calculate_grades_csv)
from instructor_task.api_helper import (check_arguments_for_rescoring, from instructor_task.api_helper import (check_arguments_for_rescoring,
encode_problem_and_student_input, encode_problem_and_student_input,
...@@ -206,3 +207,15 @@ def submit_bulk_course_email(request, course_id, email_id): ...@@ -206,3 +207,15 @@ def submit_bulk_course_email(request, course_id, email_id):
# create the key value by using MD5 hash: # create the key value by using MD5 hash:
task_key = hashlib.md5(task_key_stub).hexdigest() task_key = hashlib.md5(task_key_stub).hexdigest()
return submit_task(request, task_type, task_class, course_id, task_input, task_key) return submit_task(request, task_type, task_class, course_id, task_input, task_key)
def submit_calculate_grades_csv(request, course_id):
"""
AlreadyRunningError is raised if the course's grades are already being updated.
"""
task_type = 'grade_course'
task_class = calculate_grades_csv
task_input = {}
task_key = ""
return submit_task(request, task_type, task_class, course_id, task_input, task_key)
...@@ -12,9 +12,20 @@ file and check it in at the same time as your model changes. To do that, ...@@ -12,9 +12,20 @@ file and check it in at the same time as your model changes. To do that,
ASSUMPTIONS: modules have unique IDs, even across different module_types ASSUMPTIONS: modules have unique IDs, even across different module_types
""" """
from cStringIO import StringIO
from gzip import GzipFile
from uuid import uuid4 from uuid import uuid4
import csv
import json import json
import hashlib
import os
import os.path
import urllib
from boto.s3.connection import S3Connection
from boto.s3.key import Key
from django.conf import settings
from django.contrib.auth.models import User from django.contrib.auth.models import User
from django.db import models, transaction from django.db import models, transaction
...@@ -176,3 +187,220 @@ class InstructorTask(models.Model): ...@@ -176,3 +187,220 @@ class InstructorTask(models.Model):
def create_output_for_revoked(): def create_output_for_revoked():
"""Creates standard message to store in output format for revoked tasks.""" """Creates standard message to store in output format for revoked tasks."""
return json.dumps({'message': 'Task revoked before running'}) return json.dumps({'message': 'Task revoked before running'})
class GradesStore(object):
"""
Simple abstraction layer that can fetch and store CSV files for grades
download. Should probably refactor later to create a GradesFile object that
can simply be appended to for the sake of memory efficiency, rather than
passing in the whole dataset. Doing that for now just because it's simpler.
"""
@classmethod
def from_config(cls):
"""
Return one of the GradesStore subclasses depending on django
configuration. Look at subclasses for expected configuration.
"""
storage_type = settings.GRADES_DOWNLOAD.get("STORAGE_TYPE")
if storage_type.lower() == "s3":
return S3GradesStore.from_config()
elif storage_type.lower() == "localfs":
return LocalFSGradesStore.from_config()
class S3GradesStore(GradesStore):
"""
Grades store backed by S3. The directory structure we use to store things
is::
`{bucket}/{root_path}/{sha1 hash of course_id}/filename`
We might later use subdirectories or metadata to do more intelligent
grouping and querying, but right now it simply depends on its own
conventions on where files are stored to know what to display. Clients using
this class can name the final file whatever they want.
"""
def __init__(self, bucket_name, root_path):
self.root_path = root_path
conn = S3Connection(
settings.AWS_ACCESS_KEY_ID,
settings.AWS_SECRET_ACCESS_KEY
)
self.bucket = conn.get_bucket(bucket_name)
@classmethod
def from_config(cls):
"""
The expected configuration for an `S3GradesStore` is to have a
`GRADES_DOWNLOAD` dict in settings with the following fields::
STORAGE_TYPE : "s3"
BUCKET : Your bucket name, e.g. "grades-bucket"
ROOT_PATH : The path you want to store all course files under. Do not
use a leading or trailing slash. e.g. "staging" or
"staging/2013", not "/staging", or "/staging/"
Since S3 access relies on boto, you must also define `AWS_ACCESS_KEY_ID`
and `AWS_SECRET_ACCESS_KEY` in settings.
"""
return cls(
settings.GRADES_DOWNLOAD['BUCKET'],
settings.GRADES_DOWNLOAD['ROOT_PATH']
)
def key_for(self, course_id, filename):
"""Return the S3 key we would use to store and retrive the data for the
given filename."""
hashed_course_id = hashlib.sha1(course_id)
key = Key(self.bucket)
key.key = "{}/{}/{}".format(
self.root_path,
hashed_course_id.hexdigest(),
filename
)
return key
def store(self, course_id, filename, buff):
"""
Store the contents of `buff` in a directory determined by hashing
`course_id`, and name the file `filename`. `buff` is typically a
`StringIO`, but can be anything that implements `.getvalue()`.
This method assumes that the contents of `buff` are gzip-encoded (it
will add the appropriate headers to S3 to make the decompression
transparent via the browser). Filenames should end in whatever
suffix makes sense for the original file, so `.txt` instead of `.gz`
"""
key = self.key_for(course_id, filename)
data = buff.getvalue()
key.size = len(data)
key.content_encoding = "gzip"
key.content_type = "text/csv"
# Just setting the content encoding and type above should work
# according to the docs, but when experimenting, this was necessary for
# it to actually take.
key.set_contents_from_string(
data,
headers={
"Content-Encoding": "gzip",
"Content-Length": len(data),
"Content-Type": "text/csv",
}
)
def store_rows(self, course_id, filename, rows):
"""
Given a `course_id`, `filename`, and `rows` (each row is an iterable of
strings), create a buffer that is a gzip'd csv file, and then `store()`
that buffer.
Even though we store it in gzip format, browsers will transparently
download and decompress it. Filenames should end in `.csv`, not `.gz`.
"""
output_buffer = StringIO()
gzip_file = GzipFile(fileobj=output_buffer, mode="wb")
csv.writer(gzip_file).writerows(rows)
gzip_file.close()
self.store(course_id, filename, output_buffer)
def links_for(self, course_id):
"""
For a given `course_id`, return a list of `(filename, url)` tuples. `url`
can be plugged straight into an href
"""
course_dir = self.key_for(course_id, '')
return sorted(
[
(key.key.split("/")[-1], key.generate_url(expires_in=300))
for key in self.bucket.list(prefix=course_dir.key)
],
reverse=True
)
class LocalFSGradesStore(GradesStore):
"""
LocalFS implementation of a GradesStore. This is meant for debugging
purposes and is *absolutely not for production use*. Use S3GradesStore for
that. We use this in tests and for local development. When it generates
links, it will make file:/// style links. That means you actually have to
copy them and open them in a separate browser window, for security reasons.
This lets us do the cheap thing locally for debugging without having to open
up a separate URL that would only be used to send files in dev.
"""
def __init__(self, root_path):
"""
Initialize with root_path where we're going to store our files. We
will build a directory structure under this for each course.
"""
self.root_path = root_path
if not os.path.exists(root_path):
os.makedirs(root_path)
@classmethod
def from_config(cls):
"""
Generate an instance of this object from Django settings. It assumes
that there is a dict in settings named GRADES_DOWNLOAD and that it has
a ROOT_PATH that maps to an absolute file path that the web app has
write permissions to. `LocalFSGradesStore` will create any intermediate
directories as needed. Example::
STORAGE_TYPE : "localfs"
ROOT_PATH : /tmp/edx/grade-downloads/
"""
return cls(settings.GRADES_DOWNLOAD['ROOT_PATH'])
def path_to(self, course_id, filename):
"""Return the full path to a given file for a given course."""
return os.path.join(self.root_path, urllib.quote(course_id, safe=''), filename)
def store(self, course_id, filename, buff):
"""
Given the `course_id` and `filename`, store the contents of `buff` in
that file. Overwrite anything that was there previously. `buff` is
assumed to be a StringIO objecd (or anything that can flush its contents
to string using `.getvalue()`).
"""
full_path = self.path_to(course_id, filename)
directory = os.path.dirname(full_path)
if not os.path.exists(directory):
os.mkdir(directory)
with open(full_path, "wb") as f:
f.write(buff.getvalue())
def store_rows(self, course_id, filename, rows):
"""
Given a course_id, filename, and rows (each row is an iterable of strings),
write this data out.
"""
output_buffer = StringIO()
csv.writer(output_buffer).writerows(rows)
self.store(course_id, filename, output_buffer)
def links_for(self, course_id):
"""
For a given `course_id`, return a list of `(filename, url)` tuples. `url`
can be plugged straight into an href. Note that `LocalFSGradesStore`
will generate `file://` type URLs, so you'll need to copy the URL and
open it in a new browser window. Again, this class is only meant for
local development.
"""
course_dir = self.path_to(course_id, '')
if not os.path.exists(course_dir):
return []
return sorted(
[
(filename, ("file://" + urllib.quote(os.path.join(course_dir, filename))))
for filename in os.listdir(course_dir)
],
reverse=True
)
...@@ -19,6 +19,7 @@ a problem URL and optionally a student. These are used to set up the initial va ...@@ -19,6 +19,7 @@ a problem URL and optionally a student. These are used to set up the initial va
of the query for traversing StudentModule objects. of the query for traversing StudentModule objects.
""" """
from django.conf import settings
from django.utils.translation import ugettext_noop from django.utils.translation import ugettext_noop
from celery import task from celery import task
from functools import partial from functools import partial
...@@ -29,6 +30,7 @@ from instructor_task.tasks_helper import ( ...@@ -29,6 +30,7 @@ from instructor_task.tasks_helper import (
rescore_problem_module_state, rescore_problem_module_state,
reset_attempts_module_state, reset_attempts_module_state,
delete_problem_module_state, delete_problem_module_state,
push_grades_to_s3,
) )
from bulk_email.tasks import perform_delegate_email_batches from bulk_email.tasks import perform_delegate_email_batches
...@@ -127,3 +129,13 @@ def send_bulk_course_email(entry_id, _xmodule_instance_args): ...@@ -127,3 +129,13 @@ def send_bulk_course_email(entry_id, _xmodule_instance_args):
action_name = ugettext_noop('emailed') action_name = ugettext_noop('emailed')
visit_fcn = perform_delegate_email_batches visit_fcn = perform_delegate_email_batches
return run_main_task(entry_id, visit_fcn, action_name) return run_main_task(entry_id, visit_fcn, action_name)
@task(base=BaseInstructorTask, routing_key=settings.GRADES_DOWNLOAD_ROUTING_KEY) # pylint: disable=E1102
def calculate_grades_csv(entry_id, xmodule_instance_args):
"""
Grade a course and push the results to an S3 bucket for download.
"""
action_name = ugettext_noop('graded')
task_fn = partial(push_grades_to_s3, xmodule_instance_args)
return run_main_task(entry_id, task_fn, action_name)
...@@ -4,24 +4,27 @@ running state of a course. ...@@ -4,24 +4,27 @@ running state of a course.
""" """
import json import json
import urllib
from datetime import datetime
from time import time from time import time
from celery import Task, current_task from celery import Task, current_task
from celery.utils.log import get_task_logger from celery.utils.log import get_task_logger
from celery.states import SUCCESS, FAILURE from celery.states import SUCCESS, FAILURE
from django.contrib.auth.models import User from django.contrib.auth.models import User
from django.db import transaction, reset_queries from django.db import transaction, reset_queries
from dogapi import dog_stats_api from dogapi import dog_stats_api
from pytz import UTC
from xmodule.modulestore.django import modulestore from xmodule.modulestore.django import modulestore
from track.views import task_track from track.views import task_track
from courseware.grades import iterate_grades_for
from courseware.models import StudentModule from courseware.models import StudentModule
from courseware.model_data import FieldDataCache from courseware.model_data import FieldDataCache
from courseware.module_render import get_module_for_descriptor_internal from courseware.module_render import get_module_for_descriptor_internal
from instructor_task.models import InstructorTask, PROGRESS from instructor_task.models import GradesStore, InstructorTask, PROGRESS
from student.models import CourseEnrollment
# define different loggers for use within tasks and on client side # define different loggers for use within tasks and on client side
TASK_LOG = get_task_logger(__name__) TASK_LOG = get_task_logger(__name__)
...@@ -465,3 +468,106 @@ def delete_problem_module_state(xmodule_instance_args, _module_descriptor, stude ...@@ -465,3 +468,106 @@ def delete_problem_module_state(xmodule_instance_args, _module_descriptor, stude
track_function = _get_track_function_for_task(student_module.student, xmodule_instance_args) track_function = _get_track_function_for_task(student_module.student, xmodule_instance_args)
track_function('problem_delete_state', {}) track_function('problem_delete_state', {})
return UPDATE_STATUS_SUCCEEDED return UPDATE_STATUS_SUCCEEDED
def push_grades_to_s3(_xmodule_instance_args, _entry_id, course_id, _task_input, action_name):
"""
For a given `course_id`, generate a grades CSV file for all students that
are enrolled, and store using a `GradesStore`. Once created, the files can
be accessed by instantiating another `GradesStore` (via
`GradesStore.from_config()`) and calling `link_for()` on it. Writes are
buffered, so we'll never write part of a CSV file to S3 -- i.e. any files
that are visible in GradesStore will be complete ones.
As we start to add more CSV downloads, it will probably be worthwhile to
make a more general CSVDoc class instead of building out the rows like we
do here.
"""
start_time = datetime.now(UTC)
status_interval = 100
enrolled_students = CourseEnrollment.users_enrolled_in(course_id)
num_total = enrolled_students.count()
num_attempted = 0
num_succeeded = 0
num_failed = 0
curr_step = "Calculating Grades"
def update_task_progress():
"""Return a dict containing info about current task"""
current_time = datetime.now(UTC)
progress = {
'action_name': action_name,
'attempted': num_attempted,
'succeeded': num_succeeded,
'failed': num_failed,
'total': num_total,
'duration_ms': int((current_time - start_time).total_seconds() * 1000),
'step': curr_step,
}
_get_current_task().update_state(state=PROGRESS, meta=progress)
return progress
# Loop over all our students and build our CSV lists in memory
header = None
rows = []
err_rows = [["id", "username", "error_msg"]]
for student, gradeset, err_msg in iterate_grades_for(course_id, enrolled_students):
# Periodically update task status (this is a cache write)
if num_attempted % status_interval == 0:
update_task_progress()
num_attempted += 1
if gradeset:
# We were able to successfully grade this student for this course.
num_succeeded += 1
if not header:
header = [section['label'] for section in gradeset[u'section_breakdown']]
rows.append(["id", "email", "username", "grade"] + header)
percents = {
section['label']: section.get('percent', 0.0)
for section in gradeset[u'section_breakdown']
if 'label' in section
}
# Not everybody has the same gradable items. If the item is not
# found in the user's gradeset, just assume it's a 0. The aggregated
# grades for their sections and overall course will be calculated
# without regard for the item they didn't have access to, so it's
# possible for a student to have a 0.0 show up in their row but
# still have 100% for the course.
row_percents = [percents.get(label, 0.0) for label in header]
rows.append([student.id, student.email, student.username, gradeset['percent']] + row_percents)
else:
# An empty gradeset means we failed to grade a student.
num_failed += 1
err_rows.append([student.id, student.username, err_msg])
# By this point, we've got the rows we're going to stuff into our CSV files.
curr_step = "Uploading CSVs"
update_task_progress()
# Generate parts of the file name
timestamp_str = start_time.strftime("%Y-%m-%d-%H%M")
course_id_prefix = urllib.quote(course_id.replace("/", "_"))
# Perform the actual upload
grades_store = GradesStore.from_config()
grades_store.store_rows(
course_id,
"{}_grade_report_{}.csv".format(course_id_prefix, timestamp_str),
rows
)
# If there are any error rows (don't count the header), write them out as well
if len(err_rows) > 1:
grades_store.store_rows(
course_id,
"{}_grade_report_{}_err.csv".format(course_id_prefix, timestamp_str),
err_rows
)
# One last update before we close out...
return update_task_progress()
...@@ -75,7 +75,6 @@ def instructor_task_status(request): ...@@ -75,7 +75,6 @@ def instructor_task_status(request):
'traceback': optional, returned if task failed and produced a traceback. 'traceback': optional, returned if task failed and produced a traceback.
""" """
output = {} output = {}
if 'task_id' in request.REQUEST: if 'task_id' in request.REQUEST:
task_id = request.REQUEST['task_id'] task_id = request.REQUEST['task_id']
......
...@@ -86,6 +86,7 @@ CELERY_DEFAULT_EXCHANGE = 'edx.{0}core'.format(QUEUE_VARIANT) ...@@ -86,6 +86,7 @@ CELERY_DEFAULT_EXCHANGE = 'edx.{0}core'.format(QUEUE_VARIANT)
HIGH_PRIORITY_QUEUE = 'edx.{0}core.high'.format(QUEUE_VARIANT) HIGH_PRIORITY_QUEUE = 'edx.{0}core.high'.format(QUEUE_VARIANT)
DEFAULT_PRIORITY_QUEUE = 'edx.{0}core.default'.format(QUEUE_VARIANT) DEFAULT_PRIORITY_QUEUE = 'edx.{0}core.default'.format(QUEUE_VARIANT)
LOW_PRIORITY_QUEUE = 'edx.{0}core.low'.format(QUEUE_VARIANT) LOW_PRIORITY_QUEUE = 'edx.{0}core.low'.format(QUEUE_VARIANT)
HIGH_MEM_QUEUE = 'edx.{0}core.high_mem'.format(QUEUE_VARIANT)
CELERY_DEFAULT_QUEUE = DEFAULT_PRIORITY_QUEUE CELERY_DEFAULT_QUEUE = DEFAULT_PRIORITY_QUEUE
CELERY_DEFAULT_ROUTING_KEY = DEFAULT_PRIORITY_QUEUE CELERY_DEFAULT_ROUTING_KEY = DEFAULT_PRIORITY_QUEUE
...@@ -93,9 +94,19 @@ CELERY_DEFAULT_ROUTING_KEY = DEFAULT_PRIORITY_QUEUE ...@@ -93,9 +94,19 @@ CELERY_DEFAULT_ROUTING_KEY = DEFAULT_PRIORITY_QUEUE
CELERY_QUEUES = { CELERY_QUEUES = {
HIGH_PRIORITY_QUEUE: {}, HIGH_PRIORITY_QUEUE: {},
LOW_PRIORITY_QUEUE: {}, LOW_PRIORITY_QUEUE: {},
DEFAULT_PRIORITY_QUEUE: {} DEFAULT_PRIORITY_QUEUE: {},
HIGH_MEM_QUEUE: {},
} }
# If we're a worker on the high_mem queue, set ourselves to die after processing
# one request to avoid having memory leaks take down the worker server. This env
# var is set in /etc/init/edx-workers.conf -- this should probably be replaced
# with some celery API call to see what queue we started listening to, but I
# don't know what that call is or if it's active at this point in the code.
if os.environ.get('QUEUE') == 'high_mem':
CELERYD_MAX_TASKS_PER_CHILD = 1
########################## NON-SECURE ENV CONFIG ############################## ########################## NON-SECURE ENV CONFIG ##############################
# Things like server locations, ports, etc. # Things like server locations, ports, etc.
...@@ -318,3 +329,8 @@ TRACKING_BACKENDS.update(AUTH_TOKENS.get("TRACKING_BACKENDS", {})) ...@@ -318,3 +329,8 @@ TRACKING_BACKENDS.update(AUTH_TOKENS.get("TRACKING_BACKENDS", {}))
# Student identity verification settings # Student identity verification settings
VERIFY_STUDENT = AUTH_TOKENS.get("VERIFY_STUDENT", VERIFY_STUDENT) VERIFY_STUDENT = AUTH_TOKENS.get("VERIFY_STUDENT", VERIFY_STUDENT)
# Grades download
GRADES_DOWNLOAD_ROUTING_KEY = HIGH_MEM_QUEUE
GRADES_DOWNLOAD = ENV_TOKENS.get("GRADES_DOWNLOAD", GRADES_DOWNLOAD)
...@@ -192,6 +192,14 @@ MITX_FEATURES = { ...@@ -192,6 +192,14 @@ MITX_FEATURES = {
# Disable instructor dash buttons for downloading course data # Disable instructor dash buttons for downloading course data
# when enrollment exceeds this number # when enrollment exceeds this number
'MAX_ENROLLMENT_INSTR_BUTTONS': 200, 'MAX_ENROLLMENT_INSTR_BUTTONS': 200,
# Grade calculation started from the new instructor dashboard will write
# grades CSV files to S3 and give links for downloads.
'ENABLE_S3_GRADE_DOWNLOADS': False,
# Give course staff unrestricted access to grade downloads (if set to False,
# only edX superusers can perform the downloads)
'ALLOW_COURSE_STAFF_GRADE_DOWNLOADS': False,
} }
# Used for A/B testing # Used for A/B testing
...@@ -846,6 +854,7 @@ CELERY_DEFAULT_EXCHANGE_TYPE = 'direct' ...@@ -846,6 +854,7 @@ CELERY_DEFAULT_EXCHANGE_TYPE = 'direct'
HIGH_PRIORITY_QUEUE = 'edx.core.high' HIGH_PRIORITY_QUEUE = 'edx.core.high'
DEFAULT_PRIORITY_QUEUE = 'edx.core.default' DEFAULT_PRIORITY_QUEUE = 'edx.core.default'
LOW_PRIORITY_QUEUE = 'edx.core.low' LOW_PRIORITY_QUEUE = 'edx.core.low'
HIGH_MEM_QUEUE = 'edx.core.high_mem'
CELERY_QUEUE_HA_POLICY = 'all' CELERY_QUEUE_HA_POLICY = 'all'
...@@ -857,7 +866,8 @@ CELERY_DEFAULT_ROUTING_KEY = DEFAULT_PRIORITY_QUEUE ...@@ -857,7 +866,8 @@ CELERY_DEFAULT_ROUTING_KEY = DEFAULT_PRIORITY_QUEUE
CELERY_QUEUES = { CELERY_QUEUES = {
HIGH_PRIORITY_QUEUE: {}, HIGH_PRIORITY_QUEUE: {},
LOW_PRIORITY_QUEUE: {}, LOW_PRIORITY_QUEUE: {},
DEFAULT_PRIORITY_QUEUE: {} DEFAULT_PRIORITY_QUEUE: {},
HIGH_MEM_QUEUE: {},
} }
# let logging work as configured: # let logging work as configured:
...@@ -1061,3 +1071,12 @@ REGISTRATION_OPTIONAL_FIELDS = set([ ...@@ -1061,3 +1071,12 @@ REGISTRATION_OPTIONAL_FIELDS = set([
'mailing_address', 'mailing_address',
'goals', 'goals',
]) ])
###################### Grade Downloads ######################
GRADES_DOWNLOAD_ROUTING_KEY = HIGH_MEM_QUEUE
GRADES_DOWNLOAD = {
'STORAGE_TYPE': 'localfs',
'BUCKET': 'edx-grades',
'ROOT_PATH': '/tmp/edx-s3/grades',
}
...@@ -40,6 +40,7 @@ MITX_FEATURES['ENABLE_INSTRUCTOR_BETA_DASHBOARD'] = True ...@@ -40,6 +40,7 @@ MITX_FEATURES['ENABLE_INSTRUCTOR_BETA_DASHBOARD'] = True
MITX_FEATURES['MULTIPLE_ENROLLMENT_ROLES'] = True MITX_FEATURES['MULTIPLE_ENROLLMENT_ROLES'] = True
MITX_FEATURES['ENABLE_SHOPPING_CART'] = True MITX_FEATURES['ENABLE_SHOPPING_CART'] = True
MITX_FEATURES['AUTOMATIC_VERIFY_STUDENT_IDENTITY_FOR_TESTING'] = True MITX_FEATURES['AUTOMATIC_VERIFY_STUDENT_IDENTITY_FOR_TESTING'] = True
MITX_FEATURES['ENABLE_S3_GRADE_DOWNLOADS'] = True
FEEDBACK_SUBMISSION_EMAIL = "dummy@example.com" FEEDBACK_SUBMISSION_EMAIL = "dummy@example.com"
......
...@@ -17,14 +17,23 @@ class DataDownload ...@@ -17,14 +17,23 @@ class DataDownload
# this object to call event handlers like 'onClickTitle' # this object to call event handlers like 'onClickTitle'
@$section.data 'wrapper', @ @$section.data 'wrapper', @
# gather elements # gather elements
@$display = @$section.find '.data-display'
@$display_text = @$display.find '.data-display-text'
@$display_table = @$display.find '.data-display-table'
@$request_response_error = @$display.find '.request-response-error'
@$list_studs_btn = @$section.find("input[name='list-profiles']'") @$list_studs_btn = @$section.find("input[name='list-profiles']'")
@$list_anon_btn = @$section.find("input[name='list-anon-ids']'") @$list_anon_btn = @$section.find("input[name='list-anon-ids']'")
@$grade_config_btn = @$section.find("input[name='dump-gradeconf']'") @$grade_config_btn = @$section.find("input[name='dump-gradeconf']'")
@$calculate_grades_csv_btn = @$section.find("input[name='calculate-grades-csv']'")
# response areas
@$download = @$section.find '.data-download-container'
@$download_display_text = @$download.find '.data-display-text'
@$download_display_table = @$download.find '.data-display-table'
@$download_request_response_error = @$download.find '.request-response-error'
@$grades = @$section.find '.grades-download-container'
@$grades_request_response = @$grades.find '.request-response'
@$grades_request_response_error = @$grades.find '.request-response-error'
@grade_downloads = new GradeDownloads(@$section)
@instructor_tasks = new (PendingInstructorTasks()) @$section @instructor_tasks = new (PendingInstructorTasks()) @$section
@clear_display()
# attach click handlers # attach click handlers
# The list-anon case is always CSV # The list-anon case is always CSV
...@@ -43,8 +52,9 @@ class DataDownload ...@@ -43,8 +52,9 @@ class DataDownload
url += '/csv' url += '/csv'
location.href = url location.href = url
else else
# Dynamically generate slickgrid table for displaying student profile information
@clear_display() @clear_display()
@$display_table.text 'Loading...' @$download_display_table.text gettext('Loading...')
# fetch user list # fetch user list
$.ajax $.ajax
...@@ -52,7 +62,7 @@ class DataDownload ...@@ -52,7 +62,7 @@ class DataDownload
url: url url: url
error: std_ajax_err => error: std_ajax_err =>
@clear_display() @clear_display()
@$request_response_error.text "Error getting student list." @$download_request_response_error.text gettext("Error getting student list.")
success: (data) => success: (data) =>
@clear_display() @clear_display()
...@@ -61,12 +71,13 @@ class DataDownload ...@@ -61,12 +71,13 @@ class DataDownload
enableCellNavigation: true enableCellNavigation: true
enableColumnReorder: false enableColumnReorder: false
forceFitColumns: true forceFitColumns: true
rowHeight: 35
columns = ({id: feature, field: feature, name: feature} for feature in data.queried_features) columns = ({id: feature, field: feature, name: feature} for feature in data.queried_features)
grid_data = data.students grid_data = data.students
$table_placeholder = $ '<div/>', class: 'slickgrid' $table_placeholder = $ '<div/>', class: 'slickgrid'
@$display_table.append $table_placeholder @$download_display_table.append $table_placeholder
grid = new Slick.Grid($table_placeholder, grid_data, columns, options) grid = new Slick.Grid($table_placeholder, grid_data, columns, options)
# grid.autosizeColumns() # grid.autosizeColumns()
...@@ -78,21 +89,103 @@ class DataDownload ...@@ -78,21 +89,103 @@ class DataDownload
url: url url: url
error: std_ajax_err => error: std_ajax_err =>
@clear_display() @clear_display()
@$request_response_error.text "Error getting grading configuration." @$download_request_response_error.text gettext("Error retrieving grading configuration.")
success: (data) => success: (data) =>
@clear_display() @clear_display()
@$display_text.html data['grading_config_summary'] @$download_display_text.html data['grading_config_summary']
@$calculate_grades_csv_btn.click (e) =>
# Clear any CSS styling from the request-response areas
#$(".msg-confirm").css({"display":"none"})
#$(".msg-error").css({"display":"none"})
@clear_display()
url = @$calculate_grades_csv_btn.data 'endpoint'
$.ajax
dataType: 'json'
url: url
error: std_ajax_err =>
@$grades_request_response_error.text gettext("Error generating grades. Please try again.")
$(".msg-error").css({"display":"block"})
success: (data) =>
@$grades_request_response.text data['status']
$(".msg-confirm").css({"display":"block"})
# handler for when the section title is clicked. # handler for when the section title is clicked.
onClickTitle: -> @instructor_tasks.task_poller.start() onClickTitle: ->
# Clear display of anything that was here before
@clear_display()
@instructor_tasks.task_poller.start()
@grade_downloads.downloads_poller.start()
# handler for when the section is closed # handler for when the section is closed
onExit: -> @instructor_tasks.task_poller.stop() onExit: ->
@instructor_tasks.task_poller.stop()
@grade_downloads.downloads_poller.stop()
clear_display: -> clear_display: ->
@$display_text.empty() # Clear any generated tables, warning messages, etc.
@$display_table.empty() @$download_display_text.empty()
@$request_response_error.empty() @$download_display_table.empty()
@$download_request_response_error.empty()
@$grades_request_response.empty()
@$grades_request_response_error.empty()
# Clear any CSS styling from the request-response areas
$(".msg-confirm").css({"display":"none"})
$(".msg-error").css({"display":"none"})
class GradeDownloads
### Grade Downloads -- links expire quickly, so we refresh every 5 mins ####
constructor: (@$section) ->
@$grades = @$section.find '.grades-download-container'
@$grades_request_response = @$grades.find '.request-response'
@$grades_request_response_error = @$grades.find '.request-response-error'
@$grade_downloads_table = @$grades.find ".grade-downloads-table"
POLL_INTERVAL = 1000 * 60 * 5 # 5 minutes in ms
@downloads_poller = new window.InstructorDashboard.util.IntervalManager(
POLL_INTERVAL, => @reload_grade_downloads()
)
reload_grade_downloads: ->
endpoint = @$grade_downloads_table.data 'endpoint'
$.ajax
dataType: 'json'
url: endpoint
success: (data) =>
if data.downloads.length
@create_grade_downloads_table data.downloads
else
console.log "No grade CSVs ready for download"
error: std_ajax_err => console.error "Error finding grade download CSVs"
create_grade_downloads_table: (grade_downloads_data) ->
@$grade_downloads_table.empty()
options =
enableCellNavigation: true
enableColumnReorder: false
rowHeight: 30
forceFitColumns: true
columns = [
id: 'link'
field: 'link'
name: gettext('File Name (Newest First)')
toolTip: gettext("Links are generated on demand and expire within 5 minutes due to the sensitive nature of student grade information.")
sortable: false
minWidth: 150
cssClass: "file-download-link"
formatter: (row, cell, value, columnDef, dataContext) ->
'<a href="' + dataContext['url'] + '">' + dataContext['name'] + '</a>'
]
$table_placeholder = $ '<div/>', class: 'slickgrid'
@$grade_downloads_table.append $table_placeholder
grid = new Slick.Grid($table_placeholder, grade_downloads_data, columns, options)
grid.autosizeColumns()
# export for use # export for use
......
...@@ -43,7 +43,7 @@ create_task_list_table = ($table_tasks, tasks_data) -> ...@@ -43,7 +43,7 @@ create_task_list_table = ($table_tasks, tasks_data) ->
id: 'task_type' id: 'task_type'
field: 'task_type' field: 'task_type'
name: 'Task Type' name: 'Task Type'
minWidth: 100 minWidth: 102
, ,
id: 'task_input' id: 'task_input'
field: 'task_input' field: 'task_input'
......
...@@ -26,6 +26,13 @@ ...@@ -26,6 +26,13 @@
@include font-size(16); @include font-size(16);
} }
.file-download-link a {
font-size: 15px;
color: $link-color;
text-decoration: underline;
padding: 5px;
}
// system feedback - messages // system feedback - messages
.msg { .msg {
border-radius: 1px; border-radius: 1px;
...@@ -117,7 +124,7 @@ section.instructor-dashboard-content-2 { ...@@ -117,7 +124,7 @@ section.instructor-dashboard-content-2 {
.slickgrid { .slickgrid {
margin-left: 1px; margin-left: 1px;
color:#333333; color:#333333;
font-size:11px; font-size:12px;
font-family: verdana,arial,sans-serif; font-family: verdana,arial,sans-serif;
.slick-header-column { .slick-header-column {
...@@ -428,13 +435,26 @@ section.instructor-dashboard-content-2 { ...@@ -428,13 +435,26 @@ section.instructor-dashboard-content-2 {
line-height: 1.3em; line-height: 1.3em;
} }
.data-display { .data-download-container {
.data-display-table { .data-display-table {
.slickgrid { .slickgrid {
height: 400px; height: 400px;
} }
} }
} }
.grades-download-container {
.grade-downloads-table {
.slickgrid {
height: 300px;
padding: 5px;
}
// Disable horizontal scroll bar when grid only has 1 column. Remove this CSS class when more columns added.
.slick-viewport {
overflow-x: hidden !important;
}
}
}
} }
......
...@@ -2,23 +2,49 @@ ...@@ -2,23 +2,49 @@
<%page args="section_data"/> <%page args="section_data"/>
<h2>${_("Data Download")}</h2> <div class="data-download-container action-type-container">
<h2>${_("Data Download")}</h2>
<input type="button" name="list-profiles" value="${_("List enrolled students with profile information")}" data-endpoint="${ section_data['get_students_features_url'] }"> <div class="request-response-error msg msg-error copy"></div>
<input type="button" name="list-profiles" value="CSV" data-csv="true">
<br> <p>${_("The following button displays a list of all students enrolled in this course, along with profile information such as email address and username. The data can also be downloaded as a CSV file.")}</p>
## <input type="button" name="list-grades" value="Student grades">
## <input type="button" name="list-profiles" value="CSV" data-csv="true" class="csv"> <p><input type="button" name="list-profiles" value="${_("List enrolled students' profile information")}" data-endpoint="${ section_data['get_students_features_url'] }">
## <br> <input type="button" name="list-profiles" value="${_("Download profile information as a CSV")}" data-csv="true"></p>
## <input type="button" name="list-answer-distributions" value="Answer distributions (x students got y points)">
## <br>
<input type="button" name="dump-gradeconf" value="${_("Grading Configuration")}" data-endpoint="${ section_data['get_grading_config_url'] }">
<input type="button" name="list-anon-ids" value="${_("Get Student Anonymized IDs CSV")}" data-csv="true" class="csv" data-endpoint="${ section_data['get_anon_ids_url'] }" class="${'is-disabled' if disable_buttons else ''}">
<div class="data-display">
<div class="data-display-text"></div>
<div class="data-display-table"></div> <div class="data-display-table"></div>
<div class="request-response-error"></div>
<br>
<p>${_("Displays the grading configuration for the course. The grading configuration is the breakdown of graded subsections of the course (such as exams and problem sets), and can be changed on the 'Grading' page (under 'Settings') in Studio.")}</p>
<p><input type="button" name="dump-gradeconf" value="${_("Grading Configuration")}" data-endpoint="${ section_data['get_grading_config_url'] }"></p>
<div class="data-display-text"></div>
<br>
<p>${_("Download a CSV of anonymized student IDs by clicking this button.")}</p>
<p><input type="button" name="list-anon-ids" value="${_("Get Student Anonymized IDs CSV")}" data-csv="true" class="csv" data-endpoint="${ section_data['get_anon_ids_url'] }" class="${'is-disabled' if disable_buttons else ''}"></p>
</div>
%if settings.MITX_FEATURES.get('ENABLE_S3_GRADE_DOWNLOADS'):
<div class="grades-download-container action-type-container">
<hr>
<h2> ${_("Grade Reports")}</h2>
%if settings.MITX_FEATURES.get('ALLOW_COURSE_STAFF_GRADE_DOWNLOADS') or section_data['access']['admin']:
<p>${_("The following button will generate a CSV grade report for all currently enrolled students. For large courses, generating this report may take a few hours.")}</p>
<p>${_("The report is generated in the background, meaning it is OK to navigate away from this page while your report is generating. Generated reports appear in a table below and can be downloaded.")}</p>
<div class="request-response msg msg-confirm copy"></div>
<div class="request-response-error msg msg-warning copy"></div>
<br>
<p><input type="button" name="calculate-grades-csv" value="${_("Generate Grade Report")}" data-endpoint="${ section_data['calculate_grades_csv_url'] }"/></p>
%endif
<p><b>${_("Reports Available for Download")}</b></p>
<p>${_("File links are generated on demand and expire within 5 minutes due to the sensitive nature of student grade information. Please note that the report filename contains a timestamp that represents when your file was generated; this timestamp is UTC, not your local timezone.")}</p><br>
<div class="grade-downloads-table" data-endpoint="${ section_data['list_grade_downloads_url'] }" ></div>
</div>
%endif
%if settings.MITX_FEATURES.get('ENABLE_INSTRUCTOR_BACKGROUND_TASKS'): %if settings.MITX_FEATURES.get('ENABLE_INSTRUCTOR_BACKGROUND_TASKS'):
<div class="running-tasks-container action-type-container"> <div class="running-tasks-container action-type-container">
...@@ -26,9 +52,6 @@ ...@@ -26,9 +52,6 @@
<h2> ${_("Pending Instructor Tasks")} </h2> <h2> ${_("Pending Instructor Tasks")} </h2>
<p>${_("The status for any active tasks appears in a table below.")} </p> <p>${_("The status for any active tasks appears in a table below.")} </p>
<br /> <br />
<div class="running-tasks-table" data-endpoint="${ section_data['list_instructor_tasks_url'] }"></div> <div class="running-tasks-table" data-endpoint="${ section_data['list_instructor_tasks_url'] }"></div>
</div> </div>
%endif %endif
</div>
...@@ -5,7 +5,6 @@ ...@@ -5,7 +5,6 @@
<h2>${_("Student-specific grade inspection")}</h2> <h2>${_("Student-specific grade inspection")}</h2>
<div class="request-response-error"></div> <div class="request-response-error"></div>
<p> <p>
<!-- Doesn't work for username but this MUST work -->
${_("Specify the {platform_name} email address or username of a student here:").format(platform_name=settings.PLATFORM_NAME)} ${_("Specify the {platform_name} email address or username of a student here:").format(platform_name=settings.PLATFORM_NAME)}
<input type="text" name="student-select-progress" placeholder="${_("Student Email or Username")}"> <input type="text" name="student-select-progress" placeholder="${_("Student Email or Username")}">
</p> </p>
...@@ -26,7 +25,6 @@ ...@@ -26,7 +25,6 @@
<h2>${_("Student-specific grade adjustment")}</h2> <h2>${_("Student-specific grade adjustment")}</h2>
<div class="request-response-error"></div> <div class="request-response-error"></div>
<p> <p>
<!-- Doesn't work for username but this MUST work -->
${_("Specify the {platform_name} email address or username of a student here:").format(platform_name=settings.PLATFORM_NAME)} ${_("Specify the {platform_name} email address or username of a student here:").format(platform_name=settings.PLATFORM_NAME)}
<input type="text" name="student-select-grade" placeholder="${_("Student Email or Username")}"> <input type="text" name="student-select-grade" placeholder="${_("Student Email or Username")}">
</p> </p>
...@@ -60,18 +58,18 @@ ...@@ -60,18 +58,18 @@
<p> <p>
%if section_data['access']['instructor']: %if section_data['access']['instructor']:
<p> ${_('You may also delete the entire state of a student for the specified problem:')} </p> <p> ${_('You may also delete the entire state of a student for the specified problem:')} </p>
<input type="button" class="molly-guard" name="delete-state-single" value="${_("Delete Student State for Problem")}" data-endpoint="${ section_data['reset_student_attempts_url'] }"> <p><input type="button" class="molly-guard" name="delete-state-single" value="${_("Delete Student State for Problem")}" data-endpoint="${ section_data['reset_student_attempts_url'] }"></p>
%endif %endif
</p> </p>
%if settings.MITX_FEATURES.get('ENABLE_INSTRUCTOR_BACKGROUND_TASKS') and section_data['access']['instructor']: %if settings.MITX_FEATURES.get('ENABLE_INSTRUCTOR_BACKGROUND_TASKS') and section_data['access']['instructor']:
<p> <p>
${_("Rescoring runs in the background, and status for active tasks will appear in a table on the Course Info tab. " ${_("Rescoring runs in the background, and status for active tasks will appear in the 'Pending Instructor Tasks' table. "
"To see status for all tasks submitted for this problem and student, click on this button:")} "To see status for all tasks submitted for this problem and student, click on this button:")}
</p> </p>
<input type="button" name="task-history-single" value="${_("Show Background Task History for Student")}" data-endpoint="${ section_data['list_instructor_tasks_url'] }"> <p><input type="button" name="task-history-single" value="${_("Show Background Task History for Student")}" data-endpoint="${ section_data['list_instructor_tasks_url'] }"></p>
<div class="task-history-single-table"></div> <div class="task-history-single-table"></div>
%endif %endif
<hr> <hr>
...@@ -101,10 +99,10 @@ ...@@ -101,10 +99,10 @@
</p> </p>
<p> <p>
<p> <p>
${_("These actions run in the background, and status for active tasks will appear in a table on the Course Info tab. " ${_("The above actions run in the background, and status for active tasks will appear in a table on the Course Info tab. "
"To see status for all tasks submitted for this problem, click on this button")}: "To see status for all tasks submitted for this problem, click on this button")}:
</p> </p>
<input type="button" name="task-history-all" value="${_("Show Background Task History for Problem")}" data-endpoint="${ section_data['list_instructor_tasks_url'] }"> <p><input type="button" name="task-history-all" value="${_("Show Background Task History for Problem")}" data-endpoint="${ section_data['list_instructor_tasks_url'] }"></p>
<div class="task-history-all-table"></div> <div class="task-history-all-table"></div>
</p> </p>
</div> </div>
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment