Skip to content

E
edx-platform
Commit 336d648e by Will Daly
Options

Merge pull request #1644 from edx/will/remove-load-test 

Removed custom response load test
parents 3dfdab1a b55ffde4
Showing with 0 additions and 195 deletions
lms/djangoapps/courseware/tests/load_tests/README.md deleted 100644 → 0
# Load Testing
Scripts for load testing the courseware app,
mostly using [multimechanize](http://testutils.org/multi-mechanize/)
lms/djangoapps/courseware/tests/load_tests/custom_response/README.md deleted 100644 → 0
# Custom Response Load Test
## Optional Installations
* [memcached](http://pypi.python.org/pypi/python-memcached/): Install this
and make sure it is running, or the Capa problem will not cache results.
* [AppArmor](http://wiki.apparmor.net): Follow the instructions in
`common/lib/codejail/README` to set up the Python sandbox environment.
If you do not set up the sandbox, the tests will still execute code in the CustomResponse,
so you can still run the tests.
* [matplotlib](http://matplotlib.org): Multi-mechanize uses this to create graphs.
## Running the Tests
This test simulates student submissions for a custom response problem.
First, clear the cache:
/etc/init.d/memcached restart
Then, run the test:
multimech-run custom_response
You can configure the parameters in `customresponse/config.cfg`,
and you can change the CustomResponse script and student submissions
in `customresponse/test_scripts/v_user.py`.
## Components Under Test
Components under test:
* Python sandbox (see `common/lib/codejail`), which uses `AppArmor`
* Caching (see `common/lib/capa/capa/safe_exec/`), which uses `memcache` in production
Components NOT under test:
* Django views
* `XModule`
* gunicorn
This allows us to avoid creating courses in mongo, logging in, using CSRF tokens,
and other inconveniences. Instead, we create a capa problem (from the capa package),
pass it Django's memcache backend, and pass the problem student submissions.
Even though the test uses `capa.capa_problem.LoncapaProblem` directly,
the `capa` should not depend on Django. For this reason, we put the
test in the `courseware` Django app.
lms/djangoapps/courseware/tests/load_tests/custom_response/config.cfg deleted 100644 → 0
[global]
run_time = 240
rampup = 30
results_ts_interval = 10
progress_bar = on
console_logging = off
xml_report = off
[user_group-1]
threads = 10
script = v_user.py
[user_group-2]
threads = 10
script = v_user.py
[user_group-3]
threads = 10
script = v_user.py
lms/djangoapps/courseware/tests/load_tests/custom_response/test_scripts/v_user.py deleted 100644 → 0
""" User script for load testing CustomResponse """
from capa.tests.response_xml_factory import CustomResponseXMLFactory
import capa.capa_problem as lcp
from xmodule.x_module import ModuleSystem
import mock
import fs.osfs
import random
import textwrap
# Use memcache running locally
CACHE_SETTINGS = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': '127.0.0.1:11211'
},
}
# Configure settings so Django will let us import its cache wrapper
# Caching is the only part of Django being tested
from django.conf import settings
settings.configure(CACHES=CACHE_SETTINGS)
from django.core.cache import cache
# Script to install as the checker for the CustomResponse
TEST_SCRIPT = textwrap.dedent("""
def check_func(expect, answer_given):
return {'ok': answer_given == expect, 'msg': 'Message text'}
""")
# Submissions submitted by the student
TEST_SUBMISSIONS = [random.randint(-100, 100) for i in range(100)]
class TestContext(object):
""" One-time set up for the test that is shared across transactions.
Uses a Singleton design pattern."""
SINGLETON = None
NUM_UNIQUE_SEEDS = 20
@classmethod
def singleton(cls):
""" Return the singleton, creating one if it does not already exist."""
# If we haven't created the singleton yet, create it now
if cls.SINGLETON is None:
# Create a mock ModuleSystem, installing our cache
system = mock.MagicMock(ModuleSystem)
system.STATIC_URL = '/dummy-static/'
system.render_template = lambda template, context: "<div>%s</div>" % template
system.cache = cache
system.filestore = mock.MagicMock(fs.osfs.OSFS)
system.filestore.root_path = ""
system.DEBUG = True
# Create a custom response problem
xml_factory = CustomResponseXMLFactory()
xml = xml_factory.build_xml(script=TEST_SCRIPT, cfn="check_func", expect="42")
# Create and store the context
cls.SINGLETON = cls(system, xml)
else:
pass
# Return the singleton
return cls.SINGLETON
def __init__(self, system, xml):
""" Store context needed for the test across transactions """
self.system = system
self.xml = xml
# Construct a small pool of unique seeds
# To keep our implementation in line with the one capa actually uses,
# construct the problems, then use the seeds they generate
self.seeds = [lcp.LoncapaProblem(self.xml, 'problem_id', system=self.system).seed
for i in range(self.NUM_UNIQUE_SEEDS)]
def random_seed(self):
""" Return one of a small number of unique random seeds """
return random.choice(self.seeds)
def student_submission(self):
""" Return one of a small number of student submissions """
return random.choice(TEST_SUBMISSIONS)
class Transaction(object):
""" User script that submits a response to a CustomResponse problem """
def __init__(self):
""" Create the problem """
# Get the context (re-used across transactions)
self.context = TestContext.singleton()
# Create a new custom response problem
# using one of a small number of unique seeds
# We're assuming that the capa module is limiting the number
# of seeds (currently not the case for certain settings)
self.problem = lcp.LoncapaProblem(
self.context.xml, '1',
state=None, seed=self.context.random_seed(),
system=self.context.system,
)
def run(self):
""" Submit a response to the CustomResponse problem """
answers = {'1_2_1': self.context.student_submission()}
self.problem.grade_answers(answers)
if __name__ == '__main__':
trans = Transaction()
trans.run()
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment