EDUCATOR-11

TNL-6815

TNL-5181

An issue arose recently due to ATOMIC_REQUESTS being turned on by default. It
turns out that CohortMemberships had been somewhat relying on the old default
transaction handling in order to keep CohortMemberships and the underlying
CourseUserGroup.users values in-sync.

To fix this, I've made all updates to Cohortmemberships go through an
outer_atomic(read_committed=True) block. This, is conjunction with the already
present select_for_update(), will no longer allow 2 simultaneous requests to
modify objects in memory without sharing them. Only one process will be
touching a given CohortMembership at any given time, and all changes will be
immediately comitted to the database, where the other process will see them.

I've also included some changes to get_cohort(), add_user_to_cohort(), and
remove_user_from_cohort() in order to properly make use of the new
CohortMembership system.

This reverts commit 12130c80.
Manual testing found this to not fix the bug it claimed to fix.

What had been happening was related to the fact that ATOMIC_REQUESTS is now on,
preventing writing to the database until a request is completed.

The problem with that was that CohortMemberships, with their explicit
transaction handling, were breaking that rule and touching the database.
The CohortMembership needs to also touch the database to update the
underlying course_user_groups information, to prevent the database from
getting into a strange state.

After this commit, every time a CohortMembership is updated or created, the
corresponding course_user_groups are also updated and explicitly written to
the database. This occurs atomically with the CohortMembership, and does not
wait for the end of the request.

Please note that this is a squshed commit and the work of:
Symbolist, macdiesel, nedbat, doctoryes, muzaffaryousaf and muhammad-ammar

The code changes needed to get CohortMembership functioning properly.

The key of this change is twofold: first, CohortMemberships are unique
per-user, per-course. This is enforced at the database level. Secondly,
the updates are done using a select_for_update, which ensures atomicity.

These are the migrations needed for CohortMembership to function.
0005 establishes the table, 0006 will be used to move existing data
into the table as needed.

Per product guidance, we can just arbitrarily reassign problem users.
Implementing that decision, as well as the remainder of the data migration.

Also including a short script to re-run the 0006 migration after code changes
are live, to prevent a potential issue where the database become out-of-sync.

Also remove useless docstrings where they were added to keep pylint
quiet.

TNL-1268

TNL-181

JIRA: TNL-710

IMPORTANT: this commit converts the course_groups
package to using migrations.  When deploying to an
existing openedx instance, migration 0001 may fail
with an error indicating that the CourseUserGroup
table already exists.  If this happens, running
the 0001 migration first, with the --fake option,
is recommended.  After performing this step,
remaining migrations should work as expected.

for visual indent

This commit updates common/djangoapps.

These keys are now objects with a limited interface, and the particular
internal representation is managed by the data storage layer (the
modulestore).

For the LMS, there should be no outward-facing changes to the system.
The keys are, for now, a change to internal representation only. For
Studio, the new serialized form of the keys is used in urls, to allow
for further migration in the future.

Co-Author: Andy Armstrong <andya@edx.org>
Co-Author: Christina Roberts <christina@edx.org>
Co-Author: David Baumgold <db@edx.org>
Co-Author: Diana Huang <dkh@edx.org>
Co-Author: Don Mitchell <dmitchell@edx.org>
Co-Author: Julia Hansbrough <julia@edx.org>
Co-Author: Nimisha Asthagiri <nasthagiri@edx.org>
Co-Author: Sarina Canelake <sarina@edx.org>

[LMS-2370]

- Create new cohorts.py file
- moved code there from models.py
- implemented necessary functions.

Next: testing :)

- still needs style, more features, but basic functionality works

- lets user see and add cohorts
- needs styling
- needs to allow them to actually manage membership!