Demystifying User Retirement in the Open edX platform

This article was written by team member Kaustav Banerjee.

User Retirement in an Open edX platform is the very last stage of the user account’s lifecycle, which begins with registration of a new user. That is, it is the thing that happens when a user clicks on the “Delete My Account” button on the Account Settings page.

However, it is not a single step, but a series of steps resulting in deactivation of the user account, removal of the user’s Personally Identifiable Information (PII) from all services making up the Open edX platform, and culminating in permanently disabling the username and email address from being reused ever again.

In this blog, we will explore how the user retirement workflow is designed, the steps involved in user retirement, different ways of triggering user retirement, and how to enable reuse of a retired username and email address.

User Retirement Pipeline

The Open edX platform is not a single application, but a collection of services and IDAs — each handling a specific part of the user experience, such as course-discovery, discussion forum, eCommerce, etc.  Each of these services maintains relevant user data to correctly handle users’ requests.

This makes the process of deleting a user from the platform complicated. Merely deactivating the user account and removing their credentials is not sufficient. User data also needs to be deleted from each of these services for the platform to be compliant with regulatory frameworks such as GDPR.

To make things more complicated, running some of these services on the platform is completely optional, with the possibility of installing custom services as well. Therefore, the retirement process needs to be configurable so that it attempts to delete user data only from services which are actually being used in the platform instance.

To address all this, the retirement workflow is designed as a linear pipeline, where the user deletion request moves through a series of fully configurable States.

User Retirement States

When a user retirement is initiated, an entry is created in the UserRetirementStatus model, with the state set to PENDING. This is the very first state of the Retirement Pipeline.

In this step, hash values of the username and email address respectively are generated and stored in the UserRetirementStatus model along with the original username and email address. The email address in the User model is replaced with its hash value and the password is reset. This prevents the user from logging into the user account.

At this PENDING state, it is possible for the site administrator to cancel the retirement workflow if so requested by the user. This can be done either by using the cancel_user_retirement_request management command or by clicking on the ‘Cancel’ action button in Django admin. This restores the email address in the User model and deletes the entry from UserRetirementStatus. Cancellation of retirement workflow is not possible in any other retirement state.

The user then moves through a series of unique pairs of retirement states, each indicating that a stage or action of the retirement workflow is either underway or has been completed.

RETIRING_FORUM and FORUM_COMPLETE is one such retirement state pair. Here the RETIRING_FORUM state indicates that the forum retirement API has been triggered and deletion of user data from discussion forum service is underway and the FORUM_COMPLETE state indicates that the API has been successfully executed.

Any number of such pairs of retirement states can be configured and in any order, however each RETIRING_SERVICE state should always be succeeded by the corresponding SERVICE_COMPLETE state.

The COMPLETED, ERRORED, and ABORTED are the dead end states. That means the retirement workflow cannot proceed any further from any of these states. The dead end states, along with the PENDING state need to be mandatorily configured with all other states being optional.

The COMPLETED state is the very last state of the retirement workflow, indicating that all configured stages of the retirement process have been successfully completed.

At this state, the username is replaced in the User model with the hash values generated previously. However, the entry in the UserRetirementStatus model still contains a mapping between the hashed username/email address and the actual username/email address of the user. The retirement_cleanup API can be used to delete this entry, thereby completely removing the user’s original username/email from the platform.

The retirement workflow moves to the ERRORED state for one of two reasons — either there is an execution error during one of the retirement stages, or there is a discrepancy in the retirement states (such as the user moving from one state to a previous state).

A site admin can look into the cause of the error, fix it, and then manually move the user to a correct state, which would enable the resumption of  the retirement process.

The default list of the retirement states is defined in lms/env/common.py

This list can be configured by setting the RETIREMENT_STATES variable in /edx/etc/lms.yml
The active Retirement States are maintained using the RetirementState model.

Each state has an Execution Order Number, which determines the order of the states in the workflow. Any violation of this order of execution during the retirement process will lead to an error with the user moved to the ERRORED state.

The populate_retirement_states management command can be used to populate the RetirementState model or override the states which are already configured.

There is currently no out-of-the-box mechanism in Open edX to drive this retirement workflow.

Therefore this needs to be driven from an external service, or the tubular scripts can be used to either trigger the workflow execution manually or set up as a cron job to execute workflow at regular intervals.

Initiating User Retirement

There are a number of ways in which user retirement can be initiated:

1. Users can initiate user retirement by clicking on the “Delete My Account” button on the Account Settings page in Account MFE. This in turn calls the DeactivateLogoutView API to trigger the retirement.
2. Retirement can be initiated for multiple users simultaneously from an external service using the bulk_retire_users API.
3. A site admin can trigger retirement on behalf of a user, using the retire_user management command.

Enable Reuse of Retired username and email address

As mentioned previously, the generated hash value of the username and email address, is used to replace the corresponding original credentials in the User model at different stages of the User Retirement Workflow.

As part of the validation process during user registration, the hash value of the new username and email address is calculated and queried against the User model to check if those hash values are already present. If so, the validation fails and the user is requested to choose a different username and email address. This is how the platform ensures that a retired username and/or email address is never reused.

There may be certain use cases, however, where the reuse of the retired user credentials would be desirable or even necessary. We here at OpenCraft encountered one such use case for one of our clients. This client used a third party external service for user management. That means, the client’s users would register, manage and deactivate/delete their accounts in a third party UI, and that service would use the correct APIs of the Open edX platform in turn to register and/or retire the user accounts in the platform. Now, using the third party service, it was possible for the client’s users to delete an account and then create another account using the same credentials. However, the user registration API in the Open edX platform would throw an error when using the retired credentials. This was a big problem for our client.

We looked for an existing solution to this problem and, not having found anything, we set out to design a solution for this problem ourselves.

There were a few considerations during the design of the solution:

1. There must be other members of the Open edX community who share this requirement of reuse of retired user credentials for their different use cases. So a one-off code change for our client would not be sufficient. Our solution must be usable for the wider Open edX community.
2. The user APIs are a very core part of the Open edX platform, with almost all parts of the platform being affected by them. Therefore, any changes to the user APIs would not only require thorough regression testing, it could also be months — if not years — before they are fully reviewed, accepted, and merged with the upstream codebase. Even then, the changes would only be available in the subsequent stable release for regular platform users. We thus needed a solution that could be easily configured with existing installations and be available for use by the community right away.

This led us to create the enable_retired_username_email django app.

This app is designed to be installed along with the Open edX installation using pip:

pip install -e git+https://github.com/open-craft/enable-retired-username-email.git@v1.0#egg=enable-retired-username-email

Or by setting the EDXAPP_PRIVATE_REQUIREMENTS variable while installing the Open edX platform using ansible.

This app receives the post_save signal from the UserRetirementStatus model, and does one of two things:

1. If an entry has just been created, it replaces the hashed username with a friendly name in the form of deleted_user_<user_id>. This is done so that during the retirement workflow, all services which replace the original username with the hashed username use this friendly name instead.
2. If the entry is in the COMPLETE state, a hash of the string <email_id>+<current_timestamp> is calculated. This email address in the corresponding User model, which now contains the hash of the original email address, is replaced with this new hash value. The entry in the UserRetirementStatus model is deleted, thereby removing all traces of the original credentials and its corresponding hash values.

Voilà! We hope this is useful, and let us know via comments below or at contact [@] opencraft.com if you have any comments or questions!

Photo by Tandem X Visuals on Unsplash