This part of the documentation describes the runner package. It includes all details about the runner queue.
The celery.Celery context object.
Celery is a robust and scalable job queue. You may shift a running instance from one machine to paralleled computing platform simply by changing the backend of Celery. To learn more about this framework, you may refer to Celery Project.
The default logger for this Celery application. It is used everywhere in this package. The interface is the same as logging.Logger, so you may refer to logging for more details.
This module defines all celery.Task objects.
A Task object is a proxy for the foreground process to queue their jobs into the Celery background runner. The communications are managed by the Celery framework, and the client may not care about these details.
For example, if you want to run a long-time job in the background runner, you may decorate the job method with app.task():
@app.task
def RestartService(name):
os.system('sudo service %s restart' % name)
Then you can tell Celery to queue and run that method in your foreground process:
from tasks import RestartService
RestartService.delay('dnsmasq')
Task defined in this module are mainly used in railgun.website.codelang. You may also refer to Tasks to learn more about how to define a task, and Calling Tasks about how to call a task.
Run the given Python submission.
| Handler: | PythonHandin |
|---|---|
| Parameters: |
Run the given NetAPI submission.
| Handler: | NetApiHandin |
|---|---|
| Parameters: |
Run the given CSV data submission.
| Handler: | InputClassHandin |
|---|---|
| Parameters: |
Common pattern to run a submission. Its main function is to glue BaseHandin, BaseHost and ApiClient together.
It is guaranteed that all errors are handled and logged correctly in this method.
| Parameters: |
|
|---|
This module provides BaseHandin as the basic interface of a submission handler, as well as derived classes targeted to different submission types.
Submissions may be various in programming languages. Each programming language should have a corresponding handler, which decodes the user uploaded data (for example, extracting the archive files, or parsing the csv data), and prepares for the runner host.
The basic interface of a submission handler.
| Parameters: |
|
|---|
Run this submission and store the result. Derived classes should at least implement this.
| Returns: | A tuple of (exitcode, stdout, stderr). |
|---|
The uuid of this submission.
The programming language of this handler.
The extra options of this submission.
The original uploaded data of this submission.
CSV data submission handler, derived from BaseHandin.
| Parameters: |
|---|
NetAPI submission handler, derived from BaseHandin.
| Parameters: |
|---|
Store the user submitted url address.
Python submission handler, derived from BaseHandin.
| Parameters: |
|---|
Decode base64 file content and save to disk as a temporary file.
The file will be placed under config.TEMPORARY_DIR. If you manage the TempDiskUploadFile via with statement, then the file will be removed automatically. For example:
with TempDiskUploadFile(upload, '%s.zip' % uuid) as tempFile:
os.system('7z x "%s"' % tempeFile.path)
| Parameters: |
|---|
Whether we should ignore the system exceptions when we are trying to delete the temporary file?
Store the full path of this temporary file.
The original base64 encoded file content.
This module provides BaseHost as the basic interface of a submission handler, as well as derived classes targeted to different programming languages.
Submissions may be various in programming languages, while different languages may have different context to run. So a runner host prepares the environment, compiles and launches the submission according to its programming language.
The base interface for a runner host.
A runner host will hold a working directory under config.TEMPORARY_DIR, whose name is uuid. This directory will be automatically removed if BaseHost is managed by with statement, for example:
with BaseHost(uuid, hw, 'python') as host:
pass
| Parameters: |
|---|
Extract the given archive file into tempdir.
The FileRules defined in hw and hwcode will be validated on each file, so this method is safe.
| Parameters: | archive (Extractor) – An extractor of the submission archive file. |
|---|
Prepare the runner context by copying files from hw/code into tempdir. This method should be called before extract_handin().
Run this submission. All derived classes must implement this.
| Returns: | A tuple of (exitcode, stdout, stderr). |
|---|
Set the user and the group in host config.
The uid will be stored in config['user_id'] while the gid will be stored in config['group_id']. However, if uid is given None, all the defence based on user privileges will not take place.
| Parameters: | |
|---|---|
| Raises: | KeyError if uid or gid is a str, but does not exist in the system database. |
Spawn an external process to execute the given commands.
If the owner user of current process (runner queue) is root, and config['user_id'] != 0, the owner of tempdir will be changed to that user, and the file system mode will be changed to 0700.
| Parameters: |
|---|
The xml node of compiler parameters. You may refer to HwCode.compiler_params for more details.
The HostConfig for the process.
The xml node of runner parameters. You may refer to HwCode.runner_params for more details.
The acquired system account (name or uid).
Railgun can be configured to run multiple submissions in different system accounts simultaneously. This attribute stores the acquired system account, so that we can release it later.
If you intend to get the uid and gid of this user, access config['user_id'] and config['group_id'] instead.
The uuid of the submission.
Config values passed to hosts by environmental variables.
Prepare the environmental variables for the host process.
Values of None will be ignored, and other values will be converted into string type before being added to environ dictionary.
All the keys matching [A-Z][A-Z0-9_]* will be used directly in the new environ dictionary. All the keys in other formats will be translated into uppercase, and a prefix of RAILGUN_ will be add to the key.
For example:
>>> config = HostConfig(user_id=1, group_id=2, ignored=None,
PYTHONPATH='/usr/local/lib/python2.7')
>>> config.make_environ()
{'RAILGUN_USER_ID': '1', 'RAILGUN_GROUP_ID': '2',
'PYTHONPATH': '/usr/local/lib/python2.7'}
| Returns: | The environmental variable dictionary. |
|---|---|
| Return type: | dict |
The CSV data runner host, derived from PythonHost.
| Parameters: |
|---|
The NetAPI runner host, derived from PythonHost.
| Parameters: |
|---|
Validate the user submitted url address at compile stage.
The url address will be tested with the configured regex patterns loaded from BaseHost.compiler_params. Refer to NetAPI Judging for more details about the rules.
The Python runner host, derived from BaseHost.
This class is special, in that it does not only acts as a derived class to serve Python submission, but also acts as a base class for all Python based hosts.
For example, the InputClassHost, taking a Python script to evaluate the submitted csv data, is merely the same as a PythonHost. So it just inherits this class to implement the common functions.
| Parameters: |
|---|
The main Python script file (from BaseHost.runner_params).
Whether this host uses offline system accounts?
Store the path of Python safe runner (RAILGUN_ROOT/SafeRunner).
The timeout limit of this submission (from BaseHost.runner_params).
Railgun can be configured to run multiple submission simultaneously with different system accounts.
This module acquires the accounts from a credential server (if provided), or just uses the config.OFFLINE_USER_ID and config.ONLINE_USER_ID.
Note
Offline users will not be able to access the internet. You must use the correct type of users for different submission types.
Get a free offline system account.
| Parameters: | expires (int) – Seconds for this user to expire. |
|---|---|
| Returns: | The acquired user account name. |
| Raises: | Various Exception if the user cannot be acquired. |
Get a free online system account.
| Parameters: | expires (int) – Seconds for this user to expire. |
|---|---|
| Returns: | The acquired user account name. |
| Raises: | Various Exception if the user cannot be acquired. |
This module provides the pre-defined errors that holds the messages to describe what errors a submission produces.
The submission archive contains too many files. You may refer to extract_handin() of BaseHost to see more details.
The submitted archive file is in bad format. This error may also be raised if the user uploads an archive file with wrong extension in file name.
The system cannot extract submission archive into working directory. You may refer to extract_handin() of BaseHost to see more details.
There exist some entity that is denied by hw.xml or code.xml in the submitted archive file.
| Parameters: | fname (str) – The name of the denied entity. |
|---|
The server has come across some technique error.
Since run_handin() is wrapped with a large try-catch, all the un-expected errors can be catched. This error is then generated and reported to user.
The submission language doesn’t belong to corresponding homework.
| Parameters: | lang (str) – The provided programming language. |
|---|
The URL address submitted by user is rejected by regex patterns defined in code.xml. You may refer to compile() of NetApiHost to see more details.
The runner host produces invalid UTF-8 sequence. You should tell the students to encode their source code in UTF-8.
The base class for all submission errors.
| Parameters: |
|
|---|
The file permission of runner is not configured correctly.
The runner will check the file system permissions to guarantee that the execution environment is safe if runconfig.RUNNER_CHECK_PERM is set to True. If the validation does not pass, all the new submissions will be rejected with this error.
The system cannot copy runtime files into working directory. You may refer to prepare_hwcode() of BaseHost to see more details.
The API client for runner to communicate with website.
The runner queue and the runner host should report the results of submissions via website api. Refer to Website API for more details.
| Parameters: | baseurl (str) – The base url of website api. |
|---|
Send payload to remote server and execute given action.
| Parameters: | |
|---|---|
| Returns: | The requests.Response object. |
Store the process exitcode, standard output and standard error output of the submission.
| Parameters: |
|---|
Change the status of submission to Running.
| Parameters: | handid (str) – The uuid of the submission. |
|---|
Store the base url of website api.
Store the secret communication key.
Load the secret key from keys/commKey.txt.
| Returns: | The secret key to encrypt and decrypt API post data. |
|---|
Shortcut to report the error of a submission.
| Parameters: |
|
|---|