Source Code - Job Helpers

These are the helper methods for abstracting celery calls from the Django REST Framework Serializers. These are optional for most users, I just find them helpful because the serializers all examine a common dictionary structure instead of custom ones all over the code. The response structure is:

task_response_node = {
    "status": status,
    "err": err,
    "task_name": task_name,
    "data": data,
    "celery_enabled": celery_enabled,
    "use_cache": use_cache,
    "cache_key": cache_key
}

  1. status will be a const value from the drf_network_pipeline.pipeline.consts

    Response Status Codes

    SUCCESS = 0
FAILED = 1
ERR = 2
EX = 3
NOTRUN = 4
INVALID = 5
NOTDONE = 6

  2. err will be an empty string on SUCCESS and not-empty if there was a problem

  3. data is the result from the Celery worker (if it was used instead of python manage.py runserver 0.0.0.0:8010)

  4. use_cache is a flag meaning the results ere also cached in the cache_key for django-cacheops to use (this is not supported yet)

  5. task_name is a human readable task label for debugging in the logs

Build Task Request

drf_network_pipeline.job_utils.build_task_request.build_task_request(status=4, err='not-set', task_name='', data=None, job_id=None, celery_enabled=False, use_cache=False, cache_record=False, cache_key=None)[source]

build_task_node

Builds a common request dictionary for all Celery tasks being wrapped with the utils framework

Parameters:
  • status – task return status code
  • err – task error message for debugging
  • task_name – task label for debugging
  • data – task return data
  • job_id – task job id
  • celery_enabled – control flag for testing celery tasks
  • use_cache – use the cached record if available
  • cache_record – cache the result in redis after done
  • cache_key – cache the result in this redis key

Build Task Response

drf_network_pipeline.job_utils.build_task_response.build_task_response(status=4, err='not-set', task_name='', data=None, celery_enabled=False, use_cache=False, cache_key=None)[source]

Builds a common response dictionary for all Celery tasks being wrapped with the utils framework

Parameters:
  • status – task return status code
  • err – task error message for debugging
  • task_name – task label for debugging
  • data – task return data
  • celery_enabled – control flag for testing celery tasks
  • use_cache – use the cached record if available
  • cache_key – cache the result in this redis key

Handle Task Method

drf_network_pipeline.job_utils.handle_task_method.handle_task_method(req_node=None, task_method=None, get_result=False, delay_timeout=1.0)[source]

Wraps task invocation for easier debugging with a standardized dictionary status, error, data response

Parameters:
  • req_node – request tracking data
  • task_method – task method to run
  • get_result – get the result from task
  • delay_timeout – timeout in seconds to wait

Run Task

drf_network_pipeline.job_utils.run_task.run_task(task_method=None, task_name='please-set-name', req_data=None, get_result=False, delay_timeout=1.0, use_cache=False, cache_record=False, cache_key=None)[source]

Handles Celery sync/async task processing

Parameters:
  • task_method – requested method
  • task_name – name of the task for logging
  • req_data – requested data
  • get_result – get the result from task
  • delay_timeout – seconds to wait for the task to finish
  • use_cache – use the cached record if available
  • cache_record – cache the result in redis after done
  • cache_key – cache the result in this redis key