Source Code - ML Pipeline

These are the methods for developing with the current ML Pipeline app within the Django Rest Framework.

Constants

Constants for the ML

Building a Response Dictionary

This builds a dictionary that is published to the AntiNex Core within the MLJob’s prediction manifest. This dictionary contains how to send the results back to the core. This would allow for an environment to run many Rest APIs and reuse the same core workers.

drf_network_pipeline.pipeline.build_worker_result_node.build_worker_result_node(req=None)[source]
Parameters:req – incoming request dictionary - not used right now

Creating ML Job Stub Records for Tracking Purposes

Creates initial MLJob and MLJobResult record stub in the database

drf_network_pipeline.pipeline.create_ml_job_record.create_ml_job_record(req_data=None)[source]
Parameters:req_data – dictionary to build the MLJob and MLJobResult objects

Creating New Training Datasets

Creates an initial MLPrepare record stub in the database

drf_network_pipeline.pipeline.create_ml_prepare_record.create_ml_prepare_record(req_data=None)[source]
Parameters:req_data – dictionary to build the MLPrepare object

Process AntiNex Core Worker Results

Fills in the MLJob and MLJobResult records with the JSON response from the AntiNex Core.

drf_network_pipeline.pipeline.process_worker_results.handle_worker_results_message(body=None)[source]
Parameters:body – contents from the results
drf_network_pipeline.pipeline.process_worker_results.process_worker_results(res_node=None)[source]
Parameters:res_node – incoming request dictionary - not used right now

Celery Tasks

Celery tasks that are handled within the Django Rest API Worker when the environment variable CELERY_ENABLED is set to 1

(task)drf_network_pipeline.pipeline.tasks.task_ml_job(req_node=None)[source]
Parameters:
  • self – parent task object for bind=True
  • req_node – job utils dictionary for passing a dictionary
(task)drf_network_pipeline.pipeline.tasks.task_ml_prepare(req_node=None)[source]
Parameters:
  • self – parent task object for bind=True
  • req_node – job utils dictionary for passing a dictionary
(task)drf_network_pipeline.pipeline.tasks.task_ml_process_results(res_node=None)[source]

Core workers send results back to the REST API worker here

Parameters:
  • self – parent task object for bind=True
  • res_node – results dictionary from the core
(task)drf_network_pipeline.pipeline.tasks.task_publish_to_core(publish_node=None)[source]
Parameters:
  • self – parent task object for bind=True
  • publish_node – dictionary to send to the AntiNex Core Worker

Utility Methods

Utility methods

drf_network_pipeline.pipeline.utils.convert_to_date(value=None, format='%Y-%m-%d %H:%M:%S')[source]

param: value - datetime object param: format - string format