Source Code - ML Pipeline
These are the methods for developing with the current ML Pipeline app within the Django Rest Framework.
Constants
Constants for the ML
Building a Response Dictionary
This builds a dictionary that is published to the AntiNex Core within the MLJob’s prediction manifest. This dictionary contains how to send the results back to the core. This would allow for an environment to run many Rest APIs and reuse the same core workers.
-
drf_network_pipeline.pipeline.build_worker_result_node.
build_worker_result_node
(req=None)[source]
Parameters: | req – incoming request dictionary - not used right now |
Creating ML Job Stub Records for Tracking Purposes
Creates initial MLJob
and MLJobResult
record stub in the database
-
drf_network_pipeline.pipeline.create_ml_job_record.
create_ml_job_record
(req_data=None)[source]
Parameters: | req_data – dictionary to build the MLJob and MLJobResult objects |
Creating New Training Datasets
Creates an initial MLPrepare
record stub in the database
-
drf_network_pipeline.pipeline.create_ml_prepare_record.
create_ml_prepare_record
(req_data=None)[source]
Parameters: | req_data – dictionary to build the MLPrepare object |
Process AntiNex Core Worker Results
Fills in the MLJob
and MLJobResult
records with the JSON response from the AntiNex Core.
-
drf_network_pipeline.pipeline.process_worker_results.
handle_worker_results_message
(body=None)[source]
Parameters: | body – contents from the results |
-
drf_network_pipeline.pipeline.process_worker_results.
process_worker_results
(res_node=None)[source]
Parameters: | res_node – incoming request dictionary - not used right now |
Celery Tasks
Celery tasks that are handled within the Django Rest API Worker when the environment variable CELERY_ENABLED
is set to 1
-
(task)
drf_network_pipeline.pipeline.tasks.
task_ml_job
(req_node=None)[source]
Parameters: |
- self – parent task object for bind=True
- req_node – job utils dictionary for passing a dictionary
|
-
(task)
drf_network_pipeline.pipeline.tasks.
task_ml_prepare
(req_node=None)[source]
Parameters: |
- self – parent task object for bind=True
- req_node – job utils dictionary for passing a dictionary
|
-
(task)
drf_network_pipeline.pipeline.tasks.
task_ml_process_results
(res_node=None)[source]
Core workers send results back to the REST API worker here
Parameters: |
- self – parent task object for bind=True
- res_node – results dictionary from the core
|
-
(task)
drf_network_pipeline.pipeline.tasks.
task_publish_to_core
(publish_node=None)[source]
Parameters: |
- self – parent task object for bind=True
- publish_node – dictionary to send to the AntiNex Core Worker
|
Utility Methods
Utility methods
-
drf_network_pipeline.pipeline.utils.
convert_to_date
(value=None, format='%Y-%m-%d %H:%M:%S')[source]
param: value - datetime object
param: format - string format