Samba Safety’s data science team had long been using the power of data to inform their business. They had several skilled engineers and scientists, building insightful models that improved the quality of risk analysis on their platform. The challenges faced by this team were not related to data science. Samba’s data science team needed help connecting their existing data science workflow to a continuous delivery solution.

Samba Safety’s data science team maintained several script-like artefacts as part of their development workflow. These scripts performed several tasks, including data preprocessing, feature engineering, model creation, model tuning, and model comparison and validation. These scripts were all run manually when new data arrived into their environment for training. Additionally, these scripts didn’t perform any model versioning or hosting for inference. Samba’s data science team had developed manual workarounds to promote new models to production, but this process became time-consuming and labor-intensive.

To free up Samba’s highly skilled data science teams to innovate on new ML workloads, Samba Safety needed to automate the manual tasks associated with maintaining existing models. Furthermore, the solution needed to replicate the manual workflow used by Samba’s data science team, and make decisions about proceeding based on the outcomes of these scripts.

Finally, the solution had to integrate with their existing code base. The Samba Security data science team used a code repository solution external to AWS; the final pipeline had to be intelligent enough to trigger based on updates to their code base, which was written primarily in R.

Samba Safety relied on Firemind’s expertise and knowledge to ensure the complexities of this project could be understood and remedied, using the AWS cloud.

Firemind was chosen as a preferred partner due to our ability to work within the realms of data science, whilst exploring automation and ML pipelines.

The solution for Samba Security’s data science team was built around two ML pipelines. The first ML pipeline trained a model using Samba’s custom data preprocessing, training, and testing scripts. The resulting model artifact is deployed for batch and real-time inference to model endpoints managed by Amazon SageMaker.

The second ML pipeline facilitates the inference request to the hosted model. In this way, the pipeline for training is decoupled from the pipeline for inference. One of the complexities in this project is replicating the manual steps taken by the Samba Security data scientists. The team at Firemind used AWS Step Functions and SageMaker Processing to complete this task. Step Functions allows you to run discrete tasks in AWS using AWS Lambda functions, Amazon Elastic Kubernetes Service (Amazon EKS) workers, or in this case SageMaker. SageMaker Processing allows you to define jobs that run on managed ML instances within the SageMaker ecosystem. Each run of a Step Function job maintains its own logs, run history, and details on the success or failure of the job. The team used Step Functions and SageMaker, together with Lambda, to handle the automation of training, tuning, deployment, and inference workloads.

The only remaining piece was the continuous integration of code changes to this deployment pipeline. Firemind implemented a CodeStar project that maintained a connection to Samba Security’s existing code repository. When the industrious data science team at Samba Security posts an update to a specific branch of their code base, CodeStar picks up the changes and triggers the automation.

We were able to connect Samba Safety’s existing data science workflow to a continuous and scalable delivery solution.

The use of Amazon Sagemaker and Step Functions helped realise the full benefits of an automated workflow. Ensuring the training, tuning, deployment and inference workloads could run without instance.