Oxford Immune’s product, Algocyte®, continuously learns from the user’s health behaviour over time, building a personalised profile baseline.

This process requires a complex workflow which introduces a number of challenges. Specifically, Oxford Immune needed to isolate the best services suited for hosting a number of extraction and classification models. The CICD needed to be deployable across multiple regions whilst also having an aggregation classifier that could merge any and all results into JSON output files.

Oxford Immune had already worked with us on a similar, but less technical project, in which we established an automated white blood cell execution flow.

They were happy with the success of the previous project and wanted to use firemind’s expertise in the more technical automation of their white blood cell execution flow.

Firemind acted as both a consultancy and development agency to deliver the execution flow and the CICD flow required to manage the regional and environmental deployment of the solution.

The execution flow being delivered in scope with this project phase is for extraction and classification, specifically using Amazon Sagemaker Batch. Batch was used as the compute layer to host and process the models provided in the previous project phase, Code Review.

The SageMaker batch processes could be triggered as frequently as necessary, with an AWS Lambda integrated into the architecture to be used as the flow trigger. The pre-processing from the RBC workflow is based on images being part of an order. The desired output of the WBC Workflow is an aggregated result based on the analysis of all images in an order (an ideal scenario for using Sagemaker Batch). When the images (in an order) are ready, the WBC workflow can be triggered. This means each batch can run as fast and frequently as needed, as an order image set can be produced.

The execution flow would be used for the extraction and classification tasks of the WBC execution flow. They will write and read to an Amazon DynamoDB table, and the results will be aggregated and written out as JSON to an Amazon S3 bucket location.

The three extraction types currently defined in the code provided would be executed by a parallel state in the AWS Step Functions that allow each type to be run concurrently. The default execution would run all extraction types, for both current and future additions, although using the parameters in the triggering Lambda, it would be possible to specify the execution of a filtered selection of types.

Similar to the extraction execution, two classifiers would be executed in a parallel state in the Step Function. This will also run all classifications by default, and support the selection/filtering to run specific classification types based on the parameters set in the triggering Lambda function.

Additionally, the delivery of this solution included the CICD pipeline to deploy to all of the environments (Dev, Testing, Prod), plus any region where services are supported.

A multi-environment cloud workflow allowed for the separation of different stages of development, such as development, testing, and production environments. This isolation helped enhance security by reducing the risk of unauthorised access or changes to critical production systems. It enabled tighter control over data, access privileges, and configurations, minimising the impact of potential security breaches or errors.

Automating CI/CD pipelines in the cloud ensures consistent and repeatable processes for building, testing, and deploying applications. It helped enforce coding standards, conduct comprehensive unit and integration tests, and perform automated regression testing. This improved the software quality, increased reliability, and reduced the likelihood of introducing bugs or performance issues into production environments.

Cloud-based CI/CD automation facilitated collaboration between development, testing, and operations teams. It provided a centralised platform where team members could access and monitor the progress of different pipeline stages, view test results, and track deployments. This enhanced visibility, promotes communication, and enables teams to identify and resolve issues quickly.