AWS Only
Revitalising a Custom-Built NLP Solution With Generative AI for Archer Technologies LLC
Firemind developed a generative AI solution that identifies differences between compliance obligations and defined control standards, across thousands of data inputs. Our PoC focused on SEC regulations to start, prior to a production ready state. The application of ML to this problem is expected to provide a novel capability to Archer’s core platform whilst using generative AI to differentiate Archer’s compliance and risk management offering.
AWS ONLY ·
Revitalising a Custom-Built NLP Solution With Generative AI for Archer Technologies LLC
Firemind developed a generative AI solution that identifies differences between compliance obligations and defined control standards, across thousands of data inputs. Our PoC focused on SEC regulations to start, prior to a production ready state. The application of ML to this problem is expected to provide a novel capability to Archer’s core platform whilst using generative AI to differentiate Archer’s compliance and risk management offering.
At a glance
Archer Technologies LLC help organisations manage risk in the digital era, uniting stakeholders, integrating technologies and transforming risk into reward.
Challenge
Creating a generative AI solution that addresses the core issues with the prior solution, including document ingestion, change management and reduced false positives.
Solution
Develop a generative AI solution that identifies the differences between compliance obligations and defined control standards.
Services Used
Amazon Bedrock
AWS Lambda
Amazon S3
Outcomes
15x faster regulation sorting
100% compliance accuracy
5x reduction in human error
Business challenges
Revitalising their previous ML solution
In 2020, Archer developed a custom ML solution that automatically processed regulatory changes, making it easier for customers to keep control standards up to date.
External factors caused Archer to de-prioritise the project and focus on core business priorities. Today, Archer customers are requesting this solution to available once again.
Since the initial solution was developed, NLP techniques have advanced considerably, thanks to the capabilities of generative AI. As a result, Archer would like to revisit the core capabilities of this solution and replace their custom-built NLP solution with generative AI.
“Firemind delivered beyond our expectations, ensuring a robust, Generative AI focused infrastructure, that enabled super fast classification and compliance checks. We couldn't be happier with the outcome and look forward to pushing towards a production state.”
Julian Hirsch, Senior Engineer - Archer Technologies LLC
Solution
Building a PoC that redefines risk classification
We needed to allow Archer Technologies LLC to send documents via an AWS hosted user interface, to then be inferred by a process utilising LLMs on Amazon SageMaker and Amazon Bedrock. The output of the inference would then be stored on Amazon S3 for Archer Technologies LLC to extract RAG (red, amber, green) information for each submission.
There were 5 main stages to this project:
1. Firemind worked with Archer Technologies LLC to define a number of prompts for up to 10 controls. Archer Technologies LLC could then use this template to update further controls, ready for inference by the LLM. This will be in the format of a single column CSV file.
2. Archer Technologies LLC could then upload a zip file directly into an Amazon S3 bucket, using either the APIs, CLI, or AWS console provided.
3. AWS Lambda could then unzip the files, triggering step function processes.
4. The step functions could then clean the data, splitting compliance rows into chunks, running mapping methods and extracting defining information from the output. This information could then be mapped to an output CSV file, providing a RAG status, an explanation of the SOP’s compliance level and an explanation of why the LLM provided each RAG status.
5. Finally, the output could be saved in Amazon S3, where Archer Technologies LLC can access it via the APIs, CLI or AWS console provided.
High level UX
This project needed not only a robust architecture, but a reliable and easy to use front-end design. Our design team worked alongside Archer Technologies LLC's design department to make a solution that will simple to use and effective in it's use of prompts and data controls.
Faster compliance
Significant speed increase when looking at compliance level in RAG statuses, when compared to human resource.
Reducing human error
The PoC showed a quick advancement in accuracy and correct classification. Ensuring significant savings by automating the time needed to complete the checks.
Model Spotlight
Claude-v1
We selected Claude-v1 for this project because of its natural language processing capabilities, which aligned perfectly with Archer’s goal of automating and accelerating the regulation sorting process.
By leveraging Claude-v1, Archer was able to achieve a 15x faster regulation sorting rate, allowing for quicker and more efficient compliance checks.
The model’s precision played a crucial role in ensuring 100% compliance accuracy, which was essential for maintaining regulatory standards.
Furthermore its ability to consistently deliver accurate results led to a 5x reduction in human error, significantly improving the overall reliability of the process.
Why Firemind
Archer Technologies LLC and Firemind were brought together by Amazon Web Services (AWS) as a designated delivery partner of the AWS ML Fast Start Program. As an outcome from the ML Fast Start Program, Archer Technologies LLC had defined a need to develop a generative AI solution that identifies differences between compliance obligations and defined control standards.
As specialists in ML and generative AI, Firemind was the best choice of partner.
15x
Faster regulation sorting
100%
Compliance accuracy
Added value
Additionally, the 5x reduction in human error showcased the solution’s effectiveness in streamlining the customer’s operations and enhancing their overall compliance management capabilities.
Get in touch
Want to learn more?
Seen a specific case study or insight and want to learn more? Or thinking about your next project? Drop us a message below!