Mapping Regulatory Requirements to Policy Documents
Deep Dive Labs is developing a regulatory SaaS product. In this post, we discuss one of our core problems we are working to address in the field.
The Functional Storyline!
Adherance to Government mandated regulations is one of the core responsibilities for many organizations. For example, the Monetary Authority of Singapore (MAS) issues regulations that financial institutions, like banks, must follow. These regulations operate with the force of the law. Banks then create policy documents for their various functions, ensuring these policies align with the regulatory requirements.
The internal compliance team at a bank is responsible for ensuring that the drafted policies meet the regulatory demands. This often involves thoroughly reading and interpreting the regulation, understanding both its context and specific requirements. Additionally, some regulations include broad guidelines that need to be translated into operational procedures within the bank's policies. Deciding how to implement these guidelines is one of the challenging aspects of a compliance officer's role.
A illustrative flow for the problem statement and its final result (courtesy, Google Gemini!)
The compliance officer is seated at a desk, surrounded by stacks of papers. They are on the phone, having a heated discussion with a bank manager about implementing a new regulation. The manager is depicted as frustrated and impatient.
A lightbulb illuminates above the compliance officer's head. They have a breakthrough in understanding a complex regulatory requirement and are jotting down notes excitedly.
The compliance officer is presenting a new policy document to a group of bank executives. They are confidently explaining how the policy aligns with the regulatory requirements. The executives look impressed.
The compliance officer standing tall, surrounded by their team. They are holding a trophy labeled "Compliance Champion." The bank is celebrating their success in meeting regulatory obligations.
We explored the following problem: given a regulation and a policy document, can we identify the regulatory requirements and determine if the policy document meets these requirements?
Our solution involves two main steps:
Analyze the regulatory document to extract the stated requirements. This includes identifying mandatory requirements, soft guidelines, and categorizing them.
Compare the extracted requirements with the content of the policy document to determine if each requirement is adequately addressed.
The results of this process are then displayed in a tabular format, which is useful for auditors to review and mark off items or add comments as needed.
We began with a simple prompt:
"Analyze the attached documents and provide a table with columns labeled 'Categories,' 'Title,' and 'Detailed Requirements.'"
The Data Protection Trustmark (DPTM) is a voluntary certification for organizations to demonstrate their commitment to accountable data protection practices. This certification helps businesses gain a competitive edge and build trust with their customers and stakeholders.
Since obtaining both regulatory documents and internal policy documents for evaluation can be challenging, we turned to the Data Protection Trustmark (DPTM) in Singapore (https://www.imda.gov.sg/how-we-can-help/data-protection-trustmark-certification).
The Infocomm Media Development Authority (IMDA) has published clear certification criteria on their website, which we used in place of formal regulations. While the DPTM is not a regulation and its certification is voluntary, it served as a valuable starting point. We also used privacy policies from organizations, both certified and non-certified, as a substitute for internal policy documents.
Here is the table generated by OpenAI, which provided us with the DPTM requirements.
This completed the first step of our approach: "Analyze the regulatory document to extract the stated requirements. This includes identifying mandatory requirements, soft guidelines, and categorizing them."
For the second step, we used this prompt:
"In the DPTM Requirements table, add a column titled 'Evidence.' Analyze the attached documents and evaluate if the DPTM requirements are met according to the assertions in the document. If they are met, populate the 'Evidence' column with the relevant content from the attached documents (along with the document name) that validates the specific DPTM requirement. If the requirement is NOT met, write 'Not met' in the column."
Here is an example of a company that met the requirements according to their public privacy policy. It is important to note that certain items, like the Data Protection Impact Assessment (DPIA), are not publicly disclosed. In such cases, the model correctly noted "Not Met," which simply means that these elements were understandably absent from the public policy.
We uploaded privacy policies from various organizations, some with the certification and some without. The results were promising—not only did the model identify sections where the requirements were met, but it also pinpointed where the requirements were not met.
This marks the beginning of our journey into the Regulatory SaaS domain.
