Compliance Map: Connecting Regulations and Frameworks for Smart Audits - NODES 2024
CONTEXT
Earlier this year, we Divya and I presented our work titled “TRACE- A Framework to Assist Auditors in Evaluating Regulatory Compliance” at the Computational Legal Studies workshop at Singapore Management University. You can read about
the technical background in our earlier blog post here
our impressions from the workshop here, and
the presentation here
In TRACE, we mapped Regulatory Documents to Enterprise Policy Documents. In this work, presented at the NODES 2024 online event, we continued that work to map Regulatory Requirements to Control Frameworks. The Compliance Map is the complete mapping from Regulations to Control Frameworks to Enterprise Policy Documents.
Check out the code and the presentation in our Github: NODES24: Code Repository corresponding to Compliance Map (talk in NODES 2024)
Compliance processes often involve navigating intricate regulations and aligning them with established control frameworks, a task that can be daunting for compliance officers and auditors. Our ongoing work in this area seeks to make compliance mapping simpler and efficient, leading to reduced tedious manual work from compliance officers and auditors. The compliance landscape operates within a structured ecosystem involving three key stakeholders:
Regulators
Regulatory bodies like the Monetary Authority of Singapore (MAS) and the Infocomm Media Development Authority (IMDA) publish mandatory requirements for enterprises. These regulations ensure that businesses operate within legal and ethical boundaries.
Example:
The Data Protection Trustmark (DPTM) is a certification framework developed by the Infocomm Media Development Authority (IMDA) in Singapore to help organizations demonstrate accountable data protection practices in compliance with Singapore’s Personal Data Protection Act (PDPA).
Non-Regulatory Agencies
Organizations such as NIST (National Institute of Standards and Technology) create control frameworks like the NIST Privacy Framework. These frameworks provide guidance and standards for organizations to achieve compliance with regulatory requirements.
Example:
The NIST Privacy Framework is a voluntary tool developed by the National Institute of Standards and Technology (NIST) to help organizations manage privacy risks.
Enterprises
Businesses and organizations implement internal policies, processes, and systems to meet regulatory obligations. They generate evidence documents to demonstrate their adherence to these requirements during audits or certifications.
Example:
Enterprise evidence documents are records submitted during audits to prove compliance with regulations, standards, or internal policies.
However, connecting the regulations from regulatory bodies to the frameworks published by non-regulatory agencies is a challenge. This alignment can simplify audits, reduce redundant work, and enhance clarity for compliance teams.
Document Mapping: The Challenges
Mapping regulations to frameworks involves extracting structured data from regulatory documents and linking similar requirements in the Control framework. The process is not straightforward due to several challenges:
Format Variability: Regulatory documents are often published in diverse formats, such as PDFs or spreadsheets, each with its own structure, some prescriptive like the RMiT guidelines from Malaysia, and while others like the MAS TRM guidelines are principles based.
Information Hierarchies: Frameworks and regulations may use different hierarchies of information. For instance, the DPTM checklist is structured around principles and sub-requirements, while the NIST Privacy Framework uses a hierarchy of functions, categories, and subcategories.
Stakeholder Relevance: Requirements often apply to different stakeholders, such as internal teams or third-party vendors, requiring precise mapping for relevance.
Our approach leverages LLMs to address these complexities and ensure accurate mapping.
Document Mapping: The Process
We use the DPTM (Regulatory document) and the NIST Privacy Framework (Control Framework) as examples for illustration.
The DPTM checklist, published by IMDA, includes principles, sub-requirements, and stakeholder-specific details. It is relatively structured and self-explanatory.
The NIST Privacy Framework, on the other hand, provides a hierarchy of functions, categories, and subcategories. Its requirements often rely on contextual information derived from the broader framework.
Contextualized requirements play a crucial role in improving the accuracy and relevance of compliance mappings. While regulatory descriptions like those in the DPTM checklist are often self-explanatory, framework components, such as those in the NIST Privacy Framework, require additional context from their broader structure. By combining elements like functional objectives and categories with subcategories, a richer, more meaningful representation of each requirement is achieved. This approach not only reduces ambiguities but also ensures that mappings align more closely with organizational needs, ultimately enhancing the effectiveness of compliance efforts. Contextualized requirements are derived from the approach published by Anthropic here.
Using LLMs, we extract structured information from regulations and frameworks. For example, we process the DPTM checklist to extract regulatory requirements and the NIST Privacy Framework to identify functional objectives and contextualized descriptions. Embeddings are generated for each requirement to capture semantic meaning, enabling accurate similarity mapping.
Each regulatory requirement is compared against the framework components using cosine similarity. The top five most relevant framework requirements are identified and linked to the regulatory requirement.
In this case, the DPTM checklist, which has 50 requirements, is mapped to relevant components within the NIST Privacy Framework, which contains 100 requirements.
We store the mapped data in a graph database, allowing for a clear visualization of relationships between regulatory requirements and framework components. This visual representation helps compliance teams quickly identify areas of focus.
Key Insights from Mapping
Commonly Mapped Requirements: Certain requirements emerge as high-priority due to their relevance across multiple regulations and frameworks. For example:
Legal, regulatory, and contractual obligations often form the foundation of compliance efforts.
Policies and processes related to data privacy and governance frequently overlap between regulations and frameworks.
Unmapped Framework Components: Not all framework components are mapped to regulatory requirements. For instance, some NIST requirements, such as those addressing organizational priorities or resilience mechanisms, do not directly align with the DPTM checklist.
A Graph-Based Compliance Map
The graph-based compliance map we developed offers a comprehensive view of how regulatory requirements connect to framework components. Here’s what it enables:
Visualization of Relationships: Each regulatory requirement is linked to its relevant framework components, making it easier to understand overlaps and gaps.
Resource Allocation: Compliance teams can prioritize evidence creation for requirements that map to multiple framework components.
Gap Analysis: Identifying unmapped framework components helps organizations understand areas not covered by current regulations.
Future Directions
Our work in compliance mapping is far from complete. We aim to expand this approach to address unstructured regulations and frameworks. Upcoming initiatives include:
Mapping MAS TRM (Technology Risk Management) guidelines to the NIST 800-53 framework.
Incorporating stakeholder-specific filters to tailor mappings for internal and external teams.
Exploring new embedding techniques to enhance mapping precision further.
If compliance is a focus area for you, we invite you to join us as beta testers. Your feedback will play a vital role in shaping the future of compliance solutions. You can also connect with us on LinkedIn for updates and collaboration opportunities.
