Automated Discovery Of Audit Reports (audits.json)

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶

This Internet-Draft will expire on 8 June 2026.¶

Copyright Notice

Copyright (c) 2025 IETF Trust and the persons identified as the document authors. All rights reserved.¶

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶

Table of Contents

1. Introduction

An increasing number of regulatory regimes require organizations involved in certain business practices to provide a degree of transparency in their business operations by posting reports of audits in an accessible location. For example, New York City's 2021 Local Law 144 ([NYC-LL144]) established auditing requirements for employers who use automated decision-making tools in the employment process, but it has proved challenging to even find these audits effectively (see [Auditing-the-Audits]).¶

For a business that has a website, the natural place to provide access to the audit is on that website. This document describes a standard mechanism that can be used to point to any audit posted associated with regulatory compliance.¶

The mechanism is an audits.json summary document, found at a well-known URL, which allows the hosting domain to refer to specific business operations, relevant regulatory regimes, and their associated audit reports.¶

Adoption of this mechanism should make it easier for businesses to comply with these requirements, while also making it easier for researchers, analysts, and regulators to assess compliance and evaluate the overall effectiveness of the regulations.¶

2. Locating The audits.json Summary

This specification uses the .well-known URL space defined by [RFC8615].¶

A given domain hosts the audits.json summary in /.well-known/audits.json within the website operated by the organization.¶

If Example Corporation operats https://example.com/, then the summary report would be found at https://example.com/.well-known/audits.json.¶

3. audits.json Structure

The object served from the URL described in Section 2 will have Content-Type: application/json, and will consist of a single dictionary object with at least two top-level keys: operations and audits.¶

The overall object relationship looks like this:¶

4. IANA Considerations

IANA should register audits.json in the "Well-Known URIs" registry, with the following values:¶

• URI suffix: audits.json¶

• Change controller: IETF (is this the right choice, if this is an informational draft? is there a better group to control updates to this specification?)¶

• Specification document(s): This document¶

• Status: provisional (unless we see wider adoption, in which case we should ask for permanent)¶

• Related information: Any other examples we want to point to?¶

Appendix A. Test Vectors

{
  "operations": [
    {
      "urls": [
        "https://example.com/jobs/sysadmin",
        "https://example.com/jobs/clerk"
      ],
      "regs": [
        "https://codelibrary.amlegal.com/codes/newyorkcity/latest/NYCrules/0-0-0-138391"
      ],
      "audits": [ "fizz", "sparkle" ]
    },
    {
      "urls": [
        "https://example.com/jobs/cfo",
        "https://example.com/jobs/cto"
      ],
      "regs": [
        "https://codelibrary.amlegal.com/codes/newyorkcity/latest/NYCrules/0-0-0-138391"
      ],
      "audits": [ "fizz" ]
    }
  ],
  "audits": {
    "fizz": {
      "title": "Fizz Resume Prioritization Service",
      "urls": [
        "https://fizz.example/bias_audits/aedt_audit_2024.pdf",
        "https://fizz.example/bias_audits/aedt_2024_data.tgz"
      ],
      "date": "2024-12-13"
    },
    "sparkle": {
      "title": "Sparkle Video Analytics Assessment",
      "urls": [
        "https://sparkle.example/video_analytics_2023_audit.pdf"
      ],
      "date": "2024-01-15"
    }
  }
}

Appendix B. JSON Schema For audits.json

The following JSON Schema (see [I-D.bhutton-json-schema-01]) can be used to validate an audits.json summary file.¶

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "http://ietf.org/json_schemas/audit_schema.json",
  "title": "Audit Report Summary",
  "description": "Automated Discovery of Audit Reports",
  "type": "object",
  "properties": {
    "operations": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "urls": {
            "type": "array",
            "items": {
              "type": "string",
              "format": "uri"
            },
            "uniqueItems": true,
            "minItems": 1
          },
          "regs": {
            "type": "array",
            "items": {
              "type": "string",
              "format": "uri"
            },
            "uniqueItems": true,
            "minItems": 1
          },
          "audits": {
            "type": "array",
            "items": {
              "type": "string"
            },
            "description": "property of top-level 'audits' object",
            "uniqueItems": true,
            "minItems": 1
          }
        },
        "required": [
          "urls",
          "regs",
          "audits"
        ]
      }
    },
    "audits": {
      "type": "object",
      "additionalProperties": {
        "type": "object",
        "properties": {
          "date": {
            "type": "string",
            "format": "date"
          },
          "title": {
            "type": "string"
          },
          "urls": {
            "type": "array",
            "items": {
              "type": "string",
              "format": "uri"
            },
            "uniqueItems": true,
            "minItems": 1
          }
        },
        "required": [
          "date",
          "title",
          "urls"
        ]
      }
    }
  },
  "required": [
    "operations",
    "audits"
  ]
}

For example, you can validate this using with [JV]:¶

jv audits-schema.json audits.json ¶

Note that JSON Schema cannot represent foreign key constraints, so the fact that each /operations/*/audits must reference a named member of /audits needs to be enforced separately.¶

Appendix C. Examples of Regulatory Regimes

Please propose more pointers for this subsection!¶

• [NYC-LL144] established requirements for employers in New York City to publish audits of automated decision-making tools used for hiring¶

• Colorado's SB 205 [CO-SB205] requires developers of some artificial intelligence systems to publish reports about the design and deployment of such systems on their websites¶

Acknowledgements

Document History

This section is to be removed before publishing as an RFC.¶

Authors' Addresses