JSON Schema documentation with examples for all XARF v4 content types

Schema Reference

Introduction to JSON Schema Validation

XARF v4 uses JSON Schema (Draft 2020-12) to define and validate the structure of abuse reports. JSON Schema provides a powerful, standardized way to ensure that XARF reports are properly formatted and contain all required fields before processing.

Why JSON Schema?

• Automated Validation: Validate reports programmatically before accepting them
• Clear Documentation: Schema files serve as precise technical specifications
• Type Safety: Enforce correct data types for all fields
• Extensibility: Add custom fields while maintaining core compliance
• Interoperability: Standard format supported by many programming languages

Schema Validation Flow

1. Receive XARF report (JSON)
   ↓
2. Load appropriate schema file
   ↓
3. Validate report against schema
   ↓
4. If valid: Process report
   If invalid: Return validation errors

How to Validate XARF Reports

Using Python

import json
import jsonschema
from jsonschema import validate

# Load your XARF report
with open('report.json', 'r') as f:
    report = json.load(f)

# Load the master schema
with open('xarf-v4-master.json', 'r') as f:
    schema = json.load(f)

# Validate
try:
    validate(instance=report, schema=schema)
    print("Report is valid!")
except jsonschema.exceptions.ValidationError as e:
    print(f"Validation failed: {e.message}")

Using JavaScript/Node.js

const Ajv = require('ajv');
const addFormats = require('ajv-formats');

const ajv = new Ajv({ allErrors: true });
addFormats(ajv);

// Load schema and report
const schema = require('./xarf-v4-master.json');
const report = require('./report.json');

// Validate
const validate = ajv.compile(schema);
const valid = validate(report);

if (valid) {
  console.log('Report is valid!');
} else {
  console.log('Validation errors:', validate.errors);
}

Using Online Validator

Try the XARF Online Validator Tool - paste your JSON report and get instant validation feedback with detailed error messages.

Schema Files on GitHub

All XARF v4 schemas are open source and available on GitHub:

Repository: https://github.com/xarf/xarf-spec/tree/main/schemas/v4

Core Schema Files

File	Purpose	URL
xarf-v4-master.json	Master schema with type-specific validation	View on GitHub
xarf-core.json	Base schema with common fields for all reports	View on GitHub
types/content-base.json	Base schema for all content-category types	View on GitHub

Schema ID Base: https://xarf.org/schemas/v4/

Base Schema Structure

Core Fields (xarf-core.json)

All XARF v4 reports must include these base fields:

Required Fields

{
  "xarf_version": "4.0.0",
  "report_id": "550e8400-e29b-41d4-a716-446655440000",
  "timestamp": "2024-01-15T14:30:25Z",
  "reporter": {
    "org": "Example Security Org",
    "contact": "[email protected]",
    "domain": "example.com"
  },
  "sender": {
    "org": "Example Security Org",
    "contact": "[email protected]",
    "domain": "example.com"
  },
  "source_identifier": "192.0.2.1",
  "category": "connection",
  "type": "ddos"
}

Field	Type	Description
xarf_version	string	XARF version (pattern: ^4\.[0-9]+\.[0-9]+$)
report_id	string (UUID)	Unique report identifier (UUID v4 format)
timestamp	string (ISO 8601)	When the abuse incident occurred
reporter	object	Organization reporting the incident
reporter.org	string	Organization name (max 200 chars)
reporter.contact	string (email)	Contact email for follow-up
reporter.domain	string	Reporter’s domain name
source_identifier	string	IP address, domain, or identifier of abuse source
category	enum	Abuse category: connection, content, copyright, infrastructure, messaging, reputation, vulnerability
type	string	Specific abuse type within the category

Optional Common Fields

Field	Type	Description
source_port	integer	Source port (1-65535), critical for CGNAT environments
evidence_source	string	Quality indicator (e.g., spamtrap, honeypot, user_report)
evidence	array	Array of evidence items with base64-encoded payloads
tags	array	Namespaced tags (format: namespace:value)
confidence	number	Confidence score (0.0-1.0)
description	string	Human-readable description (max 1000 chars)
legacy_version	string	Original XARF version if converted from v3
_internal	object	Internal metadata - NEVER transmitted between systems

Evidence Item Structure

{
  "content_type": "message/rfc822",
  "description": "Original spam email with headers",
  "payload": "UmVjZWl2ZWQ6IGZyb20g...",
  "hash": "sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
  "size": 1024
}

Field	Type	Required	Description
content_type	string	Yes	MIME type of evidence
payload	string	Yes	Base64-encoded evidence data
description	string	No	Human-readable description (max 500 chars)
hash	string	No	Integrity hash (format: algorithm:hexvalue)
size	integer	No	Evidence size in bytes (max 5MB per item)

Content Types by Category

XARF v4 includes 32 specialized types organized into 7 categories.

1. Connection-Based Abuse (8 types)

Network-level attacks and suspicious connection patterns.

Type	Schema File	Description
login_attack	connection-login-attack.json	Brute force login attempts and authentication attacks
port_scan	connection-port-scan.json	Network port scanning and reconnaissance activities
ddos	connection-ddos.json	Distributed Denial of Service attacks
infected_host	connection-infected-host.json	Compromised systems participating in botnets
reconnaissance	connection-reconnaissance.json	Network reconnaissance and information gathering
scraping	connection-scraping.json	Automated content scraping and harvesting
sql_injection	connection-sql-injection.json	SQL injection attack attempts
vuln_scanning	connection-vulnerability-scan.json	Vulnerability scanning activities

Example: Minimal Valid DDoS Report

{
  "xarf_version": "4.0.0",
  "report_id": "ddos-789a0123-b456-78c9-d012-345678901234",
  "timestamp": "2024-01-15T16:55:42Z",
  "reporter": {
    "org": "DDoS Protection Service",
    "contact": "[email protected]",
    "domain": "protection.net"
  },
  "sender": {
    "org": "DDoS Protection Service",
    "contact": "[email protected]",
    "domain": "protection.net"
  },
  "source_identifier": "192.0.2.155",
  "source_port": 12345,
  "category": "connection",
  "type": "ddos",
  "protocol": "tcp",
  "first_seen": "2024-01-15T16:45:00Z"
}

Type-specific fields:

• protocol (required): tcp, udp, icmp, sctp
• first_seen (required): When attack was first observed
• destination_ip (optional): Target IP address
• destination_port (optional): Target port
• attack_vector (optional): e.g., syn_flood, udp_flood
• peak_pps (optional): Peak packets per second
• peak_bps (optional): Peak bits per second
• duration_seconds (optional): Attack duration

2. Content-Based Abuse (9 types)

Malicious or harmful content hosted or distributed online.

Type	Schema File	Description
phishing	content-phishing.json	Phishing websites and credential harvesting
malware	content-malware.json	Malware hosting and distribution
csam	content-csam.json	Child Sexual Abuse Material
csem	content-csem.json	Child Sexual Exploitation Material
exposed_data	content-exposed-data.json	Exposed sensitive data and information leaks
brand_infringement	content-brand_infringement.json	Brand impersonation and trademark violations
fraud	content-fraud.json	Fraudulent websites and scam content
remote_compromise	content-remote_compromise.json	Remote compromise and webshell infections
suspicious_registration	content-suspicious_registration.json	Suspicious domain registrations and threat indicators

Content-Base Schema

All content types inherit from content-base.json, which provides:

Required: url (string, URI format)

Optional Common Fields:

• domain: Fully qualified domain name
• registrar: Domain registrar
• nameservers: DNS nameservers
• dns_records: DNS evidence (A, AAAA, MX, TXT)
• screenshot_url: Screenshot evidence URL
• verified_at: When content was verified active
• verification_method: manual, automated_crawler, user_report, honeypot, threat_intelligence
• target_brand: Impersonated brand
• hosting_provider: Hosting provider name
• asn: Autonomous System Number
• country_code: ISO 3166-1 alpha-2 code
• ssl_certificate: SSL certificate details
• whois: WHOIS data

Example: Minimal Valid Phishing Report

{
  "xarf_version": "4.0.0",
  "report_id": "b2c3d4e5-f6g7-8901-bcde-f2345678901a",
  "timestamp": "2025-01-15T15:15:24Z",
  "reporter": {
    "org": "Phishing Detection Service",
    "contact": "[email protected]",
    "domain": "antiphishing.example"
  },
  "sender": {
    "org": "Phishing Detection Service",
    "contact": "[email protected]",
    "domain": "antiphishing.example"
  },
  "source_identifier": "203.0.113.45",
  "category": "content",
  "type": "phishing",
  "url": "https://secure-banking-login.example.com/auth"
}

Phishing-specific optional fields:

• credential_fields: Form fields on the page (e.g., ["username", "password"])
• phishing_kit: Known phishing kit identifier
• redirect_chain: URL redirect sequence
• submission_url: Where credentials are submitted
• cloned_site: Legitimate site being impersonated
• detection_evasion: Evasion techniques used
• lure_type: Social engineering lure (e.g., account_suspension, security_alert)

3. Copyright Violations (6 types)

Intellectual property infringement and unauthorized distribution.

Type	Schema File	Description
copyright	copyright-copyright.json	Generic copyright infringement and DMCA violations
p2p	copyright-p2p.json	Peer-to-peer copyright infringement (BitTorrent, etc.)
cyberlocker	copyright-cyberlocker.json	File hosting service copyright infringement
ugc_platform	copyright-ugc-platform.json	User-generated content platform infringement
link_site	copyright-link-site.json	Link aggregation site infringement
usenet	copyright-usenet.json	Usenet newsgroup copyright infringement

Example: Minimal Valid P2P Report

{
  "xarf_version": "4.0.0",
  "report_id": "p2p-789a1234-b567-89c0-d123-456789abcdef",
  "timestamp": "2024-01-15T18:30:45Z",
  "reporter": {
    "org": "Content Protection Agency",
    "contact": "[email protected]",
    "domain": "cpa.org"
  },
  "sender": {
    "org": "Content Protection Agency",
    "contact": "[email protected]",
    "domain": "cpa.org"
  },
  "source_identifier": "203.0.113.150",
  "source_port": 6881,
  "category": "copyright",
  "type": "p2p",
  "p2p_protocol": "bittorrent",
  "swarm_info": {
    "info_hash": "da39a3ee5e6b4b0d3255bfef95601890afd80709"
  }
}

P2P-specific fields:

• p2p_protocol (required): bittorrent, edonkey, gnutella, kademlia, other
• swarm_info (required): Must include info_hash or magnet_uri
• peer_info (optional): Peer ID, client version, upload/download amounts
• work_title (optional): Copyrighted work title
• rights_holder (optional): Copyright holder
• work_category (optional): movie, tv_show, music, software, ebook, game, etc.

4. Infrastructure Abuse (2 types)

Compromised or misused infrastructure and systems.

Type	Schema File	Description
botnet	infrastructure-botnet.json	Botnet infections and compromised systems
compromised_server	infrastructure-compromised-server.json	Compromised servers and infrastructure

Example: Minimal Valid Bot Report

{
  "xarf_version": "4.0.0",
  "report_id": "bot-123e4567-e89b-12d3-a456-426614174000",
  "timestamp": "2024-01-15T12:00:00Z",
  "reporter": {
    "org": "Botnet Tracking Service",
    "contact": "[email protected]",
    "domain": "bottracker.net"
  },
  "sender": {
    "org": "Botnet Tracking Service",
    "contact": "[email protected]",
    "domain": "bottracker.net"
  },
  "source_identifier": "192.0.2.50",
  "category": "infrastructure",
  "type": "bot",
  "compromise_evidence": "C2 communication observed to known Mirai C2 server"
}

Bot-specific fields:

• compromise_evidence (required): Evidence of compromise
• malware_family (optional): e.g., conficker, mirai, emotet
• c2_server (optional): C2 server domain or IP
• c2_protocol (optional): http, https, tcp, udp, dns, irc, p2p, custom
• bot_capabilities (optional): Array of capabilities (e.g., ddos, spam, proxy)

5. Messaging Abuse (2 types)

Spam and abuse via messaging platforms.

Type	Schema File	Description
spam	messaging-spam.json	Unsolicited commercial messages and unwanted email
bulk_messaging	messaging-bulk-messaging.json	Legitimate but unwanted bulk communications

Example: Minimal Valid Spam Report

{
  "xarf_version": "4.0.0",
  "report_id": "spam-123e4567-e89b-12d3-a456-426614174000",
  "timestamp": "2024-01-15T14:30:25Z",
  "reporter": {
    "org": "SpamCop",
    "contact": "[email protected]",
    "domain": "spamcop.net"
  },
  "sender": {
    "org": "SpamCop",
    "contact": "[email protected]",
    "domain": "spamcop.net"
  },
  "source_identifier": "192.0.2.123",
  "source_port": 25,
  "category": "messaging",
  "type": "spam",
  "protocol": "smtp",
  "smtp_from": "[email protected]"
}

Messaging-spam specific fields:

• protocol (required): smtp, sms, whatsapp, telegram, etc.
• smtp_from (required if protocol=smtp): SMTP envelope sender
• smtp_to (optional): SMTP recipient
• subject (optional): Message subject
• message_id (optional): Message ID from headers
• spam_indicators (optional): Object with detection indicators

6. Reputation & Intelligence (2 types)

Threat intelligence, blocklists, and reputation data.

Type	Schema File	Description
blocklist	reputation-blocklist.json	IP/domain blocklist inclusion reports
threat_intelligence	reputation-threat-intelligence.json	Threat intelligence and IOC reports

Example: Minimal Valid Blocklist Report

{
  "xarf_version": "4.0.0",
  "report_id": "bl-123e4567-e89b-12d3-a456-426614174000",
  "timestamp": "2024-01-15T10:00:00Z",
  "reporter": {
    "org": "Blocklist Service",
    "contact": "[email protected]",
    "domain": "blocklist.org"
  },
  "sender": {
    "org": "Blocklist Service",
    "contact": "[email protected]",
    "domain": "blocklist.org"
  },
  "source_identifier": "192.0.2.200",
  "category": "reputation",
  "type": "blocklist",
  "blocklist_name": "SpamhausSBL"
}

Blocklist-specific fields:

• blocklist_name (required): Name of the blocklist
• listing_reason (optional): Why IP/domain was listed
• listing_date (optional): When it was listed
• removal_info (optional): Delisting instructions

7. Vulnerabilities (3 types)

Security vulnerabilities and misconfigurations.

Type	Schema File	Description
cve	vulnerability-cve.json	Common Vulnerabilities and Exposures reports
open	vulnerability-open-service.json	Open services and exposed resources
misconfiguration	vulnerability-misconfiguration.json	Security misconfigurations and hardening issues

Example: Minimal Valid CVE Report

{
  "xarf_version": "4.0.0",
  "report_id": "vuln-123e4567-e89b-12d3-a456-426614174000",
  "timestamp": "2024-01-15T09:00:00Z",
  "reporter": {
    "org": "Vulnerability Scan Service",
    "contact": "[email protected]",
    "domain": "scanner.org"
  },
  "sender": {
    "org": "Vulnerability Scan Service",
    "contact": "[email protected]",
    "domain": "scanner.org"
  },
  "source_identifier": "192.0.2.75",
  "category": "vulnerability",
  "type": "cve",
  "cve_id": "CVE-2021-41773"
}

CVE-specific fields:

• cve_id (required): CVE identifier (e.g., CVE-2021-41773)
• cvss_score (optional): CVSS score (0.0-10.0)
• affected_product (optional): Vulnerable product/version
• exploit_available (optional): Whether exploit exists
• patch_available (optional): Whether patch is available

Schema Validation Tools

Command-Line Tools

Python - jsonschema

pip install jsonschema
python -m jsonschema -i report.json schema.json

Node.js - ajv-cli

npm install -g ajv-cli
ajv validate -s xarf-v4-master.json -d report.json --spec=draft2020

Go - gojsonschema

go get github.com/xeipuuv/gojsonschema

Libraries by Language

Language	Library	Link
Python	jsonschema	https://python-jsonschema.readthedocs.io/
JavaScript/Node.js	ajv	https://ajv.js.org/
Java	everit-org/json-schema	https://github.com/everit-org/json-schema
Go	gojsonschema	https://github.com/xeipuuv/gojsonschema
PHP	justinrainbow/json-schema	https://github.com/justinrainbow/json-schema
Ruby	json-schema	https://github.com/voxpupuli/json-schema
C#	Newtonsoft.Json.Schema	https://www.newtonsoft.com/jsonschema

Online Validation

XARF Online Validator: https://xarf.org/tools/validator/

Features:

• Paste JSON reports for instant validation
• Detailed error messages with field locations
• Example reports for each content type
• Schema version selection
• Export validation results

Advanced Topics

Custom Fields

XARF v4 allows custom fields via additionalProperties: true. Add organization-specific fields while maintaining schema compliance:

{
  "xarf_version": "4.0.0",
  "report_id": "...",
  "category": "messaging",
  "type": "spam",
  "my_org_ticket_id": "ABUSE-12345",
  "my_org_priority": "high",
  "my_org_analyst": "john.doe"
}

Best Practice: Prefix custom fields with your organization name to avoid conflicts.

Internal Metadata

Use the _internal field for operational metadata that should NEVER be transmitted:

{
  "xarf_version": "4.0.0",
  "report_id": "...",
  "category": "content",
  "type": "phishing",
  "_internal": {
    "ticket": "ABUSE-1234",
    "analyst": "jane.smith",
    "priority": "critical",
    "sla_deadline": "2024-01-16T14:30:00Z",
    "ml_confidence": 0.94
  }
}

Schema Versioning

XARF uses semantic versioning. The xarf_version field indicates which schema version to validate against:

• 4.0.0 - Initial XARF v4 release
• 4.1.0 - Minor enhancements, backward compatible
• 4.0.1 - Patch fixes, fully compatible

Migration: When XARF schemas are updated, existing reports remain valid if they comply with the base version’s requirements.

Validation Best Practices

1. Validate Early: Check reports immediately upon receipt before processing
2. Provide Clear Errors: Return specific validation errors to senders
3. Log Validation Failures: Track validation failures for debugging
4. Use Strict Mode: Enable all validation checks in your schema validator
5. Cache Schemas: Load schema files once and reuse the validator
6. Test Edge Cases: Validate against minimal and maximal valid reports
7. Version Check: Always verify xarf_version matches your supported versions

Common Validation Errors

Error	Cause	Fix
Missing required property ‘xarf_version’	Core field omitted	Add xarf_version: "4.0.0"
Invalid UUID format for report_id	Wrong UUID format	Use UUID v4 (e.g., 550e8400-e29b-41d4-a716-446655440000)
Invalid date-time format	Timestamp not ISO 8601	Use format: 2024-01-15T14:30:25Z
Category ‘content’ requires ‘url’ field	Type-specific field missing	Add required field based on content type
Invalid enum value for ‘category’	Typo or wrong category	Use valid category: messaging, content, copyright, etc.
source_port required when protocol=smtp	Conditional requirement not met	Add source_port field for SMTP reports

Next Steps

• Technical Specification - Complete XARF v4 technical details
• Common Fields Reference - Deep dive into core fields
• Sample Reports - Real-world example reports
• Best Practices - Guidelines for effective XARF implementation

Questions?

• GitHub Discussions: https://github.com/xarf/xarf-spec/discussions
• Schema Issues: https://github.com/xarf/xarf-spec/issues
• Specification: https://github.com/xarf/xarf-spec

---

Need Help? Join the community on GitHub to ask questions, share implementations, and contribute to the XARF specification.