GitHub
Home / Documentation / Technical Specification

Complete XARF v4 specification with all 32 types and field definitions

Technical Specification

Overview

XARF v4 is a comprehensive, JSON-based format for structured abuse reporting. This specification defines the complete schema, validation rules, field requirements, and all 32 types organized into 7 categories.

Key Features:

  • 32 specialized types covering modern internet abuse scenarios
  • 7 logical categories for easy classification and routing
  • Evidence-first design with cryptographic validation support
  • Real-time processing optimized for automated workflows
  • Backwards compatible with XARF v3 reports

Report Structure

Top-Level Fields

All XARF v4 reports share this common structure:

Mandatory Fields (must be present in all reports)
Field Type Format Description
xarf_version string ^4\.[0-9]+\.[0-9]+$ XARF schema version (semantic versioning)
report_id string UUID v4 Unique report identifier for tracking and deduplication
timestamp string ISO 8601 When the abuse incident occurred (UTC recommended)
reporter object - Organization that identified/complained about the abuse
sender object - Organization transmitting this report (often same as reporter)
source_identifier string IP/domain IP address, domain, or identifier of the abuse source
category string enum Abuse category (connection, content, copyright, infrastructure, messaging, reputation, vulnerability)
type string enum Specific abuse type within the category
Recommended Fields (should be included when available)
Field Type Format Description
source_port integer 1-65535 Source port number (critical for CGNAT networks)
evidence array - Structured evidence items with cryptographic validation
evidence_source string varies Quality indicator (e.g., spamtrap, honeypot, automated_scan)
confidence number 0.0-1.0 Confidence score for automated reports
Optional Fields (may be included for additional context)
Field Type Format Description
tags array namespace:value Categorization tags (e.g., malware:emotet, campaign:2024q1)
description string - Human-readable description of the abuse incident
legacy_version string “3” Original XARF version if converted from v3 format
_internal object - Internal operational metadata (never transmitted)

Reporter Object

The reporter identifies the organization that identified or complained about the abuse (the actual complainant).

Field Type Required Description
org string Yes Name of the organization that identified the abuse
contact string Yes Contact email of the complaining organization
domain string Yes Domain name of the reporting organization

Sender Object

The sender identifies the organization that is transmitting this report (the sender/infrastructure provider). In many cases, reporter and sender are the same organization, but they may differ when using reporting infrastructure providers.

Field Type Required Description
org string Yes Name of the organization sending/transmitting the report
contact string Yes Contact email for the sending organization
domain string Yes Domain name of the sending organization

When to use different reporter vs sender:

  • ISP abuse desks using third-party reporting infrastructure (e.g., ISP = reporter, Abusix = sender)
  • Brand protection companies sending reports on behalf of clients (brand = reporter, protection service = sender)
  • Anti-abuse services forwarding reports from their customers (customer = reporter, service = sender)
  • National CERTs reporting on behalf of member organizations (member = reporter, CERT = sender)

Common scenarios:

Same organization (typical):

"reporter": {
  "org": "Example Security",
  "contact": "[email protected]",
  "domain": "example.com"
},
"sender": {
  "org": "Example Security",
  "contact": "[email protected]",
  "domain": "example.com"
}

Different organizations (infrastructure provider):

"reporter": {
  "org": "Swisscom Abuse Desk",
  "contact": "[email protected]",
  "domain": "swisscom.ch"
},
"sender": {
  "org": "Abusix",
  "contact": "[email protected]",
  "domain": "abusix.com"
}

Evidence Item Object

Field Type Required Description
content_type string Yes MIME type (e.g., image/png, message/rfc822, text/plain)
payload string Yes Base64-encoded evidence data
description string Recommended Human-readable description of evidence
hash string Recommended Cryptographic hash in format algorithm:hexvalue (e.g., sha256:abc123...)
size integer Optional Size in bytes (pre-encoding)

Common Fields by Category

Different abuse categories require different context. Here are the category-specific fields:

Connection Category Fields

Mandatory: destination_ip, protocol Recommended: destination_port, attempt_count, service Optional: username, packet_count, byte_count, attack_vector

Content Category Fields

Mandatory: url Recommended: target_brand, file_hash, malware_family (when type=malware) Optional: file_size, redirect_chain, victim_count, takedown_urgency, geolocation, device, user_agent

Mandatory: work_title, rights_holder Recommended: infringing_url, work_identifier, torrent_hash Optional: file_list, swarm_size, tracker_urls

Infrastructure Category Fields

Recommended: malware_family, c2_server, first_seen, last_seen Optional: infection_vector, bot_id, compromised_service

Messaging Category Fields

Mandatory: protocol, smtp_from (when protocol=smtp), subject (when protocol=smtp) Recommended: message_id, recipient Optional: headers, body_hash

Reputation Category Fields

Mandatory: threat_type Recommended: confidence_score, first_reported, sources Optional: asn, geographic_location, threat_indicators

Vulnerability Category Fields

Mandatory: service Recommended: service_version, cve_id, cvss_score Optional: service_port, exploit_available, patch_available, cvss_vector

Content Type Categories

XARF v4 organizes its 32 types into 7 logical categories:

1. Connection-Based Abuse (8 types)

Network-level attacks and suspicious connection patterns.

Type Description
login_attack Brute force login attempts and authentication attacks
port_scan Network port scanning and reconnaissance activities
ddos Distributed Denial of Service attacks
infected_host Compromised systems participating in botnets
reconnaissance Network reconnaissance and information gathering
scraping Automated content scraping and harvesting
sql_injection SQL injection attack attempts
vuln_scanning Vulnerability scanning activities

Evidence sources: honeypot, firewall_logs, ids_detection, flow_analysis

Typical response: Immediate blocking, rate limiting, IP reputation update

2. Content-Based Abuse (9 types)

Malicious or harmful content hosted or distributed online.

Type Description
phishing Phishing websites and credential harvesting
malware Malware hosting and distribution
csam Child Sexual Abuse Material
csem Child Sexual Exploitation Material
exposed_data Exposed sensitive data and information leaks
brand_infringement Brand impersonation and trademark violations
fraud Fraudulent websites and scam content
remote_compromise Remote compromise and webshell infections
suspicious_registration Suspicious domain registrations and threat indicators

Evidence sources: crawler, user_report, automated_scan, spam_analysis

Typical response: Content takedown, domain suspension, hosting provider notification

Intellectual property infringement and unauthorized distribution.

Type Description
copyright Generic copyright infringement and DMCA violations
p2p Peer-to-peer copyright infringement (BitTorrent, etc.)
cyberlocker File hosting service copyright infringement
ugc_platform User-generated content platform infringement
link_site Link aggregation site infringement
usenet Usenet newsgroup copyright infringement

Evidence sources: automated_scan, rights_holder_report, crawler

Typical response: DMCA takedown, content removal, repeat infringer policy

4. Infrastructure Abuse (2 types)

Compromised or misused infrastructure and systems.

Type Description
botnet Botnet infections and compromised systems
compromised_server Compromised servers and infrastructure

Evidence sources: traffic_analysis, researcher_analysis, automated_discovery

Typical response: Client remediation, server hardening, service suspension

5. Messaging Abuse (2 types)

Spam and abuse via messaging platforms and channels.

Type Description
spam Unsolicited commercial messages and unwanted email
bulk_messaging Legitimate but unwanted bulk communications

Evidence sources: spamtrap, user_complaint, automated_filter, honeypot

Typical response: Email filtering, account suspension, reputation update

6. Reputation & Intelligence (2 types)

Threat intelligence, blocklists, and reputation data.

Type Description
blocklist IP/domain blocklist inclusion reports
threat_intelligence Threat intelligence and IOC reports

Evidence sources: threat_intelligence, automated_analysis, researcher_analysis

Typical response: Reputation update, threat feed integration, preventive blocking

7. Vulnerabilities (3 types)

Security vulnerabilities and misconfigurations.

Type Description
cve Common Vulnerabilities and Exposures reports
open Open services and exposed resources
misconfiguration Security misconfigurations and hardening issues

Evidence sources: vulnerability_scan, researcher_analysis, automated_discovery

Typical response: Patch notification, security advisory, configuration review

Sample JSON Reports

Example 1: Phishing Report

{
  "xarf_version": "4.0.0",
  "report_id": "550e8400-e29b-41d4-a716-446655440000",
  "timestamp": "2024-01-15T14:30:00Z",
  "category": "content",
  "type": "phishing",
  "reporter": {
    "org": "Example Bank Security",
    "contact": "[email protected]",
    "domain": "examplebank.com"
  },
  "sender": {
    "org": "Example Bank Security",
    "contact": "[email protected]",
    "domain": "examplebank.com"
  },
  "reporter_reference_id": "PHISH-2024-001",
  "source_identifier": "192.0.2.100",
  "url": "https://fake-bank-login.example.com/login",
  "target_brand": "Example Bank",
  "evidence_source": "crawler",
  "evidence": [
    {
      "content_type": "image/png",
      "description": "Screenshot of phishing page",
      "payload": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAAB...",
      "hash": "sha256:abc123def456789..."
    }
  ],
  "tags": ["phishing:banking", "severity:high"],
  "confidence": 0.95
}

Example 2: DDoS Attack Report

{
  "xarf_version": "4.0.0",
  "report_id": "123e4567-e89b-12d3-a456-426614174000",
  "timestamp": "2024-01-15T10:20:30Z",
  "category": "connection",
  "type": "ddos",
  "reporter": {
    "org": "DDoS Protection Service",
    "contact": "[email protected]",
    "domain": "ddosprotect.com"
  },
  "sender": {
    "org": "DDoS Protection Service",
    "contact": "[email protected]",
    "domain": "ddosprotect.com"
  },
  "reporter_reference_id": "DDOS-2024-789",
  "source_identifier": "198.51.100.50",
  "source_port": 12345,
  "destination_ip": "203.0.113.100",
  "destination_port": 80,
  "protocol": "tcp",
  "packet_count": 50000,
  "byte_count": 75000000,
  "evidence_source": "flow_analysis",
  "evidence": [
    {
      "content_type": "text/plain",
      "description": "Network flow data showing attack pattern",
      "payload": "VGltZXN0YW1wLCBTcmNJUCwgRHN0SUAs...",
      "hash": "sha256:def789ghi012abc345..."
    }
  ],
  "tags": ["attack:volumetric", "severity:critical"],
  "confidence": 0.98
}

Example 3: Email Spam Report

{
  "xarf_version": "4.0.0",
  "report_id": "987fcdeb-51a2-43d1-9f12-345678901234",
  "timestamp": "2024-01-15T08:45:15Z",
  "category": "messaging",
  "type": "spam",
  "reporter": {
    "org": "SpamCop",
    "contact": "[email protected]",
    "domain": "spamcop.net"
  },
  "sender": {
    "org": "SpamCop",
    "contact": "[email protected]",
    "domain": "spamcop.net"
  },
  "reporter_reference_id": "SC-2024-456",
  "source_identifier": "192.0.2.75",
  "source_port": 25,
  "protocol": "smtp",
  "smtp_from": "[email protected]",
  "subject": "Buy cheap meds online!",
  "recipient": "[email protected]",
  "evidence_source": "spamtrap",
  "evidence": [
    {
      "content_type": "message/rfc822",
      "description": "Complete spam email with headers",
      "payload": "UmVjZWl2ZWQ6IGZyb20gWzE5Mi4wLjIu...",
      "hash": "sha256:ghi345jkl678abc901..."
    }
  ],
  "tags": ["spam:pharma", "severity:medium"],
  "confidence": 0.92
}

Example 4: Malware Distribution Report

{
  "xarf_version": "4.0.0",
  "report_id": "456789ab-cdef-1234-5678-90abcdef1234",
  "timestamp": "2024-01-15T16:00:00Z",
  "category": "content",
  "type": "malware",
  "reporter": {
    "org": "Malware Analysis Lab",
    "contact": "[email protected]",
    "domain": "malware-lab.org"
  },
  "sender": {
    "org": "Malware Analysis Lab",
    "contact": "[email protected]",
    "domain": "malware-lab.org"
  },
  "reporter_reference_id": "MAL-2024-123",
  "source_identifier": "203.0.113.200",
  "url": "http://malicious-site.example/download/trojan.exe",
  "malware_family": "emotet",
  "file_hash": "sha256:5d41402abc4b2a76b9719d911017c592",
  "file_size": 245760,
  "evidence_source": "automated_scan",
  "evidence": [
    {
      "content_type": "application/octet-stream",
      "description": "Password-protected malware sample (password: infected)",
      "payload": "UEsDBBQACQAIAA...",
      "hash": "sha256:d7a8fbb307d7809469ca9abcb0082e4f8d5651e46d3cdb762d02d0bf37c9e592",
      "size": 245760
    }
  ],
  "tags": ["malware:emotet", "severity:critical"],
  "confidence": 0.99
}

{
  "xarf_version": "4.0.0",
  "report_id": "fedcba98-7654-3210-fedc-ba9876543210",
  "timestamp": "2024-01-15T12:30:00Z",
  "category": "copyright",
  "type": "p2p",
  "reporter": {
    "org": "Copyright Protection Agency",
    "contact": "[email protected]",
    "domain": "copyright-protect.org"
  },
  "sender": {
    "org": "Copyright Protection Agency",
    "contact": "[email protected]",
    "domain": "copyright-protect.org"
  },
  "reporter_reference_id": "DMCA-2024-567",
  "source_identifier": "198.51.100.150",
  "source_port": 6881,
  "work_title": "Popular Movie Title (2024)",
  "rights_holder": "Major Film Studio",
  "work_identifier": "IMDB:tt1234567",
  "torrent_hash": "info_hash:abcd1234ef567890",
  "swarm_size": 1250,
  "evidence_source": "automated_scan",
  "evidence": [
    {
      "content_type": "text/plain",
      "description": "BitTorrent peer list and metadata",
      "payload": "SW5mb0hhc2g6IGFiY2QxMjM0ZWY1Njc4...",
      "hash": "sha256:jkl901mno234abc567..."
    }
  ],
  "tags": ["copyright:film", "severity:medium"],
  "confidence": 0.94
}

Example 6: Botnet Infection Report

{
  "xarf_version": "4.0.0",
  "report_id": "321fedcb-9876-5432-10fe-dcba98765432",
  "timestamp": "2024-01-15T09:15:30Z",
  "category": "infrastructure",
  "type": "botnet",
  "reporter": {
    "org": "Botnet Research Group",
    "contact": "[email protected]",
    "domain": "botnet-watch.org"
  },
  "sender": {
    "org": "Botnet Research Group",
    "contact": "[email protected]",
    "domain": "botnet-watch.org"
  },
  "reporter_reference_id": "BOT-2024-890",
  "source_identifier": "192.0.2.250",
  "malware_family": "mirai",
  "c2_server": "evil-c2.example.com",
  "first_seen": "2024-01-10T14:20:00Z",
  "last_seen": "2024-01-15T09:00:00Z",
  "evidence_source": "traffic_analysis",
  "evidence": [
    {
      "content_type": "text/plain",
      "description": "C2 communication logs",
      "payload": "Q29ubmVjdGlvbiB0byBldmlsLWMyLmV4...",
      "hash": "sha256:mno567pqr890abc123..."
    }
  ],
  "tags": ["botnet:mirai", "severity:high"],
  "confidence": 0.88
}

Example 7: Vulnerability Report (CVE)

{
  "xarf_version": "4.0.0",
  "report_id": "789abcde-f012-3456-7890-abcdef123456",
  "timestamp": "2024-01-15T11:00:00Z",
  "category": "vulnerability",
  "type": "cve",
  "reporter": {
    "org": "Security Scanner Service",
    "contact": "[email protected]",
    "domain": "scanner-service.com"
  },
  "sender": {
    "org": "Security Scanner Service",
    "contact": "[email protected]",
    "domain": "scanner-service.com"
  },
  "reporter_reference_id": "VULN-2024-345",
  "source_identifier": "203.0.113.50",
  "service": "OpenSSL",
  "service_version": "1.0.1",
  "service_port": 443,
  "cve_id": "CVE-2014-0160",
  "cvss_score": 7.5,
  "cvss_vector": "CVSS:3.0/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:N",
  "exploit_available": true,
  "patch_available": true,
  "evidence_source": "vulnerability_scan",
  "evidence": [
    {
      "content_type": "text/plain",
      "description": "Vulnerability scan results",
      "payload": "VnVsbmVyYWJpbGl0eSBEZXRhaWxzOiBIZWFy...",
      "hash": "sha256:pqr123stu456def789..."
    }
  ],
  "tags": ["cve:CVE-2014-0160", "severity:high"],
  "confidence": 1.0
}

Evidence Handling

Supported Evidence Types

Text Evidence:

  • text/plain - Log files, configuration dumps, command output
  • text/csv - Structured data exports
  • application/json - JSON-formatted data

Message Evidence:

  • message/rfc822 - Complete email messages with headers
  • text/email - Email fragments

Image Evidence:

  • image/png - Screenshots (preferred for web content)
  • image/jpeg - Photos and screenshots
  • image/gif - Animated content

Document Evidence:

  • application/pdf - Reports and documentation
  • text/html - Webpage snapshots

Binary Evidence:

  • application/octet-stream - Malware samples, unknown binaries
  • application/zip - Archive files (must be password-protected for malware)

Evidence Size Limits

  • Per item: Maximum 5MB
  • Total per report: Maximum 15MB
  • Recommendation: Compress large text evidence before encoding

Evidence Best Practices

  1. Use cryptographic hashes - Include SHA256 hashes for all evidence
  2. Provide descriptions - Help recipients understand what they’re looking at
  3. Screenshots for web content - PNG format preferred over HTML
  4. Password-protect malware - Use password “infected” for malware archives
  5. Include full headers - For email spam, include complete RFC822 headers
  6. Base64 encode properly - No line breaks, proper padding

Validation Rules

Format Validation

  1. report_id: Must be valid UUID v4 format
  2. timestamp: Must be ISO 8601 date-time with timezone
  3. email addresses: Must be RFC 5322 compliant
  4. URLs: Must be RFC 3986 compliant
  5. IP addresses: Must be valid IPv4 or IPv6

Field Requirements

  1. All reports: Must include all mandatory fields
  2. Category-specific: Must include category-specific mandatory fields
  3. Content types: Must match pattern category-type
  4. Tags: Must follow namespace:value pattern (lowercase alphanumeric and underscore)

Evidence Validation

  1. Base64 encoding: Must be valid base64 (RFC 4648)
  2. Size limits: Individual items ≤ 5MB, total ≤ 15MB
  3. MIME types: Must be valid and match actual content
  4. Hash format: Must match algorithm:hexvalue pattern (e.g., sha256:abc123)

Validation Modes

Strict Mode:

  • Fail on unknown fields
  • Require all mandatory + recommended fields
  • Enforce strict format validation

Standard Mode (Default):

  • Require all mandatory fields
  • Warn on missing recommended fields
  • Allow unknown fields for forward compatibility

Permissive Mode:

  • Require only mandatory fields
  • No warnings for missing recommended fields
  • Suitable for gradual adoption

Version Compatibility

XARF v4 parsers should support automatic conversion of XARF v3 reports for backwards compatibility. Key differences:

XARF v3 XARF v4 Notes
Version xarf_version Format: 4.0.0
ReporterInfo reporter Restructured object with org, contact, type fields
Report.ReportType category + type Split into category and type fields
Report.SourceIp source_identifier Supports IPs and domains
Samples evidence Enhanced structure with hashes

Additional Resources

Previous: ← Introduction Next: Common Fields Reference →