Supported EDM Data Set Formats

• Enterprise DLP supports a header row in an EDM dataset.
• Enterprise DLP supports datasets in CSV and TSV formats.
  Palo Alto Networks recommends that datasets in CSV format adhere to the RFC-4180 standard.
• Palo Alto Networks recommends Atomic columns to ensure accurate matching of sensitive data.
  Atomic columns are columns containing cells that contain a discrete or unique Data Type value. For example, in your dataset you have the SSN column. One of the cells in this column contains the value "123456789;098765432." In this example, Enterprise DLP inspects for all incidents of 123456789;098765432 as a singular SSN rather than inspecting for 123456789 and 098765432 as unique incidents.
• Enterprise DLP supports up to 50 individual Data Type values in a single cell.
  The Data Types are data values recognized by Enterprise DLP. If a cell has more than 50 Data Type values recognized by Enterprise DLP, Enterprise DLP processes only the first 50 values and ignores the remaining values.
  For example, Today is August 02 2020 contains three data type values; Today and is are Alphabet data types and August 02 2020 is a Date data type.
• Enterprise DLP supports English (Latin script) and the following languages in EDM datasets:
  • Hebrew (Hebrew script): The detection engine supports all 22 letters, including the five distinct final form (Sofiot) characters (ך, ם, ן, ף, ץ) used at the end of words. Although you can include Nikud in Hebrew words, Enterprise DLP ignores these vowel points in both the EDM dataset and the inspected network traffic. While Hebrew script is supported, there are some limitations for data formats.
    • Date Processing: Limited to Gregorian numeric dates (for example, 01/02/2026). Enterprise DLP does not process traditional Hebrew calendar formats or dates where the month is written in Hebrew script (for example, "1 בפברואר 2026").
    • Email Addresses: Limited to Latin script. Enterprise DLP does not process email addresses that contain Hebrew characters.
    • Regional Identifiers: Recognition of certain sensitive identifiers, such as Social Security Numbers and Driver's License IDs, is currently restricted to formats used in the USA.
  • Chinese: Support for CJK Ideographs (Hanzi) and Bopomofo (Zhuyin Phonetic Script).
  • Japanese: Support for CJK Ideographs (Kanji), Hiragana, and Katakana.
  • Korean: Support for Hangul and CJK Ideographs (Hanja).
  • Thai: Support for Thai script.
  While Chinese, Japanese, and Korean (CJK) and Thai languages are supported, there are some limitations for data formats.
  • Currency: For the Korean won, only the currency code KRW is supported. The visual currency symbol ₩ is not supported.
  • Date Processing: Dates containing CJK or Thai characters (for example, 2026年3月30日 or 30 มีนาคม 2026) and native Thai numerals (for example, ๒๕๖๙) are not supported. Only Arabic numeric date formats are supported.
  • ID Cards and Phone Numbers: CJK national ID card numbers and phone numbers are matched as generic numbers only. Exact matching across different formatted variations (for example, hyphenated versus unhyphenated strings like 138-1234-5678 and 13812345678) is not supported.
  • Text Conversion and Normalization: Enterprise DLP does not perform Traditional to Simplified Chinese conversion, Hiragana to Katakana conversion, or Half-width to Full-width Katakana normalization (for example, half-width ｶﾀｶﾅ and full-width カタカナ will not match each other).
  • Unicode Range: Rare characters in CJK Extension B through G (range U+20000 and above) requiring surrogate pairs are not supported.
  • Encodings: Legacy encodings (such as Shift JIS, EUC-JP, GB2312, Big5, and EUC-KR) are not supported. EDM source files must use supported encoding schemes (such as UTF-8 or UTF-16).
• Enterprise DLP supports the following delimiters to separate values in scanned files:
  • Caret (^)
  • Less-than sign (<)
  • Greater-than sign (>)
  • Pipe (|)
  • Semicolon (;)
  • Tab (\t)
  • Tilde (~)
• Enterprise DLP stops inspecting an EDM dataset when any of the following limits is reached first.

  Limit	Default Maximum
  Columns and rows per dataset	30 columns and 130 million rows
  Cells per dataset	120 million
  Cells per tenant (all datasets)	500 million

  For example, an EDM dataset with 30 columns and 4 million rows (120 million cells) is within all limits, as is a dataset with 6 columns and 20 million rows (120 million cells).
  Contact Palo Alto Networks Customer Support to increase the maximum number of cells supported for your Enterprise DLP tenant.

  By request, Enterprise DLP can support up to 1 billion cells per EDM dataset and up to 2 billion cells per Enterprise DLP tenant across all EDM datasets uploaded to Enterprise DLP.
• The supported file encoding schemes are UTF-8, UTF-16, ISO-8859-1, and US-ASCII.
• The EDM CLI app removes all punctuation from data contained in the EDM dataset.

• Alphabet and Alphanumeric

  Format	Example
  Latin script: Numbers, uppercase letters, and lowercase letters Hebrew script: Numbers and Hebrew alphabet characters (including final forms) Chinese script: Numbers, CJK Ideographs (Hanzi), and Bopomofo (Zhuyin) Japanese script: Numbers, CJK Ideographs (Kanji), Hiragana, and Katakana Korean script: Numbers, Hangul, and CJK Ideographs (Hanja) Thai script: Numbers and Thai alphabet characters	ABCDEFG / abcdefg / AB123CG אבגדהוז / אב123גד / כן 中文 / ㄅㄆㄇㄈ / 123中文 日本語 / ひらがな / カタカナ 한국어 / 韓國 / 123한국어 ภาษาไทย / 123ไทย

• Bank Routing Number and Bank Account Number

  Format	Example
  Enterprise DLP requires an exact match for a bank routing number and bank account number to render a verdict.	N/A

• Brazil CPF
  Always ensure that the column data format in the source file is set to include any leading or ending zeros in the individual ID numbers. For example, 01234567891 and 12345678900.

  Format	Example
  Unmasked—11 consecutive digits	12345678901
  Nine-digit block with final separator (- or /)	123456789-01 123456789/01
  Three-part separated with final separator (- or /)	123.456.789-01 123.456.789/01 123 456 789-01 123 456 789/01

• Country Name
  Enterprise DLP supports inspection of country names in official full names, short names, and common abbreviations across English, Hebrew, Chinese, Japanese, Korean, and Thai languages.

  Format	Example
  Latin script: Full names, short names, and standard ISO abbreviations Hebrew script: Hebrew country names Chinese script: Simplified and Traditional Chinese country names Japanese script: Kanji and Katakana country names Korean script: Hangul and Hanja country names Thai script: Thai country names	United States of America / United States / USA / US מדינת ישראל (State of Israel) / ישראל (Israel) 中华人民共和国 (People's Republic of China) / 中国 (China) 日本国 (State of Japan) / 日本 (Japan) 대한국 (Republic of Korea) / 한국 (Korea) / 일본 (Japan) ราชอาณาจักรไทย (Kingdom of Thailand) / ประเทศไทย (Thailand)

• Credit Card

  Format	Example
  Between 13 to 23 digits including dashes (-).	4739-5402-9061-0638 4739540290610638

• Currency
  • Enterprise DLP supports inspection of the US Dollar (USD, $), Euro (EUR, €), New Israeli Shekel (ILS/NIS, ₪), Japanese Yen / Chinese Yuan (JPY/CNY, ¥), Korean Won (KRW), Thai Baht (THB, ฿), and Generic Number values.
  • All currencies support values from 0 to 1 trillion (1,000,000,000).
  • All currencies support negative values. Enterprise DLP matches negative values against their absolute counterparts.
  • Enterprise DLP recognizes currency formats when a currency symbol or code is within 5 spaces of the beginning or end of the number.
  • Enterprise DLP treats the following formats as currencies when a symbol or code is present. As standalone values, Enterprise DLP treats these as generic numbers.
    • Simple Integer with No Symbol or Code
    • No Symbol or Code with Cents
    • Standard Decimal Notation with No Symbol or Code

  Format	Example
  Simple Integer with Symbol	$1000, €1000, ₪1000, ¥1000, or ฿1000
  Simple Integer with Code	USD1000, EUR1000, ILS1000, NIS1000, JPY1000, CNY1000, KRW1000, or THB1000
  Simple Integer with Symbol and Space	$ 1000, € 1000, 1000 ₪, ¥ 1000, or ฿ 1000
  Simple Integer with Code and Space	USD 1000, EUR 1000, ILS 1000, 1000 NIS, JPY 1000, CNY 1000, KRW 1000, or THB 1000
  Symbol or Code with Cents	$1000.00, EUR1000.00, ₪1000.00, ¥1000.00, or ฿1000.00
  Symbol or Code Comma Notation	$1,000, EUR1,000, 1,000 ₪, ¥1,000, or or ฿1,000
  Comma Notation Symbol or Code with Cents	$1,000.00, EUR1,000.00, 1,000.00 ₪, ¥1,000.00, or or ฿1,000.00
  Symbol or Code European Dot-Comma Notation	$1.000,00, €1.000,00, ₪1.000,00, ¥1.000,00, or ฿1.000,00
  Simple Integer with No Symbol or Code	1000
  No Symbol or Code with Cents	1000.00
  Standard Decimal Notation with No Symbol or Code	1,000.00

• Date
  While Hebrew and Thai scripts are supported for general text, dates must remain in Gregorian numeric formats (for example, DD/MM/YYYY). Traditional Hebrew calendar dates (using Hebrew month names or years) and dates containing Thai script or Thai numerals are not supported.
  Enterprise DLP supports spaces, dashes (-), slashes (/), periods (.), and any combination of these as separators. Enterprise DLP detects variations of the formats described below.
  Enterprise DLP supports 20th and 21st century dates only. These are all dates starting on January 1, 1901.

  Format	Example
  Legend DD—Two-digit day of the month. For example, 02. MMM—Three letter abbreviation for month. For example, Aug. MM—Two-digit month. For example, 12. YYYY—Four-digit year. For example, 2020.
  MM-DD-YYYY MM/DD/YYYY MM.DD.YYYY MMM-DD-YYYY MMM/DD/YYYY MMM.DD.YYYY DD-MM-YYYY DD/MM/YYYY DD MM YYYY DD.MM.YYYY DD-MMM-YYYY DD/MMM/YYYY DD MMM YYYY DD.MMM.YYYY YYYY-MM-DD YYYY/MM/DD YYYY.MM.DD YYYY-MMM-DD YYYY/MMM/DD YYYY.MMM.DD	12-02-2020 12/02/2020 12.02.2020 Dec-02-2020 Dec/02/2020 Dec.02.2020 02-12-2020 02/12/2020 02 12 2020 02.12.2020 02-Dec-2020 02/Dec/2020 02 Dec 2020 02.Dec.2020 2020-12-02 2020/12/02 2020.12-02 2020-Dec-02 2020/Dec/02 2020.Dec.02

  Ambiguous Dates—An ambiguous date is a date value that Enterprise DLP can't explicitly identify or interpret as a discrete and specific DD, MM, or YYYY value. For ambiguous dates, Enterprise DLP detects and takes action on all possible interpretations of ambiguous dates. When Enterprise DLP encounters an ambiguous year value, it only checks for dates in the 20th (19xx) and 21st (20xx) centuries.
  For example, consider the ambiguous date of 1/8/1975. This date can be interpreted as January 8th, 1975 or August 1st, 1975 depending on the country.
  Common examples of ambiguous dates include:
  • Dates where the year value of the date consists of 2 digits and is greater than 32.
    Check Type: Enterprise DLP checks for every possible day/month combination.
    Example: 2/06/50
    Dates Enterprise DLP Checks For:

    Feb 6 1950
    Feb 6 2050
    June 2 1950
    June 2 2050 

  • Dates where the day and year value are ambiguous, or the month and year values are ambiguous.
    Check Type: Enterprise DLP assumes the last pair of numbers is the year value and checks for every variation of that date.
    Example: Feb/06/12
    Dates Enterprise DLP Checks For:

    Feb 6 1912
    Feb 6 2012

  • Dates where the day, month, and year values are all ambiguous.
    Check Type: Enterprise DLP assumes the last pair of numbers is the year value and checks for every variation of that date.
    Example: 2/06/12
    Dates Enterprise DLP Checks For:

    Feb 6 1912
    Feb 6 2012
    June 2 1912
    June 2 2012 

• Email
  While Hebrew script is supported for general text, only Latin script is supported for email addresses. Enterprise DLP does not process email addresses that contain Hebrew characters.

  Format	Example
  RFC5322—<emailprefix>@<emaildomain>	bill@business.com BILL@BUSINESS.COM BILL@business.com bill@BUSINESS.com

• First, Middle, Last, and Full Name

  Format	Example
  Latin script: Uppercase and lowercase letters Hebrew script: Standard and final form (Sofiot) characters Chinese script: CJK Ideographs (Hanzi) Japanese script: CJK Ideographs (Kanji), Hiragana, and Katakana Korean script: Hangul and CJK Ideographs (Hanja) Thai script: Thai alphabet characters	Bill / bill / Bill Smith / bill smith בנימין / יצחק 张伟 佐藤 / さとう 김유수 สมชาย

• IP Address (IPv4 and IPv6)

  Format	Example
  Enterprise DLP requires an exact match for IPv4 and IPv6 IP addresses to render a verdict.	N/A

• Medical Record Number

  Format	Example
  Enterprise DLP requires an exact match for Medical Record Numbers to render a verdict.	N/A

• Member ID and Reward ID

  Format	Example
  Enterprise DLP requires an exact match for Member IDs and Reward IDs to render a verdict.	N/A

• Numbers

  Format	Example
  Enterprise DLP requires an exact match for all numbers to render a verdict. Enterprise DLP removes positive signed integers (+) and treats the same as a nonsigned integer. Enterprise DLP does not remove negative signed integers (-) to differentiate between positive and negative signed integers.	SI Numbers - 1234, +1234, or -1234 Formatted Numbers—9.00 Indian Number System—12, 34, 567.89

• Phone Number

  Format	Example
  Ten-digit US phone number format only. The EDM CLI app removes the Country code, parentheses (()), dash(-), space, and periods (.).	8001234567 (800)1234567 1.800.123.4567 +1 (800)123-4567 1 800 123 4567 +1 800 123 4567 +1 800 123-4567 1-800-123-4567 1 (800) 123-4567 (800)123-4567 (800) 123 4567 800-123-4567

• Vehicle License Plates
  • Enterprise DLP supports inspection of vehicle license plates for the United States and Israel. Enterprise DLP can detect license plates containing dashes (ABC-123), license plates presented as a continuous string of characters (ABC123), and license plates containing spaces (ABC 123).
  • United States: Supports various state-issued alphanumeric patterns, typically consisting of 6 to 7 characters.
  • Israel: Supports standardized 7-digit (XX-XXX-XX) and 8-digit (XXX-XX-XXX) numeric patterns.

  Format Type	Example
  United States Alphanumeric	ABC-1234, ABC 1234
  Israel 7-Digit Numeric	12-345-67, 12 345 67
  Israel 8-Digit Numeric	123-45-678, 123 45 678

• USA Driver License

  Format	Example
  Alphanumeric	E1234567 e1234567

• USA Social Security Number
  Format:
  Enterprise DLP supports spaces, dashes (-), periods (.) as separators.

  Format	Example
  XXX-XX-XXXX XXX XX XXXX XXX.XX.XXXX XXXXXXXXX	123-45-6789 123 45 6789 123.45.6789 123456789

• UUID

  Format	Example
  RFC4122—32 hexadecimal (base-16) digits. If you are using hyphens, the total is 36 digits.	123e4567e89b12d3a456426614174000 123e4567-e89b-12d3-a456-426614174000