Focus

Prisma AIRS

Understanding Model Security Rules

Table of Contents

Understanding Model Security Scan Results

Understanding Model Security Rules

Learn about Model Security rules that scans AI/ML models for major threats while validating metadata like licenses and file formats to ensure secure model usage.

Where Can I Use This?	What Do I Need?
Prisma AIRS (Model Security)	Prisma AIRS Model Security License

Model security rules serves as the central mechanism for securing model access. Model security scanning covers several key areas: thread and metadata. To learn more about threat categories and risk mitigation strategies for AI and machine learning systems, refer Model Threats.

Following are the three major threat categories:

Threat	Description
Deserialization threats	Issues that arise when you load a model into memory.
Backdoor Threats	Issues that arise when a model was specifically designed to support alternative or hidden paths in its behavior.
Runtime Threats	Issues that arise when you use a model to perform inference.

Model security also examine specific metadata fields in models to address security considerations, such as verifying a model's license to ensure it's appropriate for commercial use. Model security rules enables you to validate the following concerns:

Metadata in Models	Security Rule Validation
Open Source License	Ensures that the model you are using is licensed for your use case.
Model Format	Ensures that the model you are using is in a format that is supported by your environment.
Model Location	Ensures that the model you are using is hosted in a location that is secure and trusted.
Verified Organizations in Hugging Face	Ensures that the model you are using is from a trusted source.
Model is Blocked	Overrides all other checks to ensure that a model is blocked no matter what.

Our managed rules integrate all of these checks to provide comprehensive security for model usage. To get started, you can explore your defaults in your Model Security at InsightsPrisma AIRSModel SecurityModel Security Groups.

How Scans, Security Groups, and Rules Work Together

With Model Security, you initiate a model scan and associate it with a security group. Model Security evaluates the model against all enabled rules within that security group to assess whether it satisfies your security requirements. This assessment relies on the results from enabled rules in the security group and the group's configured threshold.

When you scan a model as follows:


# Import the Model Security SDK/Client
from model_security_client.api import ModelSecurityAPIClient
# Load your scanner URL
scanner_url = os.environ["MODEL_SECURITY_TOKEN_ENDPOINT"] 
# Define your model's URI
model_uri = "s3://demo-models/unsafe_model.pkl"
# Set your security group's UUID
security_group_uuid = "6e2ccc3a-db57-4901-a944-ce65e064a3f1"
# Create a Model Security Client
guardian = ModelSecurityAPIClient(base_url=scanner_url)
# Scan your model
response = client.scan(model_uri=model_uri, security_group_uuid=security_group_uuid)

The response will appear as follows (showing the security rules results):


{
  "http_status_code": 200,
  "scan_status_json": {
    "aggregate_eval_outcome": "FAIL",
    "aggregate_eval_summary": {
      "critical_count": 1,
      "high_count": 0,
      "low_count": 0,
      "medium_count": 1
    },
    "violations": [
      {
        "issue": "Model file 'ykilcher_totally-harmless-model/retr0reg.gguf' is stored in an unapproved format: gguf",
        "threat": "UNAPPROVED_FORMATS",
        "operator": null,
        "module": null,
        "file": "ykilcher_totally-harmless-model/retr0reg.gguf",
        "hash": "f59ad9c65c5a74b0627eb6ca5c066b02f4a76fe6",
        "threat_description": "Model is stored in a format that your Security Group does not allow",
        "policy_name": "Stored In Approved File Format",
        "policy_instance_uuid": "34ef1ddc-0b7a-45b8-a84a-c96b1d8383d0",
        "remediation": {
          "steps": [
            "Store the model in a format approved by your organization"
          ]
        }
      },
      {
        "issue": "The model will execute remote code since it contains operator `__class__` in Jinja template.",
        "threat": "PAIT-GGUF-101",
        "operator": "__class__",
        "module": null,
        "file": "ykilcher_totally-harmless-model/retr0reg.gguf",
        "hash": "f59ad9c65c5a74b0627eb6ca5c066b02f4a76fe6",
        "threat_description": "GGUF Model Template Containing Arbitrary Code Execution Detected",
        "policy_name": "Load Time Code Execution Check",
        "policy_instance_uuid": "09780b9f-c4f7-4e0b-ad21-7ff779472283",
        "remediation": {
          "steps": [
            "Use model formats that disallow arbitrary code execution"
          ]
        }
      }
    ]
  }
}

Note the FAIL status in the aggregate_eval_outcome field. This indicates the model did not pass the scan because security rule failures surpassed your security group's threshold, with the violations field providing details about which rules were breached.

Each model security rule contains the following fields.

Rule Field	Description
Rule Name	Specifies the name of the security rule.
Rule Description	Specifies the description of the security rule.
Compatible Sources	Specifies the model source types that this rule is compatible with.
Status	Specifies the status of the security rule, either Enabled or Disabled. This can be set globally, or at the security group level.

Connecting Scans to Rules

When scanning a model, model security first identifies its format through introspection. After determining the model type, model security maps it to the taxonomy of model vulnerability threats and coordinate the specific deeper scans required for that model.

A series of specific threats like Arbitrary Code Execution At Runtime are grouped together when they are reported and are shown in a specific rule. This allows you to block specific types of threats without having to manage the complexity of all the various formats and permutations.

You can also configure common rules for all models regardless of their source type.

Security Rule Checks

Rule Name	Status	Description	Example
Runtime Code Execution Check	Enabled	This rule detects Arbitrary Code Execution that can occur during model inference through various methods.	Keras Model Lambda Layer Arbitrary Code Execution Detected at Model Run Time TensorFlow SavedModel Contains Arbitrary Code Execution at Model Run Time These attacks mean the model will execute code without your knowledge during use, making this a Critical issue to block. Learn more about this threat type here: Runtime Threats.
Known Framework Operators Check	Enabled	Machine learning model formats often include built-in operators to support common data science tasks during model operation. Some frameworks allow custom operator definitions, which poses risks when executing unknown third-party code.	When TensorFlow SavedModel Contains Unknown Operators is detected, it indicates that the model creator is using non-standard tooling approaches. For more information refer, SavedModel Contains Unknown Operators.
Model Architecture Backdoor Check	Enabled	A model's behavior can contain a backdoor embedded in its architecture, specifically through a parallel data flow path within the model. For most inputs, the model operates as expected, but for certain inputs containing a trigger, the backdoor activates and effectively alters the model's behavior.	When ONNX Model Contains Architectural Backdoor is detected, it warns you that a model has at least one non-standard path requiring further investigation. For more information refer, ONNX Model Contains Architectural Backdoor.
Load Time Code Execution Check	Enabled	This rule is similar to the runtime attacks, but these attacks execute immediately upon model loading. For example, the below python snippet is sufficient to trigger the exploit without your knowledge: with open('path_to_your_model.pkl', 'rb') as file: model = pickle.load(file)	When Pickle, Keras Lambda Layers, PyTorch models, and more are all vulnerable to this threat is detected in a model, there's no reliable method to eliminate the threat. However, you can investigate the model's history or, if it's your own model, examine the build process to identify the source of the malicious code. For more information refer, PyTorch Model Arbitrary Code Execution
Suspicious Model Components Check	Enabled	Not all rules target specific threats; some, like this one, assess a model's potential for future exploitation. This check identifies components within the model that could enable malicious code execution later. Example: Remote code execution being called by fetching external data from Protobuf files or others over the internet. A violation here should prompt you to be cautious and to evaluate all relevant components around the model before making a decision on whether or not the model is safe for use. More information can be found here: Keras Lambda Layers Can Execute Code	Remote code execution being called by fetching external data from Protobuf files or others over the internet. This violation prompts caution and thorough evaluation of all relevant model components before determining whether the model is safe for use. For more information refer, Keras Lambda Layers Can Execute Code
Stored In Approved File Format	Enabled	This rule verifies whether the model is stored in a format that you've approved within the rule's list. We recommend enabling these formats by default: Safetensors JSON By default Model Security reports that pickle and keras_metadata are not approved formats.	—

Hugging Face Model Rules

Lastly, there are rules specifically scoped to Hugging Face models. These rules target the particular metadata you control or that is consistently provided by Hugging Face.

Hugging Face Model Rule	Status	Description
License Exists	Enabled	The simplest rule in the application. This rule checks to see if the model has a license associated with it. If no license is present, then this rule will fail.
License Is Valid For Use	Enabled	This rule gives you more control for the models that your organization will run or test. As a commercial entity, you may not want to run models with non-permissive open-source licenses like GPLv3 or others. Adding licenses to the rule will expand the list of licenses that are allowed for use. A reasonable set of defaults are: apache-2.0 mit bsd-3.0 You can find a full list of license options here: Hugging Face Licenses Note we use the License identifier field to map to the license in the model metadata.
Model Is Blocked	Enabled	Sometimes you just cannot tolerate a model for whatever reason. This rule allows you to block a specific model from being used. The format for blocking a model from Hugging Face relies on the Organization and Model Name. For example opendiffusion/sentiment-check can be entered and it would block the model sentiment-check from opendiffusion. When a model is both blocked and allowed simultaneously, the block takes precedence.
Organization Verified By Hugging Face	Enabled	Hugging Face is an excellent site for the latest models from all over the world, giving you access to cutting edge research. You may want to restrict organizations from providing unverified models from Hugging Face. This prevents accidentally running models from deceptively similar sources like facebook-llama instead of the legitimate meta-llama (where legitimate meta-llama is the correct model). This rule simply checks that Hugging Face has verified the organization, if that passes the models from the organization will pass this check.
Organization Is Blocked	Enabled	If you find a particular organization that just delivers problematic models or for any other reason, you'd like to block them, this is your rule. Enter the organization name into the rule and all models from that organization will be blocked. For example facebook-llama would block ALL of the models provided by that organization.

Understanding Model Security Scan Results

Reporting New Threats

Prisma AIRS Docs

Understanding Model Security Rules

Prisma AIRS Docs

Understanding Model Security Rules

How Scans, Security Groups, and Rules Work Together

Connecting Scans to Rules

Security Rule Checks

Hugging Face Model Rules

Activation & Onboarding

Next-Generation Firewalls

Cloud-Delivered Security Services

Network Security

Visibility & Monitoring