Skip to content

DSAR Storage and Encryption

How Core stores and protects the files and personal data of Data Subject Access Requests (DSARs): a dedicated S3 "vault" bucket for temporary disclosure files, and application-level AES-256-GCM encryption on top of it. The first part sets up the bucket and its application configuration; the second covers the encryption key.

The encryption key is not settled

Whether the DSAR_ENCRYPTION_KEY in use is the one that was once committed to the Core repository, and whether it must be rotated, is being resolved in IS-132. Until that is closed, do not copy a key from anywhere: generate a fresh one with the helper below, and do not rotate a live key without reading the rotation warning. Keys never go in this page or in git.

Part 1: the S3 vault

This guide explains how to set up the S3 "Vault" for the DSAR Processor Model migration. The vault provides secure, temporary storage for sensitive data disclosure files.

Overview

The DSAR system now uses a dedicated AWS S3 bucket as a "vault" for temporary storage of sensitive data. Files are: - Never stored on disk (streamed directly to S3) - Automatically deleted after 14 days (via S3 Lifecycle policies) - Accessed via presigned URLs (15-minute validity) - Tracked in the database with expiration dates

This implements the "Processor" model where the system holds data temporarily only on the Controller's (account owner's) instructions.

AWS S3 Setup

1. Create a Dedicated S3 Bucket

aws s3 mb s3://your-org-dsar-vault-prod --region us-east-1

Bucket naming convention: {org}-dsar-vault-{environment} - Example: cookiehub-dsar-vault-prod

2. Enable Versioning (Optional)

For audit trail protection, enable versioning:

aws s3api put-bucket-versioning \
  --bucket your-org-dsar-vault-prod \
  --versioning-configuration Status=Enabled

3. Set Bucket Encryption

Enable default encryption with AES-256:

aws s3api put-bucket-encryption \
  --bucket your-org-dsar-vault-prod \
  --server-side-encryption-configuration '
  {
    "Rules": [
      {
        "ApplyServerSideEncryptionByDefault": {
          "SSEAlgorithm": "AES256"
        }
      }
    ]
  }'

4. Block Public Access

Ensure the bucket is completely private:

aws s3api put-public-access-block \
  --bucket your-org-dsar-vault-prod \
  --public-access-block-configuration \
  "BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true"

5. Set Lifecycle Policy (14-Day Auto-Purge)

Create a lifecycle policy to automatically delete files after 14 days:

aws s3api put-bucket-lifecycle-configuration \
  --bucket your-org-dsar-vault-prod \
  --lifecycle-configuration '{
    "Rules": [
      {
        "Id": "DeleteDSARDisclosuresAfter14Days",
        "Status": "Enabled",
        "Prefix": "disclosures/",
        "Expiration": {
          "Days": 14
        },
        "NoncurrentVersionExpiration": {
          "NoncurrentDays": 14
        }
      },
      {
        "Id": "CleanupIncompleteMultipartUploads",
        "Status": "Enabled",
        "Prefix": "disclosures/",
        "AbortIncompleteMultipartUpload": {
          "DaysAfterInitiation": 7
        }
      }
    ]
  }'

6. Create IAM Policy

Create an IAM policy for the application to access the vault:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DSARVaultAccess",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObject",
        "s3:HeadObject"
      ],
      "Resource": "arn:aws:s3:::your-org-dsar-vault-prod/disclosures/*"
    },
    {
      "Sid": "DSARVaultListBucket",
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket"
      ],
      "Resource": "arn:aws:s3:::your-org-dsar-vault-prod",
      "Condition": {
        "StringLike": {
          "s3:prefix": "disclosures/*"
        }
      }
    }
  ]
}

7. Create IAM User or Role

Create a dedicated IAM user or assume role for the application:

aws iam create-user --user-name dsar-vault-processor

aws iam put-user-policy \
  --user-name dsar-vault-processor \
  --policy-name dsar-vault-policy \
  --policy-document file://policy.json

aws iam create-access-key --user-name dsar-vault-processor

Note the AccessKeyId and SecretAccessKey from the output.

Application Configuration

1. Set Environment Variables

Add the following to your .env file:

# DSAR S3 Vault (Processor Model)
DSAR_VAULT_ACCESS_KEY_ID=your-iam-user-access-key
DSAR_VAULT_SECRET_ACCESS_KEY=your-iam-user-secret-key
DSAR_VAULT_REGION=us-east-1
DSAR_VAULT_BUCKET=your-org-dsar-vault-prod
DSAR_VAULT_URL=https://your-org-dsar-vault-prod.s3.amazonaws.com
DSAR_VAULT_ENDPOINT=https://s3.amazonaws.com
DSAR_VAULT_USE_PATH_STYLE_ENDPOINT=false

2. Run Database Migrations

php artisan migrate

This will: - Add expires_at column to dsar_disclosures table - Create dsar_instruction_logs table for compliance tracking

3. Test the Configuration

php artisan tinker

# Test S3 connection
Storage::disk('s3-dsar-vault')->put('test.txt', 'test content');
Storage::disk('s3-dsar-vault')->get('test.txt');  // Should return 'test content'
Storage::disk('s3-dsar-vault')->delete('test.txt');

How It Works

File Upload Flow (Processor Model)

  1. Admin completes DSAR request with attached files
  2. System creates ZIP in memory (never on disk)
  3. ZIP is streamed to S3 vault immediately
  4. Disclosure record created with S3 path, hash, and 14-day expiration
  5. Instruction log entry created documenting the Controller's instruction
  6. Email sent with presigned URL (15-minute validity) and password

File Download Flow (Presigned URLs)

  1. User clicks download link from email
  2. Application verifies request is still valid (not expired, not rejected)
  3. Application generates presigned URL (15-minute validity)
  4. User is redirected directly to S3
  5. Browser downloads directly from S3
  6. S3 Lifecycle policy automatically deletes after 14 days

Processor Model Compliance

Aspect Implementation
Data Minimization Files only in S3 vault, temp storage on local disk
Temporary Storage 14-day auto-delete via S3 lifecycle policy
Instruction Logging dsar_instruction_logs table tracks all actions
Access Control Presigned URLs prevent unauthorized access
Audit Trail Complete action logging with IP, user agent
Server Isolation Server acts as gatekeeper, not delivery truck

Monitoring and Maintenance

Monitor S3 Storage Usage

aws s3 ls s3://your-org-dsar-vault-prod/ --summarize --human-readable --recursive

Verify Lifecycle Policies

aws s3api get-bucket-lifecycle-configuration --bucket your-org-dsar-vault-prod

Check for Failed Uploads

Monitor CloudWatch logs for DsarVaultService errors:

aws logs filter-log-events \
  --log-group-name /aws/laravel/dsar \
  --filter-pattern "DsarVaultService"

Troubleshooting

Presigned URL Generation Fails

Problem: Failed to generate presigned URL

Solution: 1. Verify IAM credentials are correct 2. Check bucket name in .env 3. Ensure IAM policy includes s3:GetObject 4. Check CloudWatch logs for detailed error

Files Not Automatically Deleted

Problem: Files older than 14 days still exist in S3

Solution: 1. Verify lifecycle policy is enabled:

aws s3api get-bucket-lifecycle-configuration --bucket your-bucket
2. Note: S3 may take up to 24 hours to delete files 3. You can manually clean up older files:
aws s3 rm s3://your-bucket/disclosures/ --recursive --exclude "*" \
  --include "disclosures/*" \
  --older-than 14d

Access Denied Errors

Problem: Access Denied when uploading to S3

Solution: 1. Verify IAM user/role has correct policy 2. Check that the policy resource is correct: arn:aws:s3:::bucket-name/disclosures/* 3. Verify AWS credentials in .env 4. Test manually:

aws s3 put-object \
  --bucket your-org-dsar-vault-prod \
  --key disclosures/test.txt \
  --body /path/to/test.txt

Migration from Local Storage

For existing requests in local storage, there are two options:

  1. Keep existing files local - Only new requests use S3 vault
  2. Migrate existing data - Create an artisan command to copy to S3

To migrate existing disclosures (optional):

php artisan dsar:migrate-to-s3-vault

Security Considerations

✅ What we've implemented: - Presigned URLs with 15-minute validity (prevents sharing) - Password-protected ZIPs - Files deleted after 14 days - Instruction logging for accountability - No PII in file paths (UUIDs only) - Encrypted transport (HTTPS)

⚠️ Additional measures to consider: - Enable S3 access logging - Enable CloudTrail for API audit - Use VPC endpoints to prevent internet exposure - Implement IP whitelisting for API access - Encrypt at rest with customer-managed KMS keys - Use MFA delete protection

This implementation supports GDPR Article 15 (Right of Access) by:

  1. Data Minimization - Temporary storage only
  2. Purpose Limitation - Only for fulfilling the request
  3. Accountability - Complete instruction and action logs
  4. Lawfulness - Presigned URLs prevent unauthorized access
  5. Integrity & Confidentiality - Encrypted transport and storage

See dsar_instruction_logs table for proof of: - Who authorized the disclosure - When the instruction was given - What action was taken - Legal basis for processing

Part 2: encryption

This guide explains how to set up encryption for DSAR (Data Subject Access Request) files and personal data.

Overview

The DSAR system uses AES-256-GCM symmetric encryption to protect sensitive data. This provides both confidentiality and authenticity (authenticated encryption). A single shared key is used for all encryption and decryption operations.

Key Components

  • Encryption Key: A 256-bit (32-byte) key stored in .env as DSAR_ENCRYPTION_KEY in Laravel's base64: format
  • DsarEncryptionService: Handles encrypt/decrypt for file contents and personal data
  • Blind-Index Search: HMAC-SHA256 trigram tokens allow searching encrypted PII without decryption

What Gets Encrypted

  1. Personal data in dsar_requests table (name, email, requester_name, requester_email, details) — via Laravel's encrypted cast
  2. Uploaded files — encrypted with AES-256-GCM before S3 storage
  3. Disclosure ZIPs — encrypted before S3 storage
  4. S3 vault credentials in dsar_settings — via Laravel's encrypted cast
  5. S3 server-side encryption — AES-256 at rest in S3

Setup Instructions

1. Generate an Encryption Key

Use the static helper method to generate a key:

php artisan tinker
>>> App\Services\DsarEncryptionService::generateKey()
=> "base64:<generated by the tinker helper>"

2. Add to .env File

DSAR_ENCRYPTION_KEY=base64:<generated by the tinker helper>

Important: - The key must include the base64: prefix - Do not wrap in quotes - Do not commit the .env file to version control

3. Verify Setup

The key is loaded via config('services.dsar.encryption_key') (defined in config/services.php). To verify:

php artisan tinker
>>> $svc = App\Services\DsarEncryptionService::forAccount();
>>> $encrypted = $svc->encrypt('test');
>>> $svc->decrypt($encrypted);
=> "test"

Encryption Format

Encrypted data is stored as a base64-encoded JSON package:

{
    "v": 1,
    "iv": "<base64 encoded 96-bit IV>",
    "data": "<base64 encoded ciphertext>",
    "tag": "<base64 encoded GCM auth tag>"
}

Each encryption operation generates a unique random IV, so encrypting the same data twice produces different ciphertext.

Where Encryption Is Used

  1. File Upload (FileController::uploadStagedFile)
  2. Encrypts file content before storing in S3 vault

  3. ZIP Creation (ItemController::complete)

  4. Decrypts staged files, bundles into ZIP
  5. Encrypts final ZIP before S3 storage

  6. User Download (UserController::download)

  7. Decrypts ZIP before streaming to user

  8. Admin Preview (FileController::downloadStagedFile)

  9. Decrypts file before returning to admin

Key Rotation

To rotate the encryption key: 1. Generate a new key using DsarEncryptionService::generateKey() 2. Update DSAR_ENCRYPTION_KEY in .env 3. Warning: Files encrypted with the old key will no longer be decryptable. Ensure all active disclosures have expired before rotating.

Troubleshooting

"Encryption key must be 32 bytes" Error

The key is not the correct length. Regenerate using DsarEncryptionService::generateKey() and ensure the base64: prefix is present.

"DSAR_ENCRYPTION_KEY not configured" Error

The key is missing from .env or config/services.php is not loading it. Run php artisan config:clear and verify the key is set.

"Failed to decrypt data or authentication failed" Error

The data was encrypted with a different key, or the encrypted package has been corrupted. Check that the current DSAR_ENCRYPTION_KEY matches the one used when the data was encrypted.

Security Notes

  • The encryption key must be kept secure and never committed to version control
  • AES-256-GCM provides authenticated encryption — tampered ciphertext is detected and rejected
  • Use a secrets manager (e.g., AWS Secrets Manager) in production
  • Back up the encryption key securely — losing it means losing access to all encrypted DSAR data
  • The same key is used for all accounts; per-account keys can be implemented by extending DsarEncryptionService::forAccount()