Security Architecture

Data Security

Protecting data at rest, in transit, and in use: data classification frameworks, column-level encryption, tokenisation, data masking, PII handling, GDPR/CCPA technical controls, and access auditing.

⏱ 10 min read

What it is

Data security encompasses the controls that protect data throughout its lifecycle — creation, storage, processing, transmission, and destruction — with a focus on preventing unauthorised access, disclosure, modification, or destruction. The foundation is data classification: assigning sensitivity levels to data (Public, Internal, Confidential, Restricted) based on the harm that disclosure would cause, and applying proportionate controls at each level. Encryption at rest protects data stored on disk from physical media theft; encryption in transit (TLS) protects data from network interception; and column-level encryption (also called application-level encryption or field-level encryption) protects specific sensitive fields even within a database to which the application has legitimate read access.

Tokenisation replaces a sensitive value (credit card number, SSN) with a non-sensitive surrogate token that has no mathematical relationship to the original value and can only be de-tokenised by authorised parties with access to the token vault. It differs from encryption in that the token cannot be de-tokenised without the vault, even with the algorithm. Data masking transforms data for lower-trust environments: static masking creates a de-identified copy for development/testing; dynamic masking returns masked values in real-time to unprivileged users. PII (Personally Identifiable Information) handling requires additional controls under GDPR and CCPA: data minimisation (don't collect what you don't need), purpose limitation (don't use data beyond its stated purpose), retention limits (delete data when no longer needed), and the right to erasure (technical ability to delete all data associated with a user).

Why it exists

Data breaches are among the most costly security incidents: the IBM Cost of a Data Breach Report 2023 found an average cost of $4.45M per breach. Encryption at rest means that stolen database backups or cloud storage contents are unreadable without access to the encryption keys. Column-level encryption for the most sensitive fields (SSNs, payment card data, medical records) means that even a SQL injection attack that reads the database returns ciphertext rather than plaintext PII. GDPR/CCPA have created significant legal and financial obligations for data protection: GDPR fines can reach €20M or 4% of global revenue, whichever is higher. Technical controls (encryption, masking, access logging) provide both security and the compliance evidence required during regulatory audits.

When to use

  • All systems storing personal data (PII) — GDPR/CCPA technical controls including encryption, access logging, and erasure capability are legally required.
  • Payment card data — PCI DSS mandates tokenisation or encryption of stored card data.
  • Healthcare data — HIPAA requires encryption of PHI at rest and in transit.
  • Shared development/staging environments — production data should never be used; masked or synthetic data must be used instead.

When not to use

  • Column-level encryption on all fields: Column-level encryption prevents database-side filtering, aggregation, and indexing on encrypted fields. Apply it only to the most sensitive fields (SSN, credit card, password, medical record number) where the performance cost is justified by the protection requirement.

Typical architecture

DATA CLASSIFICATION FRAMEWORK:
  Restricted:  PII, PHI, PAN (payment), credentials
               → Column-level encryption, strict access, full audit
  Confidential: Internal financial data, trade secrets
               → Encryption at rest, role-based access
  Internal:    Business data, non-sensitive metrics
               → Standard access controls
  Public:      Published content, open data
               → No special controls

COLUMN-LEVEL ENCRYPTION PATTERN:
  Application code:
  ssn_encrypted = encrypt(ssn, kms_key_id="key-abc")
  # Stored as: AES-256-GCM ciphertext + IV + AAD
  # KMS key used for envelope encryption:
  #   data key generated per field/record,
  #   encrypted with KMS master key,
  #   stored alongside ciphertext

  Database schema:
  CREATE TABLE customers (
    id UUID PRIMARY KEY,
    email TEXT,           -- application encrypted
    ssn BYTEA,            -- column-level encrypted
    dob_year INTEGER,     -- partial (year only, not full DOB)
    payment_token TEXT    -- token, no PAN stored
  )

TOKENISATION FLOW (payment):
  Card entry: 4111-1111-1111-1111
  Application → Token Vault (PCI-compliant)
  Token Vault → returns: tok_live_abc123xyz
  Store only: tok_live_abc123xyz
  Display: ****-****-****-1111 (last 4, not token)
  For charge: send token to payment processor
  → Processor de-tokenises and processes
  PAN never stored in application database

DATA MASKING FOR NON-PRODUCTION:
  Production: email=john.doe@example.com
  Masked dev: email=user_12345@masked.local
  Production: ssn=123-45-6789
  Masked dev: ssn=XXX-XX-6789 (preserve last 4 for testing)

GDPR ERASURE (right to be forgotten):
  User requests deletion:
  1. Mark user record as deleted (soft delete)
  2. Async job: replace PII fields with null/hashed pseudonym
  3. Remove from search indexes
  4. Propagate deletion to downstream systems via event
  5. Retain non-PII data for analytics (user_id only)
  6. Log erasure completion for compliance audit

DATA ACCESS LOGGING:
  Every DB query touching PII rows emits:
  {timestamp, user_id, service, query_type,
   table, row_count, purpose_label}
  Immutable log in append-only store (WORM)
  Reviewed in quarterly access audits

Pros and cons

Pros

  • Encryption at rest ensures that stolen database backups provide no value to attackers without the encryption keys.
  • Tokenisation of payment card data removes PAN from application storage entirely, dramatically reducing PCI DSS scope.
  • Data masking in development environments eliminates the risk of PII exposure to developers and in development security incidents.
  • Data access logging creates the immutable audit trail required for compliance audits and enables detection of insider threats or compromised service accounts.

Cons

  • Column-level encryption prevents database-side predicates on encrypted fields; queries like WHERE ssn = ? cannot use indexes and require full table scans or application-side decryption.
  • Key management complexity: column-level encryption requires application-level key management, key rotation processes, and re-encryption of existing data on key rotation.
  • Data erasure in event-sourced or append-only log architectures is architecturally challenging; erasure requires crypto-shredding (destroying the encryption key for erased user's data) rather than physical deletion.
  • Data masking tools may not perfectly preserve data relationships needed for integration testing.

Implementation notes

Use envelope encryption for column-level encryption: generate a unique data encryption key (DEK) per-record or per-field, encrypt the sensitive value with the DEK, then encrypt the DEK with a KMS-managed key encryption key (KEK). Store the encrypted DEK alongside the ciphertext. This pattern means key rotation only requires re-encrypting the DEKs (lightweight), not re-encrypting all the data. AWS KMS, Google Cloud KMS, and HashiCorp Vault all support this pattern. For searchable encrypted data, consider deterministic encryption (same plaintext always produces same ciphertext, allowing equality comparison) for low-cardinality fields, or use a blind index (hash of the plaintext, indexed separately) for lookups, understanding that this leaks some information about equality.

For GDPR erasure, design the system from the start to support pseudonymisation: store a stable internal user_id as a foreign key in all data tables rather than PII like email. On erasure, the PII record is deleted or nulled, but the user_id foreign key rows in analytics and audit tables remain intact (now pseudonymous). For event streams and append-only logs, implement crypto-shredding: encrypt user-specific events with a per-user encryption key; when erasure is requested, delete the user's encryption key from KMS — all their encrypted events become unreadable.

Common failure modes

  • Encryption at rest but plaintext backups: Databases with transparent encryption but unencrypted backup files that are exported to S3 without encryption — the backup provides plaintext access.
  • Encryption keys stored alongside data: Encryption keys stored in the same database as the encrypted data provide no security — an attacker who reads the database reads both the data and the key.
  • PII in application logs: Request logging that captures HTTP bodies or query parameters may inadvertently log PII (email addresses in query strings, names in JSON bodies) — logs are often stored with weaker access controls than the database.
  • No data retention policy enforcement: Data is collected with stated retention periods (GDPR: retain for purpose duration only) but deletion jobs are never implemented, resulting in data retained indefinitely.
  • No erasure capability for event streams: Kafka topics or event logs with no crypto-shredding strategy make GDPR erasure technically infeasible.

Decision checklist

  • Is all data classified with sensitivity levels and controls applied proportionately?
  • Is encryption at rest enabled for all databases, object storage, and backups?
  • Are the most sensitive fields (SSN, PAN, medical data) protected with column-level encryption or tokenisation?
  • Is production PII masked or synthetic before use in development/testing?
  • Is there a technical erasure capability for GDPR right-to-be-forgotten requests?
  • Are data access logs collected and retained in an immutable store?

Example use cases

  • Healthcare patient records: PHI fields (diagnosis, medication, notes) encrypted at column level using AWS KMS; encrypted DEKs stored per-patient; only authorised service accounts have IAM permission to call KMS decrypt; all PHI access logged to immutable CloudTrail.
  • E-commerce checkout: Credit card numbers tokenised via Stripe or Braintree token vault; application stores only token and last 4 digits; PCI DSS scope reduced to tokenisation integration points only; full PAN never touches application servers.
  • GDPR erasure at scale: User requests account deletion; internal user_id retained in analytics tables; per-user encryption key deleted from KMS (crypto-shredding any encrypted PII in Kafka events); PII database fields nulled; erasure event logged with timestamp for compliance evidence.

Further reading