Platform APIs and Abstractions

What it is

Platform APIs are the programmatic interfaces through which developers request platform resources — a database, a Kubernetes namespace, an S3 bucket, an SSL certificate, a secret — without directly configuring the underlying cloud infrastructure. Rather than granting developers IAM access to create RDS instances (requiring AWS knowledge, security configuration, and VPC setup), the platform exposes a PostgresDatabase Custom Resource Definition (CRD). The developer applies a YAML manifest declaring the desired database size and engine version; a Kubernetes operator running in the cluster reconciles this declaration against AWS, creates the RDS instance with correct subnets, security groups, and KMS encryption, and injects the connection string as a Kubernetes Secret. The developer gets a database in 5 minutes without writing a line of Terraform.

The dominant technical approach to platform APIs in 2024 is Kubernetes as the control plane: CRDs extend the Kubernetes API with platform-specific resource types (Database, Queue, ServiceAccount, Environment), operators implement the business logic for each resource type, and all platform resources are managed through the same kubectl apply and GitOps workflows developers already use. Crossplane provides a framework for composing CRDs that provision cloud resources declaratively. The Backstage Software Catalog API provides a REST and GraphQL interface for querying the catalog programmatically, enabling integration with CI/CD, cost tools, and incident management.

Why it exists

Raw cloud APIs (AWS Console, Terraform) are powerful but require deep infrastructure expertise to use safely. An application developer who needs a PostgreSQL database for their service should not need to understand VPC subnets, security group rules, parameter groups, backup retention policies, multi-AZ configuration, encryption at rest, IAM authentication, and subnet group configuration. They should declare "I need a PostgreSQL 15 database with 50 GB storage" and receive a connection string. Platform APIs provide this abstraction: they encode the infrastructure team's expertise into a repeatable, safe interface that any developer can use.

The Kubernetes control plane model is particularly powerful because it provides desired-state reconciliation for free: operators continuously compare the declared state (a PostgresDatabase CRD with size: small) against the actual state (the RDS instance in AWS) and reconcile any drift. This means platform resources are self-healing — if someone manually modifies the RDS instance outside the CRD, the operator detects and corrects the drift. It also means platform resources are GitOps-compatible — a PR to change the CRD declaration triggers a reconciliation that updates the real infrastructure, with full audit trail.

When to use

• Use CRDs and operators when multiple teams need the same type of cloud resource (databases, queues, certificates) with consistent configuration and security defaults — the abstraction benefit scales with demand.
• Use Crossplane when the organisation wants a cloud-provider-agnostic infrastructure abstraction layer that works across AWS, GCP, and Azure with the same CRD interface.
• Use the Backstage Software Catalog API when building tooling (cost dashboards, incident routing, dependency analysis) that needs programmatic access to the service registry.
• Use resource abstraction layers when developers are spending time on infrastructure configuration that could be encoded in an operator without loss of flexibility for the common case.
• Expose escape hatches via annotation-based overrides (e.g., platform.org/custom-instance-class: db.r6g.4xlarge) to allow teams with non-standard requirements to bypass defaults without bypassing the entire abstraction.

When not to use

• Low-demand resources: Writing a Kubernetes operator for a resource type that only 2 teams need and configure differently is over-engineering; a shared Terraform module is sufficient.
• Rapidly evolving abstractions: If the underlying infrastructure is changing frequently (new cloud services, new security requirements), the overhead of maintaining CRD schemas and operator versions may exceed the benefit; abstract stable resources first.
• Very small organisations: Crossplane has significant operational complexity; organisations with fewer than 5 engineers working on infrastructure should use Terraform modules and module registries rather than a full control plane abstraction.
• Replacing the Kubernetes API entirely: Building a bespoke internal API gateway in front of Kubernetes adds complexity without Kubernetes' built-in RBAC, admission webhooks, and reconciliation — prefer CRDs over custom API servers.

Typical architecture

Platform API Layer (Kubernetes-based)
──────────────────────────────────────────────────────────
  Developer workflow
  ┌─────────────────────────────────────────────────────┐
  │  # developer declares: I need a Postgres database   │
  │  apiVersion: platform.acme.io/v1                    │
  │  kind: PostgresDatabase                             │
  │  spec:                                              │
  │    size: medium   # hides: instance class, storage  │
  │    version: "15"                                    │
  │    team: checkout                                   │
  └─────────────┬───────────────────────────────────────┘
                │ kubectl apply (or GitOps)
  Kubernetes API server
  ├── OPA Gatekeeper validates (team owns namespace?)
  ├── Admission webhook enforces required labels
  └── PostgresDatabase CRD stored in etcd

  Database Operator (reconcile loop)
  ├── Reads CRD spec
  ├── Calls Crossplane / AWS provider
  │   └── Creates: RDS instance, subnet group,
  │               security group, parameter group
  ├── Writes connection string → K8s Secret
  │   └── checkout/postgres-checkout-conn
  └── Updates CRD status (Ready: True / Provisioning)

  Backstage Software Catalog API
  ├── GET /api/catalog/entities?kind=Component
  ├── POST /api/catalog/entities (register service)
  └── Consumed by: cost dashboard, incident routing,
                   dependency graph visualisation

Pros and cons

Implementation notes

Start with the Operator SDK or Kubebuilder to scaffold operators — these frameworks handle boilerplate (reconcile loops, informer caches, leader election) and let the platform team focus on the business logic. Follow the operator pattern: the reconcile function should be idempotent (calling it multiple times with the same desired state produces the same result), and all side effects (cloud API calls) should be guarded by existence checks before creation. Use Crossplane's provider packages for cloud resources rather than writing raw AWS SDK calls in your operator — Crossplane Providers (provider-aws, provider-gcp) already handle credential management, retry logic, and status reporting for hundreds of AWS/GCP resource types.

For the Backstage Catalog API: the catalog is populated via catalog-info.yaml files in each service repository (registered by Backstage discovery) and via programmatic registration from CI/CD pipelines (POST /api/catalog/entities). Design the catalog entity schema carefully — the spec.owner, spec.system, and spec.lifecycle fields drive ownership routing and should be populated from CODEOWNERS or GitHub team data, not manually. Use Backstage's built-in Software Templates to ensure new services are always registered in the catalog at creation time, preventing catalog staleness.

Common failure modes

• Abstraction too thin: CRD merely wraps the cloud resource with no defaults — developers still need to understand every cloud parameter; the abstraction provides no value; add opinionated defaults and only expose the parameters that differ between use cases.
• Abstraction too thick: CRD hides all cloud configuration; team with a legitimate advanced requirement (read replicas in a specific region) has no way to express it; CRD has no escape hatch; team bypasses the platform entirely; design all abstractions with a documented escape hatch from day one.
• Operator single point of failure: The database operator crashes and goes into CrashLoopBackOff; all database provisioning requests queue up unanswered for 4 hours before the platform team notices; run operators with multiple replicas and leader election; add operator health to platform SLOs.
• CRD schema changes break existing resources: Platform team changes a CRD field name in v1beta1; all existing resources using v1alpha1 stop reconciling; requires conversion webhooks that were not planned for; version CRDs from the start and maintain conversion webhooks when introducing breaking changes.
• Backstage catalog stale data: A service is deleted from the codebase but its catalog entry persists; oncall engineer pages the former owner (who left the company) during an incident; implement automatic catalog cleanup when repos are archived and require catalog health checks in each team's quarterly review.

Decision checklist

• Does the resource type have enough demand (3+ teams, multiple instances) to justify writing and maintaining an operator?
• Are opinionated security defaults (encryption at rest, network isolation, backup retention) baked into the CRD defaults rather than optional fields?
• Is there a documented and functioning escape hatch for teams with legitimate non-standard configuration requirements?
• Are operators running with at least 2 replicas with leader election for high availability?
• Is CRD versioning planned for from the start, with conversion webhooks in place before v1 graduation?
• Is the Backstage catalog populated via automation (catalog-info.yaml + discovery) rather than manual registration?

Example use cases

• Database self-service via CRD: Platform team writes a PostgresDatabase operator backed by Crossplane's AWS provider; 15 product teams provision databases via kubectl apply; all databases have encryption, automated backups, and private subnets by default; database provisioning time drops from 3 days (ticket) to 8 minutes (operator reconciliation); zero misconfigured databases in production.
• Backstage catalog driving incident response: PagerDuty integration reads the Backstage catalog API to determine service ownership; all incidents are automatically routed to the correct team; MTTR for misrouted incidents drops from 45 minutes to 0; post-incident reviews show correct first notification 98% of the time, up from 74%.
• Crossplane multi-cloud environment provisioning: SaaS company must support tenant environments on both AWS and GCP; Crossplane Composite Resource encapsulates the environment (namespace + database + bucket + network); same CRD works on both clouds; environment provisioning is cloud-agnostic from the developer perspective; cloud migration risk reduced.

Related patterns

• Internal Developer Platform — platform APIs are the technical underpinning of the self-service layer in the IDP.
• Paved Roads and Golden Paths — golden paths are the UX layer on top of platform APIs.
• Platform Security — OPA/Gatekeeper admission webhooks integrate with the Kubernetes API to enforce security policies on all platform resource requests.

Further reading

• Crossplane Composition — Crossplane Docs
• Kubebuilder Introduction — Kubebuilder Book
• Software Catalog API — Backstage Docs