Skip to content

Core Concepts

Architecture Overview

Senren uses a three-component architecture designed for reliability and scalability:

graph TB
    SDK[SDK/User] -->|gRPC| CP[Control Plane]
    CP -->|PostgreSQL| DB[(State Storage)]
    CP -->|Kafka| K[Kafka Topics]
    K -->|Subscribe| RP[Regional Plane]
    RP -->|Apply| CRD[Kubernetes CRDs]
    CRD -->|Watch| CTL[Controllers]
    CTL -->|Provision| INF[Infrastructure]
    CTL -->|Update| CRD
    RP -->|Report| K
    K -->|Status| CP

Control Plane

The control plane is the central coordinator:

  • Receives user requests via gRPC
  • Stores desired state in PostgreSQL
  • Publishes state changes to Kafka
  • Consumes status updates from regions

Key Properties: - Single source of truth for desired state - Transactional storage with PostgreSQL - Reliable message delivery via outbox pattern

Regional Plane

Each Kubernetes cluster runs a regional plane:

  • Consumes state changes from Kafka
  • Computes diff between desired and current state
  • Applies CRDs to Kubernetes API
  • Watches CRD status and reports back

Key Properties: - Stateless (all state in K8s and Kafka) - Independent per region - Automatic retry on failures

Controllers

Kubernetes controllers provision actual infrastructure:

  • Watch CRDs for changes
  • Provision StatefulSets, Services, ConfigMaps
  • Update CRD status with runtime information

Key Properties: - Standard Kubernetes reconciliation pattern - Separate from regional plane (different failure domain) - Can be restarted without losing state

State-Based Synchronization

Senren uses state-based sync instead of event-based commands:

Traditional (Event-Based)

CREATE user-cache
UPDATE user-cache SET memory=1024
DELETE user-cache

Events can be lost, reordered, or replayed incorrectly.

Senren (State-Based)

{
  "version": 42,
  "databases": [
    {
      "name": "user-cache",
      "memory_mb": 1024,
      "region": "us-central"
    }
  ]
}

The complete desired state is published. Regional plane computes the diff.

Benefits: - No lost operations - Idempotent (same state = same result) - Easy to reason about - Natural conflict resolution

Resource Discovery

Platform components are discovered via CRDs, not configuration files:

apiVersion: platform.senren.ai/v1alpha1
kind: RegionalCluster
metadata:
  name: us-central
spec:
  regionId: us-central
  kafka:
    bootstrapServers:
      - kafka.us-central:9092
    topicPrefix: prod

When Terraform provisions a new cluster:

  1. Creates the cluster infrastructure
  2. Creates a RegionalCluster CRD
  3. Control plane discovers it automatically
  4. No manual configuration needed

Outbox Pattern

Senren uses the outbox pattern for reliable message delivery:

BEGIN;
  INSERT INTO registry.redis (...);
  INSERT INTO outbox (topic, payload) VALUES (...);
COMMIT;

A background processor: - Polls the outbox table - Publishes to Kafka - Marks as processed

Benefits: - Transactional consistency (DB + Kafka) - Automatic retry on Kafka failures - No lost messages

Status Reporting

Status flows back through the system:

  1. Controller checks infrastructure readiness
  2. Updates CRD .status field
  3. Regional plane watches status changes
  4. Publishes to Kafka status topic
  5. Control plane stores in database

Users can query status:

let status = client.get_redis_status("prod", "user-cache").await?;
println!("Ready: {}, Host: {}", status.ready, status.host);

Multi-Region

Resources are partitioned by region:

graph LR
    CP[Control Plane] -->|State| K[Kafka]
    K -->|us-central state| RP1[Regional Plane<br/>us-central]
    K -->|eu-west state| RP2[Regional Plane<br/>eu-west]
    RP1 --> K1[K8s us-central]
    RP2 --> K2[K8s eu-west]

Each region: - Has its own Kafka topic partition (by region ID) - Receives only its relevant state - Reports status independently - Can be deployed/updated separately

Next Steps