Scaling & sizing

The core is stateless

Every piece of state lives in PostgreSQL: sessions, API keys, cache, counters. Installation and migrations take an advisory lock, so any number of replicas can start simultaneously. Run as many core replicas as you need behind any load balancer, with no sticky sessions.

The one per-process thing is the observability buffer (up to 10,000 traces, flushed every 2 s). A replica killed without SIGTERM loses what was in it.

What uses memory

Driver	Size	Control
File upload in flight	Up to MAX_UPLOAD_SIZE (50 MB) each, fully in RAM	Lower MAX_UPLOAD_SIZE; cap concurrency
Streamed response in flight	The whole response, until it ends	Cap concurrency
JSON body	Up to 10 MB each	Fixed
Observability buffer	≤ 10,000 traces, roughly 1 KB each	Fixed
Baseline	~100 MB

Rule of thumb: baseline + concurrency × typical request footprint. The GCP installer's choice of 1 GiB with concurrency 20 assumes chat traffic with occasional uploads. For an upload-heavy workload either give 2 GiB+ or lower the concurrency per replica and add replicas.

CPU is rarely the limit: the core forwards bytes and parses JSON. DLP scanning of very large prompts is the heaviest CPU path.

Connection budget

Each core replica opens:

Pool	Max connections
Main (DATABASE_URL)	10
Observability (OBSERVABILITY_DB_POSTGRESQL)	8

replicas × 18 must fit under PostgreSQL's max_connections with room for administrators and backups. Beyond a handful of replicas, put PgBouncer in transaction mode in front of the database.

Kubernetes manifest

apiVersion: apps/v1
kind: Deployment
metadata: { name: axium-core }
spec:
  replicas: 2
  selector: { matchLabels: { app: axium-core } }
  template:
    metadata: { labels: { app: axium-core } }
    spec:
      terminationGracePeriodSeconds: 20
      imagePullSecrets: [ { name: quay } ]
      containers:
        - name: core
          image: quay.io/axiumlab/core:v0.1.0
          ports: [ { containerPort: 3000 } ]
          env:
            - { name: TRUST_PROXY, value: "true" }
            - { name: PUBLIC_BASE_URL, value: https://axium-api.your-domain.com }
            - { name: OBSERVABILITY_TYPE, value: postgres }
          envFrom:
            - secretRef: { name: axium-core }   # DATABASE_URL, MASTER_ENCRYPTION_KEY, AXIUM_LICENSE, OBSERVABILITY_DB_POSTGRESQL
          resources:
            requests: { cpu: 250m, memory: 512Mi }
            limits:   { memory: 1Gi }
          livenessProbe:  { httpGet: { path: /v1/health, port: 3000 }, periodSeconds: 15 }
          readinessProbe: { httpGet: { path: /v1/install/status, port: 3000 }, periodSeconds: 20 }
          startupProbe:   { httpGet: { path: /v1/health, port: 3000 }, periodSeconds: 5, failureThreshold: 12 }
---
apiVersion: v1
kind: Service
metadata: { name: axium-core }
spec:
  selector: { app: axium-core }
  ports: [ { port: 80, targetPort: 3000 } ]

Autoscale on CPU or, better, on concurrent requests at the ingress. Keep the ui at one or two replicas; it serves static files.

Cloud Run

The installer deploys with --memory 1Gi --cpu 1 --concurrency 20 --min-instances 0 --max-instances 4. Adjust with:

gcloud run services update axium-core --region europe-west1 \
  --min-instances 1 --max-instances 10 --concurrency 20 --memory 2Gi --timeout 900

--min-instances 1 removes cold starts (and the schema check they repeat). Remember the Cloud SQL tier: db-f1-micro allows few connections; move to a dedicated-core tier before scaling the core past two or three instances.

Database growth

Table	Grows with	Keep in check
stats.proxy_daily	apps × models × keys × days	Small; no action.
rm.request_io	cached responses	DELETE …/cache/expired on a schedule.
telemetry.proxy_traces	one row per request	Your retention job or partitioning.