Deployment Architecture

Flow ID: SY-29 Module(s): DevOps, Docker, CI/CD, Kubernetes, Plesk Complexity: Very High Last Updated: 2026-05-26

Business Overview

Ecommercen supports two first-class deployment models: traditional Plesk-based VMs and modern Kubernetes. Both are actively used in production and will coexist for the foreseeable future. Plesk is the current standard used by all clients. Kubernetes is the forward-looking option with wecare as the pilot client; full migration across all clients is expected to take 5+ years. The platform's architecture (CodeIgniter 3 + PSR-4 domain layer) runs identically on both targets -- the same codebase, same Docker images (for K8s), same .env-driven configuration.

Method 1: Plesk VM Deployment (Current Standard)

Setup Process

1. VM Provisioning: A Hetzner Cloud VM is provisioned (typically CX-series shared vCPU instances).
2. Plesk Installation: Plesk Obsidian is installed as the server management panel, providing domain management, SSL, database administration, and PHP version control.
3. Domain Setup: Each client domain is configured as a Plesk subscription with its own document root under the standard Plesk webspace directory.
4. Git Extension: The Plesk Git extension is installed. The client's Bitbucket repository is cloned into the webspace. Deployment is triggered via the Plesk UI pull/deploy button or webhook.
5. Composer Extension: The Plesk Composer extension runs composer install --no-dev after each pull to install PHP dependencies.
6. PHP Executable Path: The .env variable APP_PHP_EXECUTABLE is set to the Plesk PHP binary path (e.g., /opt/plesk/php/8.1/bin/php) since Plesk manages multiple PHP versions outside the system PATH.
7. Host Configuration: Apache virtual host directives, PHP-FPM pool settings, and cron schedules are configured through the Plesk UI.

Client Density

Typical density is 1-10 clients per VM, depending on traffic volume and resource requirements. Each client has its own:

• MariaDB database (managed via Plesk)
• PHP-FPM pool
• Document root with independent .env
• Cron schedule (configured in Plesk Scheduled Tasks)

Stack

Component	Details
Web Server	Nginx (reverse proxy) + Apache (mod_php fallback), managed by Plesk
PHP Runtime	PHP-FPM 8.1 via Plesk PHP handler
Database	MariaDB (Plesk-managed instance, shared across clients on the same VM)
SSL	Let's Encrypt via Plesk extension (automatic renewal)
Caching	File-based (FileAdapter) or APCu (ApcuAdapter) -- no Redis by default
Sessions	File-based PHP sessions (default)
Monitoring	None centralized -- per-client application/logs/ directory

Deployment Flow (Plesk)

Developer pushes tag to Bitbucket
        |
        v
Plesk Git Extension: pull latest code
        |
        v
Plesk Composer Extension: composer install --no-dev
        |
        v
Manually run: php migrator.php migrate  (if schema changes)
        |
        v
Clear cache: delete cache/container.php + application cache
        |
        v
Live (instant, no downtime mechanism)

Limitations

Area	Limitation
Scaling	No autoscaling -- vertical scaling only (resize VM)
Rollback	No automated rollback -- manual git checkout to previous tag
Logging	No centralized log aggregation -- logs on each VM's filesystem
Metrics	No monitoring stack -- Plesk basic stats only
GitOps	Manual pull/deploy -- no declarative state reconciliation
Secrets	.env files on disk -- no encrypted secret management
Isolation	Multiple clients share the same VM kernel and MariaDB instance
SSL	Per-client manual setup through Plesk UI
Database	No replication, no automatic failover

Method 2: Kubernetes Deployment (Forward Path)

Cluster Architecture

The production Kubernetes cluster runs on Hetzner Cloud in the hel1 (Helsinki) region:

Role	Count	Instance Type	CPU/RAM	Scheduling
Masters	3	cx42	Shared vCPU	Control plane + shared workloads
DB Workers	2	ccx33	Dedicated vCPU	MariaDB Galera + MaxScale only
Web Workers	4	ccx13	Dedicated vCPU	Application pods + ingress

Cluster Software:

• Distribution: RKE2 (Rancher Kubernetes Engine 2)
• CNI: Cilium with native routing and kube-proxy replacement
• IaC: Terraform for node provisioning, Ansible for RKE2 bootstrap

Docker Images

The platform builds three Docker image types, all based on Alpine Linux:

PHP Image (php.dockerfile)

A 5-stage multi-stage build that compiles PHP 8.1 from source:

Stage	Target	Description
1	php_builder_runtime	Alpine base with build dependencies (autoconf, build-base, libxml2-dev, etc.)
2	php_builder	Compiles PHP 8.1 and extensions (igbinary, phpredis 6.0.2, APCu 5.1.28, Xdebug 3.3.2) from source
3	php_base	Copies compiled binaries into a clean Alpine image with runtime configs
4	php_runtime	Minimal Alpine with only runtime shared libraries (no build tools)
5	Final images	php_fpm (production, no Xdebug), php_fpm_dev (with Xdebug), php_cli (production, no Composer/git), php_cli_dev (with Composer + git)

Key PHP extensions compiled from source: OPcache, igbinary, phpredis (with lz4 + zstd compression), APCu, Xdebug (dev only). GD is built with WebP, JPEG, and FreeType support. Additional libraries: libsodium, vips, libheif, poppler.

PHP-FPM is configured to listen on 0.0.0.0:9000 with clear_env = no (to receive environment variables from Kubernetes) and stderr error logging.

Node Image (node.dockerfile)

A lightweight 2-stage build:

Stage	Target	Description
1	node	Base node:20.16-alpine3.20
2	node_dev	Adds git and SSH client (for private Bitbucket npm dependencies)

Application Bootstrap Image (app.dockerfile)

A 3-stage build that produces the deployable application artifact:

Stage	Target	Base	Description
1	app_builder_composer	php_cli_dev	Copies application source, runs composer install --no-dev with BuildKit secret mount for auth.json
2	app_builder_node	node_dev	Copies frontend assets, runs npm run all-production (npm ci + storefront + admin builds), then removes node_modules
3	app_bootstrap	Alpine 3.20	Merges Composer and Node outputs, installs rsync, sets file permissions (755 dirs, 644 files), creates storage directories (storage/forms, storage/import, storage/invoices, storage/tmp, storage/user_langs, storage/views, storage/vouchers). storage/user_langs is kept for the legacy _user_langs.json migration entry point only — runtime override persistence has moved to the language_overrides DB table (see AD-26).

The bootstrap image's default command runs deploy.sh, which copies the application files from /usr/local/app/ to the target directory (typically an emptyDir volume at /usr/local/var/www/html).

CI/CD Pipeline (Bitbucket Pipelines)

The CI/CD pipeline is triggered on git tags matching the pattern *.*.*:

Developer pushes git tag (e.g., 4.99.5 or 4.99.5.2.wecare)
        |
        v
Bitbucket Pipelines: Tag format detection
        |
        +-- Upstream tag: 1.2.3 --> DOCKER_TAG="1.2.3", tagged as "upstream-latest"
        +-- Client tag:   1.2.3.4.clientname --> DOCKER_TAG="1.2.3.4-clientname", tagged as "clientname-latest"
        +-- Client lock:  ECOMMERCEN_CLIENT env var forces client name
        |
        v
Step 1: Build and Push Docker Image
        |
        +-- docker buildx create (BuildKit builder)
        +-- build-app.sh --> app_bootstrap target
        +-- Platform: linux/amd64
        +-- SBOM + provenance attestations enabled
        +-- Push to Docker Hub with version tag + "latest" alias
        +-- BuildKit layer caching (local)
        |
        v
Step 2: Create Release Notes
        |
        +-- Generate release-notes.md with metadata
        +-- Include docker pull command and attestation inspection command

Tag Format Rules:

Tag Pattern	Type	Docker Tag	Latest Alias
1.2.3	Upstream	1.2.3	upstream-latest
1.2.3.4.clientname	Client	1.2.3.4-clientname	clientname-latest
1.2.3.4 (with ECOMMERCEN_CLIENT)	Client (locked)	1.2.3.4-{ECOMMERCEN_CLIENT}	{ECOMMERCEN_CLIENT}-latest

Build Arguments for Client Tags:

When TAG_TYPE=client, the build receives --build-arg CLIENT_NAME and --build-arg CUSTOM_VERSION, enabling client-specific build-time customizations.

Build Metadata:

Each build produces a build-metadata.json artifact containing build number, commit SHA, tag, Docker tag, image digest, tag type, upstream version, client name, custom version, and ISO 8601 timestamp.

GitOps (Argo CD)

• Pattern: App-of-Apps with Kustomize + Helm hybrid
• Multi-tenancy: ApplicationSet generates per-client application instances from a template
• Authentication: Google SSO via Dex
• RBAC: it team has full access; dev team has read + sync permissions
• Managed services: ~24 services across the cluster (application workloads, databases, Redis, monitoring, ingress, cert-manager, sealed-secrets, etc.)

Application Workload (Per-Client Pod)

Each client runs as a Kubernetes Deployment with the following container architecture:

Pod
 |
 +-- Init Containers (run sequentially before main containers):
 |   |
 |   +-- 1. app-bootstrap (app_bootstrap image)
 |   |       Runs deploy.sh: copies compiled app from image to emptyDir volume
 |   |
 |   +-- 2. app-migrate (php_cli image)
 |           Runs init_migrate.php:
 |             - Executes Phinx migrations (migrator.php migrate)
 |             - Checks registry table for SEEDER.INITIAL_SEED_RAN
 |             - Runs InitialSeed if first deployment
 |
 +-- Main Containers (run concurrently):
 |   |
 |   +-- php-fpm (php_fpm image)
 |   |       PHP-FPM listening on port 9000
 |   |       Serves PHP requests from shared emptyDir volume
 |   |
 |   +-- nginx (sidecar)
 |           Nginx reverse proxy on port 8080
 |           Serves static files, proxies PHP to php-fpm:9000
 |           Read-only mount of the same emptyDir volume
 |
 +-- Shared Volume: emptyDir (populated by app-bootstrap init container)

Argo CD Sync Hook: prepare_infrastructure.php

Runs as a Kubernetes Job during Argo CD sync (before the main deployment rolls out). This PHP script uses the Kubernetes API directly via service account credentials:

Action	Details
App Secrets	Reads the app Kubernetes Secret. On first sync (no ecommercen.com/has-been-initialized annotation), generates APP_ENCRYPTION_KEY, APP_ADMIN_PWD, and APP_ADMIN_ADVISABLE_PWD (16-byte random hex). Marks secret as initialized. Subsequent syncs skip this step.
Database Secrets	Rotates APP_DB_DEFAULT_PWD on every sync (base64-encoded 16-byte random hex). Always updates the secret's resourceVersion.
Keycloak Integration	If APP_ADMIN_SSO_ENABLED=true: authenticates to Keycloak admin API via client credentials, creates or updates an OpenID Connect client for the e-shop's admin SSO. Stores Keycloak client ID/secret in a dedicated Kubernetes Secret. Supports sync and teardown modes.

KEDA Autoscaling

• Trigger: Traefik request rate per pod (15 requests/second/pod threshold) + CPU utilization (70%)
• Range: 4 minimum replicas, 16 maximum replicas
• Scaler: KEDA ScaledObject with Prometheus metrics from Traefik

Version-Gated Cron Jobs

The init_cron.php script runs as an init container before any cron job pod starts:

1. Calls the web service's health endpoint (e.g., http://app-svc:80/_healthz/ready)
2. Parses the JSON response to extract the running version
3. Compares against the cron image's version from application/config/version.php
4. Refuses to start if versions do not match (prevents stale cron jobs from running during a rolling update)
5. Retries up to 5 times with 1-second intervals (configurable via --retries and --timeout flags)

Data Layer

MariaDB

• Operator: MariaDB Operator for Kubernetes
• SaaS clients: Galera cluster (synchronous multi-master replication)
• Dedicated clients: Async primary-replica replication (2-node)
• MaxScale: MariaDB MaxScale proxy with readwritesplit router and automatic failover
• TCP Ingress: MySQL port exposed via Traefik TCP route (for ERP systems connecting externally)

Redis

Dual Redis instances with Sentinel for high availability:

Instance	Eviction Policy	Purpose
Cache Redis	allkeys-lru	Application cache (L2 tier), CDN purge queues
Session Redis	volatile-lru	PHP session storage (sticky, TTL-bound)

Application configuration:

# Cache Redis (L2)
APP_CACHE_L2_ADAPTER=Advisable\Cache\Adapter\RedisAdapter
APP_CACHE_L2_CONNECTION=tcp://redis-cache:6379?prefix={client}

# Session Redis
APP_SESS_DRIVER=redis
APP_SESS_SAVE_PATH=tcp://redis-session:6379?prefix={client}

Observability

Component	Tool	Purpose
Metrics	Prometheus (kube-prometheus-stack)	Cluster and application metrics collection
Dashboards	Grafana	Visualization, alerting, and dashboards
Logs	Loki	Centralized log aggregation from all pods
Access	Cloudflared tunnels	Secure access to 12 management UIs without public exposure

Secrets Management

• Sealed Secrets (Bitnami): All Kubernetes Secrets are encrypted at rest in git using kubeseal. Only the cluster's private key can decrypt them.
• DB password rotation: prepare_infrastructure.php rotates the database password on every Argo CD sync, ensuring credentials change with each deployment.
• Encryption keys: APP_ENCRYPTION_KEY is generated once on first sync and never rotated (annotated with ecommercen.com/has-been-initialized).
• Keycloak secrets: SSO client credentials are generated on first sync and persisted. Subsequent syncs use the stored values as source of truth.

Networking

Component	Details
Ingress Controller	Traefik (deployed as LoadBalancer service via Hetzner Load Balancer)
HTTP/HTTPS	Standard web traffic with TLS termination at Traefik
TCP Routes	MySQL access for ERP systems via MaxScale through Traefik TCP entrypoints
TLS Certificates	cert-manager with Let's Encrypt (HTTP-01 challenge for web domains, DNS-01 via Cloudflare for wildcard/internal domains)

OPcache Preloading

In the K8s PHP-FPM image, OPcache preloading is available via opcache_preload.php. This script runs once at PHP-FPM master startup and compiles PHP files into shared memory:

• Vendor classes: All Composer classmap entries under vendor/ are preloaded (safe, no side effects)
• App classes: All PSR-4 namespaces under src/ are preloaded (skips tests/ and patches/)
• CI3 classes excluded: Legacy CodeIgniter classes are intentionally skipped because load_class() relies on require_once side effects that conflict with preloaded class definitions

Wecare: Reference Implementation

The first client on Kubernetes is wecare (new.wecare.gr), serving as the reference implementation for the K8s deployment path.

Aspect	Configuration
Database	Dedicated 2-node MariaDB with async replication + MaxScale
Redis	Dual Redis instances (cache + session) with Sentinel
Autoscaling	KEDA: 4-16 pods based on Traefik request rate (15 req/s/pod) + CPU (70%)
Ingress	Traefik -> TLS termination -> PHP-FPM + Nginx sidecar
Storage	S3-compatible object storage (Hetzner Object Storage) via FILES_STORAGE_DISK=s3
SSO	Keycloak admin SSO via OpenID Connect
Observability	Full Prometheus + Grafana + Loki stack

Local Development (Docker Compose)

The .docker/integration/ directory provides a Docker Compose stack for local development that mirrors both deployment targets:

# Development mode (default): bind-mount source, live reloading
./compose.sh up -d

# Production-like mode: uses app_bootstrap image, no source mounts
COMPOSE_ENV=prod ./compose.sh up -d

# Database variants
DB_MODE=mariadb ./compose.sh up -d    # MariaDB (default)
DB_MODE=vitess ./compose.sh up -d     # Vitess (experimental)

# Redis variants
REDIS_MODE=standalone ./compose.sh up -d  # Single Redis (default)
REDIS_MODE=cluster ./compose.sh up -d     # Redis Cluster

Development stack (web-dev.compose.yml): Bind-mounts the project root into the PHP-FPM container for live code changes. Includes Xdebug configuration.

Production-like stack (web-prod.compose.yml): Uses the app_bootstrap image to populate a named volume, then runs init_migrate.php before starting PHP-FPM and Nginx -- exactly mirroring the K8s init container workflow.

Configuration

Key Repositories

Repository	Location	Purpose
Main application	devteamadvisable/adveshop4 (Bitbucket)	Platform source code, Docker images, CI/CD
K8s manifests	Separate Argo CD-managed repository	Kustomize + Helm manifests, sealed secrets
Client repos	Forks of adveshop4 per client	Client-specific code in custom/ and application/

Key Files

File/Directory	Purpose
.docker/images/php.dockerfile	5-stage PHP image build (PHP 8.1.31, Alpine 3.20)
.docker/images/app.dockerfile	3-stage application bootstrap image build
.docker/images/node.dockerfile	Node 20.16 image for frontend builds
.docker/scripts/build/build-common.sh	Shared build logic (PHP_VERSION=8.1.32, IMAGE_VERSION=1.0.0, BuildKit flags)
.docker/scripts/build/build-php.sh	Builds all PHP image targets and exports reference configs
.docker/scripts/build/build-app.sh	Builds the app_bootstrap image
.docker/scripts/build/build-node.sh	Builds node and node_dev images
.docker/scripts/deploy/deploy.sh	Init container script: copies app files to target directory
.docker/scripts/deploy/init_common.php	Shared utilities: logging, script execution, version loading
.docker/scripts/deploy/init_migrate.php	Runs Phinx migrations + conditional InitialSeed
.docker/scripts/deploy/init_cron.php	Version-gated cron readiness check
.docker/scripts/deploy/prepare_infrastructure.php	Argo CD sync hook: secrets rotation, Keycloak setup
.docker/scripts/deploy/opcache_preload.php	OPcache preload script for PHP-FPM startup
.docker/integration/compose.sh	Docker Compose wrapper with env/db/redis mode selection
bitbucket-pipelines.yml	CI/CD pipeline definition
.env.example	Reference for all environment variables
application/config/version.php	Application version (MM.VV.PPP.CCC format, e.g., 04.99.005.000)

Environment Variables (Deployment-Specific)

Variable	Plesk	K8s	Description
ENVIRONMENT	production	production	Runtime environment
APP_PHP_EXECUTABLE	/opt/plesk/php/8.1/bin/php	(default php)	Path to PHP binary
APP_CACHE_L2_ADAPTER	FileAdapter (typical)	RedisAdapter	Persistent cache backend
APP_CACHE_L2_CONNECTION	../cache/	tcp://redis:6379?prefix={client}	Cache connection string
APP_CACHE_L1_ADAPTER	auto (APCu if available)	auto (APCu)	Pod-local cross-request cache
APP_SESS_DRIVER	files	redis	Session storage driver
APP_SESS_SAVE_PATH	(PHP default)	tcp://redis:6379?prefix={client}	Session store connection
FILES_STORAGE_DISK	local	s3 (typical)	File storage backend
APP_SAAS_MANAGED	false	true	Whether managed by SaaS platform
APP_ADMIN_SSO_ENABLED	false	true (if Keycloak)	Admin SSO via Keycloak

Comparison Matrix

Capability	Plesk VM	Kubernetes
Provisioning	Manual (Hetzner UI + Plesk)	IaC (Terraform + Ansible)
Deployment	Git pull + Composer install	Docker image build + Argo CD sync
Scaling	Vertical only (resize VM)	Horizontal autoscaling (KEDA 4-16 pods)
Rollback	Manual git checkout	Argo CD rollback (instant, image-based)
Database HA	Single instance, no failover	Galera/replication + MaxScale auto-failover
Caching	File-based or APCu	Redis with Sentinel (dual instances)
Sessions	File-based	Redis with Sentinel
Secrets	.env on disk	Sealed Secrets (encrypted in git)
SSL	Plesk Let's Encrypt extension	cert-manager (HTTP-01 + DNS-01)
Monitoring	None centralized	Prometheus + Grafana + Loki
Log Aggregation	Local files only	Loki (centralized)
Multi-tenancy	Multiple clients per VM	Namespace isolation per client
Zero-Downtime Deploy	No	Yes (rolling update + init containers)
Cost Per Client	Lower (shared VM)	Higher (dedicated pods + infra)
Operational Complexity	Low	High

Business Rules

1. Both methods are first-class: Neither is deprecated. Plesk is the operational standard for all current clients. Kubernetes is the strategic direction with wecare as the pilot.
2. Same codebase: The application code is identical across both deployment methods. All differences are handled via .env configuration.
3. Version-gated cron: In K8s, init_cron.php prevents stale cron jobs from executing during rolling updates by comparing the cron pod's version against the web service's version.
4. Secrets generated once: APP_ENCRYPTION_KEY and Keycloak credentials are generated on first Argo CD sync only. The ecommercen.com/has-been-initialized annotation prevents regeneration.
5. DB password rotated per sync: Unlike encryption keys, database passwords are rotated on every Argo CD sync via prepare_infrastructure.php.
6. Tag format determines image scope: Upstream tags (1.2.3) produce a generic image tagged upstream-latest. Client tags (1.2.3.4.clientname) produce client-specific images tagged {clientname}-latest and may include client-specific build args.
7. Init container ordering is critical: In K8s, app-bootstrap must complete before app-migrate, and app-migrate must complete before PHP-FPM starts. This is enforced by Kubernetes init container sequential execution.
8. OPcache preloading skips CI3: The preload script intentionally excludes legacy CodeIgniter classes because load_class() depends on require_once side effects that break when classes are already in shared memory.
9. Client repos use ECOMMERCEN_CLIENT: When set as a pipeline variable, this forces all tags (even upstream-format ones) to build as client images, simplifying client repo CI configuration.
10. SBOM attestations: All production Docker images include BuildKit SBOM and provenance attestations for supply chain security.

Related Flows

• SY-01 Cron Framework -- Cron scheduling and execution model (version-gated in K8s via init_cron.php)
• SY-07 Cache Management -- L1/L2 cache architecture (FileAdapter on Plesk, RedisAdapter on K8s)
• SY-22 Job Manager -- Background job execution (same jobs run on both deployment targets)
• SY-26 Circuit Breaker -- External service resilience (cache-backed state requires Redis on K8s)
• SY-27 Deferred Task Runner -- Post-response fire-and-forget tasks
• SY-28 Storage Abstraction -- File storage (local disk on Plesk, S3 on K8s)