Storage Abstraction Layer

Flow ID: SY-28 Module(s): Storage, UploadFileOptions, UploadService, storageJobs, iqvia Complexity: Medium Last Updated: 2026-04-28

Business Overview

The Storage Abstraction Layer decouples all file operations (read, write, delete, move, list) from the underlying storage backend. Instead of scattering file_get_contents() and unlink() calls throughout the codebase, every file operation goes through a unified Storage class backed by League Flysystem 3.x. This lets a deployment run on local disk during development yet switch to S3-compatible object storage in production with a single environment variable change -- no code modifications required.

Three drivers are supported: local filesystem, S3 (any S3-compatible API including MinIO and Hetzner Object Storage), and SFTP (used for the IQVIA pharmaceutical data integration). The layer is organized around named storage types (logical purposes like files or iqvia) each containing one or more disks (backend configurations like local, s3, s3Demo, sftp). A global storage() helper provides singleton access with static instance caching, meaning the same disk connection is reused across a request.

Architecture

storage($type, $disk)          <-- global helper (singleton cache)
       |
       v
 Advisable\Storage\Storage     <-- facade over Flysystem
       |
       +-- createFilesystem()
       |        |
       |        +-- 'local'  --> LocalFilesystemAdapter (FCPATH root)
       |        +-- 's3'     --> AwsS3V3Adapter (S3Client + bucket + prefix)
       |        +-- 'sftp'   --> SftpAdapter (SftpConnectionProvider)
       |
       v
 League\Flysystem\Filesystem   <-- Flysystem 3.x unified API

Key Components

Component	Path	Role
Storage	src/Storage/Storage.php	Main class: wraps Flysystem, creates adapters from config, exposes put, get, delete, move, exists, url, size, listContents, and stream variants
storage() helper	application/helpers/storage_helper.php	Global function returning singleton Storage instances keyed by type:disk
Storage config	application/config/storage.php	Defines storage types and their disk configurations
UploadFileOptions	ecommercen/libraries/UploadFileOptions.php	Value object pairing an uploaded file with its target storage type/disk
AdvUploadFileNameGenerator	ecommercen/libraries/AdvUploadFileNameGenerator.php	Generates unique filenames, checking existence via storage()->exists()
AdvUploadValidator	ecommercen/libraries/AdvUploadValidator.php	Validates uploaded files (size, type, dimensions) before storage
Advuploader	application/libraries/Advuploader.php	Legacy upload library routing through UploadFileOptions and Storage
UploadService	src/Domains/Support/Upload/UploadService.php	Modern domain-layer upload service (DI-managed) delegating to storage()
UploadFieldConfig	src/Domains/Support/Upload/UploadFieldConfig.php	Value object for upload field configuration (folder, allowed types, max size)
UploadException	src/Domains/Support/Upload/UploadException.php	Exception thrown on upload validation/storage failure
Upload container	src/Domains/Support/Upload/container.php	DI registration for UploadService
Controller (base)	ecommercen/core/Controller.php	Loads storage helper and config on every request

Storage Migration Jobs

Component	Path	Role
AdvCopyLocalToS3	ecommercen/storageJobs/AdvCopyLocalToS3.php	Copies all files under /files/ from local disk to S3, skipping files that already exist on S3
AdvCopyDemoFiles	ecommercen/storageJobs/AdvCopyDemoFiles.php	Copies files from s3Demo disk to s3 disk (demo data provisioning)
AdvDeleteLocalFiles	ecommercen/storageJobs/AdvDeleteLocalFiles.php	Deletes the local files/ directory after migration to S3
AdvDeleteS3Files	ecommercen/storageJobs/AdvDeleteS3Files.php	Deletes all files from S3 under /files/ (cleanup utility)
AdvUpgradeStorage	ecommercen/storageJobs/AdvUpgradeStorage.php	Relocates legacy folder structures (forms, invoices, vouchers, import) to new paths
AdvInternalMoveFolder	ecommercen/storageJobs/AdvInternalMoveFolder.php	Moves a local folder and its contents to a new local path

Code Flow

Disk Selection and Adapter Creation

1. Caller requests storage: storage('files') or storage('files', 's3') or storage('iqvia').
2. Singleton check: The storage() helper in application/helpers/storage_helper.php maintains a static $instances array keyed by "type:disk". If a matching instance exists, it is returned immediately.
3. Construction: new Storage($type, $disk) loads application/config/storage.php via CodeIgniter's config loader.
4. Default disk resolution: If no $disk was passed, the type's default key is used (e.g., env('FILES_STORAGE_DISK', 'local') for the files type).
5. Adapter creation (createFilesystem()): A switch on $diskConfig['driver'] selects the adapter:
   • local: Creates LocalFilesystemAdapter rooted at FCPATH (the public web root). The Filesystem is configured with public_url set to APP_CDN_HOST or APP_BASE_URL.
   • s3: Creates an S3Client with credentials from env vars, then wraps it in AwsS3V3Adapter with the configured bucket and prefix.
   • sftp: Creates SftpConnectionProvider with host, username, password, private key, passphrase, port, and timeout, then wraps it in SftpAdapter with the configured root path.
6. Ready: The resulting Filesystem instance is stored in $this->filesystem and all subsequent operations delegate to it.

File Upload Flow (Legacy)

1. HTTP upload arrives: $_FILES[$postFileKey] contains the uploaded file.
2. Validation: AdvUploadValidator checks size limits, file type allowlist, and image dimensions.
3. Options construction: An UploadFileOptions is created with storageType: 'files' and the target path.
4. Filename generation: AdvUploadFileNameGenerator generates an MD5-hashed filename (if encrypt is true) and checks storage()->exists() to avoid collisions. If a collision is found and overwrite is false, it appends incrementing suffixes (up to maxFilenameIncrement).
5. Write: $uploadFileOptions->storage()->put($path . $filename, file_get_contents($tmpName)) writes the file to the configured storage backend.
6. Result: The generated filename is returned to the caller for database persistence.

File Upload Flow (Modern)

1. Controller prepares field configs: An array of UploadFieldConfig objects is built, each specifying folder, allowedTypes, and maxSize.
2. UploadService::processUploads(): Iterates over field configs, skipping empty $_FILES entries.
3. Per-field upload: For each file, validates with UploadValidator, generates a filename via UploadFileNameGenerator, and writes via storage('files')->put().
4. Error handling: Any failure throws UploadException (extends ValidationException) with per-field error details.

File Deletion (Common Pattern)

Throughout the codebase, file deletion follows a consistent pattern:

try {
    storage()->delete("files/{subdirectory}/{$filename}");
} catch (\Exception $exception) {
    log_message('error', "Failed to delete file ...: " . $exception->getMessage());
}

This pattern appears in models for products, categories, sliders, slides, blog posts, blog categories, blog authors, documents, events, and more. Deletions are always wrapped in try/catch blocks, logging failures but not aborting the parent operation.

Storage Migration (Local to S3)

1. AdvCopyLocalToS3::execute(): Creates a local source and s3 destination via storage(disk: 'local') and storage(disk: 's3').
2. Diff loop: Calls filesToCopy() which lists all files under /files/ on both source and destination, computes array_diff(), and returns files present on local but missing on S3.
3. Copy: For each file in the diff, reads from source with $source->get($file) and writes to destination with $destination->put($file, $content).
4. Iterative: The while (!empty($filesToCopy = ...)) loop continues until no files remain, handling cases where new files appear during migration.

IQVIA SFTP Upload

1. AdvIqviaUpload::executeCommand(): Determines the date range and filename for the CSV export.
2. Data extraction: Queries order data and enriches with product barcodes.
3. CSV generation: Builds a pipe-delimited CSV string with IQVIA-specific column format.
4. Upload: storage('iqvia')->put($filename, $csv) writes the CSV directly to the IQVIA SFTP server.

Video Stream Integration

BunnyStream receives a Storage instance via its constructor. It uses:

• $this->storage->exists($filePath) to verify video files exist before upload
• $this->storage->size($filePath) to determine file size for upload timeout calculation
• $this->storage->getStream($filePath) to stream video content to the Bunny CDN API

Data Model

No dedicated database tables. The storage layer operates on the filesystem (local, S3, or SFTP). File references are stored as relative paths in various entity tables:

Table	Column(s)	File Path Pattern
product_media	uri	files/products/{filename}
product_categories	image, banner_image	files/product_category/{filename}, files/categories/{filename}
slides	image	files/slides/{filename}
slideshow_groups	image	files/slides/{filename}
blog	banner_image	files/blogs/{filename}
blog_categories	banner_image	files/blogs/{filename}
blog_authors	banner_image	files/blogs/{filename}
documents, documents_mui	image, file	files/documents/{filename}
videos	thumbnail	files/youtube/{id}.jpg
event_categories	image	files/event_category/{filename}
vendors	image, banner	files/vendors/{filename}

Configuration

Storage Config (application/config/storage.php)

The config defines a types array. Each type has a default disk and a disks map:

Type	Default Disk	Available Disks	Purpose
files	env('FILES_STORAGE_DISK', 'local')	local, s3, s3Demo	All application file storage (images, documents, uploads)
iqvia	env('IQVIA_STORAGE_DISK', 'sftp')	sftp	IQVIA pharmaceutical data CSV uploads

Disk Configurations

files.local

Setting	Value	Description
driver	local	Local filesystem adapter
root	FCPATH	Web root directory (public/)
visibility	public	Files are publicly accessible
public_url	APP_CDN_HOST or APP_BASE_URL	URL prefix for url() method

files.s3

Setting	Env Var	Description
driver	--	s3
key	FILES_S3_ACCESS_KEY_ID	AWS/S3 access key
secret	FILES_S3_SECRET_ACCESS_KEY	AWS/S3 secret key
region	FILES_S3_DEFAULT_REGION	AWS region
bucket	FILES_S3_BUCKET	S3 bucket name
endpoint	FILES_S3_ENDPOINT	Custom endpoint (for non-AWS S3-compatible services)
use_path_style_endpoint	FILES_S3_AWS_USE_PATH_STYLE_ENDPOINT	Path-style URLs (for MinIO, etc.)
prefix	FILES_S3_PREFIX	Key prefix within bucket
url	FILES_S3_URL	Public URL for the bucket

files.s3Demo

Shares the same S3 credentials and bucket as files.s3 but with separate URL and prefix:

Setting	Env Var	Description
url	DEMO_FILES_S3_URL	Demo-specific public URL
prefix	DEMO_FILES_S3_PREFIX	Demo-specific key prefix within shared bucket

iqvia.sftp

Setting	Env Var	Default	Description
driver	--	sftp	SFTP adapter
host	IQVIA_HOST	--	SFTP server hostname
username	IQVIA_USER	--	SFTP username
password	IQVIA_PWD	--	SFTP password
port	IQVIA_PORT	22	SFTP port
root	IQVIA_ROOT	/Home/advisable	Remote root directory
timeout	IQVIA_TIMEOUT	30	Connection timeout in seconds
privateKey	IQVIA_PRIVATE_KEY	--	SSH private key (alternative to password auth)
passphrase	IQVIA_PASSPHRASE	--	Passphrase for the private key

Environment Variables

Variable	Default	Description
FILES_STORAGE_DISK	local	Active disk for the files storage type
FILES_S3_ACCESS_KEY_ID	--	S3 access key
FILES_S3_SECRET_ACCESS_KEY	--	S3 secret key
FILES_S3_DEFAULT_REGION	--	S3 region
FILES_S3_BUCKET	--	S3 bucket name
FILES_S3_ENDPOINT	--	Custom S3 endpoint URL
FILES_S3_AWS_USE_PATH_STYLE_ENDPOINT	false	Use path-style S3 URLs
FILES_S3_URL	--	Public URL for S3 bucket
FILES_S3_PREFIX	''	Key prefix in S3 bucket
DEMO_FILES_S3_URL	{endpoint}/{bucket}/{prefix}	Public URL for demo S3 files
DEMO_FILES_S3_PREFIX	''	Key prefix for demo files
APP_CDN_HOST	--	CDN URL for local storage url() method
APP_BASE_URL	--	Fallback URL when no CDN is configured
IQVIA_STORAGE_DISK	sftp	Active disk for the iqvia storage type
IQVIA_HOST	--	IQVIA SFTP server
IQVIA_USER	--	IQVIA SFTP username
IQVIA_PWD	--	IQVIA SFTP password
IQVIA_PORT	22	IQVIA SFTP port
IQVIA_ROOT	/Home/advisable	IQVIA remote root path
IQVIA_TIMEOUT	30	IQVIA SFTP timeout (seconds)
IQVIA_PRIVATE_KEY	--	IQVIA SSH private key
IQVIA_PASSPHRASE	--	IQVIA SSH key passphrase

Storage API

The Storage class exposes these methods, all delegating to the underlying Filesystem:

Method	Signature	Description
put	put(string $path, string $contents, array $options = []): void	Write string contents to a file
putStream	putStream(string $path, $contents, array $options = []): void	Write a stream resource to a file
get	get(string $path): string	Read file contents as string
getStream	getStream(string $path)	Read file contents as a stream resource
exists	exists(string $path): bool	Check if a file exists
delete	delete(string $path): void	Delete a file
move	move(string $source, string $destination, array $config = []): void	Move/rename a file
url	url(string $path): string	Get the public URL for a file
listContents	listContents(string $path, bool $deep = false): iterable	List files and directories (returns Flysystem StorageAttributes objects)
size	size(string $path): int	Get file size in bytes

Consumers

The storage layer is consumed across the entire codebase. Key consumer categories:

Upload Operations

• Advuploader -- legacy file upload library (application/libraries/Advuploader.php)
• UploadService -- modern upload service (src/Domains/Support/Upload/UploadService.php)
• Editor controller -- WYSIWYG editor image uploads (application/controllers/Editor.php)
• AdvUploadImagesFromZipFile -- bulk product image import from ZIP (ecommercen/job/libraries/AdvUploadImagesFromZipFile.php)

Delete Operations

• Adv_product_category_model -- product category image deletion
• Adv_categories_model -- category banner image deletion
• Adv_sliders_model, Adv_slide_model -- slider/slide image deletion
• Adv_blog_model, Adv_blog_category_model, Adv_blog_author_model -- blog image deletion
• Adv_documents_model, Adv_documents_admin -- document file deletion
• Adv_event_categories_model -- event category image deletion

Read/Existence Operations

• VideoManager -- checks file existence before video upload (src/VideoStream/VideoManager.php)
• BunnyStream -- reads file stream for video upload to Bunny CDN (src/VideoStream/Stream/BunnyStream.php)
• AdvUploadFileNameGenerator -- checks file existence to avoid filename collisions
• AdvBlockBuilderController -- lists builder assets via listContents() (ecommercen/builder/controllers/AdvBlockBuilderController.php)
• Adv_slide_admin -- checks storage for YouTube thumbnail existence

Move/Rename Operations

• AdvUploadImagesFromZipFile -- renames imported product images with hashed filenames
• Image_names controller -- file rename utility (application/controllers/Image_names.php)

Write Operations (Non-Upload)

• FaviconImageGenerator -- writes generated favicon PNG images and manifest.json via storage()->put() (src/Favicon/FaviconImageGenerator.php). Previously manifest.json was written via raw fopen()/fwrite(), silently broken on S3/SFTP deployments; all writes now route through the storage abstraction.
• Adv_video_model -- saves YouTube thumbnail images (ecommercen/video/models/Adv_video_model.php)
• Adv_slide_admin -- saves YouTube thumbnails for slides

External Integrations

• AdvIqviaUpload -- uploads CSV sales data to IQVIA SFTP (ecommercen/iqvia/libraries/AdvIqviaUpload.php)

Client Extension Points

• Custom storage types: Client repos can add new storage types to application/config/storage.php (e.g., a client-specific SFTP endpoint for ERP file exchange).
• Disk override via env: Set FILES_STORAGE_DISK=s3 in .env to switch all file operations to S3 without code changes.
• Upload service override: Create Custom\Domains\Support\Upload\UploadService and alias it in custom/Domains/container.php to customize upload validation or filename generation.
• Storage type in UploadFileOptions: Pass a custom storageType and storageDisk when constructing UploadFileOptions to route specific uploads to different backends.
• Storage jobs: The migration jobs in ecommercen/storageJobs/ can be extended or replaced in client repos to handle client-specific migration workflows.

Business Rules

1. Singleton instances: The storage() helper caches instances per type:disk key. A given storage connection is created once per request, preventing redundant adapter initialization.
2. Default disk fallback: When no disk is specified, the type's default config value is used, which reads from an env var with a hardcoded fallback (e.g., local for files, sftp for IQVIA).
3. Encrypted filenames: By default, UploadFileOptions sets encrypt: true, causing AdvUploadFileNameGenerator to hash filenames with md5(uniqid(mt_rand())) to prevent conflicts and obscure original names.
4. Collision avoidance: When a filename collision is detected, the generator appends incrementing numeric suffixes (1, 2, ... up to maxFilenameIncrement which defaults to 100). If all increments are exhausted, an empty string is returned and the upload fails.
5. Delete-safe pattern: All file deletions are wrapped in try/catch blocks. A failed deletion logs an error but does not abort the parent database operation (e.g., record deletion proceeds even if the file delete fails).
6. Local root is FCPATH: The local disk root is the web-accessible public/ directory. Files are stored under files/ subdirectories (e.g., files/products/, files/slides/).
7. CDN-aware URLs: The local adapter's url() method returns URLs prefixed with APP_CDN_HOST if set, falling back to APP_BASE_URL. This supports transparent CDN integration.
8. S3 path-style endpoints: The use_path_style_endpoint option supports non-AWS S3-compatible services (MinIO, Hetzner Object Storage) that require path-style URLs.
9. Demo file isolation: The s3Demo disk shares the same S3 bucket and credentials as s3 but uses a separate prefix and URL, enabling demo data to coexist with production data in a single bucket.
10. SFTP authentication: The IQVIA SFTP adapter supports both password and private key authentication. Both can be configured simultaneously; the adapter uses whichever is available.
11. Storage migration is incremental: AdvCopyLocalToS3 only copies files missing from the destination. It can be run multiple times safely (idempotent).
12. No DI registration for Storage: The Storage class is not registered in the Symfony DI container. It is always accessed via the storage() global helper or instantiated directly (new Storage()).
13. Helper auto-loaded: The storage helper and storage config are loaded in ecommercen/core/Controller.php, making storage() available in all controller contexts without explicit loading.

Dependencies

Package	Version	Purpose
league/flysystem	^3.0 (3.32.0 installed)	Core filesystem abstraction
league/flysystem-aws-s3-v3	^3.0	S3 adapter
league/flysystem-sftp-v3	^3.0	SFTP adapter
aws/aws-sdk-php	(transitive)	S3 client
phpseclib/phpseclib	(transitive)	SSH/SFTP connections

Related Flows

• SY-18 Image Upload (ZIP) -- Bulk product image import using storage for file operations
• SY-07 Cache Management -- File-based cache (separate from storage layer, but both file-centric)
• AD-02 Product Management -- Product image uploads via storage
• AD-21 Slider Management -- Slide image management using storage
• AD-12 Blog Admin -- Blog image uploads and deletions
• AD-30 Video Management -- Video file operations via BunnyStream + Storage
• SY-29 Deployment Architecture -- local disk storage on Plesk vs. S3 on K8s