GuideContainers

Guide

Containers

The container API of every workload - images, environment, ports, resources, probes, lifecycle, security - and sidecars and init containers as classes.

Deployments, StatefulSets, Jobs and CronJobs run containers. This page covers the container API they share.

One class, two roles

The base classes AbstractContainerDeployment, AbstractContainerStatefulSet, AbstractContainerJob and AbstractContainerCronJob implement the container API themselves: the manifest class is also its pod's main container. For the usual workload, one meaningful container per pod, that makes one readable class. Pod-level methods (replicas(), volumes()) and container-level methods (image(), env()) sit side by side. The container is named app unless containerName() says otherwise.

Images

Image is a value object built from a name, or parsed from a full reference with Image::fromString('registry.example.com/acme/api:1.4.0'):

src/K8S/Shop/Manifest/ApiDeployment.phnsrc/K8S/Shop/Manifest/ApiDeployment.php
image(): Image => Image::fromName('api')
    ->setPrefix('acme')
    ->setRegistryUrl('registry.example.com')
    ->setTag($this->imageTag);

imagePullPolicy(): ImagePullPolicy => ImagePullPolicy::Always;
public function image(): Image
{
    return Image::fromName('api')
        ->setPrefix('acme')
        ->setRegistryUrl('registry.example.com')
        ->setTag($this->imageTag);
}

public function imagePullPolicy(): ImagePullPolicy
{
    return ImagePullPolicy::Always;
}

imagePullPolicy() returns the ImagePullPolicy enum (Always, Never, IfNotPresent); the default is IfNotPresent. Taking the tag through the constructor lets the pipeline decide which build deploys while the class decides what it is (Configuration and DI).

Environment variables

EnvConfigurator covers every source of a variable. Sources that are manifests are named by class; ConfigMaps and Secrets has the details:

src/K8S/Shop/Manifest/ApiDeployment.phnsrc/K8S/Shop/Manifest/ApiDeployment.php
env(EnvConfigurator $env): void {
    $env->var('LOG_FORMAT', 'json')                                    // a literal
        ->addConfigMapClass(ApiConfigMap::class)                      // every key of a ConfigMap
        ->varFromSecretClass('DB_PASSWORD', DatabaseSecret::class, DatabaseSecret::KEY_PASSWORD) // one key
        ->addSecret('payments-provider', external: true)              // a Secret this project does not render
        ->varFromPod('POD_NAME', PodField::metadataName())            // the downward API
        ->varFromContainerResources('MEMORY_LIMIT', ContainerResourcesField::memoryLimits());
}
public function env(EnvConfigurator $env): void
{
    $env->var('LOG_FORMAT', 'json')                                    // a literal
        ->addConfigMapClass(ApiConfigMap::class)                      // every key of a ConfigMap
        ->varFromSecretClass('DB_PASSWORD', DatabaseSecret::class, DatabaseSecret::KEY_PASSWORD) // one key
        ->addSecret('payments-provider', external: true)              // a Secret this project does not render
        ->varFromPod('POD_NAME', PodField::metadataName())            // the downward API
        ->varFromContainerResources('MEMORY_LIMIT', ContainerResourcesField::memoryLimits());
}

external: true marks a name this project does not render, such as a Secret an operator writes; without it, a name that matches no manifest fails the render.

Ports

src/K8S/Shop/Manifest/ApiDeployment.phnsrc/K8S/Shop/Manifest/ApiDeployment.php
ports(PortsConfigurator $ports): void {
    $ports->add(self::PORT)->setName('http');
    $ports->add(9090)->setName('metrics')->setProtocol(IPProtocol::TCP);
}
public function ports(PortsConfigurator $ports): void
{
    $ports->add(self::PORT)->setName('http');
    $ports->add(9090)->setName('metrics')->setProtocol(IPProtocol::TCP);
}

Name ports: Services, probes and routes read better with http than with 8080.

Resources

CPU and memory are values, not strings:

src/K8S/Shop/Manifest/ApiDeployment.phnsrc/K8S/Shop/Manifest/ApiDeployment.php
resources(ContainerResourcesInterface $resources): void {
    $resources
        ->requestCPU(CPU::millicores(250))
        ->requestMemory(Memory::mebibytes(256))
        ->limitCPU(CPU::cores(1))
        ->limitMemory(Memory::mebibytes(512));
}
public function resources(ContainerResourcesInterface $resources): void
{
    $resources
        ->requestCPU(CPU::millicores(250))
        ->requestMemory(Memory::mebibytes(256))
        ->limitCPU(CPU::cores(1))
        ->limitMemory(Memory::mebibytes(512));
}

CPU and Memory compare and add (add(), greaterThan(), lowerThan(), equals()), so code can derive one quantity from another, or a subscriber can refuse a production container below a floor. resources() is one of the two methods with environment variants by default: resourcesProd() sizes production (Environments).

Probes

src/K8S/Shop/Manifest/ApiDeployment.phnsrc/K8S/Shop/Manifest/ApiDeployment.php
probes(ProbesConfigurator $probes): void {
    $probes->readinessProbe()
        ->setInitialDelaySeconds(5)
        ->setPeriodSeconds(15)
        ->action()->httpGet(self::PORT)->setPath('/health');

    $probes->livenessProbe()
        ->setPeriodSeconds(20)
        ->setFailureThreshold(3)
        ->action()->tcpSocket(self::PORT);

    $probes->startupProbe()
        ->setPeriodSeconds(10)
        ->setFailureThreshold(30)
        ->action()->exec(['bin/healthcheck']);
}
public function probes(ProbesConfigurator $probes): void
{
    $probes->readinessProbe()
        ->setInitialDelaySeconds(5)
        ->setPeriodSeconds(15)
        ->action()->httpGet(self::PORT)->setPath('/health');

    $probes->livenessProbe()
        ->setPeriodSeconds(20)
        ->setFailureThreshold(3)
        ->action()->tcpSocket(self::PORT);

    $probes->startupProbe()
        ->setPeriodSeconds(10)
        ->setFailureThreshold(30)
        ->action()->exec(['bin/healthcheck']);
}

Each probe has the usual settings (setTimeoutSeconds(), setSuccessThreshold() and the ones above) and one action(): httpGet($port) with setPath(), setScheme(), addHeader(); tcpSocket($port); or exec($command).

Lifecycle and security

src/K8S/Shop/Manifest/ApiDeployment.phnsrc/K8S/Shop/Manifest/ApiDeployment.php
lifecycle(LifecycleConfigurator $lifecycle): void => $lifecycle->preStop()->exec(['sleep', '5']);

securityContext(SecurityContextConfigurator $context): void {
    $context->runAsNonRoot(true)
        ->runAsUser(1000)
        ->setReadOnlyRootFilesystem(true)
        ->allowPrivilegeEscalation(false);

    $context->capabilities()->drop(['ALL']);
}
public function lifecycle(LifecycleConfigurator $lifecycle): void
{
    $lifecycle->preStop()->exec(['sleep', '5']);
}

public function securityContext(SecurityContextConfigurator $context): void
{
    $context->runAsNonRoot(true)
        ->runAsUser(1000)
        ->setReadOnlyRootFilesystem(true)
        ->allowPrivilegeEscalation(false);

    $context->capabilities()->drop(['ALL']);
}

command() and args() receive a StringList:

src/K8S/Shop/Manifest/WorkerDeployment.phnsrc/K8S/Shop/Manifest/WorkerDeployment.php
namespace App\K8S\Shop\Manifest;

use K8Schema\Collection\StringList;
use Klass8s\Core\Container\Image\Image;
use Klass8s\Core\Deployment\AbstractContainerDeployment;

class WorkerDeployment extends AbstractContainerDeployment
{
    static shortName(): string => 'worker';

    image(): Image => Image::fromName('acme/shop-api')->setTag('1.4.0');

    command(StringList $command): void => $command->addAll(['php', 'bin/console']);

    args(StringList $args): void => $args->addAll(['messenger:consume', 'async', '--time-limit=3600']);
}
<?php

declare(strict_types=1);

namespace App\K8S\Shop\Manifest;

use K8Schema\Collection\StringList;
use Klass8s\Core\Container\Image\Image;
use Klass8s\Core\Deployment\AbstractContainerDeployment;

class WorkerDeployment extends AbstractContainerDeployment
{
    public static function shortName(): string
    {
        return 'worker';
    }

    public function image(): Image
    {
        return Image::fromName('acme/shop-api')->setTag('1.4.0');
    }

    public function command(StringList $command): void
    {
        $command->addAll(['php', 'bin/console']);
    }

    public function args(StringList $args): void
    {
        $args->addAll(['messenger:consume', 'async', '--time-limit=3600']);
    }
}

Sidecars

A container that is not the main one is a class of its own extending AbstractContainer: the same container API, no manifest identity. Return it from the workload's containers():

src/K8S/Shop/Container/LogShipperContainer.phnsrc/K8S/Shop/Container/LogShipperContainer.php
namespace App\K8S\Shop\Container;

use Klass8s\Core\Container\AbstractContainer;
use Klass8s\Core\Container\Image\Image;

// A sidecar: the container API without a manifest. Not a service; constructed where it is used.
class LogShipperContainer extends AbstractContainer
{
    containerName(): string => 'logs';

    image(): Image => Image::fromName('fluent/fluent-bit')->setTag('3.1');
}
<?php

declare(strict_types=1);

namespace App\K8S\Shop\Container;

use Klass8s\Core\Container\AbstractContainer;
use Klass8s\Core\Container\Image\Image;

// A sidecar: the container API without a manifest. Not a service; constructed where it is used.
class LogShipperContainer extends AbstractContainer
{
    public function containerName(): string
    {
        return 'logs';
    }

    public function image(): Image
    {
        return Image::fromName('fluent/fluent-bit')->setTag('3.1');
    }
}
src/K8S/Shop/Manifest/ApiDeployment.phnsrc/K8S/Shop/Manifest/ApiDeployment.php
containers(): iterable {
    yield $this;                     // the main container: this class
    yield new LogShipperContainer();
}
public function containers(): iterable
{
    yield $this;                     // the main container: this class
    yield new LogShipperContainer();
}

Container classes are not services: you construct them, often with arguments. They live in the App's Container/ directory, which config/services.yaml excludes from service discovery (Installation). Because a sidecar is a class, one definition serves every workload that needs it, and a subscriber can add it to every pod without any workload naming it.

Init containers

The same, from initContainers():

src/K8S/Shop/Container/WaitForDatabaseContainer.phnsrc/K8S/Shop/Container/WaitForDatabaseContainer.php
namespace App\K8S\Shop\Container;

use K8Schema\Collection\StringList;
use Klass8s\Core\Container\AbstractContainer;
use Klass8s\Core\Container\Image\Image;

// An init container that waits until the database answers; the host comes from whoever uses it.
class WaitForDatabaseContainer extends AbstractContainer
{
    __construct(private string $host, private int $port = 5432) {}

    containerName(): string => 'wait-for-database';

    image(): Image => Image::fromName('busybox')->setTag('1.37');

    command(StringList $command): void
        => $command->addAll(['sh', '-c', sprintf('until nc -z %s %d; do sleep 2; done', $this->host, $this->port)]);
}
<?php

declare(strict_types=1);

namespace App\K8S\Shop\Container;

use K8Schema\Collection\StringList;
use Klass8s\Core\Container\AbstractContainer;
use Klass8s\Core\Container\Image\Image;

// An init container that waits until the database answers; the host comes from whoever uses it.
class WaitForDatabaseContainer extends AbstractContainer
{
    public function __construct(private string $host, private int $port = 5432) {}

    public function containerName(): string
    {
        return 'wait-for-database';
    }

    public function image(): Image
    {
        return Image::fromName('busybox')->setTag('1.37');
    }

    public function command(StringList $command): void
    {
        $command->addAll(['sh', '-c', sprintf('until nc -z %s %d; do sleep 2; done', $this->host, $this->port)]);
    }
}
src/K8S/Shop/Manifest/MigrationJob.phnsrc/K8S/Shop/Manifest/MigrationJob.php
namespace App\K8S\Shop\Manifest;

use App\K8S\Shop\Container\WaitForDatabaseContainer;
use K8Schema\Collection\StringList;
use Klass8s\Core\Container\Image\Image;
use Klass8s\Core\Job\AbstractContainerJob;

class MigrationJob extends AbstractContainerJob
{
    static shortName(): string => 'migration';

    image(): Image => Image::fromName('acme/shop-api')->setTag('1.4.0');

    initContainers(): iterable {
        yield new WaitForDatabaseContainer('postgres.database.svc.cluster.local');
    }

    command(StringList $command): void => $command->addAll(['php', 'bin/console', 'doctrine:migrations:migrate', '--no-interaction']);
}
<?php

declare(strict_types=1);

namespace App\K8S\Shop\Manifest;

use App\K8S\Shop\Container\WaitForDatabaseContainer;
use K8Schema\Collection\StringList;
use Klass8s\Core\Container\Image\Image;
use Klass8s\Core\Job\AbstractContainerJob;

class MigrationJob extends AbstractContainerJob
{
    public static function shortName(): string
    {
        return 'migration';
    }

    public function image(): Image
    {
        return Image::fromName('acme/shop-api')->setTag('1.4.0');
    }

    public function initContainers(): iterable
    {
        yield new WaitForDatabaseContainer('postgres.database.svc.cluster.local');
    }

    public function command(StringList $command): void
    {
        $command->addAll(['php', 'bin/console', 'doctrine:migrations:migrate', '--no-interaction']);
    }
}
Renderedbin/console k8s:render:apps shop
# File: /shop/migration.job.yaml
apiVersion: batch/v1
kind: Job
metadata:
  annotations:
    klass8s.dev/app: shop
    klass8s.dev/deploy-priority: '0'
  labels:
    app.kubernetes.io/name: shop
    app.kubernetes.io/component: migration
  name: shop-migration
spec:
  suspend: false
  template:
    metadata:
      annotations:
        klass8s.dev/env-sources-checksum: d41d8cd98f00b204e9800998ecf8427e
        klass8s.dev/volume-sources-checksum: d41d8cd98f00b204e9800998ecf8427e
      labels:
        app.kubernetes.io/name: shop
        app.kubernetes.io/component: migration
    spec:
      containers:
        -
          command:
            - php
            - bin/console
            - 'doctrine:migrations:migrate'
            - '--no-interaction'
          image: 'acme/shop-api:1.4.0'
          imagePullPolicy: IfNotPresent
          name: app
      initContainers:
        -
          command:
            - sh
            - '-c'
            - 'until nc -z postgres.database.svc.cluster.local 5432; do sleep 2; done'
          image: 'busybox:1.37'
          imagePullPolicy: IfNotPresent
          name: wait-for-database
      restartPolicy: Never

The way out

configureContainer(Container $container) receives the generated klass8s/k8schema container after every configurator has run, for any field the configurators lack.

Next