StorageCluster


The Portworx Operator manages the complete lifecycle of a Portworx cluster. It provides easy installation and configuration managment for Portworx. The Portworx cluster configuration if specified by a Kubernetes CRD (CustomResourceDefinition) called StorageCluster. The StorageCluster object acts as the definition of the Portworx Cluster.

As shown in the below diagram, it manages the Portworx platform consisting of the Portworx nodes, STORK, Lighthouse and other components that make running stateful applications seamless for the users.

Portworx Operator

The StorageCluster provides a Kubernetes native experience to manage a Portworx cluster just like any other application in Kubernetes. Simply creating or editing a StorageCluster object will result in the Operator creating or updating the Portworx cluster in the background.

It is recommended to use the Portworx Spec Generator to create a StorageCluster spec. The spec generator will walk you through different options which you select based on your environment to generate a StorageCluster spec.

If you want to generate the StorageCluster spec manually, you can refer to the following examples and the schema description of StorageCluster.

StorageCluster Examples

Here are some sample Portworx configurations for reference. You can set various fields as per your environment and cluster requirements.

  • Portworx with Internal KVDB and all unused devices on the system
apiVersion: core.libopenstorage.org/v1alpha1
kind: StorageCluster
metadata:
  name: portworx
  namespace: kube-system
spec:
  image: portworx/oci-monitor:2.1.2
  kvdb:
    internal: true
  storage:
    useAll: true
  • Portworx with external ETCD, with STORK and Lighthouse enabled.
apiVersion: core.libopenstorage.org/v1alpha1
kind: StorageCluster
metadata:
  name: portworx
  namespace: kube-system
spec:
  image: portworx/oci-monitor:2.1.2
  kvdb:
    endpoints:
    - etcd:http://etcd-1.net:2379
    - etcd:http://etcd-2.net:2379
    - etcd:http://etcd-3.net:2379
    authSecret: px-etcd-auth
  stork:
    enabled: true
    image: openstorage/stork:2.2.4
    args:
      health-monitor-interval: "100"
  userInterface:
    enabled: true
    image: portworx/px-lighthouse:2.0.4
  • Portworx with update and delete strategies and placement rules
apiVersion: core.libopenstorage.org/v1alpha1
kind: StorageCluster
metadata:
  name: portworx
  namespace: kube-system
spec:
  image: portworx/oci-monitor:2.1.2
  updateStrategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 20%
  deleteStrategy:
    type: UninstallAndWipe
  placement:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: px/enabled
            operator: NotIn
            values:
            - "false"
          - key: node-role.kubernetes.io/master
            operator: DoesNotExist
  • Portworx with custom image registry, network interfaces and misc options
apiVersion: core.libopenstorage.org/v1alpha1
kind: StorageCluster
metadata:
  name: portworx
  namespace: kube-system
spec:
  image: portworx/oci-monitor:2.1.2
  imagePullPolicy: Always
  imagePullSecret: regsecret
  customImageRegistry: docker.private.io/repo
  network:
    dataInterface: eth1
    mgmtInterface: eth1
  secretsProvider: vault
  runtimeOptions:
    num_io_threads: "10"
  env:
  - name: VAULT_ADDRESS
    value: "http://10.0.0.1:8200"

StorageCluster Schema

spec.image (string)

Specify the Portworx monitor image (Example: portworx/oci-monitor:2.1.2)

spec.imagePullPolicy (string)

Image pull policy for all the images deployed by the operator. Default: Always

spec.imagePullSecret (string)

Image pull secret is the name of the secret in the same namespace as the StorageCluster. This is used for pulling images from a secure registry.

spec.customImageRegistry (string)

Specify a custom container registry server that will be used to Docker images. You may include the repository as well (Example: myregistry.net:5443, myregistry.net/myrepository)

spec.secretsProvider (string)

Name of the secrets provider that Portworx will connect to for features like volume encryption, cloudsnap, etc. Default: k8s (Kubernetes secrets)

spec.runtimeOptions (map[string]string)

Runtime options is a map string keys and values used to overwrite Portworx runtime options.

spec.featureGates (map[string]string)

Feature gates is a map string key and values to enable/disable Portworx features.

spec.env[] (object array)

Env is a list of Kubernetes like environment variables. Just like how environment variables are provided in Kubernetes, you can directly give values or import from a source like Secret, ConfigMap, etc.

KVDB configuration

This section contains all the details to configure the key-value database used by Portworx. If endpoints are not specified, the Operator starts Portworx with internal KVDB.

spec.kvdb.internal (boolean)

If you want to use Portworx’s internal kvdb, you can set this option.

spec.kvdb.endpoints[] (string array)

If using external key-value database like ETCD, Consul, specify the endpoints to connect to it in this list. If the endpoints are specified, then spec.kvdb.internal is ignored and the external KVDB will be used.

spec.kvdb.authSecret (string)

KVDB Auth secret is the name of the secret in the same namespace as StorageCluster. This secret should have information needed to authenticate with the KVDB. It could username/password for basic authentication or certificate information or ACL token.

  • If using username/password for authentication the secret should have keys called username and password.
  • If using certificates for authentication the secret should have keys called kvdb-ca.crt, kvdb.crt and kvdb.key for CA certificate, certificate and corresponding certificate key respectively.
  • If using ACL token for authentication the secret should have key called acl-token.

Storage configuration

This section contains all the details to configure the storage for the Portworx cluster. If no devices are specified, the Operator starts Portworx with spec.storage.useAll set to true.

spec.storage.useAll (boolean)

This tells Portworx to use all available, unformatted, unpartioned devices. It will be ignored if spec.storage.devices is not empty.

spec.storage.useAllWithPartitions (boolean)

This tells Portworx to use all available unformatted devices. It will be ignored if spec.storage.devices is not empty.

spec.storage.forceUseDisks (boolean)

This tells Portworx to use the devices even if there is file system present on it. Note that the drives may be wiped before using.

spec.storage.devices[] (string array)

Devices is an array of devices that should be used by Portworx.

spec.storage.journalDevice (string)

Device that will be used for journaling by Portworx.

spec.storage.systemMetadataDevice (string)

Device that will be used for storing metadata by Portworx. It is recommended to have a system metadata device when using internal KVDB for better performance.

Cloud storage configuration

This section contains all the details to configure the storage in cloud. This enables Portworx to manage cloud disks automatically for the user based on given specs. spec.storage takes precedence over this section. Make sure spec.storage is empty when specifying cloud storage.

spec.cloudStorage.deviceSpecs[] (string array)

Device specs is a list of storage device specs. A cloud disk will be created for every spec in the list.

spec.cloudStorage.journalDeviceSpec (string)

Device spec for device that will be used for journaling by Portworx.

spec.cloudStorage.systemMetadataDeviceSpec (string)

Device spec for device that will be used for storing metadata by Portworx. It is recommended to have a system metadata device when using internal KVDB for better performance.

spec.cloudStorage.maxStorageNodesPerZone (uint32)

Specify maximum number of storage nodes per zone. Portworx will not provision drives for additional nodes in the zone and start them as compute only nodes.

spec.cloudStorage.maxStorageNodes (uint32)

Specify maximum number of total storage nodes. Portworx will not provision drives for additional nodes in the cluster and start them as compute only nodes. It is recommended to use maxStorageNodesPerZone as a best practice.

Network configuration

This section contains network information needed by Portworx. If nothing is specified, Portworx will auto detect and choose network interfaces.

spec.network.dataInterface (string)

Data interface allows users to override the auto selected network interace for data traffic.

spec.network.mgmtInterface (string)

Management interface allows users to override the auto selected network interace for control plane traffic.

Placement Rules

Placement lets your override where Portworx will be deployed. By default Portworx gets deployed on all worker nodes.

spec.placement.nodeAffinity (object)

Kubernetes like node affinity to restrict Portworx on certain nodes.

Update Strategy

Similar to Kubernetes DaemonSet, this allows you to specify a update strategy for Portworx updates.

spec.updateStrategy.type (object)

Type of update strategy to use. Currently it supports RollingUpdate and OnDelete. Default: RollingUpdate.

spec.updateStrategy.rollingUpdate.maxUnavailable (intOrString)

Similar to Kubernetes rolling update strategy you can specify this section for rolling updates. The default value is 1 which means only one node will be down at a given point of time. You can specify a static number of percentage value. (Example: 3, 30%, etc)

Delete/Uninstall Strategy

This section contains information on how to uninstall the Portworx cluster.

spec.deleteStrategy.type

Type of delete strategy to use. Deleting the Portworx StorageCluster object will trigger this delete strategy. By default there is no delete strategy, which means only the Kubernetes components deployed by the Operator will be removed leaving the Portworx systemd service running without disturbing the apps running on it. Currently there are two supported delete strategies:

  • Uninstall - Uninstall all Portworx components from the system leaving the devices and KVDB intact.
  • UninstallAndWIpe - Uninstall all Portworx components from the system and also wipe the devices and metadata from KVDB.

Stork configuration

This section contains information to enable and manage stork deployment through the Portworx Operator.

spec.stork.enabled (boolean)

You can enable/disable STORK by toggling this flag at any given time.

spec.stork.image (string)

Specify the STORK image.

spec.stork.args (map[string]string)

A map of string keys and values to override the default STORK arguments or to add new arguments.

spec.stork.env[] (object array)

A list of Kubernetes like environment variables that need to be passed to STORK.

Lighthouse configuration

This section contains information to enable and manage Lighthouse deployment through the Portworx Operator.

spec.userInterface.enabled (boolean)

You can enable/disable Lighthouse by toggling this flag at any given time.

spec.userInterface.image (string)

Specify the Lighthouse image.


Last edited: Thursday, Aug 8, 2019