The Portworx Operator manages the complete lifecycle of a Portworx cluster. It provides easy installation and configuration managment for Portworx. The Portworx cluster configuration if specified by a Kubernetes CRD (CustomResourceDefinition) called StorageCluster. The StorageCluster object acts as the definition of the Portworx Cluster.
As shown in the below diagram, it manages the Portworx platform consisting of the Portworx nodes, STORK, Lighthouse and other components that make running stateful applications seamless for the users.
The StorageCluster provides a Kubernetes native experience to manage a Portworx cluster just like any other application in Kubernetes. Simply creating or editing a StorageCluster object will result in the Operator creating or updating the Portworx cluster in the background.
It is recommended to use the Portworx Spec Generator to create a StorageCluster spec. The spec generator will walk you through different options which you select based on your environment to generate a StorageCluster spec.
Here are some sample Portworx configurations for reference. You can set various fields as per your environment and cluster requirements.
- Portworx with Internal KVDB and all unused devices on the system
apiVersion: core.libopenstorage.org/v1alpha1 kind: StorageCluster metadata: name: portworx namespace: kube-system spec: image: portworx/oci-monitor:2.1.2 kvdb: internal: true storage: useAll: true
- Portworx with external ETCD, with STORK and Lighthouse enabled.
apiVersion: core.libopenstorage.org/v1alpha1 kind: StorageCluster metadata: name: portworx namespace: kube-system spec: image: portworx/oci-monitor:2.1.2 kvdb: endpoints: - etcd:http://etcd-1.net:2379 - etcd:http://etcd-2.net:2379 - etcd:http://etcd-3.net:2379 authSecret: px-etcd-auth stork: enabled: true image: openstorage/stork:2.2.4 args: health-monitor-interval: "100" userInterface: enabled: true image: portworx/px-lighthouse:2.0.4
- Portworx with update and delete strategies and placement rules
apiVersion: core.libopenstorage.org/v1alpha1 kind: StorageCluster metadata: name: portworx namespace: kube-system spec: image: portworx/oci-monitor:2.1.2 updateStrategy: type: RollingUpdate rollingUpdate: maxUnavailable: 20% deleteStrategy: type: UninstallAndWipe placement: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: px/enabled operator: NotIn values: - "false" - key: node-role.kubernetes.io/master operator: DoesNotExist
- Portworx with custom image registry, network interfaces and misc options
apiVersion: core.libopenstorage.org/v1alpha1 kind: StorageCluster metadata: name: portworx namespace: kube-system spec: image: portworx/oci-monitor:2.1.2 imagePullPolicy: Always imagePullSecret: regsecret customImageRegistry: docker.private.io/repo network: dataInterface: eth1 mgmtInterface: eth1 secretsProvider: vault runtimeOptions: num_io_threads: "10" env: - name: VAULT_ADDRESS value: "http://10.0.0.1:8200"
Specify the Portworx monitor image (Example: portworx/oci-monitor:2.1.2)
Image pull policy for all the images deployed by the operator. Default: Always
Image pull secret is the name of the secret in the same namespace as the StorageCluster. This is used for pulling images from a secure registry.
Specify a custom container registry server that will be used to Docker images. You may include the repository as well (Example: myregistry.net:5443, myregistry.net/myrepository)
Name of the secrets provider that Portworx will connect to for features like volume encryption, cloudsnap, etc. Default: k8s (Kubernetes secrets)
Runtime options is a map string keys and values used to overwrite Portworx runtime options.
Feature gates is a map string key and values to enable/disable Portworx features.
spec.env (object array)
Env is a list of Kubernetes like environment variables. Just like how environment variables are provided in Kubernetes, you can directly give values or import from a source like Secret, ConfigMap, etc.
This section contains all the details to configure the key-value database used by Portworx. If endpoints are not specified, the Operator starts Portworx with internal KVDB.
If you want to use Portworx’s internal kvdb, you can set this option.
spec.kvdb.endpoints (string array)
If using external key-value database like ETCD, Consul, specify the endpoints to connect to it in this list.
If the endpoints are specified, then
spec.kvdb.internal is ignored and the external KVDB will be used.
KVDB Auth secret is the name of the secret in the same namespace as StorageCluster. This secret should have information needed to authenticate with the KVDB. It could username/password for basic authentication or certificate information or ACL token.
- If using username/password for authentication the secret should have keys called
- If using certificates for authentication the secret should have keys called
kvdb.keyfor CA certificate, certificate and corresponding certificate key respectively.
- If using ACL token for authentication the secret should have key called
This section contains all the details to configure the storage for the Portworx cluster. If no devices are
specified, the Operator starts Portworx with
spec.storage.useAll set to true.
This tells Portworx to use all available, unformatted, unpartioned devices. It will be ignored if
spec.storage.devices is not empty.
This tells Portworx to use all available unformatted devices. It will be ignored if
spec.storage.devices is not empty.
This tells Portworx to use the devices even if there is file system present on it. Note that the drives may be wiped before using.
spec.storage.devices (string array)
Devices is an array of devices that should be used by Portworx.
Device that will be used for journaling by Portworx.
Device that will be used for storing metadata by Portworx. It is recommended to have a system metadata device when using internal KVDB for better performance.
Cloud storage configuration
This section contains all the details to configure the storage in cloud. This enables Portworx
to manage cloud disks automatically for the user based on given specs.
spec.storage takes precedence
over this section. Make sure
spec.storage is empty when specifying cloud storage.
spec.cloudStorage.deviceSpecs (string array)
Device specs is a list of storage device specs. A cloud disk will be created for every spec in the list.
Device spec for device that will be used for journaling by Portworx.
Device spec for device that will be used for storing metadata by Portworx. It is recommended to have a system metadata device when using internal KVDB for better performance.
Specify maximum number of storage nodes per zone. Portworx will not provision drives for additional nodes in the zone and start them as compute only nodes.
Specify maximum number of total storage nodes. Portworx will not provision drives for additional
nodes in the cluster and start them as compute only nodes. It is recommended to use
as a best practice.
This section contains network information needed by Portworx. If nothing is specified, Portworx will auto detect and choose network interfaces.
Data interface allows users to override the auto selected network interace for data traffic.
Management interface allows users to override the auto selected network interace for control plane traffic.
Placement lets your override where Portworx will be deployed. By default Portworx gets deployed on all worker nodes.
Kubernetes like node affinity to restrict Portworx on certain nodes.
Similar to Kubernetes DaemonSet, this allows you to specify a update strategy for Portworx updates.
Type of update strategy to use. Currently it supports RollingUpdate and OnDelete. Default: RollingUpdate.
Similar to Kubernetes rolling update strategy you can specify this section for rolling updates. The default value is 1 which means only one node will be down at a given point of time. You can specify a static number of percentage value. (Example: 3, 30%, etc)
This section contains information on how to uninstall the Portworx cluster.
Type of delete strategy to use. Deleting the Portworx StorageCluster object will trigger this delete strategy. By default there is no delete strategy, which means only the Kubernetes components deployed by the Operator will be removed leaving the Portworx systemd service running without disturbing the apps running on it. Currently there are two supported delete strategies:
- Uninstall - Uninstall all Portworx components from the system leaving the devices and KVDB intact.
- UninstallAndWIpe - Uninstall all Portworx components from the system and also wipe the devices and metadata from KVDB.
This section contains information to enable and manage stork deployment through the Portworx Operator.
You can enable/disable STORK by toggling this flag at any given time.
Specify the STORK image.
A map of string keys and values to override the default STORK arguments or to add new arguments.
spec.stork.env (object array)
A list of Kubernetes like environment variables that need to be passed to STORK.
This section contains information to enable and manage Lighthouse deployment through the Portworx Operator.
You can enable/disable Lighthouse by toggling this flag at any given time.
Specify the Lighthouse image.