Portworx install on PKS on vSphere using shared datastores

Pre-requisites

• This page assumes you have a running etcd cluster. If not, return to Installing etcd on PKS.

Architecture

Below diagram gives an overview of the Portworx architecture on vSphere using shared datastores.

• Portworx runs as a Daemonset hence each Kubernetes minion/worker will have the Portworx daemon running.
• Based on the given spec by the end user, Portworx on each node will create it’s disk on the configured shared datastore(s) or datastore cluster(s).
• Portworx will aggregate all of the disks and form a single storage cluster. End users can carve PVCs (Persistent Volume Claims), PVs (Persistent Volumes) and Snapshots from this storage cluster.
• Portworx tracks and manages the disks that it creates. So in a failure event, if a new VM spins up, Portworx on the new VM will be able to attach to the same disk that was previously created by the node on the failed VM.

ESXi datastore preparation

Create one or more shared datastore(s) or datastore cluster(s) which is dedicated for Portworx storage. Use a common prefix for the names of the datastores or datastore cluster(s). We will be giving this prefix during Portworx installation later in this guide.

Portworx installation

Step 1: vCenter user for Portworx

You will need to provide Portworx with a vCenter server user that will need to either have the full Admin role or, for increased security, a custom-created role with the following minimum vSphere privileges:

• Datastore
  • Browse datastore
  • Low level file operations
  • Remove file
• Host
  • Local operations
  • Reconfigure virtual machine
• Virtual machine
  • Change Configuration
  • Add existing disk
  • Add new disk
  • Add or remove device
  • Advanced configuration
  • Change Settings
  • Modify device settings
  • Remove disk

If you create a custom role as above, make sure to select “Propagate to children” when assigning the user to the role.

Step 2: Create a Kubernetes secret with your vCenter user and password

Update the following items in the Secret template below to match your environment:

1. VSPHERE_USER: Use output of printf <vcenter-server-user> | base64
2. VSPHERE_PASSWORD: Use output of printf <vcenter-server-password> | base64

apiVersion: v1
kind: Secret
metadata:
  name: px-vsphere-secret
  namespace: kube-system
type: Opaque
data:
  VSPHERE_USER: YWRtaW5pc3RyYXRvckB2c3BoZXJlLmxvY2Fs
  VSPHERE_PASSWORD: cHgxLjMuMEZUVw==

kubectl apply the above spec after you update the above template with your user and password.

Step 3: Generate rest of the specs

vSphere environment details

Export following env variables based on your vSphere environment. These variables will be used in a later step when generating the yaml spec.

# Hostname or IP of your vCenter server
export VSPHERE_VCENTER=myvcenter.net

# Prefix of your shared ESXi datastore(s) names. Portworx will use datastores who names match this prefix to create disks.
export VSPHERE_DATASTORE_PREFIX=mydatastore-

# Change this to the port number vSphere services are running on if you have changed the default port 443
export VSPHERE_VCENTER_PORT=443

Disk templates

A disk template defines the VMDK properties that Portworx will use as a reference for creating the actual disks out of which Portworx will create the virtual volumes for your PVCs.

Following example will create a 150GB zeroed thick vmdk on each VM.

export VSPHERE_DISK_TEMPLATE=type=zeroedthick,size=150

The template follows the following format:

"type=<vmdk type>,size=<size of the vmdk>"

• type: Supported types are thin, zeroedthick , eagerzeroedthick
• size: This is the size of the VMDK in GiB

Generating the spec if using secure etcd

Now generate the spec with the following curl command.

VER=$(kubectl version --short | awk -Fv '/Server Version: /{print $3}')
curl -fsL -o px-spec.yaml "https://install.portworx.com/2.1?kbver=$VER&c=portworx-demo-cluster&b=true&st=k8s&pks=true&vsp=true&ds=$VSPHERE_DATASTORE_PREFIX&vc=$VSPHERE_VCENTER&s=%22$VSPHERE_DISK_TEMPLATE%22"

Wipe Portworx installation

Below are the steps to wipe your entire Portworx installation on PKS.

1. Run cluster-scoped wipe: curl -fsL https://install.portworx.com/px-wipe | bash -s -- -T pks
2. Go to each virtual machine and delete the additional vmdks Portworx created in the shared datastore.