Install on Docker Standalone
runC
containers. Please refer to the Portworx on Kubernetes page if you want to install Portworx on Kubernetes.
Why OCI
Running Portworx as a runC container eliminates any cyclical dependencies between a Docker container consuming storage from the Portworx container. It also enables you to run your Linux containers without a Docker daemon completely, while still getting all of the advantages of a Linux container and cloud native storage from Portworx.
Install
Prerequisites
- SYSTEMD: The installation below assumes the systemd package is installed on your system (i.e. systemctl command works).
- Note, if you are running Ubuntu 16.04, CentoOS 7 or CoreOS v94 (or newer) the “systemd” is already installed and no actions will be required.
- SCHEDULERS: If you are installing PX into Kubernetes or Mesosphere DC/OS cluster, we recommend to install the scheduler-specific Portworx package, which provides tighter integration, and better overall user experience.
- FIREWALL: Ensure ports 9001-9022 are open between the cluster nodes that will run Portworx.
- NTP: Ensure all nodes running PX are time-synchronized, and NTP service is configured and running.
- KVDB: Please have a clustered key-value database (etcd or consul) installed and ready. For etcd installation instructions refer this doc.
- STORAGE: At least one of the PX-nodes should have extra storage available, in a form of unformatted partition or a disk-drive. Also please note that storage devices explicitly given to Portworx (ie.
px-runc ... -s /dev/sdb -s /dev/sdc3
) will be automatically formatted by PX.
The installation and setup of PX OCI bundle is a 3-step process:
- Install PX OCI bits
- Configure PX OCI
- Enable and start Portworx service
Step 1: Install the PX OCI bundle
Portworx provides a Docker based installation utility to help deploy the PX OCI bundle. This bundle can be installed by running the following Docker container on your host system:
# Uncomment appropriate `REL` below to select desired Portworx release
REL="/2.1" # 2.1 portworx release
#REL="/2.0" # 2.0 portworx release
latest_stable=$(curl -fsSL "https://install.portworx.com$REL/?type=dock&stork=false&aut=false" | awk '/image: / {print $2}')
# Download OCI bits (reminder, you will still need to run `px-runc install ..` after this step)
sudo docker run --entrypoint /runc-entry-point.sh \
--rm -i --privileged=true \
-v /opt/pwx:/opt/pwx -v /etc/pwx:/etc/pwx \
$latest_stable
Step 2: Configure PX under runC
Now that you have downloaded and installed the PX OCI bundle, you can use the px-runc install
command from the bundle to configure systemd to start PX runC.
The px-runc command is a helper-tool that does the following:
- Prepares the OCI directory for runc
- Prepares the runC configuration for PX
- Uses systemd to start the PX OCI bundle
Installation example:
sudo /opt/pwx/bin/px-runc install -c MY_CLUSTER_ID \
-k etcd://myetc.company.com:2379 \
-s /dev/xvdb -s /dev/xvdc
Command-line arguments
General options
-c [REQUIRED] Specifies the cluster ID that this PX instance is to join
-k [REQUIRED] Points to your key value database, such as an etcd cluster or a consul cluster
-s [REQUIRED unless -a is used] Specifies the various drives that PX should use for storing the data
-e key=value [OPTIONAL] Specify extra environment variables
-v <dir:dir[:shared,ro]> [OPTIONAL] Specify extra mounts
-d <ethX> [OPTIONAL] Specify the data network interface
-m <ethX> [OPTIONAL] Specify the management network interface
-z [OPTIONAL] Instructs PX to run in zero storage mode
-f [OPTIONAL] Instructs PX to use an unmounted drive even if it has a filesystem on it
-a [OPTIONAL] Instructs PX to use any available, unused and unmounted drives
-A [OPTIONAL] Instructs PX to use any available, unused and unmounted drives or partitions
-j [OPTIONAL] Specifies a journal device for PX. Specify a persistent drive like /dev/sdc or use auto (recommended)
-x <swarm|kubernetes> [OPTIONAL] Specify scheduler being used in the environment
-r <portnumber> [OPTIONAL] Specifies the portnumber from which PX will start consuming. Ex: 9001 means 9001-9020
- additional PX-OCI -specific options:
-oci <dir> [OPTIONAL] Specify OCI directory (default: /opt/pwx/oci)
-sysd <file> [OPTIONAL] Specify SystemD service file (default: /etc/systemd/system/portworx.service)
KVDB options
-userpwd <user:passwd> [OPTIONAL] Username and password for ETCD authentication
-ca <file> [OPTIONAL] Specify location of CA file for ETCD authentication
-cert <file> [OPTIONAL] Specify location of certificate for ETCD authentication
-key <file> [OPTIONAL] Specify location of certificate key for ETCD authentication
-acltoken <token> [OPTIONAL] ACL token value used for Consul authentication
Secrets options
-secret_type <aws|dcos|docker|k8s|kvdb|vault> [OPTIONAL] Specify the secret type to be used by Portworx for cloudsnap and encryption features.
-cluster_secret_key <id> [OPTIONAL] Specify the cluster wide secret key to be used when using AWS KMS or Vault for volume encryption.
Sharedv4 options
To enable sharedv4 type of volumes provide the following portworx input argument:
-enable-shared-and-shared-v4 Enables both regular and NFSv4 volume sharing
Environment variables
PX_HTTP_PROXY [OPTIONAL] If running behind an HTTP proxy, set the PX_HTTP_PROXY variables to your HTTP proxy.
PX_HTTPS_PROXY [OPTIONAL] If running behind an HTTPS proxy, set the PX_HTTPS_PROXY variables to your HTTPS proxy.
PX_ENABLE_CACHE_FLUSH [OPTIONAL] Enable cache flush deamon. Set PX_ENABLE_CACHE_FLUSH=true.
-e
option
Below is an example install command with extra “PX_ENABLE_CACHE_FLUSH” environment variable:
sudo /opt/pwx/bin/px-runc install -e PX_ENABLE_CACHE_FLUSH=yes \
-c MY_CLUSTER_ID -k etcd://myetc.company.com:2379 -s /dev/xvdb
Examples
Installing Portworx using etcd:
px-runc install -k etcd://my.company.com:2379 -c MY_CLUSTER_ID -s /dev/sdc -s /dev/sdb2 {{ include.sched-flags }}
px-runc install -k etcd://70.0.1.65:2379 -c MY_CLUSTER_ID -s /dev/sdc -d enp0s8 -m enp0s8 {{ include.sched-flags }}
Installing Portworx using consul:
px-runc install -k consul://my.company.com:8500 -c MY_CLUSTER_ID -s /dev/sdc -s /dev/sdb2 {{ include.sched-flags }}
px-runc install -k consul://70.0.2.65:8500 -c MY_CLUSTER_ID -s /dev/sdc -d enp0s8 -m enp0s8 {{ include.sched-flags }}
Modifying the PX configuration
After the initial installation, you can modify the PX configuration file at /etc/pwx/config.json
(see details) and restart Portworx using systemctl restart portworx
.
Step 3: Starting PX runC
Once you install the PX OCI bundle and systemd configuration from the steps above, you can start and control PX runC directly via systemd.
Below commands reload systemd configurations, enable and starts the Portworx service.
sudo systemctl daemon-reload
sudo systemctl enable portworx
sudo systemctl start portworx
Upgrading the PX OCI bundle
To upgrade the OCI bundle, simply re-run the installation Step 1 with the --upgrade
option. After the upgrade, you will need to restart the Portworx service.
Below command upgrades your installation to the latest stable Portworx version:
latest_stable=$(curl -fsSL 'https://install.portworx.com?type=dock&stork=false&aut=false' | awk '/image: / {print $2}')
sudo docker run --entrypoint /runc-entry-point.sh \
--rm -i --privileged=true \
-v /opt/pwx:/opt/pwx -v /etc/pwx:/etc/pwx \
$latest_stable --upgrade
sudo systemctl restart portworx
Uninstalling the PX OCI bundle
To uninstall the PX OCI bundle, please run the following:
# 1: Remove systemd service (if any)
sudo systemctl stop portworx
sudo systemctl disable portworx
sudo rm -f /etc/systemd/system/portworx*
# NOTE: if the steps below fail, please reboot the node, and repeat the steps 2..5
# 2: Unmount oci (if required)
grep -q '/opt/pwx/oci /opt/pwx/oci' /proc/self/mountinfo && sudo umount /opt/pwx/oci
# 3: Remove binary files
sudo rm -fr /opt/pwx
# 4: [OPTIONAL] Remove configuration files. Doing this means UNRECOVERABLE DATA LOSS.
sudo chattr -ie /etc/pwx/.private.json
sudo rm -fr /etc/pwx
Logging and Log files
The systemd(1) uses a very flexible logging mechanism, where logs can be viewed using the journalctl
command.
For example:
# Monitor the Portworx logs
sudo journalctl -f -u portworx
# Get a slice of Portworx logs
sudo journalctl -u portworx --since 09:00 --until "1 hour ago"
However, if you prefer to capture Portworx service logs in a separate log file, you will need to modify your host system as follows:
# Create a rsyslogd(8) rule to separate out the PX logs:
sudo cat > /etc/rsyslog.d/23-px-runc.conf << _EOF
:programname, isequal, "px-runc" /var/log/portworx.log
& stop
_EOF
# Create logrotate(8) configuration to periodically rotate the logs:
sudo cat > /etc/logrotate.d/portworx << _EOF
/var/log/portworx.log {
daily
rotate 7
compress
notifempty
missingok
postrotate
/usr/bin/pkill -HUP syslogd 2> /dev/null || true
endscript
}
_EOF
# Signal syslogd to reload the configurations:
sudo pkill -HUP syslogd
Advanced usage: Interactive/Foreground mode
Alternatively, one might prefer to first start the PX interactively (for example, to verify the configuration parameters were OK and the startup was successful), and then install it as a service:
# Invoke PX interactively, abort with CTRL-C when confirmed it's running:
sudo /opt/pwx/bin/px-runc run -c MY_CLUSTER_ID \
-k etcd://myetc.company.com:2379 \
-s /dev/xvdb
[...]
> time="2017-08-18T20:34:23Z" level=info msg="Cloud backup schedules setup done"
> time="2017-08-18T20:34:23Z" level=info msg="Starting REST service on socket : /run/docker/plugins/pxd.sock"
> time="2017-08-18T20:34:23Z" level=info msg="Starting REST service on socket : /var/lib/osd/driver/pxd.sock"
> time="2017-08-18T20:34:23Z" level=info msg="PX is ready on Node: 53f5e87b... CLI accessible at /opt/pwx/bin/pxctl."
[ hit Ctrl-C ]