dcos storage profile

dcos storage profile

Manage volume profiles.

Synopsis

Different workloads typically have different storage requirements. For workloads that primarily store data for archiving purposes, it often makes sense to use HDDs. For query-intensive workloads, it often makes sense to use low-latency storage such as SSDs or NVMe devices. Even the term “SSDs” is broad, and some workloads may benefit from write-optimized vs. read-optimized SSDs within the same cluster or even the same node. The DC/OS Storage Service lets you distinguish between these different categories using “volume profiles”.

A volume profile selects storage capacity from suitable volume providers. A volume profile can only select storage from one kind of volume provider. For example, a single volume profile can select “lvm” volume providers or "ebs" volume providers, but not both. For more information regarding volume providers, run dcos storage provider --help.

The volume profile also lets you define volume configuration that will apply to all volumes created using that volume profile. The specific volume configuration that may be provided depends on the volume plugin (e.g., lvm) and is described in the “Volume plugins” section for that plugin in the documentation: </mesosphere/dcos/services/storage/>

Let’s use the scenarios described above to walk through the process.

Dan has three workloads. The first is a blob store used to archive data. The second is a MongoDB framework that must serve many requests per second but receives comparatively few writes. The third is a Cassandra framework used to consume sensor data at a very high rate and needs to be optimized for writing.

Dan’s DC/OS cluster consists of masters, agents and public agents. On each agent there are several storage devices. For this example we will assume that every agent has the same device composition, but nothing prevents Dan from having different numbers of devices on each agent.

Every agent has the following devices:

• /dev/xvda: the OS is installed on this device
• /dev/xvdb, /dev/xvdc: HDDs intended for archiving data
• /dev/xvdd, /dev/xvde: SSDs optimized for read performance
• /dev/xvdf, /dev/xvdg: SSDs optimized for write performance

Dan wants to:

• have his blob store consume volumes carved from the HDDs
• have MongoDB consume volumes carved from the read-optimized SSDs
• have Cassandra consume volumes carved from the write-optimized SSDs

The first thing he needs to do is create a separate “lvm” volume provider for each of the physical device collections. He creates them as follows:

# First he lists the storage devices available in his cluster.
dcos storage device list

NODE                                     NAME   STATUS  ROTATIONAL  TYPE  FSTYPE  MOUNTPOINT
c67efa5d-34fa-4bc5-8b21-2a5e0bd52385-S1  xvda   ONLINE  false       disk  -       -
c67efa5d-34fa-4bc5-8b21-2a5e0bd52385-S1  xvda1  ONLINE  false       part  xfs     /
c67efa5d-34fa-4bc5-8b21-2a5e0bd52385-S1  xvdb   ONLINE  true        disk  -       -
c67efa5d-34fa-4bc5-8b21-2a5e0bd52385-S1  xvdc   ONLINE  true        disk  -       -
c67efa5d-34fa-4bc5-8b21-2a5e0bd52385-S1  xvdd   ONLINE  false       disk  -       -
c67efa5d-34fa-4bc5-8b21-2a5e0bd52385-S1  xvde   ONLINE  false       disk  -       -
c67efa5d-34fa-4bc5-8b21-2a5e0bd52385-S1  xvdf   ONLINE  false       disk  -       -
c67efa5d-34fa-4bc5-8b21-2a5e0bd52385-S1  xvdg   ONLINE  false       disk  -       -
...

# We are working with a single node so let's reuse its node ID.
export NODE_ID=c67efa5d-34fa-4bc5-8b21-2a5e0bd52385-S1

# Create a 'lvm' volume provider that groups xvdb and xvdc into a LVM volume
# group and exposes that VG's storage capacity to DC/OS.
cat <<EOF | dcos storage provider create
{
    "name": "hdd-${NODE_ID}",
    "description": "high-latency archival devices",
    "spec": {
        "plugin": {
            "name": "lvm",
            "config-version": "latest"
        },
        "node": "${NODE_ID}",
        "plugin-configuration": {
            "devices": ["xvdb", "xvdc"]
        },
        "labels": {
            "latency": "high"
        }
    }
}
EOF

# Create a 'lvm' volume provider that groups xvdd and xvde into a LVM volume
# group and exposes that VG's storage capacity to DC/OS. We label this provider
# as "read-optimized" as it consists of devices that we know are optimized for
# reading and we would like to target them for certain workloads.
cat <<EOF | dcos storage provider create
{
    "name": "ssd-ro-${NODE_ID}",
    "description": "low-latency read-optimized devices",
    "spec": {
        "plugin": {
            "name": "lvm",
            "config-version": "latest"
        },
        "node": "${NODE_ID}",
        "plugin-configuration": {
            "devices": ["xvdd", "xvde"]
        },
        "labels": {
          "latency": "low",
          "read-optimized": "true"
        }
    }
}
EOF

# Create a 'lvm' volume provider that groups xvdf and xvdg into a LVM volume
# group and exposes that VG's storage capacity to DC/OS. We label this provider
# as "write-optimized" as it consists of devices that we know are optimized for
# writing and we would like to target them for certain workloads.
cat <<EOF | dcos storage provider create
{
    "name": "ssd-wo-${NODE_ID}",
    "description": "low-latency write-optimized devices",
    "spec": {
        "plugin": {
            "name": "lvm",
            "config-version": "latest"
        },
        "node": "${NODE_ID}",
        "plugin-configuration": {
            "devices": ["xvdf", "xvdg"]
        },
        "labels": {
          "latency": "low",
          "write-optimized": "true"
        }
    }
}
EOF

Dan now has three volume providers: hdd-[NODE_ID], ssd-ro-[NODE_ID] and ssd-wo-[NODE_ID]. In order to create volumes for his workloads, he now needs to create volume profiles. He can create many volume profiles. The same volume profile can target multiple volume providers. Different volume profiles can target the same volume providers.

The first volume profile he creates is called any-lvm. This volume profile selects any lvm volume provider when creating a volume and applies no additional configuration to the volumes it creates. This lets Dan quickly create volumes for testing purposes without caring about what volume providers (i.e., what devices) the volume will be created on. Volumes of this type may be created on any of the volume providers, which is fine for short-lived test volumes.

cat <<EOF | dcos storage profile create
{
    "name": "any-lvm",
    "description": "any node-local storage",
    "spec": {
        "provider-selector": {
            "plugin": "lvm"
        },
        "mount": {}
    }
}
EOF

The second volume profile he creates is called slow. This volume profile selects only volume providers that are labelled as {"latency": "high"}. This is a catch-all volume profile for when he wants to create test volumes that have no redundancy or high performance requirements. This is useful for creating volumes for workloads that are under development and ensures that the high performance storage devices are kept for production workloads.

cat <<EOF | dcos storage profile create
{
    "name": "slow",
    "description": "HDD-backed node-local storage",
    "spec": {
        "provider-selector": {
            "plugin": "lvm",
            "matches": {
                "labels": {
                    "latency": "high"
                }
            }
        },
        "mount": {}
    }
}
EOF

The third volume profile he creates is called archive. This volume profile selects only volume providers that are labelled as {"latency": "high"}. It also includes a “volume-configuration” section that specifies the type field for volumes created using this profile as raid1. This means that any volume created using this volume profile will be mirrored across the two underlying devices using the LVM RAID functionality for logical volumes. The available volume-configuration fields are given in the documentation for the lvm plugin in the “Volume plugins” section of the DC/OS Storage Service documentation: </mesosphere/dcos/services/storage/>

cat <<EOF | dcos storage profile create
{
    "name": "archive",
    "description": "HDD-backed RAID-1 node-local storage",
    "spec": {
        "provider-selector": {
            "plugin": "lvm",
            "matches": {
                "labels": {
                    "latency": "high"
                }
            }
        },
        "volume-configuration": {
            "type": "raid1"
        }
        "mount": {}
    }
}
EOF

The fourth volume profile he creates is called fast-read. This volume profile selects only volume providers that are labelled as {"latency": "low"} as well as {"read-optimized": "true"}. Volumes creating using this volume profile will be created on the ssd-ro-[NODE_ID] volume provider as LVM logical volumes on the LVM volume group associated with that volume provider.

cat <<EOF | dcos storage profile create
{
    "name": "fast-read",
    "description": "SSD-backed RAID-1 read-optimized node-local storage",
    "spec": {
        "provider-selector": {
            "plugin": "lvm",
            "matches": {
                "labels": {
                    "latency": "low",
                    "read-optimized": "true"
                }
            }
        },
        "volume-configuration": {
            "type": "raid1"
        }
        "mount": {}
    }
}
EOF

The fifth volume profile he creates is called fast-write. This volume profile selects only volume providers that are labelled as {"latency": "low"} as well as {"write-optimized": "true"}. Volumes creating using this volume profile will be created on the ssd-wo-[NODE_ID] volume provider as LVM logical volumes on the LVM volume group associated with that volume provider.

cat <<EOF | dcos storage profile create
{
    "name": "fast-write",
    "description": "SSD-backed RAID-1 write-optimized node-local storage",
    "spec": {
        "provider-selector": {
            "plugin": "lvm",
            "matches": {
                "labels": {
                    "latency": "low",
                    "write-optimized": "true"
                }
            }
        },
        "volume-configuration": {
            "type": "raid1"
        }
        "mount": {}
    }
}
EOF

Dan can now create volumes using these volume profiles as follows.

# Create a volume on any volume provider using the 'any-lvm' profile.
dcos storage volume create --name tmpvol-1 --capacity 1G --profile any-lvm

# Create a HDD-backed volume using the 'slow' profile.
dcos storage volume create --name cheapvol-1 --capacity 10G --profile slow

# Create a RAID-1 HDD-backed volume using the 'archive' profile
dcos storage volume create --name blobvol-1 --capacity 100G --profile archive

# Create a RAID-1 volume on read-optimized SSD using the 'fast-read' profile.
dcos storage volume create --name mongovol-1 --capacity 200G \
  --profile fast-read

# Create a RAID-1 volume on write-optimized SSD using the 'fast-write' profile.
dcos storage volume create --name cassandravol-1 --capacity 200G \
  --profile fast-write

dcos storage profile [flags]

Options inherited from parent commands