Slurm configuration

For what each setting does and how to change it safely, see Configure Slurm.

Read a cluster’s Slurm configuration: GET /clusters/{clusterId}/scheduler-configuration
Replace a cluster’s Slurm configuration: PUT /clusters/{clusterId}/scheduler-configuration
Read the organization default: GET /organizations/{orgId}/scheduler-configuration
Replace the organization default: PUT /organizations/{orgId}/scheduler-configuration

These endpoints apply to clusters that run Slurm. They accept only people, not service accounts.

The configuration object

Every field is optional. A field that has never been set is omitted.

Field	Type	Description
`slurmPartitions`	string	Partition definitions for `slurm.conf`. Replaces the generated partitions. Cluster only.
`slurmExtraConfig`	string	Lines appended to `slurm.conf`.
`cgroupExtraConfig`	string	Lines appended to `cgroup.conf`.
`gresConf`	string	Contents of `gres.conf`. Replaces the generated file. Cluster only.
`prolog`	object	Prolog scripts, keyed by file name. Each must start with `#!`.
`epilog`	object	Epilog scripts, keyed by file name. Each must start with `#!`.
`controllerEtc`	object	Extra files in the controller’s `/etc/slurm`, keyed by file name.
`loginS6Packages`	string	Script run when a login node starts.
`computeS6Packages`	string	Script run when a compute node starts. Changing it restarts every compute node and ends running jobs.

For the organization default, slurmPartitions and gresConf are not accepted.

Read a cluster’s Slurm configuration

GET /clusters/{clusterId}/scheduler-configuration

API: Platform. Permission: clusters.configure.

Parameters

Name	In	Type	Required	Description
`clusterId`	path	string	yes	Cluster ID.
`businessOrgId`	query	string	yes	Organization ID.

Response

A configuration object.

Example

andromeda call platform GET "/clusters/$CLUSTER_ID/scheduler-configuration?businessOrgId=$ORG_ID"

{
  "slurmExtraConfig": "MaxJobCount=20000\n",
  "prolog": {"10-gpu-check.sh": "#!/bin/sh\nnvidia-smi -L >/dev/null || exit 1\n"}
}

Errors

Status	Code	When
403	`forbidden`	You lack `clusters.configure`, or you called with a service account.
404	`not_found`	No cluster with this ID belongs to the organization.
422	`invalid_request`	`businessOrgId` is missing.

Replace a cluster’s Slurm configuration

PUT /clusters/{clusterId}/scheduler-configuration

API: Platform. Permission: clusters.configure. Replaces the whole configuration with the body. A field you omit is cleared, so read the configuration first and send it back with your changes. Sending a configuration identical to the stored one changes nothing. Otherwise the cluster’s spec.schedulerGeneration increases and the cluster applies the new configuration. Applying it closes open SSH sessions on the cluster.

Parameters

Name	In	Type	Required	Description
`clusterId`	path	string	yes	Cluster ID.
`businessOrgId`	query	string	yes	Organization ID.

Request body

A configuration object.

Response

200 with the configuration as stored.

Example

Raise the job limit, and keep the rest of the configuration from the read example, saved as slurm.json:

jq '.slurmExtraConfig = "MaxJobCount=50000\n"' slurm.json |
  andromeda call --data @- \
    platform PUT "/clusters/$CLUSTER_ID/scheduler-configuration?businessOrgId=$ORG_ID"

Errors

Status	Code	When
400	`invalid_scheduler_config`	A setting failed validation. `message` says which. See Validation.
403	`forbidden`	You lack `clusters.configure`, or you called with a service account.
404	`not_found`	No cluster with this ID belongs to the organization.
422	`invalid_request`	`businessOrgId` is missing, or the body is not a configuration object.

Validation

Rule	Example that fails
`prolog` and `epilog` scripts start with `#!`.	`{"prolog": {"check.sh": "exit 0"}}`
File names contain only letters, digits, `-`, `_`, and `.`.	`{"prolog": {"gpu/check.sh": "#!/bin/sh\ntrue"}}`
`controllerEtc` does not contain a generated file: `slurm.conf`, `cgroup.conf`, `gres.conf`, `plugstack.conf`.	`{"controllerEtc": {"slurm.conf": "..."}}`
Each script or file is at most 64 KiB.
`prolog`, `epilog`, and `controllerEtc` together are at most 256 KiB.

slurmPartitions, slurmExtraConfig, cgroupExtraConfig, gresConf, and the package scripts are not validated. An error in them surfaces on the cluster, not in this response, and can stop slurmctld.

Read the organization default

GET /organizations/{orgId}/scheduler-configuration

API: Platform. Permission: clusters.configure. Returns the configuration new clusters in the organization start with. An organization with no default returns {}.

Parameters

Name	In	Type	Required	Description
`orgId`	path	string	yes	Organization ID.

Example

andromeda call platform GET "/organizations/$ORG_ID/scheduler-configuration"

{"loginS6Packages": "#!/bin/sh\napt-get install -y tmux htop\n"}

Replace the organization default

PUT /organizations/{orgId}/scheduler-configuration

API: Platform. Permission: clusters.configure. Replaces the whole default. Clusters created afterwards start with it. Existing clusters are not changed. The body is a configuration object without slurmPartitions or gresConf. The response is 200 with the default as stored. Validation is the same as for a cluster.

Example

jq -n '{loginS6Packages: "#!/bin/sh\napt-get install -y tmux htop\n"}' |
  andromeda call --data @- platform PUT "/organizations/$ORG_ID/scheduler-configuration"

Errors

Status	Code	When
400	`invalid_scheduler_config`	A setting failed validation.
403	`forbidden`	You lack `clusters.configure`, or you called with a service account.
422	`invalid_request`	The body contains a field the default does not accept, such as `slurmPartitions` or `gresConf`.

Overview

Business API

Platform API

The configuration object

Read a cluster’s Slurm configuration

Parameters

Response

Example

Errors

Replace a cluster’s Slurm configuration

Parameters

Request body

Response

Example

Errors

Validation

Read the organization default

Parameters

Example

Replace the organization default

Example

Errors

​The configuration object

​Read a cluster’s Slurm configuration

​Parameters

​Response

​Example

​Errors

​Replace a cluster’s Slurm configuration

​Parameters

​Request body

​Response

​Example

​Errors

​Validation

​Read the organization default

​Parameters

​Example

​Replace the organization default

​Example

​Errors

The configuration object

Read a cluster’s Slurm configuration

Parameters

Response

Example

Errors

Replace a cluster’s Slurm configuration

Parameters

Request body

Response

Example

Errors

Validation

Read the organization default

Parameters

Example

Replace the organization default

Example

Errors