Ceph: Difference between revisions
imported>C4lliope No edit summary |
imported>C4lliope No edit summary |
||
| Line 4: | Line 4: | ||
The Ceph Nix package has been hard-pressed to keep up with Ceph, as filesystem concerns are a larger challenge on NixOS than on other Linux lineages. Please make a Wiki account and add your experiences, if you made progress running a modern Ceph version. | The Ceph Nix package has been hard-pressed to keep up with Ceph, as filesystem concerns are a larger challenge on NixOS than on other Linux lineages. Please make a Wiki account and add your experiences, if you made progress running a modern Ceph version. | ||
Describe your ceph user, alongside your normal login user: | |||
<syntaxhighlight lang="nix"> | |||
users.users = { | |||
mesh = { isNormalUser = true; extraGroups = [ "wheel" "docker" ]; }; | |||
ceph = { isNormalUser = true; extraGroups = [ "wheel" "ceph" ]; }; | |||
}; | |||
users.groups.ceph = {}; | |||
</syntaxhighlight> | |||
Be sure you rebuild so you can assign some paths to the <code>ceph</code> user. | |||
Make a UUID using <code>uuidgen</code> and describe your Ceph nodes: | |||
<syntaxhighlight lang="nix"> | |||
services.ceph = { | |||
global.fsid = "4b687c5c-5a20-4a77-8774-487989fd0bc7"; | |||
osd = { | |||
enable = true; | |||
daemons = ["0"]; | |||
}; | |||
mon = { | |||
enable = false; | |||
extraConfig = { | |||
"mon initial members" = "mesh-a,mesh-b,mesh-c"; | |||
"mon host" = "10.0.0.11,10.0.0.12,10.0.0.13"; | |||
}; | |||
}; | |||
}; | |||
</syntaxhighlight> | |||
Make your OSD volume; run these commands on each node: | |||
(based on https://docs.ceph.com/en/quincy/install/manual-deployment/ ) | |||
<syntaxhighlight lang="bash"> | |||
export IP=<your-node-IP-on-local-LAN> | |||
export FSID=4b687c5c-5a20-4a77-8774-487989fd0bc7 | |||
sudo -u ceph mkdir -p /etc/ceph | |||
sudo -u ceph mkdir -p /var/lib/ceph/bootstrap-osd | |||
sudo -u ceph mkdir -p /tmp/monmap | |||
sudo -u ceph mkdir -p /var/lib/ceph/mon/ceph-$(hostname) | |||
sudo -u ceph mkdir /var/lib/ceph/mon/ceph-mon-$(hostname) | |||
sudo ceph-authtool --create-keyring /etc/ceph/ceph.client.admin.keyring --gen-key -n client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' | |||
sudo mkdir -p /var/lib/ceph/bootstrap-osd && sudo ceph-authtool --create-keyring /var/lib/ceph/bootstrap-osd/ceph.keyring --gen-key -n client.bootstrap-osd --cap mon 'profile bootstrap-osd' --cap mgr 'allow r' | |||
sudo ceph-authtool /tmp/ceph.mon.keyring --import-keyring /etc/ceph/ceph.client.admin.keyring sudo ceph-authtool /tmp/ceph.mon.keyring --import-keyring /var/lib/ceph/bootstrap-osd/ceph.keyring | |||
sudo chown ceph:ceph /tmp/ceph.mon.keyring | |||
sudo monmaptool --create --add mesh-a $IP --fsid $FSID /tmp/monmap | |||
sudo -u ceph ceph-mon --mkfs -i mon-$(hostname) --monmap /tmp/monmap --keyring /tmp/ceph.mon.keyring | |||
</syntaxhighlight> | |||
Bind all Ceph OSD shares using systemd (based on <code>u/imspacekitteh</code>'s example): | Bind all Ceph OSD shares using systemd (based on <code>u/imspacekitteh</code>'s example): | ||