Netdata

netdata is a metrics tool, which comes with a lot of sane preconfiguration. It contains of various plugins, which need to be enabled sometimes with additional effort.

Adding node

• Install the netdata package and enable the service.
• When adding new node in the web interface you get a token, write that token to /var/lib/netdata/cloud.d/token
• As root run nix-shell -p netdata --run "netdata-claim.sh"

Newer web UI

Netdata comes with an old unmaintained web UI that is accesible at port 19999. Netdata Inc. will not fix any bugs in the old UI and it may to become more and more broken as time goes on. There is, however, a newer maintained, but unfree proprietary web UI that can be optionally enabled to replace the old UI. To take it into use, override netdata's package:

services.netdata.package = pkgs.netdata.override {
  withCloudUi = true;
};

In addition, you need to allow unfree packages.

Streaming node setup

Ensure you choose appropriate access control for your nodes.

Receiver node

services.netdata.configDir."stream.conf" =
  let
    mkChildNode = apiKey: allowFrom: ''
      [${apiKey}]
        enabled = yes
        default history = <a value of your choice>
        default memory mode = dbengine # a good default
        health enabled by default = auto
        allow from = ${allowFrom}
    '';
  in pkgs.writeText "stream.conf" ''
    [stream]
      # This won't stream by itself, except if the receiver is a sender too, which is possible in netdata model.
      enabled = no
      enable compression = yes

    # An allowed sender node
    ${mkChildNode "an API key" "an allowed IP"}
  '';

Sender node

services.netdata.configDir."stream.conf" = pkgs.writeText "stream.conf" ''
  [stream]
    enabled = yes
    destination = receiver-hostname-or-ip-address:19999
    api key = any string that is set also on the receiver side
'';

If you don't need any web UI and want to consume minimal resources on the sender node, use:

services.netdata = {
  config = {
    global = { "memory mode" = "none"; };
    web = {
      mode = "none";
      "accept a streaming request every seconds" = 0;
    };
  };
};

This way, it won't spawn any web UI, neither store any metric locally.

Python Plugins

nvidia-smi

To enable the nvidia-smi plugin you have to make sure nvidia-smi can be called by netdata.

systemd.services.enable = true;
systemd.services.netdata.path = [pkgs.linuxPackages.nvidia_x11];
services.netdata.configDir."python.d.conf" = pkgs.writeText "python.d.conf" ''
  nvidia_smi: yes
'';

samba

To enable samba plugin additional permissions and configurations will need to be set.

services.netdata.configDir."python.d.conf" = pkgs.writeText "python.d.conf" ''
  samba: yes
'';

# add samba and sudo to path of python plugin
systemd.services.netdata.path = [  pkgs.samba "/run/wrappers" ];

# permit to run sudo smbstatus -P
security.sudo.extraConfig = ''
  netdata ALL=(root) NOPASSWD: ${pkgs.samba}/bin/smbstatus
'';

# as documented here : https://github.com/netdata/netdata/blob/master/system/netdata.service.in
# review capabilityset above if other plugins are non functional
systemd.services.netdata.serviceConfig.CapabilityBoundingSet = ["CAP_SETGID"];

# enable profiling
services.samba.extraConfig = ''
  smbd profiling level = on
'';