netdata is a metrics tool, which comes with a lot of sane preconfiguration. It contains of various plugins, which need to be enabled sometimes with additional effort.

Adding node

Install the netdata package and enable the service.
When adding new node in the web interface you get a token, write that token to /var/lib/netdata/cloud.d/token
As root run nix-shell -p netdata --run "netdata-claim.sh"

Streaming node setup

Ensure you choose appropriate access control for your nodes.

Receiver node

  services.netdata.configDir."stream.conf" =
     let
        mkChildNode = apiKey: allowFrom: ''
          [${apiKey}]
            enabled = yes
            default history = <a value of your choice>
            default memory mode = dbengine # a good default
            health enabled by default = auto
            allow from = ${allowFrom}
        '';
      in pkgs.writeText "stream.conf" ''
        [stream]
          # This won't stream by itself, except if the receiver is a sender too, which is possible in netdata model.
          enabled = no
          enable compression = yes

        # An allowed sender node
        ${mkChildNode "an API key" "an allowed IP"}
      '';

Sender node

  services.netdata.configDir."stream.conf" = pkgs.writeText "stream.conf" ''
      [stream] 
        enabled = yes
        destination = receiver-hostname-or-ip-address:19999
        api key = any string that is set also on the receiver side
    '';

If you don't need any web UI and want to consume minimal resources on the sender node, use:

  services.netdata = {
    config = {
      global = { "memory mode" = "none"; };
      web = {
        mode = "none";
        "accept a streaming request every seconds" = 0;
      };
    };
  };

This way, it won't spawn any web UI, neither store any metric locally.

Python Plugins

nvidia-smi

To enable the nvidia-smi plugin you have to make sure nvidia-smi can be called by netdata.

systemd.services.enable = true;
systemd.services.netdata.path = [pkgs.linuxPackages.nvidia_x11];
services.netdata.configDir."python.d.conf" = pkgs.writeText "python.d.conf" ''
  nvidia_smi: yes
'';

samba

To enable samba plugin additional permissions and configurations will need to be set.

services.netdata.configDir."python.d.conf" = pkgs.writeText "python.d.conf" ''
  samba: yes
'';

# add samba and sudo to path of python plugin
systemd.services.netdata.path = [  pkgs.samba "/run/wrappers" ];

# permit to run sudo smbstatus -P
security.sudo.extraConfig = ''
  netdata ALL=(root) NOPASSWD: ${pkgs.samba}/bin/smbstatus
'';

# as documented here : https://github.com/netdata/netdata/blob/master/system/netdata.service.in
# review capabilityset above if other plugins are non functional
systemd.services.netdata.serviceConfig.CapabilityBoundingSet = ["CAP_SETGID"];

# enable profiling
services.samba.extraConfig = ''
smbd profiling level = on
'';