Go: Difference between revisions
imported>Mic92 go static linking |
Tags: Undo Mobile edit Mobile web edit |
||
(20 intermediate revisions by 15 users not shown) | |||
Line 1: | Line 1: | ||
[https://golang.org/ Go] is a statically-typed language with syntax loosely derived from that of C, adding garbage collected memory management, type safety, some dynamic-typing capabilities, additional built-in types such as variable-length arrays and key-value maps, and a large standard library. | [https://golang.org/ Go] is a statically-typed language with syntax loosely derived from that of C, adding garbage collected memory management, type safety, some dynamic-typing capabilities, additional built-in types such as variable-length arrays and key-value maps, and a large standard library. | ||
==Using cgo on NixOS== | == buildGoModule == | ||
nixpkgs includes a library function called '''buildGoModule''' ([https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/go/module.nix implementation]) See [https://nixos.org/manual/nixpkgs/stable/#sec-language-go nixpkgs manual '''Language: Go'''] | |||
`buildGoModule` uses the version of Go that's included in `nixpkgs` to build the software. | |||
==== Using a specific version of Go ==== | |||
To build for a specific version of Go, you may need to find the appropriate `pkgs.buildGoXXXModule` function to use. | |||
This function may not be present in the version of nixpkgs that you're using, for example, `buildGo122Module` is not available in `github:NixOS/nixpkgs/nixos-23.05`, but is available in `github:NixOS/nixpkgs/nixos-unstable`. | |||
==== Subpackages ==== | |||
By default, `buildGoModule` will attempt to build the `main` package that's in the root of the source code location. | |||
However, it's a common pattern in Go applications to have binaries within the `./cmd/binary-name` directory instead. | |||
Setting the `subPackages` attribute to be a list of the packages to build supports this pattern. | |||
==== Example (downloading source code from Github) ==== | |||
The following `flake.nix` demonstrates how to build a Go module, where the source code is located in Github. To use it, copy this file as `flake.nix` into an empty directory on your computer, and run `nix build`. Nix will download the source code, including dependencies, and produce a `./result` folder containing a `ziti` binary. | |||
Running `nix shell` will create a shell, where you can execute the `ziti` binary.<syntaxhighlight lang="nix"> | |||
{ | |||
description = "OpenZiti"; | |||
inputs = { | |||
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable"; | |||
}; | |||
outputs = { self, nixpkgs }: | |||
let | |||
allSystems = [ | |||
"x86_64-linux" # 64-bit Intel/AMD Linux | |||
"aarch64-linux" # 64-bit ARM Linux | |||
"x86_64-darwin" # 64-bit Intel macOS | |||
"aarch64-darwin" # 64-bit ARM macOS | |||
]; | |||
forAllSystems = f: nixpkgs.lib.genAttrs allSystems (system: f { | |||
pkgs = import nixpkgs { inherit system; }; | |||
}); | |||
in | |||
{ | |||
packages = forAllSystems ({ pkgs }: { | |||
default = pkgs.buildGo122Module rec { | |||
pname = "openziti"; | |||
version = "1.0.0"; | |||
subPackages = [ "ziti" ]; | |||
src = pkgs.fetchFromGitHub { | |||
owner = "openziti"; | |||
repo = "ziti"; | |||
rev = "v${version}"; | |||
sha256 = "sha256-2li/+XWKk+lybB1DE0unKvQrA0pKE9VIRFoEYMcbLS8="; | |||
}; | |||
vendorHash = "sha256-uyjQd5kB61UEKSl1Qf1Gu6Fr40l4KixHSnEtTMq58Vc="; | |||
}; | |||
}); | |||
}; | |||
} | |||
</syntaxhighlight> | |||
==== Example (local source) ==== | |||
If you want to build a local project with Nix, replace the `src` attribute to be the local directory, e.g.:<syntaxhighlight lang="nix"> | |||
some-package = buildGoModule { | |||
src = ./. | |||
}; | |||
</syntaxhighlight> | |||
==== Monorepo support ==== | |||
the <tt>go.mod</tt> file must be in the source root for <tt>buildGoModule</tt>. | |||
to change the source root, use | |||
<syntaxhighlight lang=nix> | |||
some-package = buildGoModule { | |||
src = fetchFromGitHub { | |||
# ... | |||
} + "/path/to/module"; | |||
# ... | |||
}; | |||
</syntaxhighlight> | |||
=== buildGoPackage === | |||
If no <tt>go.mod</tt> file is available, '''buildGoPackage''' ([https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/go/package.nix implementation]) can be used. | |||
Dependencies must be specified manually in a <tt>deps.nix</tt> file, | |||
which is linked with | |||
<syntaxhighlight lang=nix> | |||
some-package = buildGoPackage { | |||
# ... | |||
goDeps = ./deps.nix; | |||
}; | |||
</syntaxhighlight> | |||
== Using cgo on NixOS == | |||
On NixOS, include files and libraries aren't kept in a system-wide search path. If a Go program uses cgo and attempts to include C header files, or link against libraries, compilation is likely to fail. | On NixOS, include files and libraries aren't kept in a system-wide search path. If a Go program uses cgo and attempts to include C header files, or link against libraries, compilation is likely to fail. | ||
Line 18: | Line 108: | ||
If you intend to compile against glibc statically (such as via <tt>go build -ldflags "-s -w -linkmode external -extldflags -static"</tt>), add <tt>glibc.static</tt> to the list of packages passed to <tt>nix-shell</tt>. | If you intend to compile against glibc statically (such as via <tt>go build -ldflags "-s -w -linkmode external -extldflags -static"</tt>), add <tt>glibc.static</tt> to the list of packages passed to <tt>nix-shell</tt>. | ||
If you encounter [https://github.com/go-delve/delve/issues/3085 this issue] and receive an error about _FORTIFY_SOURCE when running delve (for example in VSCode), put <tt>hardeningDisable = [ "fortify" ];</tt> inside shell.nix or in the <tt>mkShell</tt> invocation argument like this: | |||
<syntaxhighlight lang="nix"> | |||
pkgs.mkShell { | |||
hardeningDisable = [ "fortify" ]; | |||
buildInputs = [ pkgs.go_1_18 ]; | |||
}; | |||
</syntaxhighlight> | |||
== Compile go program with static compile flag == | == Compile go program with static compile flag == | ||
If <code>go build -ldflags "-s -w -linkmode external -extldflags -static"</code> fails on NixOS, with the error message <code>cannot find `-lpthread</code> and <code>cannot find -lc</code> - it is because the linker cannot find static glibc to link with. You need to have glibc.static in your environment (and have CFLAGS/LDFLAGS adjusted accordingly). | If <code>go build -ldflags "-s -w -linkmode external -extldflags -static"</code> fails on NixOS, with the error message <code>cannot find `-lpthread</code> and <code>cannot find -lc</code> - it is because the linker cannot find static glibc to link with. You need to have glibc.static in your environment (and have CFLAGS/LDFLAGS adjusted accordingly). | ||
One way to achieve this is to have something like the following as | One way to achieve this is to have something like the following as <code>shell.nix</code> and run the compilation in a nix-shell: | ||
<syntaxHighlight lang=nix> | <syntaxHighlight lang=nix> | ||
Line 27: | Line 125: | ||
devEnv = stdenv.mkDerivation { | devEnv = stdenv.mkDerivation { | ||
name = "dev"; | name = "dev"; | ||
buildInputs = [ stdenv | buildInputs = [ stdenv go glibc.static ]; | ||
CFLAGS="-I${pkgs.glibc.dev}/include"; | CFLAGS="-I${pkgs.glibc.dev}/include"; | ||
LDFLAGS="-L${pkgs.glibc}/lib"; | LDFLAGS="-L${pkgs.glibc}/lib"; | ||
}; | |||
} | |||
</syntaxHighlight> | |||
== Compile go program with static compile flag (take 2) == | |||
Linking against glibc.static does not really work because glibc does not really like static linking. You get a warning like <code>warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking</code>. To really create a static build, use musl. Example based on buildGoModule example from documentation: | |||
<syntaxHighlight lang=nix> | |||
pet = buildGoModule rec { | |||
pname = "pet"; | |||
version = "0.3.4"; | |||
src = fetchFromGitHub { | |||
owner = "knqyf263"; | |||
repo = "pet"; | |||
rev = "v${version}"; | |||
sha256 = "0m2fzpqxk7hrbxsgqplkg7h2p7gv6s1miymv3gvw0cz039skag0s"; | |||
}; | |||
vendorSha256 = "1879j77k96684wi554rkjxydrj8g3hpp0kvxz03sd8dmwr3lh83j"; | |||
ldflags = [ | |||
"-s -w -X github.com/knqyf263/pet/cmd.version=${version}" | |||
]; | |||
nativeBuildInputs = [musl]; | |||
CGO_ENABLED = 0; | |||
ldflags = [ | |||
"-linkmode external" | |||
"-extldflags '-static -L${musl}/lib'" | |||
]; | |||
meta = with lib; { | |||
description = "Simple command-line snippet manager, written in Go"; | |||
homepage = "https://github.com/knqyf263/pet"; | |||
license = licenses.mit; | |||
maintainers = with maintainers; [ kalbasit ]; | |||
}; | }; | ||
} | } | ||
Line 35: | Line 173: | ||
[[Category:Languages]] | [[Category:Languages]] | ||
[[Category:Applications]] |
Latest revision as of 10:47, 1 November 2024
Go is a statically-typed language with syntax loosely derived from that of C, adding garbage collected memory management, type safety, some dynamic-typing capabilities, additional built-in types such as variable-length arrays and key-value maps, and a large standard library.
buildGoModule
nixpkgs includes a library function called buildGoModule (implementation) See nixpkgs manual Language: Go
`buildGoModule` uses the version of Go that's included in `nixpkgs` to build the software.
Using a specific version of Go
To build for a specific version of Go, you may need to find the appropriate `pkgs.buildGoXXXModule` function to use.
This function may not be present in the version of nixpkgs that you're using, for example, `buildGo122Module` is not available in `github:NixOS/nixpkgs/nixos-23.05`, but is available in `github:NixOS/nixpkgs/nixos-unstable`.
Subpackages
By default, `buildGoModule` will attempt to build the `main` package that's in the root of the source code location.
However, it's a common pattern in Go applications to have binaries within the `./cmd/binary-name` directory instead.
Setting the `subPackages` attribute to be a list of the packages to build supports this pattern.
Example (downloading source code from Github)
The following `flake.nix` demonstrates how to build a Go module, where the source code is located in Github. To use it, copy this file as `flake.nix` into an empty directory on your computer, and run `nix build`. Nix will download the source code, including dependencies, and produce a `./result` folder containing a `ziti` binary.
Running `nix shell` will create a shell, where you can execute the `ziti` binary.
{
description = "OpenZiti";
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
};
outputs = { self, nixpkgs }:
let
allSystems = [
"x86_64-linux" # 64-bit Intel/AMD Linux
"aarch64-linux" # 64-bit ARM Linux
"x86_64-darwin" # 64-bit Intel macOS
"aarch64-darwin" # 64-bit ARM macOS
];
forAllSystems = f: nixpkgs.lib.genAttrs allSystems (system: f {
pkgs = import nixpkgs { inherit system; };
});
in
{
packages = forAllSystems ({ pkgs }: {
default = pkgs.buildGo122Module rec {
pname = "openziti";
version = "1.0.0";
subPackages = [ "ziti" ];
src = pkgs.fetchFromGitHub {
owner = "openziti";
repo = "ziti";
rev = "v${version}";
sha256 = "sha256-2li/+XWKk+lybB1DE0unKvQrA0pKE9VIRFoEYMcbLS8=";
};
vendorHash = "sha256-uyjQd5kB61UEKSl1Qf1Gu6Fr40l4KixHSnEtTMq58Vc=";
};
});
};
}
Example (local source)
If you want to build a local project with Nix, replace the `src` attribute to be the local directory, e.g.:
some-package = buildGoModule {
src = ./.
};
Monorepo support
the go.mod file must be in the source root for buildGoModule. to change the source root, use
some-package = buildGoModule {
src = fetchFromGitHub {
# ...
} + "/path/to/module";
# ...
};
buildGoPackage
If no go.mod file is available, buildGoPackage (implementation) can be used. Dependencies must be specified manually in a deps.nix file, which is linked with
some-package = buildGoPackage {
# ...
goDeps = ./deps.nix;
};
Using cgo on NixOS
On NixOS, include files and libraries aren't kept in a system-wide search path. If a Go program uses cgo and attempts to include C header files, or link against libraries, compilation is likely to fail.
In order to expose header files and libraries in environment variable search paths, nix-shell can be used to enter an environment which provides the requested development dependencies.
For example, suppose a Go program includes <sys/capability.h> (provided by libcap), and links against libcap. To obtain an environment in which the program can be compiled, run:
$ nix-shell -p libcap go gcc
You can verify the presence of the necessary environment variables via the following command:
$ export | egrep 'NIX_.*(LDFLAGS|COMPILE|LINK)'
If you intend to compile against glibc statically (such as via go build -ldflags "-s -w -linkmode external -extldflags -static"), add glibc.static to the list of packages passed to nix-shell.
If you encounter this issue and receive an error about _FORTIFY_SOURCE when running delve (for example in VSCode), put hardeningDisable = [ "fortify" ]; inside shell.nix or in the mkShell invocation argument like this:
pkgs.mkShell {
hardeningDisable = [ "fortify" ];
buildInputs = [ pkgs.go_1_18 ];
};
Compile go program with static compile flag
If go build -ldflags "-s -w -linkmode external -extldflags -static"
fails on NixOS, with the error message cannot find `-lpthread
and cannot find -lc
- it is because the linker cannot find static glibc to link with. You need to have glibc.static in your environment (and have CFLAGS/LDFLAGS adjusted accordingly).
One way to achieve this is to have something like the following as shell.nix
and run the compilation in a nix-shell:
with import <nixpkgs> {}; {
devEnv = stdenv.mkDerivation {
name = "dev";
buildInputs = [ stdenv go glibc.static ];
CFLAGS="-I${pkgs.glibc.dev}/include";
LDFLAGS="-L${pkgs.glibc}/lib";
};
}
Compile go program with static compile flag (take 2)
Linking against glibc.static does not really work because glibc does not really like static linking. You get a warning like warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
. To really create a static build, use musl. Example based on buildGoModule example from documentation:
pet = buildGoModule rec {
pname = "pet";
version = "0.3.4";
src = fetchFromGitHub {
owner = "knqyf263";
repo = "pet";
rev = "v${version}";
sha256 = "0m2fzpqxk7hrbxsgqplkg7h2p7gv6s1miymv3gvw0cz039skag0s";
};
vendorSha256 = "1879j77k96684wi554rkjxydrj8g3hpp0kvxz03sd8dmwr3lh83j";
ldflags = [
"-s -w -X github.com/knqyf263/pet/cmd.version=${version}"
];
nativeBuildInputs = [musl];
CGO_ENABLED = 0;
ldflags = [
"-linkmode external"
"-extldflags '-static -L${musl}/lib'"
];
meta = with lib; {
description = "Simple command-line snippet manager, written in Go";
homepage = "https://github.com/knqyf263/pet";
license = licenses.mit;
maintainers = with maintainers; [ kalbasit ];
};
}