Data Science workgroup - Revision history

DoggoBit at 17:58, 8 June 2025

2025-06-08T17:58:49Z

← Older revision		Revision as of 17:58, 8 June 2025
Line 1:		Line 1:
	{{outdated\|Other than site-wide fixes, this page has not seen recent updates.}}		{{delete\|reason=This workgroup may no longer be active. Last substantial page edit before import.}}{{outdated\|Other than site-wide fixes, this page has not seen recent updates.}}
	This workgroup is dedicated towards improving the state of the data science stack in Nixpkgs. This includes work on packages and modules for scientific computation, artificial intelligence and data processing, as well as data science IDEs.		This workgroup is dedicated towards improving the state of the data science stack in Nixpkgs. This includes work on packages and modules for scientific computation, artificial intelligence and data processing, as well as data science IDEs.

DoggoBit: DoggoBit moved page Workgroup:DataScience to Data Science workgroup: We shouldn't be using "namespace-looking" page titles.

2025-06-08T17:53:40Z

DoggoBit moved page Workgroup:DataScience to Data Science workgroup: We shouldn't be using "namespace-looking" page titles.

← Older revision		Revision as of 17:53, 8 June 2025
(No difference)

Phanirithvij: fix old link with latest nixpkgs master link

2024-09-15T06:08:50Z

fix old link with latest nixpkgs master link

← Older revision		Revision as of 06:08, 15 September 2024
Line 88:		Line 88:


	with such highlights as ~~@aborsu's~~ [https://github.com/~~aborsu~~/nixpkgs/blob/~~22ef965da38cc5e3457fe2d848b8a789cb6ad207~~/nixos/modules/services/development/jupyter/default.nix Jupyter kernels written in Nix]:		with such highlights as [https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/development/jupyter/default.nix Jupyter kernels written in Nix]:

	{{file\|./modules/datasci.nix\|nix\|<nowiki>		{{file\|./modules/datasci.nix\|nix\|<nowiki>

Samueldr at 00:11, 5 April 2024

2024-04-05T00:11:52Z

← Older revision		Revision as of 00:11, 5 April 2024
Line 1:		Line 1:
			{{outdated\|Other than site-wide fixes, this page has not seen recent updates.}}
	This workgroup is dedicated towards improving the state of the data science stack in Nixpkgs. This includes work on packages and modules for scientific computation, artificial intelligence and data processing, as well as data science IDEs.		This workgroup is dedicated towards improving the state of the data science stack in Nixpkgs. This includes work on packages and modules for scientific computation, artificial intelligence and data processing, as well as data science IDEs.


	=== JupyterLab ===		=== JupyterLab ===

imported>Precise Librarian: "stdenv.lib" has been removed in favor of just "lib"

2022-07-06T09:18:24Z

"stdenv.lib" has been removed in favor of just "lib"

← Older revision		Revision as of 09:18, 6 July 2022
Line 60:		Line 60:

	# labextensions		# labextensions
	${pkgs~~.stdenv~~.lib.concatMapStrings		${pkgs.lib.concatMapStrings
	(s: "jupyter labextension install --no-build --app-dir=$TEMPDIR ${s}; ")		(s: "jupyter labextension install --no-build --app-dir=$TEMPDIR ${s}; ")
	(pkgs.lib.unique		(pkgs.lib.unique

imported>0x4A6F at 16:14, 6 August 2021

2021-08-06T16:14:52Z

← Older revision		Revision as of 16:14, 6 August 2021
Line 127:		Line 127:
	== Channels ==		== Channels ==

	[~~irc~~://~~irc~~.~~freenode.net~~/nixos~~-data ''~~#nixos~~-data''~~ on ~~Freenode~~]		[https://matrix.to/#/#datascience:nixos.org #datascience:nixos.org on Matrix]

	== People ==		== People ==

	[[User:Ixxie\|Ixxie]]		[[User:Ixxie\|Ixxie]]

imported>Ariutta: Mentioned JupyterWith. Clarified that PR #49807 was not merged.

2019-12-02T21:40:46Z

Mentioned JupyterWith. Clarified that PR #49807 was not merged.

← Older revision		Revision as of 21:40, 2 December 2019
Line 4:		Line 4:

	=== JupyterLab ===		=== JupyterLab ===
	[https://github.com/NixOS/nixpkgs/pull/49807 ~~Work was done (but not merged)~~] to easily deploy arbitrary kernels and jupyter extensions with nix. There are some limitations due to jupyterlab extensions relying heavily on npm and webpack to compile the javascript modules. Thus an unpure setup is easiest to get it working. ~~The~~ following `default.nix` shell ~~will~~ install 20 jupyerlab extension + 4 kernels (c, python, go, and ansible). All that ~~needs~~ to be edited by user is kernels, additionalExtensions, and buildInputs. The rest is automatic and ~~will~~ launch a jupyterlab instance for you.		The [https://github.com/tweag/jupyterWith JupyterWith] repo "provides a Nix-based framework for the definition of declarative and reproducible Jupyter environments. These environments include JupyterLab - configurable with extensions - the classic notebook, and configurable Jupyter kernels."

			Alternatively, [https://github.com/NixOS/nixpkgs/pull/49807 there is an unmerged pull request] with work to easily deploy arbitrary kernels and jupyter extensions with nix. There are some limitations due to jupyterlab extensions relying heavily on npm and webpack to compile the javascript modules. Thus an unpure setup was considered easiest to get it working. If the pull request were merged, the following `default.nix` shell would install 20 jupyerlab extension + 4 kernels (c, python, go, and ansible). All that would need to be edited by user would be kernels, additionalExtensions, and buildInputs. The rest would be automatic and would launch a jupyterlab instance for you.

	<syntaxhighlight lang="nix">		<syntaxhighlight lang="nix">

imported>Ariutta: Reference unmerged pull request

2019-12-02T21:30:39Z

Reference unmerged pull request

← Older revision		Revision as of 21:30, 2 December 2019
Line 4:		Line 4:

	=== JupyterLab ===		=== JupyterLab ===
			[https://github.com/NixOS/nixpkgs/pull/49807 Work was done (but not merged)] to easily deploy arbitrary kernels and jupyter extensions with nix. There are some limitations due to jupyterlab extensions relying heavily on npm and webpack to compile the javascript modules. Thus an unpure setup is easiest to get it working. The following `default.nix` shell will install 20 jupyerlab extension + 4 kernels (c, python, go, and ansible). All that needs to be edited by user is kernels, additionalExtensions, and buildInputs. The rest is automatic and will launch a jupyterlab instance for you.
	Work ~~has been~~ done to easily deploy arbitrary kernels and jupyter extensions with nix. There are some limitations due to jupyterlab extensions relying heavily on npm and webpack to compile the javascript modules. Thus an unpure setup is easiest to get it working. The following `default.nix` shell will install 20 jupyerlab extension + 4 kernels (c, python, go, and ansible). All that needs to be edited by user is kernels, additionalExtensions, and buildInputs. The rest is automatic and will launch a jupyterlab instance for you.

	<syntaxhighlight lang="nix">		<syntaxhighlight lang="nix">

imported>Costrouc: Adding jupyterlab instructions with nix-shell

2018-11-12T22:26:59Z

Adding jupyterlab instructions with nix-shell

← Older revision		Revision as of 22:26, 12 November 2018
Line 1:		Line 1:

	This workgroup is dedicated towards improving the state of the data science stack in Nixpkgs. This includes work on packages and modules for scientific computation, artificial intelligence and data processing, as well as data science IDEs.		This workgroup is dedicated towards improving the state of the data science stack in Nixpkgs. This includes work on packages and modules for scientific computation, artificial intelligence and data processing, as well as data science IDEs.


			=== JupyterLab ===

			Work has been done to easily deploy arbitrary kernels and jupyter extensions with nix. There are some limitations due to jupyterlab extensions relying heavily on npm and webpack to compile the javascript modules. Thus an unpure setup is easiest to get it working. The following `default.nix` shell will install 20 jupyerlab extension + 4 kernels (c, python, go, and ansible). All that needs to be edited by user is kernels, additionalExtensions, and buildInputs. The rest is automatic and will launch a jupyterlab instance for you.

			<syntaxhighlight lang="nix">
			{ pkgs ? import <nixpkgs> {}, pythonPackages ? pkgs.python36Packages }:

			let kernels = [
			pkgs.python36Packages.ansible-kernel
			pythonPackages.jupyter-c-kernel
			pkgs.gophernotes
			];

			additionalExtensions = [
			"@jupyterlab/toc"
			"@jupyterlab/fasta-extension"
			"@jupyterlab/geojson-extension"
			"@jupyterlab/katex-extension"
			"@jupyterlab/mathjax3-extension"
			"@jupyterlab/plotly-extension"
			"@jupyterlab/vega2-extension"
			"@jupyterlab/vega3-extension"
			"@jupyterlab/xkcd-extension"
			"jupyterlab-drawio"
			"@jupyterlab/hub-extension"
			"jupyterlab_bokeh"
			];
			in
			pkgs.mkShell rec {
			buildInputs = [
			### Base Packages
			pythonPackages.jupyterlab pkgs.nodejs

			### Extensions
			pythonPackages.ipywidgets
			pythonPackages.ipydatawidgets
			pythonPackages.ipywebrtc
			pythonPackages.pythreejs
			pythonPackages.ipyvolume
			pythonPackages.jupyterlab-git
			pythonPackages.jupyterlab-latex
			pythonPackages.ipyleaflet
			pythonPackages.ipympl
			] ++ kernels;

			shellHook = ''
			TEMPDIR=$(mktemp -d -p /tmp)
			mkdir -p $TEMPDIR
			cp -r ${pkgs.python36Packages.jupyterlab}/share/jupyter/lab/* $TEMPDIR
			chmod -R 755 $TEMPDIR
			echo "$TEMPDIR is the app directory"

			# kernels
			export JUPYTER_PATH="${pkgs.lib.concatMapStringsSep ":" (p: "${p}/share/jupyter/") kernels}"

			# labextensions
			${pkgs.stdenv.lib.concatMapStrings
			(s: "jupyter labextension install --no-build --app-dir=$TEMPDIR ${s}; ")
			(pkgs.lib.unique
			((pkgs.lib.concatMap
			(d: pkgs.lib.attrByPath ["passthru" "jupyterlabExtensions"] [] d)
			buildInputs) ++ additionalExtensions)) }
			jupyter lab build --app-dir=$TEMPDIR

			# start jupyterlab
			jupyter lab --app-dir=$TEMPDIR
			'';

			}
			</syntaxhighlight>

	Some recent examples of work done on libraries:		Some recent examples of work done on libraries:

imported>Ixxie at 18:49, 7 June 2018

2018-06-07T18:49:24Z

← Older revision		Revision as of 18:49, 7 June 2018
Line 2:		Line 2:
	This workgroup is dedicated towards improving the state of the data science stack in Nixpkgs. This includes work on packages and modules for scientific computation, artificial intelligence and data processing, as well as data science IDEs.		This workgroup is dedicated towards improving the state of the data science stack in Nixpkgs. This includes work on packages and modules for scientific computation, artificial intelligence and data processing, as well as data science IDEs.

	~~There have been some~~ recent examples of work done on libraries:		Some recent examples of work done on libraries:

	* [https://github.com/NixOS/nixpkgs/pulls?utf8=%E2%9C%93&q=is%3Apr+nlp+ nlp]		* [https://github.com/NixOS/nixpkgs/pulls?utf8=%E2%9C%93&q=is%3Apr+nlp+ nlp]