Noisemaker - Revision history

imported>Raboof: how to use shell.nix instead of venv with a nixpkgs that has a recent tensorflow

2020-08-21T20:51:17Z

how to use shell.nix instead of venv with a nixpkgs that has a recent tensorflow

← Older revision		Revision as of 20:51, 21 August 2020
Line 1:		Line 1:
	https://noisemaker.readthedocs.io/en/latest/		https://noisemaker.readthedocs.io/en/latest/

	~~At the time of writing noisemaker is still on~~ tensorflow 1.~~3, which isn't available~~ for python 3.7 (~~only up to 3.6, https://pypi.org/project/tensorflow/1.3.0/#files). Let~~'s ~~just use tensorflow 2 instead (https://github~~.~~com/aayars/py-noisemaker/pull/4):~~		Right now nixpkgs contains tensorflow 2.1.0 but has it disabled for python 3.8 (as it's not officially supported upstream either).

	~~<syntaxHighlight lang~~=~~console>~~		== with tensorflow 2.3.0 from a branch ==
	~~$ git clone git@~~github.com~~:aayars~~/py-noisemaker py-noisemaker
	~~$ cd py-noisemaker~~		Using nixpkgs from https://github.com/NixOS/nixpkgs/pull/95824 , you can update setup.py to change '==' versions to '>=' for the dependencies for which NixOS is shipping newer versions, and then use the following shell.nix:
	~~$ hub remote add raboof~~
	~~$ git fetch raboof~~		<syntaxHighlight lang=nix>
	~~$ git checkout tensorflow2~~		with import <nixpkgs> {};
			with python3Packages;

			buildPythonPackage rec {
			name = "noisemaker";
			src = /path/to/py-noisemaker;
			propagatedBuildInputs = [
			virtualenv
			pillow
			tensorflow_2
			click
			];
			}
	</syntaxHighlight>		</syntaxHighlight>

	~~Right now nixpkgs contains~~ tensorflow 2.1.0 ~~but has it disabled for python 3.8 (as it's not officially supported upstream either).~~ 'python3 -m venv venv' doesn't seem to work with python 3.7, so I'm sticking with 3.8.		== with tensorflow 2.3.0 from pip in a venv ==

			'python3 -m venv venv' doesn't seem to work with python 3.7, so I'm sticking with 3.8.

	noisemaker wants exactly version 6.2.0 of Pillow. The pillow in nixpkgs is a different version, so this will be fetched and built with setup.py. To make sure it can correctly find the C libraries it uses, we use fshUserEnv and add some development headers:		noisemaker wants exactly version 6.2.0 of Pillow. The pillow in nixpkgs is a different version, so this will be fetched and built with setup.py. To make sure it can correctly find the C libraries it uses, we use fshUserEnv and add some development headers:
Line 55:		Line 69:

	(this will generate 'noise.png')		(this will generate 'noise.png')

	~~== Next steps ==~~

	~~It'd be nice to package noisemaker more 'nix natively'. Looking into installing tensorflow from nix instead of pip.~~

imported>Mic92: fix markup

2020-07-31T12:27:16Z

fix markup

← Older revision		Revision as of 12:27, 31 July 2020
Line 3:		Line 3:
	At the time of writing noisemaker is still on tensorflow 1.3, which isn't available for python 3.7 (only up to 3.6, https://pypi.org/project/tensorflow/1.3.0/#files). Let's just use tensorflow 2 instead (https://github.com/aayars/py-noisemaker/pull/4):		At the time of writing noisemaker is still on tensorflow 1.3, which isn't available for python 3.7 (only up to 3.6, https://pypi.org/project/tensorflow/1.3.0/#files). Let's just use tensorflow 2 instead (https://github.com/aayars/py-noisemaker/pull/4):

	git clone git@github.com:aayars/py-noisemaker py-noisemaker		<syntaxHighlight lang=console>
	cd py-noisemaker		$ git clone git@github.com:aayars/py-noisemaker py-noisemaker
	hub remote add raboof		$ cd py-noisemaker
	git fetch raboof		$ hub remote add raboof
	git checkout tensorflow2		$ git fetch raboof
			$ git checkout tensorflow2
			</syntaxHighlight>

	Right now nixpkgs contains tensorflow 2.1.0 but has it disabled for python 3.8 (as it's not officially supported upstream either). 'python3 -m venv venv' doesn't seem to work with python 3.7, so I'm sticking with 3.8.		Right now nixpkgs contains tensorflow 2.1.0 but has it disabled for python 3.8 (as it's not officially supported upstream either). 'python3 -m venv venv' doesn't seem to work with python 3.7, so I'm sticking with 3.8.
Line 13:		Line 15:
	noisemaker wants exactly version 6.2.0 of Pillow. The pillow in nixpkgs is a different version, so this will be fetched and built with setup.py. To make sure it can correctly find the C libraries it uses, we use fshUserEnv and add some development headers:		noisemaker wants exactly version 6.2.0 of Pillow. The pillow in nixpkgs is a different version, so this will be fetched and built with setup.py. To make sure it can correctly find the C libraries it uses, we use fshUserEnv and add some development headers:

			<syntaxHighlight lang=nix>
	{ pkgs ? import <nixpkgs> {} }:		{ pkgs ? import <nixpkgs> {} }:

Line 29:		Line 32:
	];		];
	}).env		}).env
			</syntaxHighlight>

	Then create noisemaker in a venv:		Then create noisemaker in a venv:

	python3 -m venv venv		<syntaxHighlight lang=console>
	source venv/bin/activate		$ python3 -m venv venv
	python3 setup.py develop		$ source venv/bin/activate
			$ python3 setup.py develop
			</syntaxHighlight>

	Add a newer tensorflow from pip:		Add a newer tensorflow from pip:

	pip install tensorflow		<syntaxHighlight lang=console>
			$ pip install tensorflow
			</syntaxHighlight>

	And run noisemaker:		And run noisemaker:

	noisemaker		<syntaxHighlight lang=console>
			$ noisemaker
			</syntaxHighlight>

	(this will generate 'noise.png')		(this will generate 'noise.png')

	## Next steps		== Next steps ==

	It'd be nice to package noisemaker more 'nix natively'. Looking into installing tensorflow from nix instead of pip.		It'd be nice to package noisemaker more 'nix natively'. Looking into installing tensorflow from nix instead of pip.

imported>Raboof: document how to build noisemaker

2020-07-31T10:42:59Z

document how to build noisemaker

New page

https://noisemaker.readthedocs.io/en/latest/

At the time of writing noisemaker is still on tensorflow 1.3, which isn't available for python 3.7 (only up to 3.6, https://pypi.org/project/tensorflow/1.3.0/#files). Let's just use tensorflow 2 instead (https://github.com/aayars/py-noisemaker/pull/4):

git clone git@github.com:aayars/py-noisemaker py-noisemaker
cd py-noisemaker
hub remote add raboof
git fetch raboof
git checkout tensorflow2

Right now nixpkgs contains tensorflow 2.1.0 but has it disabled for python 3.8 (as it's not officially supported upstream either). 'python3 -m venv venv' doesn't seem to work with python 3.7, so I'm sticking with 3.8.

noisemaker wants exactly version 6.2.0 of Pillow. The pillow in nixpkgs is a different version, so this will be fetched and built with setup.py. To make sure it can correctly find the C libraries it uses, we use fshUserEnv and add some development headers:

{ pkgs ? import <nixpkgs> {} }:

let
my-python-packages = python-packages: with python-packages; [
];
in
(pkgs.buildFHSUserEnv {
name = "noisemaker";

targetPkgs = pkgs: [
(pkgs.python3.withPackages my-python-packages)
pkgs.zlib.dev
pkgs.libjpeg.dev
pkgs.gcc
];
}).env

Then create noisemaker in a venv:

python3 -m venv venv
source venv/bin/activate
python3 setup.py develop

Add a newer tensorflow from pip:

pip install tensorflow

And run noisemaker:

noisemaker

(this will generate 'noise.png')

## Next steps

It'd be nice to package noisemaker more 'nix natively'. Looking into installing tensorflow from nix instead of pip.