Installing a Python module locally

I decided to try to install a Python module in my home directory. I decided to start with the pygraphviz module.

Attempt 1: using 'pip install'
To install a Python module in my home directory using 'pip install', I think I should be able to type for example, on the Sanger farm:
(in /nfs/users/nfs_a/alc/Documents/PythonModules)
% pip install --install-option="--prefix=/nfs/users/nfs_a/alc/Documents/PythonModules" pygraphviz
This made a subdirectory /nfs/users/nfs_a/alc/Documents/PythonModules/share/doc/pygraphviz-1.3.1
However, this doesn't seem to have installed the module properly, I'm not sure why..

What did Beckett say? 'No matter. Try again. Fail again. Fail better'.. 

Attempt 2: using 'setup.py'
As a second attempt, I went to the pygraphfiz download page, and downloaded the source for the module:
% wget https://pypi.python.org/packages/source/p/pygraphviz/pygraphviz-1.3.1.tar.gz#md5=7f690295dfe77edaa9e552d09d98d279
Then:
% tar -xvf pygraphviz-1.3.1.tar.gz
Put the installed version of the module in ~alc/Documents/PythonModulesInstall, and tell the installer where to find graphviz (this is an extra requirement for pygraphviz, it didn't work when I let it try to find graphviz by itself):
% python3 setup.py install --home=/nfs/users/nfs_a/alc/Documents/PythonModulesInstall --include-path=/usr/include/graphviz/ --library-path=/usr/lib/graphviz
This seemed to run ok. It made a directory ~alc/Documents/PythonModulesInstall/lib/python/pygraphviz.

Now try running the tests:
% python3 setup_egg.py nosetests
This didn't work for me, as there didn't seem to be any setup_egg.py file. I must be missing something obvious. Oh well!

Now try importing the module:
% cd ~alc/Documents/PythonModulesInstall/lib/python/
% python3
>> import pygraphviz
>> dir(pygraphviz)
['AGraph', 'Attribute', 'DotError', 'Edge', 'ItemAttribute', 'Node', '__all__', '__author__', '__builtins__', '__cached__', '__date__', '__doc__', '__file__', '__initializing__', '__license__', '__loader__', '__name__', '__package__', '__path__', '__revision__', '__version__', 'absolute_import', 'agraph', 'division', 'graphviz', 'print_function', 'release', 'test', 'tests', 'version']
>> print(pygraphviz.__version__)
1.3.1

It's working, yay!

21-Nov-2018: Another example: installing pysvg:
This time in my Python directory (/nfs/users/nfs_a/alc/Documents/git/Python) I tried:
% pip install --install-option="--prefix=/nfs/users/nfs_a/alc/Documents/PythonModules" pysvg

Then I can see a directory pysvg in /nfs/users/nfs_a/alc/Documents/PythonModules/lib/python3.6/site-packages/

9-Mar-2022: using 'conda install'

Today I wanted to install scipy on the Sanger farm. In /nfs/users/nfs_a/alc/Documents/git/Python I typed:

% conda install scipy 

and it told me:

environment location: /lustre/scratch118/infgen/team133/alc/000_FUGI_CGrunau_Chromatin/RunningGDA/miniconda

The following packages will be downloaded: 

scipy-1.7.3

I can now see a directory: /lustre/scratch118/infgen/team133/alc/000_FUGI_CGrunau_Chromatin/RunningGDA/miniconda/pkgs/scipy-1.7.3-py39hc147768_0

When I typed:

% python3

and then 

>> import scipy

it seemed to work fine!

 

6-Dec-2022: using a virtual environment

I found a nice website at Sanger on the Sanger internal wiki about using a virtual environment to install Python modules. I was able to use this to install the PyPDF2 module:

(in my directory /nfs/users/nfs_a/alc/Documents/git/Python):

First I set up a virtual environment called 'vibriowatch_pypdf':

% virtualenv -p /usr/bin/python3 vibriowatch_pypdf

Then I activated it:

% source vibriowatch_pypdf/bin/activate
Then I installed PyPDF2 in that virtual environment:
% pip install PyPDF2

Then I can use the PyPDF2 module in a script that imports that Python module, e.g.

% python3 extract_data_from_pdf_file.py Monir2022_SuppTable1.pdf

If I need to run PyPDF2 from another directory on the Sanger farm, I seem to have to activate the 'vibriowatch_pypdf' environment first:

% source /nfs/users/nfs_a/alc/Documents/git/Python/vibriowatch_pypdf/bin/activate

When I want to finish using the virtual environment, I need to type:

% deactivate

That was easy, hurray!

 

Another example (30-Mar-2023): 

I wanted to run the VISTA tool developed by Corin Yeats at Pathogenwatch. First I downloaded it:

% git clone https://github.com/pathogenwatch-oss/vista/

It was put in a local directory VISTA.

I also downloaded the data files references.fasta and metadata.toml from https://github.com/pathogenwatch-oss/vista/tree/main/data

Format the database for VISTA:

% cp vista/data/references.fasta vista/data/references
% makeblastdb -in vista/data/references -input_type fasta -dbtype nucl

Then I tried to run it:

% python3 vista/vista.py 2010EL_1786.fasta vista/data > result.json

where 2010EL_1786.fasta is an example input fasta file for a genome.

It told me that it needed a Python module 'toml' and 'Bio':

To install this, in my directory /nfs/users/nfs_a/alc/Documents/git/Python, I set up a virtual environment called 'vibriowatch_vista':

% virtualenv -p /usr/bin/python3 vibriowatch_vista

Then I activated it:

% source vibriowatch_vista/bin/activate     

Then I installed toml:

% pip install toml         

Then I installed Bio:

% pip install Bio

Then back in the directory where I had downloaded VISTA from git, I activated the virtual environment:

% source /nfs/users/nfs_a/alc/Documents/git/Python/vibriowatch_vista/bin/activate

Then I was able to run VISTA ok:

% python3 vista/vista.py 2010EL_1786.fasta vista/data > result.json

This gave a result.json file with the output. Hurray! I have installed the Python libraries toml and Bio and run VISTA!