Using virtualenvwrapper lets me setup different python workspaces without changing the default environment for anyone else using the computer.
I’m running mySQL, Solr and iPython on the remote computer and using the iPython notebook in my local browser via a ssh tunnel.
Install all the system support libraries
These are required by the python modules I install below. If you remove some of the modules, you may not need all of these support libraries.
sudo apt-get install python-dev libfreetype6-dev liblapack-dev libatlas-dev \ libxml2-dev libxslt-dev gfortran uuid-dev libmysqlclient-dev
iPython needs ZeroMQ for notebooks.
wget http://download.zeromq.org/zeromq-2.2.0.tar.gz tar -zxvf zeromq-2.2.0.tar.gz cd zeromq-2.2.0 configure make make check sudo make install sudo ldconfig
Install virtualenv and virtualenvwrapper
Not sure if these steps are necessary because I went back and forward trying different approaches:
sudo apt-get install python-virtualenv sudo apt-get install python-pip
Uninstall debian squeeze default virtualenvwrapper as it is old:
sudo apt-get remove virtualenvwrapper
Make sure to delete /etc/bash_completion.d/virtualenvwrapper which is virtualenvwrapper.sh from the original debian install of virtualenvwrapper. It is not removed when you unistall virtualenvwrapper from debian.
sudo pip install virtualenvwrapper
to install the latest version.
Add lines to your .bashrc to define where the virtual environments should be
export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Devel source /usr/local/bin/virtualenvwrapper.sh
I also created the Devel directory, although I’m not sure if that was necessary.
Install ipython in virtual environment
Create a virtual environment and install ipython and all of the modules I’m using
into it. You may not want all of these modules.
mkvirtualenv --no-site-packages brendatest pip -E $VIRTUAL_ENV install ipython pip -E $VIRTUAL_ENV install tornado pip -E $VIRTUAL_ENV install numpy pip -E $VIRTUAL_ENV install scipy pip -E $VIRTUAL_ENV install pandas pip -E $VIRTUAL_ENV install matplotlib pip -E $VIRTUAL_ENV install gensim pip -E $VIRTUAL_ENV install pyzmq pip -E $VIRTUAL_ENV install sunburnt pip -E $VIRTUAL_ENV install lxml pip -E $VIRTUAL_ENV install pytz pip -E $VIRTUAL_ENV install httplib2 pip -E $VIRTUAL_ENV install MySQL-python
Although I’m now using sunburnt, some of my earlier tests use solrpy so install that too:
pip -E $VIRTUAL_ENV install solrpy
Updating to development version of ipython
Copy the base setup so can go back to it:
cpvirtualenv brendatest brendabase
This will not copy the working directory under
Upgrade to the most recent pip in the virtual environment
pip -E $VIRTUAL_ENV install --upgrade iphython
For me this changed pip from 0.7.2 to 1.1
New version doesn’t use the -E option anymore. So just use:
pip install git+git://github.com/ipython/ipython.git
Setup virtual environment for every login
I added: workon brendatest
to the bottom of my .bashrc file so the brendatest environment is always activated when I log in directly.
As well as putting my database into mySQL on morphia, I need to add a readonly user matching that defined in the Solr db-data-config.xml on morphia for solr access to the mySQL database. I also need to add a user on morphia with a password to give iPython programs access to database.
Remote iPython access
Having finished setting up ipython, it is possible to run it in a ssh terminal window, but it is using the w3m browser. I want to view it in chrome on my local computer instead. So I’ve written some scripts to do this. At the server end I have have a script to start solr:
#!/bin/sh cd java -Xms1024M -Xmx10240M -jar start.jar
and one to start iPython notebook.
Because using -t with ssh doesn’t load your user profile (.bash_rc for me), need to set up the virtualenvwrapper environment in this script.
#!/bin/bash export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Devel source /usr/local/bin/virtualenvwrapper.sh workon brendatest ipython notebook --pylab inline
Then I have a startup and shutdown script on my macbookpro. I’m using certificate authentication for ssh, so my ssh commands don’t have any password information.
Replacein the scripts below with your username and remote computer url for ssh. For example firstname.lastname@example.org
The startup script sets up the port redirection for ipython notebook, start solr and ipython remotely and then start chrome with the ipython notebook index page.
The shutdown script relies on the fact that I donâ€™t have anything else running on morphia. This is probably not the best approach.
#!/bin/sh # want to automate setting up of port forwarding from morphia.makehackvoid.com and start up of solr and ipython on morphia before opening ipython notebook page in chrome on my computer. # setup redirection for ipython notebook ssh -f -L 8888:localhost:8888 -N # on morphia... ssh -f ./startSolr.sh # delay here to keep solr startup messages seperate from ipython ones - not actually needed just cosmetic sleep 1 ssh -f b ./startIPython.sh # open browser for http://localhost:8888/ to access remote notebook # need to add something here to keep window open until I want to exit ipython sleep 1 /usr/bin/open -a "/Applications/Google Chrome.app" 'http://localhost:8888/'
#!/bin/sh # now shut everything back down. ssh 'ps h -u brenda -o pid | xargs kill'
I’d like to find a way to have the shutdown happen automatically when I close the terminal window that the startup script has run in (or maybe a way of having the startup script wait to be told to quit?)