I’ve been playing with the MakeHackVoid Sun computer (Morphia) to see how much faster some of my data mining programs run when given access to 64G of memory instead of 4G (not to mention the processors). The short answer is “a lot”!
So I started thinking about off loading the last part of the backend, which is the ipython notebook server. I don’t think that ipython notebook is putting much load on my computer, especially compared to what mysql and solr were causing. But I’m just interested to see if it is possible.
It would mean only opening a single ssh tunnel for the notebook instead of two for mysql and solr 🙂
For ipython notebook I need to install a much more recent ipython and then all the python libraries that I’m using. To keep the server stable, I’ll need to install all of these in my user space. Angus suggested that I look at virtualenvwrapper. Last night I installed it on Morphia and read these useful introduction pages:
- introduction by the author, Doug Hellmann. (originally in Python Magazine Volume 2 Issue 5 , May, 2008)
- virtualenvwrapper documentation
- Presentation: pip and virtualenv by Rich Leland
Looks like it will do what I want. I’ll report again when I’ve tried to use it – hopefully soon!