Moving to brenda.moon.net.au
Today I’m moving my website from www.moon.net.au to brenda.moon.net.au. I think I’ve got it all working, but please tell me if you notice any problems.
Category: Computers
Pandas timeseries plot – setting x-axis major and minor ticks and labels
I’ve asked this question on StackOverflow (http://stackoverflow.com/questions/12945971/pandas-timeseries-plot-setting-x-axis-major-and-minor-ticks-and-labels), but couldn’t include images because I haven’t posted on stackOverflow before. So here it is, with the images.
I want to be able to set the major and minor xticks and their labels for a time series graph plotted from a Pandas time series object.
The Pandas 0.9 “what’s new” page says: “you can either use to_pydatetime or register a converter for the Timestamp type” but I can’t work out how to do that so that I can use the matplotlib ax.xaxis.set_major_locator ax.xaxis.set_major_formatter (and minor) commands.
If I use them without converting the pandas times, the x-axis ticks and labels end up wrong.
By using the ‘xticks’ parameter I can pass the major ticks to pandas.plot, and then set the major tick labels. I can’t work out how to do the minor ticks using this approach. (I can set the labels on the default minor ticks set by pandas.plot)
Here is my test code:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
import pandas print 'pandas.__version__ is ', pandas.__version__ print 'matplotlib.__version__ is ', matplotlib.__version__ dStart = datetime.datetime(2011,5,1) # 1 May dEnd = datetime.datetime(2011,7,1) # 1 July dateIndex = pandas.date_range(start=dStart, end=dEnd, freq='D') print "1 May to 1 July 2011", dateIndex testSeries = pandas.Series(data=np.random.randn(len(dateIndex)), index=dateIndex) ax = plt.figure(figsize=(7,4), dpi=300).add_subplot(111) testSeries.plot(ax=ax, style='v-', label='first line') # using MatPlotLib date time locators and formatters doesn't work with new pandas datetime index ax.xaxis.set_minor_locator(matplotlib.dates.WeekdayLocator(byweekday=(1),interval=1)) ax.xaxis.set_minor_formatter(matplotlib.dates.DateFormatter('%d\n%a')) ax.xaxis.grid(True, which="minor") ax.xaxis.grid(False, which="major") ax.xaxis.set_major_formatter(matplotlib.dates.DateFormatter('\n\n\n%b%Y')) plt.show() # set the major xticks and labels through pandas ax2 = plt.figure(figsize=(7,4), dpi=300).add_subplot(111) xticks = pandas.date_range(start=dStart, end=dEnd, freq='W-Tue') print "xticks: ", xticks testSeries.plot(ax=ax2, style='-v', label='second line', xticks=xticks.to_pydatetime()) ax2.set_xticklabels([x.strftime('%a\n%d\n%h\n%Y') for x in xticks]); # set the text of the first few minor ticks created by pandas.plot # ax2.set_xticklabels(['a','b','c','d','e'], minor=True) # remove the minor xtick labels set by pandas.plot ax2.set_xticklabels([], minor=True) # turn the minor ticks created by pandas.plot off # plt.minorticks_off() plt.show() print testSeries['6/1/2011':'6/7/2011'] |
and it’s output:
|
1 2 3 4 5 |
pandas.__version__ is 0.9.1.dev-3de54ae matplotlib.__version__ is 1.1.1 1 May to 1 July 2011 <class 'pandas.tseries.index.DatetimeIndex'> [2011-05-01 00:00:00, ..., 2011-07-01 00:00:00] Length: 62, Freq: D, Timezone: None |
|
1 2 3 |
xticks: <class 'pandas.tseries.index.DatetimeIndex'> [2011-05-03 00:00:00, ..., 2011-06-28 00:00:00] Length: 9, Freq: W-TUE, Timezone: None |
|
1 2 3 4 5 |
2011-06-04 -0.199393 2011-06-05 -0.043118 2011-06-06 0.477771 2011-06-07 -0.033207 Freq: D |
Learning how to install python packages
In an earlier post I described my python setup for Debian, this time I’ve kept notes while working yesterday to get a stable python environment on Mac OS X Mountain Lion (10.8.1). I’ve used the notes to write up the steps that worked, there were a lot of dead ends along the way that I haven’t included.
Since the upgrade to Mountain Lion, I’ve had a lot of problems with my python environment not working, and in trying to sort it out I’d somehow managed to end up with a broken easy_install. I think I achieved that by downloading and installing the latest command line tools from the Apple Developer website instead of through XCode. So I’d recommend sticking with the one installed through XCode. But at that stage I wasn’t taking notes about what I was trying, so I’m not sure.
I fixed easy_install by installing distribute:
|
1 2 |
curl -O http://python-distribute.org/distribute_setup.py sudo python distribute_setup.py |
I noticed that my path had /usr/bin ahead of /usr/local/bin, but this should probably be the other way around.
|
1 |
/usr/local/git/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin:/usr/texbin: |
This is being set by /etc/paths  and the ~/.bashrc
Changed /etc/paths to have:
|
1 |
/usr/local/git/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/X11/bin:/usr/texbin |
Then started setting up my ipython, scipy and pandas environment. Using “distribute” in the virtual environment seems to work better at building some packages.
|
1 2 3 4 5 |
mkvirtualenv --distribute brmTest2 pip install tornado pip install ipython pip install numpy pip install scipy |
The scipy install failed, and I fixed it by installing the dev version as suggested here – http://www.thisisthegreenroom.com/2012/compiling-scipy-on-mountain-lion/
|
1 2 3 |
pip install -e git+https://github.com/scipy/scipy#egg=scipy-dev pip install pandas pip install matplotlib |
The matplotlib install failed, so I then followed these instructions -
https://gist.github.com/1860902Â (but not step 5) to install X11 and pkg-config as follows:
download and install X11 from
|
1 2 |
http://xquartz.macosforge.org/trac/wiki sudo xcode-select -switch /Applications/Xcode.app/Contents/Developer |
logged out and in to activate X11.
|
1 |
brew install pkg-config |
(not sure this actually helped)
The x11 libraries are not on the path (which I thought was what pkg-config was meant to fix?) Fixed by using:
|
1 |
export CFLAGS="-I/usr/X11/include -I/usr/X11/include/freetype2 -I/usr/X11/include/libpng12" |
then
|
1 2 |
pip install matplotlib pip install pyzmq |
So I now have a virtual environment with the right Python packages installed, and I understand a bit more about working out how to install packages when they fail. I still need to work out what the difference is between pip, distribute and easy_install.