Different branches, different IPython notebooks

Lately, I've been working on several different features of the the flotilla code at once. It was getting frustrating because I'd have quite a few IPython notebooks open, testing the different features, but these features take some computation time and I would want to work on another feature while one was computing. But, this became a problem beause I install the package using "editable" or "develop" mode using pip install -e ., which, instead of copying the code from the source directory to Python's site-packages directory, makes a soft link from site-packages. So any changes you make in your source files, they get automatically updated in the "installed" package! Coupled with IPython's autoreload, this becomes an incredibly powerful development tool, as you can test out new functions in IPython without reinstalling packages or restarting your kernel! HUGE time savings! (Thanks to Jeff Tratner who I met last year at PyData for showing me this)

Anyways, the "editable" mode and autoreload sometimes collide, and you get this kind of error:

TypeError: super(type, obj): obj must be an instance or subtype of type

This is a wonderfully cryptic error. It happens because, as far as I understand autoreload, IPython, and Python's internals (and the developers that know better please correct me if I'm wrong), Python hashes all objects when importing a library. We use a lot of object orientation in flotilla, so when we change some code in the parent class, then the child is no longer an instance of the old hash of the parent class. I kept getting these errors when switching between branches and notebooks and got so tired of it that I finally for help on Twitter.

Paul Ivanov, a matplotlib and IPython developer, responded!

After some clarification, I started using it. Here's what I did. First, let's navigate to my flotilla directory.

In [1]:
cd ~/workspace-git/flotilla
/Users/olga/workspace-git/flotilla

I keep all my IPython notebooks in Dropbox just in case my computer dies, then my research can still live. Let's clone the branch filter_splicing_on_expression to the flotilla sub-folder of my Dropbox, i.e. to ~/Dropbox/ipython/flotilla/filter_splicing_on_expression.

The idea behind this next command is,

git clone --shared -b <branch name> <original repo location> <new repo location>

Since I'm already in the folder where my original repo is located, I can indicate that with a period, "."

In [3]:
! git clone --shared -b filter_splicing_on_expression . ~/Dropbox/ipython/flotilla/filter_splicing_on_expression
Cloning into '/Users/olga/Dropbox/ipython/flotilla/filter_splicing_on_expression'...
remote: Counting objects: 8010, done.
remote: Compressing objects: 100% (2889/2889), done.
remote: Total 8010 (delta 5757), reused 7107 (delta 5013)
Receiving objects: 100% (8010/8010), 85.46 MiB | 34.05 MiB/s, done.
Resolving deltas: 100% (5757/5757), done.
Checking connectivity... done.

Now we've cloned only the filter_splicing_on_expression branch

In [6]:
cd /Users/olga/workspace-git/flotilla
/Users/olga/workspace-git/flotilla

In [7]:
ls
CONTRIBUTING.md   LICENSE           README.md         ROADMAP.md        doc/              examples/         flotilla.png      setup.cfg         talks.md
INSTALL.md        Makefile          README.rst        TODO.md           docker/           flotilla/         requirements.txt  setup.py          testing/

In [11]:
! git status
On branch master
Your branch is up-to-date with 'origin/master'.

nothing to commit, working directory clean

Now let's check out the "shared branch" I made in ~/Dropbox/ipython/flotilla/filter_splicing_on_expression/.

In [12]:
cd /Users/olga/Dropbox/ipython/flotilla/filter_splicing_on_expression/
/Users/olga/Dropbox/ipython/flotilla/filter_splicing_on_expression

In [13]:
ls
CONTRIBUTING.md   LICENSE           README.md         ROADMAP.md        doc/              examples/         flotilla.png      setup.cfg         talks.md
INSTALL.md        Makefile          README.rst        TODO.md           docker/           flotilla/         requirements.txt  setup.py          testing/

In [14]:
! git status
On branch filter_splicing_on_expression
Your branch is up-to-date with 'origin/filter_splicing_on_expression'.

nothing to commit, working directory clean

This folder has just the branch I want! And, as Paul pointed out, since the package I want, flotilla, exists in the local directory of the notebook, the local one gets imported first!

I run most of my IPython notebooks from ~/Dropbox/ipython, so I already had a notebook instance running, and I could see all my flotilla directories in the notebook.

And now I can work on individual feature branches without worrying that they're colliding! Thanks Paul! I was amazed by this when I found out and wanted to share. I hope this helped you!

Go Top
comments powered by Disqus