HOWTO: Get a Mac, Python, Homebrew, PyEnv, and Bash aliases to play nicely

Okay, I’ll be blunt: I’m a Linux guy. I know, shocker.

I’ve recently moved to an awesome new job and part of that role will be an area of developer advocacy which will require me to go to meetups and tech conferences from time to time, and talk about how freakin' awesome my new employer is. And they are, srsly. You should sign up and use it if you’re thinking about building a news/activity feed/timeline in your app.

I’m not a fan of using a Mac. Hate is a pretty strong word, and my emotions towards Mac machines doesn’t run quite that deep, but it’s close. But in the world of awesome tools like Keynote, and how everyone else at these conferences use a Mac for connectivity, I figure I’ll play along.

My biggest complaint isn’t even necessarily about the Mac, it could just be my ignorance of how to combine some of these tools. But Python version management is not as easy as Homebrew/PyEnv would have you think. So I set out today to get my Mac to play nicely with how I want to use Python on my rig.

UPDATE: I was asked at a recent meetup why I don’t just use Docker. The short answer is that I do a lot of cross-version testing of SDKs and sample code given to clients at work, and need to make sure that we are aware of which versions of Python we support.

In the following instructions, I reference your home folder on your Mac as /Users/yourusername/ which of course is likely not your actual home folder, so go ahead and substitute yourusername for your actual username on your system.

Step One: Homebrew and Dev tools

If you’re a developer on a Mac, you’ve likely already installed Homebrew on your system. But in the off chance that you’re new and haven’t gotten that far, go install it. Then, do these commands:

$ brew install python  # installs latest 2.7.x
$ brew install python3 # installs latest 3.x
$ brew install pyenv   # installs pyenv, duh

You’ll probably need to [/install-xcode](install XCode) to get a gcc compiler. Sorry (not sorry) in advance for the many gigabytes you’re about to download just to get gcc. Don’t you wish you were on Linux yet?

Step Two: PyEnv versions of Python

You’d think something as simple as pyenv install 2.7.12 would be pretty awesome, right? It is, but day to day use of pyenv makes my head hurt. But go ahead and install some other version of Python, like 2.7.11:

$ pyenv install 2.7.11

Step Three: virtualenvwrapper

Virtua-whatchamacallit? It’s a great tool for managing virtual shell environments for Python. Until tonight, I was a big, big believer in virtualenv-burrito but it’s got some serious problems with making virtual environments for Python 3 when Python 2 is your base OS-level Python interpreter. To get the non-burrito utility installed, do this:

$ pip install virtualenvwrapper

There will be some setup afterward. If you have a .bashrc or .bash_profile file in your home directory, add the following lines at the end:

export WORKON_HOME=$HOME/.virtualenvs
export PROJECT_HOME=$HOME/src
source /usr/local/bin/virtualenvwrapper.sh

Assumptions I’m making on your behalf here:

1. You want all of your virtual environments to be hidden away nicely under /Users/yourusername/.virtualenvs/
2. You keep all of your source code on your Mac under /Users/yourusername/src/

Feel free to adjust those paths as necessary.

Step Four: Reset your environment

This part will need more explanation. At this point, being the Linux fanboy that I am, I created an /opt/ folder on my system and symlinked all of my homebrew-installed Python versions under it:

$ sudo mkdir -p /opt/python
# you'll probably be prompted for your laptop password here
# so I hope you didn't just copy and paste this whole block
# of code text 🙂

# this step will give your user permission to write things into this path without needing sudo any more
$ sudo chown -R yourusername /opt

# this will be useful later in the post:
$ ln -s /usr/local/Cellar/python/2.7.12/ /opt/python/2
$ ln -s /usr/local/Cellar/python/2.7.12/ /opt/python/2.7.12
$ ln -s /usr/local/Cellar/python/3.5.2/ /opt/python/3
$ ln -s /usr/local/Cellar/python/3.5.2/ /opt/python/3.5.2

Those last four steps make a symbolic link (aka ‘symlink') from the Homebrew path for Python 2 and 3 into the /opt/python path we made. Effectively, we’re making fake paths to /opt/python/2/ and /opt/python/2.7.12/ and so on. As of the time of this writing, Python 2.7.12 and 3.5.2 were the latest installs via Homebrew.

But what about the version we installed (2.7.11) above in the section about pyenv? Pyenv installs its versions of Python under a completely different path, so you’ll want to do something like the following:

$ ln -s /Users/yourusername/.pyenv/versions/2.7.11/ /opt/python/2.7.11

This will make additional symlinks under /opt/python for all of pyenv’s versions of Python as well.

Step Five: Magic

Okay, this is where the fun begins. With virtualenvwrapper, you can easily set up virtual environments for different projects, but what if you absolutely must use a specific version of Python? Well, thanks to the previous steps, all we need is a little help from the Bash shell that’s already on your Mac. (If you’re a zsh user, I’ll go ahead and presume you’re bright enough to figure out the zsh equivalent of the following work)

At the bottom of your .bashrc or .bash_profile file, add the additional lines:

if [ -f ~/.bash_aliases.sh ]; then
. ~/.bash_aliases.sh
fi

Next, we’re going to create a new file in your home folder called .bash_aliases.sh and add the following code:

# make 'mkvirtualenv' aliases for every version of python we have under /opt/python
for py_version in `ls -1 /opt/python`; do
  MAJORVERSION=`echo $py_version | cut -c1`
  alias mkvenv$py_version="mkvirtualenv -p /opt/python/$py_version/bin/python$MAJORVERSION"
done

Now at the prompt, you can run this:

$ source ~/.bashrc #substitute .bash_profile if you have that instead

Now when you type ‘mkv' and hit TAB twice for tab-completion, you should see the following:

$ mkv
mkvenv2 mkvenv2.7.11 mkvenv2.7.12 mkvenv3 mkvenv3.5.2 mkvirtualenv

With this alone, you can run mkvenv2 to make a virtual environment using the ‘default' for Python 2 (which we symlinked to /opt/python/2/ from the installation path for Python 2.7.12), or if you want Python 3 you can run mkvenv3 instead. Actual usage would look something like:

$ mkvenv3 py3

… to create a virtual environment based on Python 3.5.2. If you wanted to make a virtual environment specific to Python 2.7.11 which we installed using pyenv:

$ mkvenv2.7.11 myvenvname

These virtual environments will be installed under your home folder within /Users/yourusername/.virtualenvs which is the WORKON_PATH set set in your environment earlier. The joy in this is you won’t have to worry any longer about forgetting to add your virtual environment to an exclusion list for git or other VCS software you use for your project and accidentally check in several megabytes of packages.

With virtualenvwrapper, you can simply type workon at a prompt to get a list of all of your Python virtual environments, and then activate them with:

$ workon myenvname

Then you can use pip etc to install your modules/packages and those packages will only get installed within your virtual environment.

Part Six: How I actually use this at work:

I have a number of virtualenvs set up for our tooling, so I do something like this:

$ for i in `workon | grep streamtest`; do
> workon $i
> # run my tests
> done; deactivate

This loops through all of my testing virtualenvs, runs my testing for each, and since workon would activate the last virtualenv in the list, I manually deactivate the virtualenv when finished.