August 27, 2016
Okay, I’ll be blunt: I’m a Linux guy. I know, shocker.
I’ve recently moved to an awesome new job and part of that role will be an area of developer advocacy which will require me to go to meetups and tech conferences from time to time, and talk about how freakin’ awesome my new employer is. And they are, srsly. You should sign up and use it if you’re thinking about building a news/activity feed/timeline in your app.
I’m not a fan of using a Mac. Hate is a pretty strong word, and my emotions towards Mac machines doesn’t run quite that deep, but it’s close. But in the world of awesome tools like Keynote, and how everyone else at these conferences use a Mac for connectivity, I figure I’ll play along.
My biggest complaint isn’t even necessarily about the Mac, it could just be my ignorance of how to combine some of these tools. But Python version management is not as easy as Homebrew/PyEnv would have you think. So I set out today to get my Mac to play nicely with how I want to use Python on my rig.
UPDATE: I was asked at a recent meetup why I don’t just use Docker. The short answer is that I do a lot of cross-version testing of SDKs and sample code given to clients at work, and need to make sure that we are aware of which versions of Python we support.
In the following instructions, I reference your home folder on your Mac as
/Users/yourusername/ which of course is likely
not your actual home folder, so go ahead and substitute
yourusername for your actual username on your system.
Step One: Homebrew and Dev tools
If you’re a developer on a Mac, you’ve likely already installed Homebrew on your system. But in the off chance that you’re new and haven’t gotten that far, go install it. Then, do these commands:
$ brew install python # installs latest 2.7.x $ brew install python3 # installs latest 3.x $ brew install pyenv # installs pyenv, duh
You’ll probably need to install XCode to get a gcc compiler. Sorry (not sorry) in advance for the many gigabytes you’re about to download just to get gcc. Don’t you wish you were on Linux yet?
Step Two: PyEnv versions of Python
You’d think something as simple as
pyenv install 2.7.12 would be pretty awesome, right? It is, but day to day use of
pyenv makes my head hurt. But go ahead and install some other version of Python, like 2.7.11:
$ pyenv install 2.7.11
Step Three: virtualenvwrapper
Virtua-whatchamacallit? It’s a great tool for managing virtual shell environments for Python. Until tonight, I was a big, big believer in virtualenv-burrito but it’s got some serious problems with making virtual environments for Python 3 when Python 2 is your base OS-level Python interpreter. To get the non-burrito utility installed, do this:
$ pip install virtualenvwrapper
There will be some setup afterward. If you have a
.bash_profile file in your home directory, add the following lines at the end:
export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/src source /usr/local/bin/virtualenvwrapper.sh
Assumptions I’m making on your behalf here:
1. You want all of your virtual environments to be hidden away nicely under
2. You keep all of your source code on your Mac under
Feel free to adjust those paths as necessary.
Step Four: Reset your environment
This part will need more explanation. At this point, being the Linux fanboy that I am, I created an
/opt/ folder on my
system and symlinked all of my homebrew-installed Python versions under it:
$ sudo mkdir -p /opt/python # you'll probably be prompted for your laptop password here # so I hope you didn't just copy and paste this whole block # of code text 🙂 # this step will give your user permission to write things into this path without needing sudo any more $ sudo chown -R yourusername /opt # this will be useful later in the post: $ ln -s /usr/local/Cellar/python/2.7.12/ /opt/python/2 $ ln -s /usr/local/Cellar/python/2.7.12/ /opt/python/2.7.12 $ ln -s /usr/local/Cellar/python/3.5.2/ /opt/python/3 $ ln -s /usr/local/Cellar/python/3.5.2/ /opt/python/3.5.2
Those last four steps make a symbolic link (aka ‘symlink’) from the Homebrew path for Python 2 and 3 into the
path we made. Effectively, we’re making fake paths to
/opt/python/2.7.12/ and so on. As of the time
of this writing, Python 2.7.12 and 3.5.2 were the latest installs via Homebrew.
But what about the version we installed (2.7.11) above in the section about pyenv? Pyenv installs its versions of Python under a completely different path, so you’ll want to do something like the following:
$ ln -s /Users/yourusername/.pyenv/versions/2.7.11/ /opt/python/2.7.11
This will make additional symlinks under
/opt/python for all of pyenv’s versions of Python as well.
Step Five: Magic
Okay, this is where the fun begins. With virtualenvwrapper, you can easily set up virtual environments for different projects, but what if you absolutely must use a specific version of Python? Well, thanks to the previous steps, all we need is a little help from the Bash shell that’s already on your Mac. (If you’re a zsh user, I’ll go ahead and presume you’re bright enough to figure out the zsh equivalent of the following work)
At the bottom of your
.bash_profile file, add the additional lines:
if [ -f ~/.bash_aliases.sh ]; then . ~/.bash_aliases.sh fi
Next, we’re going to create a new file in your home folder called
.bash_aliases.sh and add the following code:
# make 'mkvirtualenv' aliases for every version of python we have under /opt/python for py_version in `ls -1 /opt/python`; do MAJORVERSION=`echo $py_version | cut -c1` alias mkvenv$py_version="mkvirtualenv -p /opt/python/$py_version/bin/python$MAJORVERSION" done
Now at the prompt, you can run this:
$ source ~/.bashrc #substitute .bash_profile if you have that instead
Now when you type ‘mkv’ and hit TAB twice for tab-completion, you should see the following:
$ mkv mkvenv2 mkvenv2.7.11 mkvenv2.7.12 mkvenv3 mkvenv3.5.2 mkvirtualenv
With this alone, you can run
mkvenv2 to make a virtual environment using the ‘default’ for Python 2 (which we
/opt/python/2/ from the installation path for Python 2.7.12), or if you want Python 3 you can run
mkvenv3 instead. Actual usage would look something like:
$ mkvenv3 py3
… to create a virtual environment based on Python 3.5.2. If you wanted to make a virtual environment specific to Python 2.7.11 which we installed using pyenv:
$ mkvenv2.7.11 myvenvname
These virtual environments will be installed under your home folder within
/Users/yourusername/.virtualenvs which is
the WORKON_PATH set set in your environment earlier. The joy in this is you won’t have to worry any longer about
forgetting to add your virtual environment to an exclusion list for git or other VCS software you use for your project
and accidentally check in several megabytes of packages.
With virtualenvwrapper, you can simply type
workon at a prompt to get a list of all of your Python virtual
environments, and then activate them with:
$ workon myenvname
Then you can use pip etc to install your modules/packages and those packages will only get installed within your virtual environment.
Part Six: How I actually use this at work:
I have a number of virtualenvs set up for our tooling, so I do something like this:
$ for i in `workon | grep streamtest`; do > workon $i > # run my tests > done; deactivate
This loops through all of my testing virtualenvs, runs my testing for each, and since workon would activate the last
virtualenv in the list, I manually
deactivate the virtualenv when finished.