Today in this post we are going to talk about one of the most important thing for Python developers that is very much overlooked and that is the use of what are called Python Virtual Environments. This is a tool that I didn’t really learn about and find any value in until I had been working as an engineer for about 2 years and had been doing a fair amount of software development. It wasn’t until one of the PyTexas conferences I had attended that I realized the immense value in using Python Environments. If you have heard about them but never really found use for them, hopefully there is something of value that you’ll find in this post. If this is something completely new or foreign to you, at the end we will give a brief tutorial on how to use them.
Now before we can properly get started, everything that I’ll be showing off will be by using the Anaconda distribution of Python which has it’s own syntax for creating python environments. If you are not familiar with Anaconda, check out a post about why we prefer to use Anaconda over other distributions. Nothing that will be shared in this post will be specific to using Anaconda over any other distribution, however some of the syntax that will be demonstrated will be specific to the Anaconda ecosystem; we’ll try and point out the differences where possible. Without further ado, lets start talking abut what Python environments are and why you should really be using them.
If you have ever done web development or developed software/applications designed to run on multiple platforms, you know how important it is to have multiple test environments. For example, when developing web apps, you should be testing your app on multiple browsers (IE: Chrome behaves differently than say Internet Exploder Explorer), multiple operating systems (this should be given), as well as multiple devices to cover all possible screen sizes as well as interfaces that a user will be using to interact with your application. Well, in Python, there is something very similar but specifically for development called environments. For the majority of the Python world, most people will be using either Virtualenv or Conda to create and manage their Python environment.
The main purpose of having Python virtual environments is to be able to manage project dependencies. With using a Python virtual environment, you can create an isolated environment in which you can install the required Python packages for your specific project without impacting other Python virtual environments or the root Python path. The reason you want this are two fold, one it makes creating documentation for your project pretty simple and straightforward. When you are listing your dependencies, all you have to do is just list the packages in the environment associated with the project. The second is it keeps your Python system as a whole clean. This last one may be me just being a little neurotic about keeping everything super organized, but it makes it easier to sleep at night knowing exactly what I have installed on my system.
The next big reason working with Python virtual environments is that if you want to quickly spin up the same Python development environment you are working with, somewhere else without going through the tedium of starting from scratch. Let’s take the theoretical example that I’m as a developer you have NEVER encountered, and that is of when you are collaborating with someone, on a project and you want to ensure you are using the same tools. With GitHub and other similar services, it makes it really easy to share the source code that you are developing via all the features we have become accustomed to in our version control system of choice. But, how do you make sure that you both are using the same packages let alone using the same versions of said packages? Enter virtual environments. Here you are able to bundle up your virtual environment and send it to a friend, colleague, or coworker to ensure that you are both using the same setup. Besides being incredibly practical in ensuring consistency; virtual environments are incredibly handy when debugging or validating something under specific circumstances. We will talk more about how share your virtual environments in the tutorial section below.
Now the next really good reason to be using virtual environments is to preserve known good working environments. Now fortunately for me I have never really run into the problem of upgrading a package and then it being incompatible with much of what I’m working on. That said, I know it is a real thing because it has happened to me in different scenarios when it comes to C++ and stuff during the early days of ROS. Actually, I take that back. Switching some stuff from Python 2.7 to 3.4 was a little rough but it was to be expected as it was a big change between the two. Anyways, little is as frustrating as having everything breaking down simply because you ran an update/upgrade. Isn’t that the whole point of staying up to date; having stuff fixed, not broken? In an ideal world that’s how it should go. However, if there is a piece of code written by a human, it has bugs in it somewhere and finding those bugs can either be done by users or by developers. Sadly a lot of stuff gets out into the wild without having been tested all that much. By having a dedicated Python virtual environment, you can make it possible to have the best of both worlds. In one Python virtual environment, you can have it set up exactly how you need it to, in order to run some weird piece of code that requires certain versions of python packages to run correctly while at the same time have a python virtual environment that allows you to have the latest and greatest of everything.
All in all, Python virtual environments are really something that you should be using if you haven’t already adopted it as part of your work flow in doing development. Pretty much whenever I start a new project that requires Python development; I create a dedicated virtual environment just for that project.
Now for those who are completely unfamiliar with how to use virtual environments, on to the tutorial section. Now before we do so you’ll either want to install Anaconda Python (as alluded to above) or you’ll need to perform a pip install of virtualenv. For instructions on how to do either of these check out the following: