How to setup Vagrant + Chef + Pycharm or... "how to become a Python happy coder"

Today I'm going to write about how you can setup Vagrant (for those of you that still don't know what Vagrant is, check this link) and Pycharm (to me the best Python IDE out there) to be able to debug remote applications with Pycharm that are running in Vagrant. Also, we will have a quick look at Chef, that will help you configure your Vagrant stuff way easier.

Sure, why not?

Vagrant and Chef

So unless you've been living in a cave for the last couple years, you probably already heard about Vagrant and Chef. If you have some experience working with virtual images, Vagrant will become second nature to you pretty quickly. Actually, Vagrant can run on top of Virtualbox or VMWare, so make your choice.

So you might be wondering why should you even bother about learning Vagrant when you're all covered using a typical VM solution like VirtualBox or VMWare. Well, I'd say, first: is always fun to learn new stuff :D, and second, Vagrant can do things you normally can't with a VirtualBox or VMWare, such as integrate with CM tools like Puppet or Chef, allowing you to customize the instance creation, deployment, etc.

That said, in my case I'll be using Virtualbox, so let's get on with installing Virtualbox and Vagrant:

sudo apt-get update
sudo apt-get install virtualbox
wget https://dl.bintray.com/mitchellh/vagrant/vagrant_1.7.2_x86_64.deb
sudo dpkg -i vagrant_1.7.2._x86_64.deb

As you can see, I'm installing the 64bits Debian based version of Vagrant, but you can choose several flavours in Vagrant site. That said, we have all that we need to start creating our first Vagrant instance.

Running Vagrant

Vagrant depends on a config file named Vagrantfile, and that's basically all you need to know about it (more on this later). So let's create our first Vagrant instance.

mkdir tmp
cd tmp
vagrant init precise64 http://files.vagrantup.com/precise64.box

What we've done here is creating a Vagrant instance using one of the available images from Vagrant, more precisely, the one selected is a Ubuntu Precise 64bits. This has also created a Vagrantfile in this directory, lets take a closer look:

# -*- mode: ruby -*-
# vi: set ft=ruby :

# All Vagrant configuration is done below. The "2" in Vagrant.configure
# configures the configuration version (we support older styles for
# backwards compatibility). Please don't change it unless you know what
# you're doing.
Vagrant.configure(2) do |config|
  # The most common configuration options are documented and commented below.
  # For a complete reference, please see the online documentation at
  # https://docs.vagrantup.com.

  # Every Vagrant development environment requires a box. You can search for
  # boxes at https://atlas.hashicorp.com/search.
  config.vm.box = "precise64"

  # The url from where the 'config.vm.box' box will be fetched if it
  # doesn't already exist on the user's system.
  config.vm.box_url = "http://files.vagrantup.com/precise64.box"

  # Create a forwarded port mapping which allows access to a specific port
  # within the machine from a port on the host machine. In the example below,
  # accessing "localhost:8080" will access port 80 on the guest machine.
  # config.vm.network "forwarded_port", guest: 80, host: 8080

  # Create a private network, which allows host-only access to the machine
  # using a specific IP.
  # config.vm.network "private_network", ip: ""

  # Create a public network, which generally matched to bridged network.
  # Bridged networks make the machine appear as another physical device on
  # your network.
  # config.vm.network "public_network"

  # Share an additional folder to the guest VM. The first argument is
  # the path on the host to the actual folder. The second argument is
  # the path on the guest to mount the folder. And the optional third
  # argument is a set of non-required options.
  # config.vm.synced_folder "../data", "/vagrant_data"

  # Provider-specific configuration so you can fine-tune various
  # backing providers for Vagrant. These expose provider-specific options.
  # Example for VirtualBox:
  # config.vm.provider "virtualbox" do |vb|
  #   # Display the VirtualBox GUI when booting the machine
  #   vb.gui = true
  #   # Customize the amount of memory on the VM:
  #   vb.memory = "1024"
  # end
  # View the documentation for the provider you are using for more
  # information on available options.

  # Define a Vagrant Push strategy for pushing to Atlas. Other push strategies
  # such as FTP and Heroku are also available. See the documentation at
  # https://docs.vagrantup.com/v2/push/atlas.html for more information.
  # config.push.define "atlas" do |push|
  # end

  # Enable provisioning with a shell script. Additional provisioners such as
  # Puppet, Chef, Ansible, Salt, and Docker are also available. Please see the
  # documentation for more information about their specific syntax and use.
  # config.vm.provision "shell", inline: <<-shell data-blogger-escaped--y="" data-blogger-escaped-apache2="" data-blogger-escaped-apt-get="" data-blogger-escaped-end="" data-blogger-escaped-install="" data-blogger-escaped-pre="" data-blogger-escaped-shell="" data-blogger-escaped-sudo="" data-blogger-escaped-update=""> 

As you can see this is written in Ruby, and is basically used to specify VM config options: resources, port forwarding, etc. Check Vagrant docs for a full explanation on the available options for this file.

You can simply start you vagrant instance with:

vagrant up

It will take some time the first time you run this, since it has to download the image and initialize everything, but future calls to vagrant up should be quicker. You can also stop the machine using halt, reload it, destroy it, etc. One of the coolest things in Vagrant is sshing the machine:

vagrant ssh

And you're in. Vagrant will automatically mount a /vagrant directory that is mapped to where your Vagrantfile is in your local machine. The second coolest thing in Vagrant is the automatic update to whatever you thrown in that folder. If you modify something from your local machine, you'll see it right away within Vagrant and vice-versa.

That said, its time to customize this Vagrant instance so it comes with several things packed, I'm thinking of a basic Python dev environment, but you can virtually install anything you want on this Vagrant instance. To do this, you have basically two options: Chef and Puppet. Word says that Chef is better, but I just chose it because it was the first one I've tried and it seems to be widely supported by default in Vagrant.

Installing Chef

So Chef is used to automatize the installation, configuration and data deployments within a Vagrant instance. Chef uses the concept of cookbooks and recipes to install several stuff: so you have a recipe for installing say Apache, another for installing Java JDK, and so on.

Ramsay approves...hopefully

In this case we're going to use  Chef-Solo, which is an open source client version of the Chef client that doesn't require a Chef server functional. This is more than enough for the purpose of this article. If you want a deeper introduction to a full fledged Chef installation, I recommend reading this wonderful set of articles from Jason Grimes.

The best place to find Chef resources and available cookbooks is here. Now in our case we're going to install a plain Python environment so we can start writing Python code right away.

But first things first, we need to install Chef, this step is not absolutely mandatory, but required if you plan to use knife tool:

sudo apt-get install chef

Now we need to tell Vagrant that we plan to use Chef-solo. Easy enough, Vagrant has a great integration with Chef-solo, so simply add these lines to your Vagrant file and you should be able to use Chef.

config.vm.provision :chef_solo do |chef|
        chef.add_recipe "python"

We are basically telling Vagrant to use Chef-solo and to add the recipe that installs python. What we need now is to have the cookbook for Python. I haven't been able to use knife in a way that automatically resolves cookbook dependencies, so we're left all alone to download each cookbook on our own and making sure we have all that we need. Let's check what the python cookbook dependencies are:

  • build-essential
  • yum
Thus, we will need the build-essential cookbook and the yum cookbook.  Yum cookbook has a dependency on yum-epel, so you'll need that one too. To make your life easier, I've zipped all required cookbooks in this file, so you'll only need to extract it where your Vagrantfile is.

So once we have all the required cookbooks in our cookbooks folder, we should be ready to try Vagrant.

vagrant reload
vagrant provision

We're all set, if we ssh to Vagrant, we can see that we have Python and pip ready to use, sweet! Time for coding!

A final note though...I'd recommend using something more convenient for this, such ash Berkshelf. This seeems like a much more decent and maintainable way to resolve cookbook dependencies. I've tried to use it with Chef-solo, but it looks that to work properly it need a full-fledged Chef, so no deal for me, but is worth a look.

Remote debugging with Pycharm

Pycharm is a Python IDE from Jetbrains. To put it simple, is the best Python IDE I've ever tried, and I've tried some of them. The bad thing is that is not free...well you can get a Community Edition that is free (pretty much like the Java counterpart IntellijIDEA), but to be honest, for the price of the paid one, is worth every penny. 

We've all been there...right?

That said, let's see how we can debug a Python application that is running within a Vagrant instance, in other words, remote debugging, since your Vagrant instance is a "remote" machine.

Setting up Vagrant

So let's suppose we have an existing project located where we created our Vagrant instance previously. Nothing complex, a simple hello world will be enough for this. So let's start!

1.- Open Pycharm, go to File -> Open and select the folder where you created the Vagrantfile (or create a new one if you haven't done so yet)

Open test project

Probably I've already mentioned, but placing your test project right where your Vagrantfile is, will automatically place the test project file within Vagrant in a folder called /vagrant. You can customize this to more complex situations, mapping any folder in your local system to be available in Vagrant, but this is enough for us now.

2.- Now we need to specify a remote Python interpreter for this project (basically, the one within Vagrant). So go to File-Settings and click on the create new interpreter icon and select 'Add Remote':


3.- In the next window, select Vagrant and the folder where you created the Vagrantfile, leave the rest as default and click Ok.

Remote interpreter config

4.- Make sure the Remote Python from Vagrant is selected in the interpreted drop down list.

Drop-down list

5.- Make sure Vagrant is up, go to Tools -> Vagrant -> Up....isn't this awesome integration?

Vagrant tools

6.- We need to create a Run/Debug configuration for our test project, so go to Run - Edit configurations, name it, select the main script and make sure the remote interpreter is selected:

Debug configuration

7.- Now we can set up a breakpoint in the main script and see if we can debug remotely.

Adding breakpoint
8.- Then hit on the Debug button and we should be debugging our first Vagrant resident Python app....yeeha!


Final toughts

So all this may sound like a lot of hassle when you are working on your own, you can skip all this by simply using virtualenv. This is correct and this is overkill if you plan to work on simple or personal projects, but as soon as you need to work with a small-medium team, having some vagrant configuration that only requires you to have vagrant installled to get you up and writing code in seconds is priceless. 

Other than that, there are alternatives to this approach out there, like Docker. While I haven't tested it yet myself, it looks like debate is not Vagrant vs Docker, but they seem to be able to co-exist pretty well. In any case, we live in a wonderful age where developers have a lot of options when it comes to avoiding all the headaches that setting up specific dev environments can lead to. So choose your weapons!

No comments:

Post a Comment