• Home

  • Custom Ecommerce
  • Application Development
  • Database Consulting
  • Cloud Hosting
  • Systems Integration
  • Legacy Business Systems
  • Security & Compliance
  • GIS

  • Expertise

  • About Us
  • Our Team
  • Clients
  • Blog
  • Careers

  • CasePointer

  • VisionPort

  • Contact
  • Our Blog

    Ongoing observations by End Point Dev people

    Installing Python in local directory

    Szymon Lipiński

    By Szymon Lipiński
    February 20, 2013

    On one of our client’s Ubuntu 10.04 machines, I needed to upgrade Python from 2.6 to 2.7. Unfortunately, after installing Python 2.7 from apt the virtualenv, version 1.4.5, did not work correctly. This bug was fixed in a newer virtualenv version, however there were no Ubuntu packages available.

    I thought about trying something else: why not install all the software locally in my home directory on the server? When virtualenv is used to create a new environment, it copies the Python executable in to the virtualenv directory.

    First I install pythonbrew, which is great software for installing many different Python versions in a local directory.

    $ curl -kL http://xrl.us/pythonbrewinstall | bash

    Then I activate pythonbrew with:

    $ source "$HOME/.pythonbrew/etc/bashrc"

    And install the Python version I want:

    $ pythonbrew install 2.7.3

    The installation took a couple of minutes. The script downloaded the tarball with the Python source code for the required version, compiled it and installed. It was writing all the information into a log file, which I was looking at by running the command below in another console:

    $ tail -f $HOME/.pythonbrew/log/build.log

    You can also add the following lines to ~/.bashrc to activate this python after starting new bash session.

    [[ -s "$HOME/.pythonbrew/etc/bashrc" ]] && source $HOME/.pythonbrew/etc/bashrc
    pythonbrew switch 2.7.3

    I run the below command to activate the pythonbrew script:

    $ source $HOME/.pythonbrew/etc/bashrc

    The python version changed:

    $ python --version
    Python 2.6.5
    $ source $HOME/.pythonbrew/etc/bashrc
    $ python --version
    Python 2.7.2

    As you can see, Python from my local installation is used:

    $ which python

    The only thing left is to create the virtual environment for the new Python version. I use virtualenvwrapper for managing virtualenv, so the obvious way to create a new environment is:

    $ mkvirtualenv --no-site-packages envname

    Unfortunately, it creates an environment with the wrong Python version:

    $ which python
    $ python --version
    Python 2.6.5

    So let’s try to tell virtualenvwrapper which Python file should be used:

    $ deactivate
    $ rmvirtualenv envname
    Removing envname...
    $ mkvirtualenv --no-site-packages -p /home/szymon/.pythonbrew/pythons/Python-2.7.3/bin/python envname

    Unfortunately this ended with an error:

    Running virtualenv with interpreter /home/szymon/.pythonbrew/pythons/Python-2.7.3/bin/python
    New python executable in envname/bin/python
    Traceback (most recent call last):
      File "/home/szymon/.virtualenvs/envname/lib/python2.7/site.py", line 67, in <module>
        import os
      File "/home/szymon/.virtualenvs/envname/lib/python2.7/os.py", line 49, in <module>
        import posixpath as path
      File "/home/szymon/.virtualenvs/envname/lib/python2.7/posixpath.py", line 17, in <module>
        import warnings
    ImportError: No module named warnings
    ERROR: The executable envname/bin/python is not functioning
    ERROR: It thinks sys.prefix is '/home/szymon/.virtualenvs' (should be '/home/szymon/.virtualenvs/envname')
    ERROR: virtualenv is not compatible with this system or executable

    The problem is that the virtualenv version, used by virtualenvwrapper, doesn’t work with Python 2.7. As I wrote at the begining, there is no newer version available via apt. The solution is pretty simple. Let’s just install the newer virtualenvwrapper and virtualenv version using pip.

    $ pip install virtualenv
    Requirement already satisfied: virtualenv in /usr/lib/pymodules/python2.6
    Installing collected packages: virtualenv
    Successfully installed virtualenv

    As you can see, there is a problem. The problem is that there is used pip from system installation. There is no pip installed in my local Python 2.7 version. However there is easy_install:

    $ which pip
    $ which easy_install

    So let’s use it for installing virtualenv and virtualenvwrapper:

    $ easy_install virtualenv virtualenvwrapper

    I’ve checked the whole installation procedure once again, it turned out that there was some network error while downloading pip, but unfortunately I didn’t notice the error. If everything is OK, then pip should be installed, and you should be able to install virtualenv using pip as well with:

    $ pip install virtualenv virtualenvwrapper

    Cool, let’s check which version is installed:

    $ which virtualenv
    $ virtualenv --version

    Before creating the brand new virtual environment, I have to activate the new virtualenvwrapper. I have the following line in my ~/.bashrc file:

    source /usr/local/bin/virtualenvwrapper.sh

    I just have to change it to the below line and login once again:

    source /home/szymon/.pythonbrew/pythons/Python-2.7.3/bin/virtualenvwrapper.sh

    Let’s now create the virtual environment using the brand new Python version:

    $ mkvirtualenv --no-site-packages -p $HOME/.pythonbrew/pythons/Python-2.7.3/bin/python envname

    I want to use this environment each time I log into this server, so I’ve added this line to my ~/.bashrc:

    workon envname

    Let’s check if it works. I’ve logged out and logged in to my account on this server before running the following commands:

    $ which python
    $ python --version
    Python 2.7.3

    Looks like everything is OK.