Data Loading...
python-guide (1) Flipbook PDF
python-guide (1)
119 Views
62 Downloads
FLIP PDF 4.66MB
Python Guide Documentation Release 0.0.1
Kenneth Reitz
Dec 21, 2018
Contents
1
. . . . . . . . . .
3 3 6 7 10 11 13 15 17 18 21
2
Python Development Environments 2.1 Your Development Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Further Configuration of pip and Virtualenv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
25 26 32
3
Writing Great Python Code 3.1 Structuring Your Project 3.2 Code Style . . . . . . . 3.3 Reading Great Code . . 3.4 Documentation . . . . . 3.5 Testing Your Code . . . 3.6 Logging . . . . . . . . . 3.7 Common Gotchas . . . 3.8 Choosing a License . . .
35 35 47 59 60 64 69 72 77
4
Getting Started with Python 1.1 Picking a Python Interpreter (3 vs 2) . 1.2 Properly Installing Python . . . . . . 1.3 Installing Python 3 on Mac OS X . . 1.4 Installing Python 3 on Windows . . . 1.5 Installing Python 3 on Linux . . . . . 1.6 Installing Python 2 on Mac OS X . . 1.7 Installing Python 2 on Windows . . . 1.8 Installing Python 2 on Linux . . . . . 1.9 Pipenv & Virtual Environments . . . 1.10 Lower level: virtualenv . . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
Scenario Guide for Python Applications 4.1 Network Applications . . . . . . . 4.2 Web Applications & Frameworks . 4.3 HTML Scraping . . . . . . . . . . 4.4 Command-line Applications . . . . 4.5 GUI Applications . . . . . . . . . . 4.6
If you have OS X 10.12 (Sierra) or older use this line instead export PATH=/usr/local/bin:/usr/local/sbin:$PATH
Now, we can install Python 3: $ brew install python
This will take a minute or two.
1.3.2 Pip Homebrew installs pip pointing to the Homebrew’d Python 3 for you.
1.3.3 Working with Python 3 At this point, you have the system Python 2.7 available, potentially the Homebrew version of Python 2 installed, and the Homebrew version of Python 3 as well.
8
Chapter 1. Getting Started with Python
Python Guide Documentation, Release 0.0.1
$ python
will launch the Homebrew-installed Python 3 interpreter. $ python2
will launch the Homebrew-installed Python 2 interpreter (if any). $ python3
will launch the Homebrew-installed Python 3 interpreter. If the Homebrew version of Python 2 is installed then pip2 will point to Python 2. If the Homebrew version of Python 3 is installed then pip will point to Python 3. The rest of the guide will assume that python references Python 3. # Do I have a Python 3 installed? $ python --version Python 3.7.1 # Success!
1.3.4 Pipenv & Virtual Environments The next step is to install Pipenv, so you can install dependencies and manage virtual environments. A Virtual Environment is a tool to keep the dependencies required by different projects in separate places, by creating virtual Python environments for them. It solves the “Project X depends on version 1.x but, Project Y needs 4.x” dilemma, and keeps your global site-packages directory clean and manageable. For example, you can work on a project which requires Django 1.10 while also maintaining a project which requires Django 1.8. So, onward! To the Pipenv & Virtual Environments docs!
This page is a remixed version of another guide, which is available under the same license.
1.3. Installing Python 3 on Mac OS X
9
Python Guide Documentation, Release 0.0.1
1.4 Installing Python 3 on Windows
First, follow the installation instructions for Chocolatey. It’s a community system packager manager for Windows 7+. (It’s very much like Homebrew on OS X.) Once done, installing Python 3 is very simple, because Chocolatey pushes Python 3 as the default. choco install python
Once you’ve run this command, you should be able to launch Python directly from to the console. (Chocolatey is fantastic and automatically adds Python to your path.)
1.4.1 Setuptools + Pip The two most crucial third-party Python packages are setuptools and pip, which let you download, install and uninstall any compliant Python software product with a single command. It also enables you to add this network installation capability to your own Python software with very little work. All supported versions of Python 3 include pip, so just make sure it’s up to date: python -m pip install -U pip
1.4.2 Pipenv & Virtual Environments The next step is to install Pipenv, so you can install dependencies and manage virtual environments. A Virtual Environment is a tool to keep the dependencies required by different projects in separate places, by creating virtual Python environments for them. It solves the “Project X depends on version 1.x but, Project Y needs 4.x” dilemma, and keeps your global site-packages directory clean and manageable. 10
Chapter 1. Getting Started with Python
Python Guide Documentation, Release 0.0.1
For example, you can work on a project which requires Django 2.0 while also maintaining a project which requires Django 1.8. So, onward! To the Pipenv & Virtual Environments docs!
This page is a remixed version of another guide, which is available under the same license.
1.5 Installing Python 3 on Linux
This document describes how to install Python 3.6 on Ubuntu Linux machines. To see which version of Python 3 you have installed, open a command prompt and run $ python3 --version
If you are using Ubuntu 16.10 or newer, then you can easily install Python 3.6 with the following commands: $ sudo apt-get update $ sudo apt-get install python3.6
If you’re using another version of Ubuntu (e.g. the latest LTS release), we recommend using the deadsnakes PPA to install Python 3.6: $ $ $ $
sudo sudo sudo sudo
apt-get install software-properties-common add-apt-repository ppa:deadsnakes/ppa apt-get update apt-get install python3.6
1.5. Installing Python 3 on Linux
11
Python Guide Documentation, Release 0.0.1
If you are using other Linux distribution, chances are you already have Python 3 pre-installed as well. If not, use your distribution’s package manager. For example on Fedora, you would use dnf : $ sudo dnf install python3
Note that if the version of the python3 package is not recent enough for you, there may be ways of installing more recent versions as well, depending on you distribution. For example installing the python36 package on Fedora 25 to get Python 3.6. If you are a Fedora user, you might want to read about multiple Python versions available in Fedora.
1.5.1 Working with Python 3 At this point, you may have system Python 2.7 available as well. $ python
This will launch the Python 2 interpreter. $ python3
This will launch the Python 3 interpreter.
1.5.2 Setuptools & Pip The two most crucial third-party Python packages are setuptools and pip. Once installed, you can download, install and uninstall any compliant Python software product with a single command. It also enables you to add this network installation capability to your own Python software with very little work. Python 2.7.9 and later (on the python2 series), and Python 3.4 and later include pip by default. To see if pip is installed, open a command prompt and run $ command -v pip
To install pip, follow the official pip installation guide - this will automatically install the latest version of setuptools. Note that on some Linux distributions including Ubuntu and Fedora the pip command is meant for Python 2, while the pip3 command is meant for Python 3. $ command -v pip3
However, when using virtual environments (described below), you don’t need to care about that.
1.5.3 Pipenv & Virtual Environments The next step is to install Pipenv, so you can install dependencies and manage virtual environments. A Virtual Environment is a tool to keep the dependencies required by different projects in separate places, by creating virtual Python environments for them. It solves the “Project X depends on version 1.x but, Project Y needs 4.x” dilemma, and keeps your global site-packages directory clean and manageable. For example, you can work on a project which requires Django 1.10 while also maintaining a project which requires Django 1.8. 12
Chapter 1. Getting Started with Python
Python Guide Documentation, Release 0.0.1
So, onward! To the Pipenv & Virtual Environments docs!
This page is a remixed version of another guide, which is available under the same license.
1.6 Installing Python 2 on Mac OS X
Note: Check out our guide for installing Python 3 on OS X. The latest version of Mac OS X, High Sierra, comes with Python 2.7 out of the box. You do not need to install or configure anything else to use Python. Having said that, I would strongly recommend that you install the tools and libraries described in the next section before you start building Python applications for real-world use. In particular, you should always install Setuptools, as it makes it much easier for you to install and manage other third-party Python libraries. The version of Python that ships with OS X is great for learning, but it’s not good for development. The version shipped with OS X may be out of date from the official current Python release, which is considered the stable production version.
1.6.1 Doing it Right Let’s install a real version of Python. Before installing Python, you’ll need to install a C compiler. The fastest way is to install the Xcode Command Line Tools by running xcode-select --install. You can also download the full version of Xcode from the Mac App Store, or the minimal but unofficial OSX-GCC-Installer package. 1.6. Installing Python 2 on Mac OS X
13
Python Guide Documentation, Release 0.0.1
Note: If you already have Xcode installed, do not install OSX-GCC-Installer. In combination, the software can cause issues that are difficult to diagnose.
Note: If you perform a fresh install of Xcode, you will also need to add the commandline tools by running xcode-select --install on the terminal. While OS X comes with a large number of Unix utilities, those familiar with Linux systems will notice one key component missing: a decent package manager. Homebrew fills this void. To install Homebrew, open Terminal or your favorite OS X terminal emulator and run $ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/ ˓→install/master/install)"
The script will explain what changes it will make and prompt you before the installation begins. Once you’ve installed Homebrew, insert the Homebrew directory at the top of your PATH environment variable. You can do this by adding the following line at the bottom of your ~/.profile file export PATH="/usr/local/bin:/usr/local/sbin:$PATH"
Now, we can install Python 2.7: $ brew install python@2
Because python@2 is a “keg”, we need to update our PATH again, to point at our new installation: export PATH="/usr/local/opt/python@2/libexec/bin:$PATH"
Homebrew names the executable python2 so that you can still run the system Python via the executable python. $ python -V $ python2 -V $ python3 -V
# Homebrew installed Python 3 interpreter (if installed) # Homebrew installed Python 2 interpreter # Homebrew installed Python 3 interpreter (if installed)
1.6.2 Setuptools & Pip Homebrew installs Setuptools and pip for you. Setuptools enables you to download and install any compliant Python software over a network (usually the Internet) with a single command (easy_install). It also enables you to add this network installation capability to your own Python software with very little work. pip is a tool for easily installing and managing Python packages, that is recommended over easy_install. It is superior to easy_install in several ways, and is actively maintained. $ pip2 -V # pip pointing to the Homebrew installed Python 2 interpreter $ pip -V # pip pointing to the Homebrew installed Python 3 interpreter (if ˓→installed)
14
Chapter 1. Getting Started with Python
Python Guide Documentation, Release 0.0.1
1.6.3 Virtual Environments A Virtual Environment (commonly referred to as a ‘virtualenv’) is a tool to keep the dependencies required by different projects in separate places, by creating virtual Python environments for them. It solves the “Project X depends on version 1.x but, Project Y needs 4.x” dilemma, and keeps your global site-packages directory clean and manageable. For example, you can work on a project which requires Django 1.10 while also maintaining a project which requires Django 1.8. To start using this and see more information: Virtual Environments docs.
This page is a remixed version of another guide, which is available under the same license.
1.7 Installing Python 2 on Windows
Note: Check out our guide for installing Python 3 on Windows. First, download the latest version of Python 2.7 from the official website. If you want to be sure you are installing a fully up-to-date version, click the Downloads > Windows link from the home page of the Python.org web site . The Windows version is provided as an MSI package. To install it manually, just double-click the file. The MSI package format allows Windows administrators to automate installation with their standard tools. By design, Python installs to a directory with the version number embedded, e.g. Python version 2.7 will install at C:\Python27\, so that you can have multiple versions of Python on the same system without
1.7. Installing Python 2 on Windows
15
Python Guide Documentation, Release 0.0.1
conflicts. Of course, only one interpreter can be the default application for Python file types. It also does not automatically modify the PATH environment variable, so that you always have control over which copy of Python is run. Typing the full path name for a Python interpreter each time quickly gets tedious, so add the directories for your default Python version to the PATH. Assuming that your Python installation is in C:\Python27\, add this to your PATH: C:\Python27\;C:\Python27\Scripts\
You can do this easily by running the following in powershell: [Environment]::SetEnvironmentVariable("Path", "$env:Path;C:\Python27\; ˓→C:\Python27\Scripts\", "User")
This is also an option during the installation process. The second (Scripts) directory receives command files when certain packages are installed, so it is a very useful addition. You do not need to install or configure anything else to use Python. Having said that, I would strongly recommend that you install the tools and libraries described in the next section before you start building Python applications for real-world use. In particular, you should always install Setuptools, as it makes it much easier for you to use other third-party Python libraries.
1.7.1 Setuptools + Pip The two most crucial third-party Python packages are setuptools and pip. Once installed, you can download, install and uninstall any compliant Python software product with a single command. It also enables you to add this network installation capability to your own Python software with very little work. Python 2.7.9 and later (on the python2 series), and Python 3.4 and later include pip by default. To see if pip is installed, open a command prompt and run $ command -v pip
To install pip, follow the official pip installation guide - this will automatically install the latest version of setuptools.
1.7.2 Virtual Environments A Virtual Environment is a tool to keep the dependencies required by different projects in separate places, by creating virtual Python environments for them. It solves the “Project X depends on version 1.x but, Project Y needs 4.x” dilemma, and keeps your global site-packages directory clean and manageable. For example, you can work on a project which requires Django 1.10 while also maintaining a project which requires Django 1.8. To start using this and see more information: Virtual Environments docs.
This page is a remixed version of another guide, which is available under the same license.
16
Chapter 1. Getting Started with Python
Python Guide Documentation, Release 0.0.1
1.8 Installing Python 2 on Linux
Note: Check out our guide for installing Python 3 on Linux. The latest versions of CentOS, Red Hat Enterprise Linux (RHEL) and Ubuntu come with Python 2.7 out of the box. To see which version of Python you have installed, open a command prompt and run $ python2 --version
However, with the growing popularity of Python 3, some distributions, such as Fedora, don’t come with Python 2 pre-installed. You can install the python2 package with your distribution package manager: $ sudo dnf install python2
You do not need to install or configure anything else to use Python. Having said that, I would strongly recommend that you install the tools and libraries described in the next section before you start building Python applications for real-world use. In particular, you should always install Setuptools and pip, as it makes it much easier for you to use other third-party Python libraries.
1.8.1 Setuptools & Pip The two most crucial third-party Python packages are setuptools and pip. Once installed, you can download, install and uninstall any compliant Python software product with a single command. It also enables you to add this network installation capability to your own Python software with very little work.
1.8. Installing Python 2 on Linux
17
Python Guide Documentation, Release 0.0.1
Python 2.7.9 and later (on the python2 series), and Python 3.4 and later include pip by default. To see if pip is installed, open a command prompt and run $ command -v pip
To install pip, follow the official pip installation guide - this will automatically install the latest version of setuptools.
1.8.2 Virtual Environments A Virtual Environment is a tool to keep the dependencies required by different projects in separate places, by creating virtual Python environments for them. It solves the “Project X depends on version 1.x but, Project Y needs 4.x” dilemma, and keeps your global site-packages directory clean and manageable. For example, you can work on a project which requires Django 1.10 while also maintaining a project which requires Django 1.8. To start using this and see more information: Virtual Environments docs. You can also use virtualenvwrapper to make it easier to manage your virtual environments.
This page is a remixed version of another guide, which is available under the same license. • Using Virtualenvs with Pipenv:
1.9 Pipenv & Virtual Environments
This tutorial walks you through installing and using Python packages. 18
Chapter 1. Getting Started with Python
Python Guide Documentation, Release 0.0.1
It will show you how to install and use the necessary tools and make strong recommendations on best practices. Keep in mind that Python is used for a great many different purposes, and precisely how you want to manage your dependencies may change based on how you decide to publish your software. The guidance presented here is most directly applicable to the development and deployment of network services (including web applications), but is also very well suited to managing development and testing environments for any kind of project. Note: This guide is written for Python 3, however, these instructions should work fine on Python 2.7—if you are still using it, for some reason.
1.9.1 Make sure you’ve got Python & pip Before you go any further, make sure you have Python and that it’s available from your command line. You can check this by simply running: $ python --version
You should get some output like 3.6.2. If you do not have Python, please install the latest 3.x version from python.org or refer to the Installing Python section of this guide. Note: If you’re newcomer and you get an error like this: >>> python Traceback (most recent call last): File "", line 1, in NameError: name 'python' is not defined
It’s because this command is intended to be run in a shell (also called a terminal or console). See the Python for Beginners getting started tutorial for an introduction to using your operating system’s shell and interacting with Python. Additionally, you’ll need to make sure you have pip available. You can check this by running: $ pip --version
If you installed Python from source, with an installer from python.org, or via Homebrew you should already have pip. If you’re on Linux and installed using your OS package manager, you may have to install pip separately.
1.9.2 Installing Pipenv Pipenv is a dependency manager for Python projects. If you’re familiar with Node.js’ npm or Ruby’s bundler, it is similar in spirit to those tools. While pip can install Python packages, Pipenv is recommended as it’s a higher-level tool that simplifies dependency management for common use cases. Use pip to install Pipenv: $ pip install --user pipenv
Note: This does a user installation to prevent breaking any system-wide packages. If pipenv isn’t available in your shell after installation, you’ll need to add the user base’s binary directory to your PATH. 1.9. Pipenv & Virtual Environments
19
Python Guide Documentation, Release 0.0.1
On Linux and macOS you can find the user base binary directory by running python -m site --user-base and adding bin to the end. For example, this will typically print ~/.local (with ~ expanded to the absolute path to your home directory) so you’ll need to add ~/.local/bin to your PATH. You can set your PATH permanently by modifying ~/.profile. On Windows you can find the user base binary directory by running py -m site --user-site and replacing site-packages with Scripts. For example, this could return C:\Users\Username\App pip "$@" }
After saving the changes and sourcing your ~/.bashrc file you can now install packages globally by running gpip install. You can change the name of the function to anything you like, just keep in mind that you will have to use that name when trying to install packages globally with pip.
2.2.2 Caching packages for future use Every developer has preferred libraries and when you are working on a lot of different projects, you are bound to have some overlap between the libraries that you use. For example, you may be using the requests library in a lot of different projects. It is surely unnecessary to re-download the same packages/libraries each time you start working on a new project (and in a new virtual environment as a result). Fortunately, starting with version 6.0, pip provides an on-by-default caching mechanism that doesn’t need any configuration. When using older versions, you can configure pip in such a way that it tries to reuse already installed packages, too. On Unix systems, you can add the following line to your .bashrc or .bash_profile file. export PIP_DOWNLOAD_CACHE=$HOME/.pip/cache
You can set the path to anywhere you like (as long as you have write access). After adding this line, source your .bashrc (or .bash_profile) file and you will be all set. Another way of doing the same configuration is via the pip.conf or pip.ini files, depending on your system. If you are on Windows, you can add the following line to your pip.ini file under [global] settings: download-cache = %HOME%\pip\cache
Similarly, on Unix systems you should simply add the following line to your pip.conf file under [global] settings:
2.2. Further Configuration of pip and Virtualenv
33
Python Guide Documentation, Release 0.0.1
download-cache = $HOME/.pip/cache
Even though you can use any path you like to store your cache, it is recommended that you create a new folder in the folder where your pip.conf or pip.ini file lives. If you don’t trust yourself with all of this path voodoo, just use the values provided here and you will be fine.
34
Chapter 2. Python Development Environments
CHAPTER
3
Writing Great Python Code
This part of the guide focuses on the best-practices for writing Python code.
3.1 Structuring Your Project
35
Python Guide Documentation, Release 0.0.1
By “structure” we mean the decisions you make concerning how your project best meets its objective. We need to consider how to best leverage Python’s features to create clean, effective code. In practical terms, “structure” means making clean code whose logic and dependencies are clear as well as how the files and folders are organized in the filesystem. Which functions should go into which modules? How does )), 3)
Mock has many other ways you can configure it and control its behavior. mock
68
Chapter 3. Writing Great Python Code
Python Guide Documentation, Release 0.0.1
3.6 Logging
The logging module has been a part of Python’s Standard Library since version 2.3. It is succinctly described in PEP 282. The documentation is notoriously hard to read, except for the basic logging tutorial. Logging serves two purposes: • Diagnostic logging records events related to the application’s operation. If a user calls in to report an error, for example, the logs can be searched for context. • Audit logging records events for business analysis. A user’s transactions can be extracted and combined with other user details for reports or to optimize a business goal.
3.6.1 . . . or Print? The only time that print is a better option than logging is when the goal is to display a help statement for a command line application. Other reasons why logging is better than print: • The log record, which is created with every logging event, contains readily available diagnostic information such as the file name, full path, function, and line number of the logging event. • Events logged in included modules are automatically accessible via the root logger to your application’s logging stream, unless you filter them out. • Logging can be selectively silenced by using the method logging.Logger.setLevel() or disabled by setting the attribute logging.Logger.disabled to True.
3.6. Logging
69
Python Guide Documentation, Release 0.0.1
3.6.2 Logging in a Library Notes for configuring logging for a library are in the logging tutorial. Because the user, not the library, should dictate what happens when a logging event occurs, one admonition bears repeating: Note: It is strongly advised that you do not add any handlers other than NullHandler to your library’s loggers. Best practice when instantiating loggers in a library is to only create them using the __name__ global variable: the logging module creates a hierarchy of loggers using dot notation, so using __name__ ensures no name collisions. Here is an example of best practice from the requests source – place this in your __init__.py: import logging logging.getLogger(__name__).addHandler(logging.NullHandler())
3.6.3 Logging in an Application The twelve factor app, an authoritative reference for good practice in application development, contains a section on logging best practice. It emphatically advocates for treating log events as an event stream, and for sending that event stream to standard output to be handled by the application environment. There are at least three ways to configure a logger: • Using an INI-formatted file: – Pro: possible to update configuration while running using the function logging.config. listen() to listen on a socket. – Con: less control (e.g. custom subclassed filters or loggers) than possible when configuring a logger in code. • Using a dictionary or a JSON-formatted file: – Pro: in addition to updating while running, it is possible to load from a file using the json module, in the standard library since Python 2.6. – Con: less control than when configuring a logger in code. • Using code: – Pro: complete control over the configuration. – Con: modifications require a change to source code. Example Configuration via an INI File Let us say the file is named logging_config.ini. More details for the file format are in the logging configuration section of the logging tutorial. [loggers] keys=root [handlers] keys=stream_handler [formatters] keys=formatter (continues on next page)
70
Chapter 3. Writing Great Python Code
Python Guide Documentation, Release 0.0.1
(continued from previous page)
[logger_root] level=DEBUG handlers=stream_handler [handler_stream_handler] class=StreamHandler level=DEBUG formatter=formatter args=(sys.stderr,) [formatter_formatter] format=%(asctime)s %(name)-12s %(levelname)-8s %(message)s
Then use logging.config.fileConfig() in the code: import logging from logging.config import fileConfig fileConfig('logging_config.ini') logger = logging.getLogger() logger.debug('often makes a very good meal of %s', 'visiting tourists')
Example Configuration via a Dictionary As of Python 2.7, you can use a dictionary with configuration details. PEP 391 contains a list of the mandatory and optional elements in the configuration dictionary. import logging from logging.config import dictConfig logging_config = dict( version = 1, formatters = { 'f': {'format': '%(asctime)s %(name)-12s %(levelname)-8s %(message)s'} }, handlers = { 'h': {'class': 'logging.StreamHandler', 'formatter': 'f', 'level': logging.DEBUG} }, root = { 'handlers': ['h'], 'level': logging.DEBUG, }, ) dictConfig(logging_config) logger = logging.getLogger() logger.debug('often makes a very good meal of %s', 'visiting tourists')
3.6. Logging
71
Python Guide Documentation, Release 0.0.1
Example Configuration Directly in Code import logging logger = logging.getLogger() handler = logging.StreamHandler() formatter = logging.Formatter( '%(asctime)s %(name)-12s %(levelname)-8s %(message)s') handler.setFormatter(formatter) logger.addHandler(handler) logger.setLevel(logging.DEBUG) logger.debug('often makes a very good meal of %s', 'visiting tourists')
3.7 Common Gotchas
For the most part, Python aims to be a clean and consistent language that avoids surprises. However, there are a few cases that can be confusing to newcomers. Some of these cases are intentional but can be potentially surprising. Some could arguably be considered language warts. In general, what follows is a collection of potentially tricky behavior that might seem strange at first glance, but is generally sensible once you’re aware of the underlying cause for the surprise.
72
Chapter 3. Writing Great Python Code
Python Guide Documentation, Release 0.0.1
3.7.1 Mutable Default Arguments Seemingly the most common surprise new Python programmers encounter is Python’s treatment of mutable default arguments in function definitions. What You Wrote def append_to(element, to=[]): to.append(element) return to
What You Might Have Expected to Happen my_list = append_to(12) print(my_list) my_other_list = append_to(42) print(my_other_list)
A new list is created each time the function is called if a second argument isn’t provided, so that the output is: [12] [42]
What Does Happen [12] [12, 42]
A new list is created once when the function is defined, and the same list is used in each successive call. Python’s default arguments are evaluated once when the function is defined, not each time the function is called (like it is in say, Ruby). This means that if you use a mutable default argument and mutate it, you will and have mutated that object for all future calls to the function as well. What You Should Do Instead Create a new object each time the function is called, by using a default arg to signal that no argument was provided (None is often a good choice). def append_to(element, to=None): if to is None: to = [] to.append(element) return to
Do not forget, you are passing a list object as the second argument.
3.7. Common Gotchas
73
Python Guide Documentation, Release 0.0.1
When the Gotcha Isn’t a Gotcha Sometimes you can specifically “exploit” (read: use as intended) this behavior to maintain state between calls of a function. This is often done when writing a caching function.
3.7.2 Late Binding Closures Another common source of confusion is the way Python binds its variables in closures (or in the surrounding global scope). What You Wrote def create_multipliers(): return [lambda x : i * x for i in range(5)]
What You Might Have Expected to Happen for multiplier in create_multipliers(): print(multiplier(2))
A list containing five functions that each have their own closed-over i variable that multiplies their argument, producing: 0 2 4 6 8
What Does Happen 8 8 8 8 8
Five functions are created; instead all of them just multiply x by 4. Python’s closures are late binding. This means that the values of variables used in closures are looked up at the time the inner function is called. Here, whenever any of the returned functions are called, the value of i is looked up in the surrounding scope at call time. By then, the loop has completed and i is left with its final value of 4. What’s particularly nasty about this gotcha is the seemingly prevalent misinformation that this has something to do with lambdas in Python. Functions created with a lambda expression are in no way special, and in fact the same exact behavior is exhibited by just using an ordinary def:
74
Chapter 3. Writing Great Python Code
Python Guide Documentation, Release 0.0.1
def create_multipliers(): multipliers = [] for i in range(5): def multiplier(x): return i * x multipliers.append(multiplier) return multipliers
What You Should Do Instead The most general solution is arguably a bit of a hack. Due to Python’s aforementioned behavior concerning evaluating default arguments to functions (see Mutable Default Arguments), you can create a closure that binds immediately to its arguments by using a default arg like so: def create_multipliers(): return [lambda x, i=i : i * x for i in range(5)]
Alternatively, you can use the functools.partial function: from functools import partial from operator import mul def create_multipliers(): return [partial(mul, i) for i in range(5)]
When the Gotcha Isn’t a Gotcha Sometimes you want your closures to behave this way. Late binding is good in lots of situations. Looping to create unique functions is unfortunately a case where they can cause hiccups.
3.7.3 Bytecode (.pyc) Files Everywhere! By default, when executing Python code from files, the Python interpreter will automatically write a bytecode version of that file to disk, e.g. module.pyc. These .pyc files should not be checked into your source code repositories. Theoretically, this behavior is on by default for performance reasons. Without these bytecode files present, Python would re-generate the bytecode every time the file is loaded. Disabling Bytecode (.pyc) Files Luckily, the process of generating the bytecode is extremely fast, and isn’t something you need to worry about while developing your code. Those files are annoying, so let’s get rid of them! $ export PYTHONDONTWRITEBYTECODE=1
3.7. Common Gotchas
75
Python Guide Documentation, Release 0.0.1
With the $PYTHONDONTWRITEBYTECODE environment variable set, Python will no longer write these files to disk, and your development environment will remain nice and clean. I recommend setting this environment variable in your ~/.profile. Removing Bytecode (.pyc) Files Here’s nice trick for removing all of these files, if they already exist: $ find . -type f -name "*.py[co]" -delete -or -type d -name "__pycache__" -delete
Run that from the root directory of your project, and all .pyc files will suddenly vanish. Much better. Version Control Ignores If you still need the .pyc files for performance reasons, you can always add them to the ignore files of your version control repositories. Popular version control systems have the ability to use wildcards defined in a file to apply special rules. An ignore file will make sure the matching files don’t get checked into the repository. Git uses .gitignore while Mercurial uses .hgignore. At the minimum your ignore files should look like this. syntax:glob *.py[cod] __pycache__/
# This line is not needed for .gitignore files. # Will match .pyc, .pyo and .pyd files. # Exclude the whole folder
You may wish to include more files and directories depending on your needs. The next time you commit to the repository, these files will not be included.
76
Chapter 3. Writing Great Python Code
Python Guide Documentation, Release 0.0.1
3.8 Choosing a License
Your source publication needs a license. In the US, if no license is specified, users have no legal right to download, modify, or distribute. Furthermore, people can’t contribute to your code unless you tell them what rules to play by. Choosing a license is complicated, so here are some pointers: Open source. There are plenty of open source licenses available to choose from. In general, these licenses tend to fall into one of two categories: 1. licenses that focus more on the user’s freedom to do with the software as they please (these are the more permissive open source licenses such as the MIT, BSD, and Apache) 2. licenses that focus more on making sure that the code itself — including any changes made to it and distributed along with it — always remains free (these are the less permissive free software licenses such as the GPL and LGPL) The latter are less permissive in the sense that they don’t permit someone to add code to the software and distribute it without also including the source code for their changes. To help you choose one for your project, there’s a license chooser; use it. More Permissive • PSFL (Python Software Foundation License) – for contributing to Python itself • MIT / BSD / ISC – MIT (X11) – New BSD
3.8. Choosing a License
77
Python Guide Documentation, Release 0.0.1
– ISC • Apache Less Permissive: • LGPL • GPL – GPLv2 – GPLv3 A good overview of licenses with explanations of what one can, cannot, and must do using a particular software can be found at tl;drLegal.
78
Chapter 3. Writing Great Python Code
CHAPTER
4
Scenario Guide for Python Applications
This part of the guide focuses on tool and module advice based on different scenarios.
4.1 Network Applications
79
Python Guide Documentation, Release 0.0.1
4.1.1 HTTP The Hypertext Transfer Protocol (HTTP) is an application protocol for distributed, collaborative, hypermedia information systems. HTTP is the foundation of ) templateEnv = Environment( loader=templateLoader ) template = templateEnv.get_template(TEMPLATE_FILE) # List for famous movie rendering movie_list = [[1,"The Hitchhiker's Guide to the Galaxy"],[2,"Back to future"],[3, ˓→"Matrix"]] (continues on next page)
4.2. Web Applications & Frameworks
85
Python Guide Documentation, Release 0.0.1
(continued from previous page)
# template.render() returns a string which contains the rendered html html_output = template.render(list=movie_list, title="Here is my favorite movie list") # Handler for main page class MainHandler(tornado.web.RequestHandler): def get(self): # Returns rendered template string to the browser request self.write(html_output) # Assign handler to the server root (127.0.0.1:PORT/) application = tornado.web.Application([ (r"/", MainHandler), ]) PORT=8884 if __name__ == "__main__": # Setup the server application.listen(PORT) tornado.ioloop.IOLoop.instance().start()
The base.html file can be used as base for all site pages which are for example implemented in the content block.
{{title}} - My Webpage
{# In the next line the content from the site.html template will be added #} {% block content %}{% endblock %}
{% block footer %} © Copyright 2013 by you. {% endblock %}
The next listing is our site page (site.html) loaded in the Python app which extends base.html. The content block is automatically set into the corresponding block in the base.html page. {% extends "base.html" %} {% block content %}
{{title}}
{{ list_title }}
- {% for item in list %}
- {{ item[0]}} : {{ item[1]}} {% endfor %}
(continues on next page)
86
Chapter 4. Scenario Guide for Python Applications
Python Guide Documentation, Release 0.0.1
(continued from previous page)
{% endblock %}
Jinja2 is the recommended templating library for new Python web applications. Chameleon Chameleon Page Templates are an HTML/XML template engine implementation of the Template Attribute Language (TAL), TAL Expression Syntax (TALES), and Macro Expansion TAL (Metal) syntaxes. Chameleon is available for Python 2.5 and up (including 3.x and PyPy), and is commonly used by the Pyramid Framework. Page Templates add within your document structure special element attributes and text markup. Using a set of simple language constructs, you control the document flow, element repetition, text replacement, and translation. Because of the attribute-based syntax, unrendered page templates are valid HTML and can be viewed in a browser and even edited in WYSIWYG editors. This can make round-trip collaboration with designers and prototyping with static files in a browser easier. The basic TAL language is simple enough to grasp from an example:
Hello, World!
The pattern for text insertion is common enough that if you do not require strict validity in your unrendered templates, you can replace it with a more terse and readable syntax that uses the pattern ${expression}, as follows:
Hello, ${world}!
| ${row.capitalize()} ${col} |
But keep in mind that the full Default Text syntax also allows for default content in the unrendered template. Being from the Pyramid world, Chameleon is not widely used.
4.2. Web Applications & Frameworks
87
Python Guide Documentation, Release 0.0.1
Mako Mako is a template language that compiles to Python for maximum performance. Its syntax and API are borrowed from the best parts of other templating languages like Django and Jinja2 templates. It is the default template language included with the Pylons and Pyramid web frameworks. An example template in Mako looks like:
To render a very basic template, you can do the following: from mako.template import Template print(Template("hello ${))
Mako is well respected within the Python web community.
88
Chapter 4. Scenario Guide for Python Applications
Python Guide Documentation, Release 0.0.1
References
4.3 HTML Scraping
4.3.1 Web Scraping Web sites are written using HTML, which means that each web page is a structured document. Sometimes it would be great to obtain some >Carson Busses $29.95
Knowing this we can create the correct XPath query and use the lxml xpath function like this: #This will create a list of buyers: buyers = tree.xpath('//div[@title="buyer-name"]/text()') #This will create a list of prices prices = tree.xpath('//span[@]/text()')
Let’s see what we got exactly: print 'Buyers: ', buyers print 'Prices: ', prices Buyers: ['Carson Busses', 'Earl E. Byrd', 'Patty Cakes', 'Derri Anne Connecticut', 'Moe Dess', 'Leda Doggslife', 'Dan Druff', 'Al Fresco', 'Ido Hoe', 'Howie Kisses', 'Len Lease', 'Phil Meup', 'Ira Pent', 'Ben D. Rules', 'Ave Sectomy', 'Gary Shattire', 'Bobbi Soks', 'Sheila Takya', 'Rose Tattoo', 'Moe Tell'] Prices: ['$29.95', '$8.37', '$15.26', '$19.25', '$19.25', '$13.99', '$31.57', '$8.49', '$14.47', '$15.86', '$11.11', '$15.98', '$16.27', '$7.50', '$50.85', '$14.26', '$5.68', '$15.00', '$114.07', '$10.09']
Congratulations! We have successfully scraped all the ?>
can be loaded like this: import untangle obj = untangle.parse('path/to/file.xml')
and then you can get the child element’s name attribute like this: obj.root.child['name']
untangle also supports loading XML from a string or a URL.
4.14. XML parsing
123
Python Guide Documentation, Release 0.0.1
4.14.2 xmltodict xmltodict is another simple library that aims at making XML feel like working with JSON. An XML file like this:
elements more elements
element as well
can be loaded into a Python dict like this: import xmltodict with open('path/to/file.xml') as fd: doc = xmltodict.parse(fd.read())
and then you can access elements, attributes, and values like this: doc['mydocument']['@has'] # == u'an attribute' doc['mydocument']['and']['many'] # == [u'elements', u'more elements'] doc['mydocument']['plus']['@a'] # == u'complex' doc['mydocument']['plus']['#text'] # == u'element as well'
xmltodict also lets you roundtrip back to XML with the unparse function, has a streaming mode suitable for handling files that don’t fit in memory, and supports XML namespaces.
124
Chapter 4. Scenario Guide for Python Applications
Python Guide Documentation, Release 0.0.1
4.15 JSON
The json library can parse JSON from strings or files. The library parses JSON into a Python dictionary or list. It can also convert Python dictionaries or lists into JSON strings.
4.15.1 Parsing JSON Take the following string containing JSON data: json_string = '{"first_name": "Guido", "last_name":"Rossum"}'
It can be parsed like this: import json parsed_json = json.loads(json_string)
and can now be used as a normal dictionary: print(parsed_json['first_name']) "Guido"
You can also convert the following to JSON: d = { 'first_name': 'Guido', 'second_name': 'Rossum', (continues on next page)
4.15. JSON
125
Python Guide Documentation, Release 0.0.1
(continued from previous page)
'titles': ['BDFL', 'Developer'], } print(json.dumps(d)) '{"first_name": "Guido", "last_name": "Rossum", "titles": ["BDFL", "Developer"]}'
4.15.2 simplejson The json library was added to Python in version 2.6. If you’re using an earlier version of Python, the simplejson library is available via PyPI. simplejson mimics the json standard library. It is available so that developers that use older versions of Python can use the latest features available in the json lib. You can start using simplejson when the json library is not available by importing simplejson under a different name: import simplejson as json
After importing simplejson as json, the above examples will all work as if you were using the standard json library.
4.16 Cryptography
126
Chapter 4. Scenario Guide for Python Applications
Python Guide Documentation, Release 0.0.1
4.16.1 cryptography cryptography is an actively developed library that provides cryptographic recipes and primitives. It supports Python 2.6-2.7, Python 3.3+, and PyPy. cryptography is divided into two layers of recipes and hazardous materials (hazmat). The recipes layer provides a simple API for proper symmetric encryption and the hazmat layer provides low-level cryptographic primitives. Installation $ pip install cryptography
Example Example code using high level symmetric encryption recipe: from cryptography.fernet import Fernet key = Fernet.generate_key() cipher_suite = Fernet(key) cipher_text = cipher_suite.encrypt(b"A really secret message. Not for prying eyes.") plain_text = cipher_suite.decrypt(cipher_text)
4.16.2 GPGME bindings The GPGME Python bindings provide Pythonic access to GPG Made Easy, a C API for the entire GNU Privacy Guard suite of projects, including GPG, libgcrypt, and gpgsm (the S/MIME engine). It supports Python 2.6, 2.7, 3.4, and above. Depends on the SWIG C interface for Python as well as the GnuPG software and libraries. A more comprehensive GPGME Python Bindings HOWTO is available with the source, and an HTML version is available at http://files.au.adversary.org. Python 3 sample scripts from the examples in the HOWTO are also provided with the source and are accessible at gnupg.org. Available under the same terms as the rest of the GnuPG Project: GPLv2 and LGPLv2.1, both with the “or any later version” clause. Installation Included by default when compiling GPGME if the configure script locates a supported python version (which it will if it’s in $PATH during configuration). Example import gpg # Encryption to public key specified a_key = input("Enter the fingerprint filename = input("Enter the filename with open(filename, "rb") as afile: text = afile.read() c = gpg.core.Context(armor=True) rkey = list(c.keylist(pattern=a_key,
in rkey. or key ID to encrypt to: ") to encrypt: ")
secret=False)) (continues on next page)
4.16. Cryptography
127
Python Guide Documentation, Release 0.0.1
(continued from previous page)
ciphertext, result, sign_result = c.encrypt(text, recipients=rkey, always_trust=True, add_encrypt_to=True) with open("{0}.asc".format(filename), "wb") as bfile: bfile.write(ciphertext) # Decryption with corresponding secret key # invokes gpg-agent and pinentry. with open("{0}.asc".format(filename), "rb") as cfile: plaintext, result, verify_result = gpg.Context().decrypt(cfile) with open("new-{0}".format(filename), "wb") as dfile: dfile.write(plaintext) # Matching the data. # Also running a diff on filename and the new filename should match. if text == plaintext: print("Hang on ... did you say *all* of GnuPG? Yep.") else: pass
4.16.3 PyCrypto PyCrypto is another library, which provides secure hash functions and various encryption algorithms. It supports Python version 2.1 through 3.3. Installation $ pip install pycrypto
Example from Crypto.Cipher import AES # Encryption encryption_suite = AES.new('This is a key123', AES.MODE_CBC, 'This is an IV456') cipher_text = encryption_suite.encrypt("A really secret message. Not for prying eyes. ˓→") # Decryption decryption_suite = AES.new('This is a key123', AES.MODE_CBC, 'This is an IV456') plain_text = decryption_suite.decrypt(cipher_text)
128
Chapter 4. Scenario Guide for Python Applications
Python Guide Documentation, Release 0.0.1
4.17 Machine Learning
Python has a vast number of libraries for data analysis, statistics, and Machine Learning itself, making it a language of choice for many data scientists. Some widely used packages for Machine Learning and other data science applications are listed below.
4.17.1 SciPy Stack The SciPy stack consists of a bunch of core helper packages used in data science for statistical analysis and visualising data. Because of its huge number of functionalities and ease of use, the Stack is considered a must-have for most data science applications. The Stack consists of the following packages (link to documentation given): 1. NumPy 2. SciPy library 3. Matplotlib 4. IPython 5. pandas 6. Sympy 7. nose The stack also comes with Python bundled in, but has been excluded from the above list.
4.17. Machine Learning
129
Python Guide Documentation, Release 0.0.1
Installation For installing the full stack, or individual packages, you can refer to the instructions given here. NB: Anaconda is highly preferred and recommended for installing and maintaining data science packages seamlessly.
4.17.2 scikit-learn Scikit is a free and open source machine learning library for Python. It offers off-the-shelf functions to implement many algorithms like linear regression, classifiers, SVMs, k-means, Neural Networks, etc. It also has a few sample datasets which can be directly used for training and testing. Because of its speed, robustness, and ease of, it’s one of the most widely-used libraries for many Machine Learning applications. Installation Through PyPI: pip install -U scikit-learn
Through conda: conda install scikit-learn
scikit-learn also comes shipped with Anaconda (mentioned above). For more installation instructions, refer to this link. Example For this example, we train a simple classifier on the Iris dataset, which comes bundled in with scikit-learn. The dataset takes four features of flowers: sepal length, sepal width, petal length, and petal width, and classifies them into three flower species (labels): setosa, versicolor, or virginica. The labels have been represented as numbers in the dataset: 0 (setosa), 1 (versicolor), and 2 (virginica). We shuffle the Iris dataset and divide it into separate training and testing sets, keeping the last 10 data points for testing and rest for training. We then train the classifier on the training set and predict on the testing set. from sklearn.datasets import load_iris from sklearn import tree from sklearn.metrics import accuracy_score import numpy as np #loading the iris dataset iris = load_iris() x = iris.data #array of the data y = iris.target #array of labels (i.e answers) of each data entry #getting label names i.e the three flower species y_names = iris.target_names #taking random indices to split the dataset into train and test test_ids = np.random.permutation(len(x)) (continues on next page)
130
Chapter 4. Scenario Guide for Python Applications
Python Guide Documentation, Release 0.0.1
(continued from previous page)
#splitting data and labels into train and test #keeping last 10 entries for testing, rest for training x_train = x[test_ids[:-10]] x_test = x[test_ids[-10:]] y_train = y[test_ids[:-10]] y_test = y[test_ids[-10:]] #classifying using decision tree clf = tree.DecisionTreeClassifier() #training (fitting) the classifier with the training set clf.fit(x_train, y_train) #predictions on the test dataset pred = clf.predict(x_test) print pred #predicted labels i.e flower species print y_test #actual labels print (accuracy_score(pred, y_test))*100 #prediction accuracy
Since we’re splitting randomly and the classifier trains on every iteration, the accuracy may vary. Running the above code gives: [0 1 1 1 0 2 0 2 2 2] [0 1 1 1 0 2 0 2 2 2] 100.0
The first line contains the labels (i.e. flower species) of the testing data as predicted by our classifier, and the second line contains the actual flower species as given in the dataset. We thus get an accuracy of 100% this time. More on scikit-learn can be read in the documentation.
4.17. Machine Learning
131
Python Guide Documentation, Release 0.0.1
4.18 Interfacing with C/C++ Libraries
4.18.1 C Foreign Function Interface CFFI provides a simple to use mechanism for interfacing with C from both CPython and PyPy. It supports two modes: an inline ABI compatibility mode (example provided below), which allows you to dynamically load and run functions from executable modules (essentially exposing the same functionality as LoadLibrary or dlopen), and an API mode, which allows you to build C extension modules. ABI Interaction 1 2 3 4 5 6 7
from cffi import FFI ffi = FFI() ffi.cdef("size_t strlen(const char*);") clib = ffi.dlopen(None) length = clib.strlen("String to be evaluated.") # prints: 23 print("{}".format(length))
4.18.2 ctypes ctypes is the de facto standard library for interfacing with C/C++ from CPython, and it provides not only full access to the native C interface of most major operating systems (e.g., kernel32 on Windows, or libc on *nix), but also provides
132
Chapter 4. Scenario Guide for Python Applications
Python Guide Documentation, Release 0.0.1
support for loading and interfacing with dynamic libraries, such as DLLs or shared objects, at runtime. It does bring along with it a whole host of types for interacting with system APIs, and allows you to rather easily define your own complex types, such as structs and unions, and allows you to modify things such as padding and alignment, if needed. It can be a bit crufty to use, but in conjunction with the struct module, you are essentially provided full control over how your data types get translated into something usable by a pure C(++) method. Struct Equivalents MyStruct.h 1 2 3 4
struct my_struct { int a; int b; };
MyStruct.py 1 2 3 4
import ctypes class my_struct(ctypes.Structure): _fields_ = [("a", c_int), ("b", c_int)]
4.18.3 SWIG SWIG, though not strictly Python focused (it supports a large number of scripting languages), is a tool for generating bindings for interpreted languages from C/C++ header files. It is extremely simple to use: the consumer simply needs to define an interface file (detailed in the tutorial and documentations), include the requisite C/C++ headers, and run the build tool against them. While it does have some limits (it currently seems to have issues with a small subset of newer C++ features, and getting template-heavy code to work can be a bit verbose), it provides a great deal of power and exposes lots of features to Python with little effort. Additionally, you can easily extend the bindings SWIG creates (in the interface file) to overload operators and built-in methods, effectively re- cast C++ exceptions to be catchable by Python, etc. Example: Overloading __repr__ MyClass.h 1 2 3 4 5 6 7
#include class MyClass { private: std::string name; public: std::string getName(); };
myclass.i 1
%include "string.i"
2 3 4 5 6
%module myclass %{ #include #include "MyClass.h" (continues on next page)
4.18. Interfacing with C/C++ Libraries
133
Python Guide Documentation, Release 0.0.1
(continued from previous page) 7
%}
8 9 10 11 12 13 14
%extend MyClass { std::string __repr__() { return $self->getName(); } }
15 16
%include "MyClass.h"
4.18.4 Boost.Python Boost.Python requires a bit more manual work to expose C++ object functionality, but it is capable of providing all the same features SWIG does and then some, to include providing wrappers to access PyObjects in C++, extracting SWIG wrapper objects, and even embedding bits of Python into your C++ code.
134
Chapter 4. Scenario Guide for Python Applications
CHAPTER
5
Shipping Great Python Code
This part of the guide focuses on deploying your Python code.
5.1 Packaging Your Code
135
Python Guide Documentation, Release 0.0.1
Package your code to share it with other developers. For example, to share a library for other developers to use in their application, or for development tools like ‘py.test’. An advantage of this method of distribution is its well established ecosystem of tools such as PyPI and pip, which make it easy for other developers to download and install your package either for casual experiments, or as part of large, professional systems. It is a well-established convention for Python code to be shared this way. If your code isn’t packaged on PyPI, then it will be harder for other developers to find it and to use it as part of their existing process. They will regard such projects with substantial suspicion of being either badly managed or abandoned. The downside of distributing code like this is that it relies on the recipient understanding how to install the required version of Python, and being able and willing to use tools such as pip to install your code’s other dependencies. This is fine when distributing to other developers, but makes this method unsuitable for distributing applications to end-users. The Python Packaging Guide provides an extensive guide on creating and maintaining Python packages.
5.1.1 Alternatives to Packaging To distribute applications to end-users, you should freeze your application. On Linux, you may also want to consider creating a Linux distro package (e.g. a .deb file for Debian or Ubuntu.)
5.1.2 For Python Developers If you’re writing an open source Python module, PyPI , more properly known as The Cheeseshop, is the place to host it. Pip vs. easy_install Use pip. More details here. Personal PyPI If you want to install packages from a source other than PyPI (say, if your packages are proprietary), you can do it by hosting a simple HTTP server, running from the directory which holds those packages which need to be installed. Showing an example is always beneficial For example, if you want to install a package called MyPackage.tar.gz, and assuming this is your directory structure: • archive – MyPackage * MyPackage.tar.gz Go to your command prompt and type: $ cd archive $ python -m SimpleHTTPServer 9000
This runs a simple HTTP server running on port 9000 and will list all packages (like MyPackage). Now you can install MyPackage using any Python package installer. Using pip, you would do it like:
136
Chapter 5. Shipping Great Python Code
Python Guide Documentation, Release 0.0.1
$ pip install --extra-index-url=http://127.0.0.1:9000/ MyPackage
Having a folder with the same name as the package name is crucial here. I got fooled by that, one time. But if you feel that creating a folder called MyPackage and keeping MyPackage.tar.gz inside that is redundant, you can still install MyPackage using: $ pip install
http://127.0.0.1:9000/MyPackage.tar.gz
pypiserver pypiserver is a minimal PyPI compatible server. It can be used to serve a set of packages to easy_install or pip. It includes helpful features like an administrative command (-U) which will update all its packages to their latest versions found on PyPI. S3-Hosted PyPi One simple option for a personal PyPI server is to use Amazon S3. A prerequisite for this is that you have an Amazon AWS account with an S3 bucket. 1. Install all your requirements from PyPi or another source 2. Install pip2pi • pip install git+https://github.com/wolever/pip2pi.git 3. Follow pip2pi README for pip2tgz and dir2pi commands • pip2tgz packages/ YourPackage (or pip2tgz packages/ -r requirements.txt) • dir2pi packages/ 4. Upload the new files • Use a client like Cyberduck to sync the entire packages folder to your s3 bucket. • Make sure you upload packages/simple/index.html as well as all new files and directories. 5. Fix new file permissions • By default, when you upload new files to the S3 bucket, they will have the wrong permissions set. • Use the Amazon web console to set the READ permission of the files to EVERYONE. • If you get HTTP 403 when trying to install a package, make sure you’ve set the permissions correctly. 6. All done • You can now install your package with pip install --index-url=http://your-s3-bucket/ packages/simple/ YourPackage.
5.1.3 For Linux Distributions Creating a Linux distro package is arguably the “right way” to distribute code on Linux. Because a distribution package doesn’t include the Python interpreter, it makes the download and install about 2-12 MB smaller than freezing your application. Also, if a distribution releases a new security update for Python, then your application will automatically start using that new version of Python. 5.1. Packaging Your Code
137
Python Guide Documentation, Release 0.0.1
The bdist_rpm command makes producing an RPM file for use by distributions like Red Hat or SuSE trivially easy. However, creating and maintaining the different configurations required for each distribution’s format (e.g. .deb for Debian/Ubuntu, .rpm for Red Hat/Fedora, etc.) is a fair amount of work. If your code is an application that you plan to distribute on other platforms, then you’ll also have to create and maintain the separate config required to freeze your application for Windows and OS X. It would be much less work to simply create and maintain a single config for one of the cross platform freezing tools, which will produce stand-alone executables for all distributions of Linux, as well as Windows and OS X. Creating a distribution package is also problematic if your code is for a version of Python that isn’t currently supported by a distribution. Having to tell some versions of Ubuntu end-users that they need to add the ‘dead-snakes’ PPA using sudo apt-repository commands before they can install your .deb file makes for an extremely hostile user experience. Not only that, but you’d have to maintain a custom equivalent of these instructions for every distribution, and worse, have your users read, understand, and act on them. Having said all that, here’s how to do it: • Fedora • Debian and Ubuntu • Arch Useful Tools • fpm • alien • dh-virtualenv (for APT/DEB omnibus packaging)
138
Chapter 5. Shipping Great Python Code
Python Guide Documentation, Release 0.0.1
5.2 Freezing Your Code
“Freezing” your code is creating a single-file executable file to distribute to end-users, that contains all of your application code as well as the Python interpreter. Applications such as ‘Dropbox’, ‘Eve Online’, ‘Civilization IV’, and BitTorrent clients do this. The advantage of distributing this way is that your application will “just work”, even if the user doesn’t already have the required version of Python (or any) installed. On Windows, and even on many Linux distributions and OS X, the right version of Python will not already be installed. Besides, end-user software should always be in an executable format. Files ending in .py are for software engineers and system administrators. One disadvantage of freezing is that it will increase the size of your distribution by about 2–12 MB. Also, you will be responsible for shipping updated versions of your application when security vulnerabilities to Python are patched.
5.2.1 Alternatives to Freezing Packaging your code is for distributing libraries or tools to other developers. On Linux, an alternative to freezing is to create a Linux distro package (e.g. .deb files for Debian or Ubuntu, or .rpm files for Red Hat and SuSE.) Todo: Fill in “Freezing Your Code” stub
5.2. Freezing Your Code
139
Python Guide Documentation, Release 0.0.1
5.2.2 Comparison of Freezing Tools Solutions and platforms/features supported: Solution bbFreeze py2exe pyInstaller cx_Freeze py2app
Windows yes yes yes
Linux OS X yes yes no no yes yes
Python 3 no yes yes
License MIT MIT GPL
One-file mode no yes yes
Zipfile import yes yes no
Eggs pkg_resources support yes yes no no yes no
yes no
yes no
yes yes
PSF MIT
no no
yes yes
yes yes
yes yes
no yes
Note: Freezing Python code on Linux into a Windows executable was only once supported in PyInstaller and later dropped.
Note: All solutions need a Microsoft Visual C++ to be installed on the target machine, except py2app. Only PyInstaller makes a self-executable exe that bundles the appropriate DLL when passing --onefile to Configure.py.
5.2.3 Windows bbFreeze Prerequisite is to install Python, Setuptools and pywin32 dependency on Windows. 1. Install bbfreeze: $ pip install bbfreeze
2. Write most basic bb_setup.py from bbfreeze import Freezer freezer = Freezer(distdir='dist') freezer.addScript('foobar.py', gui_only=True) freezer()
Note: This will work for the most basic one file scripts. For more advanced freezing you will have to provide include and exclude paths like so: freezer = Freezer(distdir='dist', includes=['my_code'], excludes=['docs'])
3. (Optionally) include icon freezer.setIcon('my_awesome_icon.ico')
4. Provide the Microsoft Visual C++ runtime DLL for the freezer. It might be possible to append your sys.path with the Microsoft Visual Studio path but I find it easier to drop msvcp90.dll in the same folder where your script resides. 140
Chapter 5. Shipping Great Python Code
Python Guide Documentation, Release 0.0.1
5. Freeze! $ python bb_setup.py
py2exe Prerequisite is to install Python on Windows. The last release of py2exe is from the year 2014. There is not active development. 1. Download and install http://sourceforge.net/projects/py2exe/files/py2exe/ 2. Write setup.py (List of configuration options): from distutils.core import setup import py2exe setup( windows=[{'script': 'foobar.py'}], )
3. (Optionally) include icon 4. (Optionally) one-file mode 5. Generate .exe into dist directory: $ python setup.py py2exe
6. Provide the Microsoft Visual C++ runtime DLL. Two options: globally install dll on target machine or distribute dll alongside with .exe. PyInstaller Prerequisite is to have installed Python, Setuptools and pywin32 dependency on Windows. • Most basic tutorial • Manual
5.2.4 OS X py2app PyInstaller PyInstaller can be used to build Unix executables and windowed apps on Mac OS X 10.6 (Snow Leopard) or newer. To install PyInstaller, use pip: $ pip install pyinstaller
To create a standard Unix executable, from say script.py, use: $ pyinstaller script.py
This creates:
5.2. Freezing Your Code
141
Python Guide Documentation, Release 0.0.1
• a script.spec file, analogous to a make file • a build folder, that holds some log files • a dist folder, that holds the main executable script, and some dependent Python libraries all in the same folder as script.py. PyInstaller puts all the Python libraries used in script.py into the dist folder, so when distributing the executable, distribute the whole dist folder. The script.spec file can be edited to customise the build, with options such as: • bundling data files with the executable • including run-time libraries (.dll or .so files) that PyInstaller can’t infer automatically • adding Python run-time options to the executable Now script.spec can be run with pyinstaller (instead of using script.py again): $ pyinstaller script.spec
To create a standalone windowed OS X application, use the --windowed option: $ pyinstaller --windowed script.spec
This creates a script.app in the dist folder. Make sure to use GUI packages in your Python code, like PyQt or PySide, to control the graphical parts of the app. There are several options in script.spec related to Mac OS X app bundles here. For example, to specify an icon for the app, use the icon=\path\to\icon.icns option.
5.2.5 Linux bbFreeze PyInstaller
142
Chapter 5. Shipping Great Python Code
CHAPTER
6
Additional Notes
This part of the guide, which is mostly prose, begins with some background information about Python, and then focuses on next steps.
143
Python Guide Documentation, Release 0.0.1
6.1 Introduction
From the official Python website: Python is a general-purpose, high-level programming language similar to Tcl, Perl, Ruby, Scheme, or Java. Some of its main key features include: • very clear, readable syntax Python’s philosophy focuses on readability, from code blocks delineated with significant whitespace to intuitive keywords in place of inscrutable punctuation. • extensive standard libraries and third party modules for virtually any task Python is sometimes described with the words “batteries included” because of its extensive standard library, which includes modules for regular expressions, file IO, fraction handling, object serialization, and much more. Additionally, the Python Package Index is available for users to submit their packages for widespread use, similar to Perl’s CPAN. There is a thriving community of very powerful Python frameworks and tools like the Django web framework and the NumPy set of math routines. • integration with other systems Python can integrate with Java libraries, enabling it to be used with the rich Java environment that corporate programmers are used to. It can also be extended by C or C++ modules when speed is of the essence. • ubiquity on computers Python is available on Windows, *nix, and Mac. It runs wherever the Java virtual machine runs, and the reference implementation CPython can help bring Python to wherever there is a working C compiler. • friendly community 144
Chapter 6. Additional Notes
Python Guide Documentation, Release 0.0.1
Python has a vibrant and large community which maintains wikis, conferences, countless repositories, mailing lists, IRC channels, and so much more. Heck, the Python community is even helping to write this guide!
6.1.1 About This Guide Purpose The Hitchhiker’s Guide to Python exists to provide both novice and expert Python developers a best practice handbook for the installation, configuration, and usage of Python on a daily basis. By the Community This guide is architected and maintained by Kenneth Reitz in an open fashion. This is a community-driven effort that serves one purpose: to serve the community. For the Community All contributions to the Guide are welcome, from Pythonistas of all levels. If you think there’s a gap in what the Guide covers, fork the Guide on GitHub and submit a pull request. Contributions are welcome from everyone, whether they’re an old hand or a first-time Pythonista, and the authors to the Guide will gladly help if you have any questions about the appropriateness, completeness, or accuracy of a contribution. To get started working on The Hitchhiker’s Guide, see the Contribute page.
6.1. Introduction
145
Python Guide Documentation, Release 0.0.1
6.2 The Community
6.2.1 BDFL Guido van Rossum, the creator of Python, is often referred to as the BDFL — the Benevolent Dictator For Life.
6.2.2 Python Software Foundation The mission of the Python Software Foundation is to promote, protect, and advance the Python programming language, and to support and facilitate the growth of a diverse and international community of Python programmers. Learn More about the PSF.
6.2.3 PEPs PEPs are Python Enhancement Proposals. They describe changes to Python itself, or the standards around it. There are three different types of PEPs (as defined by PEP 1): Standards Describes a new feature or implementation. Informational Describes a design issue, general guidelines, or information to the community. Process Describes a process related to Python.
146
Chapter 6. Additional Notes
Python Guide Documentation, Release 0.0.1
Notable PEPs There are a few PEPs that could be considered required reading: • PEP 8: The Python Style Guide. Read this. All of it. Follow it. • PEP 20: The Zen of Python. A list of 19 statements that briefly explain the philosophy behind Python. • PEP 257: Docstring Conventions. Gives guidelines for semantics and conventions associated with Python docstrings. You can read more at The PEP Index. Submitting a PEP PEPs are peer-reviewed and accepted/rejected after much discussion. Anyone can write and submit a PEP for review. Here’s an overview of the PEP acceptance workflow:
6.2.4 Python Conferences The major events for the Python community are developer conferences. The two most notable conferences are PyCon, which is held in the US, and its European sibling, EuroPython. A comprehensive list of conferences is maintained at pycon.org.
6.2.5 Python User Groups User Groups are where a bunch of Python developers meet to present or talk about Python topics of interest. A list of local user groups is maintained at the Python Software Foundation Wiki.
6.2.6 Online Communities PythonistaCafe is an invite-only, online community of Python and software development enthusiasts helping each other succeed and grow. Think of it as a club of mutual improvement for Pythonistas where a broad range of programming questions, career advice, and other topics are discussed every day. 6.2. The Community
147
Python Guide Documentation, Release 0.0.1
6.2.7 Python Job Boards Python Jobs HQ is a Python job board, by Python Developers for Python Developers. The site aggregates Python job postings from across the web and also allows employers to post Python job openings directly on the site.
6.3 Learning Python
6.3.1 Beginner The Python Tutorial This is the official tutorial. It covers all the basics, and offers a tour of the language and the standard library. Recommended for those who need a quick-start guide to the language. The Python Tutorial Real Python Real Python is a repository of free and in-depth Python tutorials created by a diverse team of professional Python developers. At Real Python you can learn all things Python from the ground up. Everything from the absolute basics of Python, to web development and web scraping, to data visualization, and beyond. Real Python
148
Chapter 6. Additional Notes
Python Guide Documentation, Release 0.0.1
Python Basics pythonbasics.org is an introductiory tutorial for beginners. The tutorial includes exercises. It covers the basics and there are also in-depth lessons like object oriented programming and regular expressions. Python basics Python for Beginners thepythonguru.com is a tutorial focused on beginner programmers. It covers many Python concepts in depth. It also teaches you some advanced constructs of Python like lambda expressions and regular expressions. And last it finishes off with the tutorial “How to access MySQL db using Python” Python for Beginners Learn Python Interactive Tutorial Learnpython.org is an easy non-intimidating way to get introduced to Python. The website takes the same approach used on the popular Try Ruby website. It has an interactive Python interpreter built into the site that allows you to go through the lessons without having to install Python locally. Learn Python Python for You and Me If you want a more traditional book, Python For You and Me is an excellent resource for learning all aspects of the language. Python for You and Me Learn Python Step by Step Techbeamers.com provides step-by-step tutorials to teach Python. Each tutorial is supplemented with logically added coding snippets and equips with a follow-up quiz on the subject learned. There is a section for Python interview questions to help job seekers. You can also read essential Python tips and learn best coding practices for writing quality code. Here, you’ll get the right platform to learn Python quickly. Learn Python Basic to Advanced Online Python Tutor Online Python Tutor gives you a visual step-by-step representation of how your program runs. Python Tutor helps people overcome a fundamental barrier to learning programming by understanding what happens as the computer executes each line of a program’s source code. Online Python Tutor Invent Your Own Computer Games with Python This beginner’s book is for those with no programming experience at all. Each chapter has the source code to a small game, using these example programs to demonstrate programming concepts to give the reader an idea of what programs “look like”.
6.3. Learning Python
149
Python Guide Documentation, Release 0.0.1
Invent Your Own Computer Games with Python Hacking Secret Ciphers with Python This book teaches Python programming and basic cryptography for absolute beginners. The chapters provide the source code for various ciphers, as well as programs that can break them. Hacking Secret Ciphers with Python Learn Python the Hard Way This is an excellent beginner programmer’s guide to Python. It covers “hello world” from the console to the web. Learn Python the Hard Way Crash into Python Also known as Python for Programmers with 3 Hours, this guide gives experienced developers from other languages a crash course on Python. Crash into Python Dive Into Python 3 Dive Into Python 3 is a good book for those ready to jump in to Python 3. It’s a good read if you are moving from Python 2 to 3 or if you already have some experience programming in another language. Dive Into Python 3 Think Python: How to Think Like a Computer Scientist Think Python attempts to give an introduction to basic concepts in computer science through the use of the Python language. The focus was to create a book with plenty of exercises, minimal jargon, and a section in each chapter devoted to the subject of debugging. While exploring the various features available in the Python language the author weaves in various design patterns and best practices. The book also includes several case studies which have the reader explore the topics discussed in the book in greater detail by applying those topics to real-world examples. Case studies include assignments in GUI programming and Markov Analysis. Think Python Python Koans Python Koans is a port of Edgecase’s Ruby Koans. It uses a test-driven approach to provide an interactive tutorial teaching basic Python concepts. By fixing assertion statements that fail in a test script, this provides sequential steps to learning Python. For those used to languages and figuring out puzzles on their own, this can be a fun, attractive option. For those new to Python and programming, having an additional resource or reference will be helpful. Python Koans
150
Chapter 6. Additional Notes
Python Guide Documentation, Release 0.0.1
More information about test driven development can be found at these resources: Test Driven Development A Byte of Python A free introductory book that teaches Python at the beginner level, it assumes no previous programming experience. A Byte of Python for Python 2.x A Byte of Python for Python 3.x Learn to Program in Python with Codeacademy A Codeacademy course for the absolute Python beginner. This free and interactive course provides and teaches the basics (and beyond) of Python programming while testing the user’s knowledge in between progress. This course also features a built-in interpreter for receiving instant feedback on your learning. Learn to Program in Python with Codeacademy Code the blocks Code the blocks provides free and interactive Python tutorials for beginners. It combines Python programming with a 3D environment where you “place blocks” and construct structures. The tutorials teach you how to use Python to create progressively more elaborate 3D structures, making the process of learning Python fun and engaging. Code the blocks
6.3.2 Intermediate Python Tricks: The Book Discover Python’s best practices with simple examples and start writing even more beautiful + Pythonic code. Python Tricks: The Book shows you exactly how. You’ll master intermediate and advanced-level features in Python with practical examples and a clear narrative. Python Tricks: The Book Effective Python This book contains 59 specific ways to improve writing Pythonic code. At 227 pages, it is a very brief overview of some of the most common adapations programmers need to make to become efficient intermediate level Python programmers. Effective Python
6.3.3 Advanced Pro Python This book is for intermediate to advanced Python programmers who are looking to understand how and why Python works the way it does and how they can take their code to the next level. Pro Python
6.3. Learning Python
151
Python Guide Documentation, Release 0.0.1
Expert Python Programming Expert Python Programming deals with best practices in programming Python and is focused on the more advanced crowd. It starts with topics like decorators (with caching, proxy, and context manager case studies), method resolution order, using super() and meta-programming, and general PEP 8 best practices. It has a detailed, multi-chapter case study on writing and releasing a package and eventually an application, including a chapter on using zc.buildout. Later chapters detail best practices such as writing documentation, test-driven development, version control, optimization, and profiling. Expert Python Programming A Guide to Python’s Magic Methods This is a collection of blog posts by Rafe Kettler which explain ‘magic methods’ in Python. Magic methods are surrounded by double underscores (i.e. __init__) and can make classes and objects behave in different and magical ways. A Guide to Python’s Magic Methods Note: Rafekettler.com is currently down; you can go to their GitHub version directly. Here you can find a PDF version: A Guide to Python’s Magic Methods (repo on GitHub)
6.3.4 For Engineers and Scientists A Primer on Scientific Programming with Python A Primer on Scientific Programming with Python, written by Hans Petter Langtangen, mainly covers Python’s usage in the scientific field. In the book, examples are chosen from mathematics and the natural sciences. A Primer on Scientific Programming with Python Numerical Methods in Engineering with Python Numerical Methods in Engineering with Python, written by Jaan Kiusalaas, puts the emphasis on numerical methods and how to implement them in Python. Numerical Methods in Engineering with Python
6.3.5 Miscellaneous Topics Problem Solving with Algorithms and Data Structures Problem Solving with Algorithms and Data Structures covers a range of data structures and algorithms. All concepts are illustrated with Python code along with interactive samples that can be run directly in the browser. Problem Solving with Algorithms and Data Structures
152
Chapter 6. Additional Notes
Python Guide Documentation, Release 0.0.1
Programming Collective Intelligence Programming Collective Intelligence introduces a wide array of basic machine learning and data mining methods. The exposition is not very mathematically formal, but rather focuses on explaining the underlying intuition and shows how to implement the algorithms in Python. Programming Collective Intelligence Transforming Code into Beautiful, Idiomatic Python Transforming Code into Beautiful, Idiomatic Python is a video by Raymond Hettinger. Learn to take better advantage of Python’s best features and improve existing code through a series of code transformations: “When you see this, do that instead.” Transforming Code into Beautiful, Idiomatic Python Fullstack Python Fullstack Python offers a complete top-to-bottom resource for web development using Python. From setting up the web server, to designing the front-end, choosing a database, optimizing/scaling, etc. As the name suggests, it covers everything you need to build and run a complete web app from scratch. Fullstack Python PythonistaCafe PythonistaCafe is an invite-only, online community of Python and software development enthusiasts helping each other succeed and grow. Think of it as a club of mutual improvement for Pythonistas where a broad range of programming questions, career advice, and other topics are discussed every day. PythonistaCafe
6.3.6 References Python in a Nutshell Python in a Nutshell, written by Alex Martelli, covers most cross-platform Python usage, from its syntax to built-in libraries to advanced topics such as writing C extensions. Python in a Nutshell The Python Language Reference This is Python’s reference manual. It covers the syntax and the core semantics of the language. The Python Language Reference
6.3. Learning Python
153
Python Guide Documentation, Release 0.0.1
Python Essential Reference Python Essential Reference, written by David Beazley, is the definitive reference guide to Python. It concisely explains both the core language and the most essential parts of the standard library. It covers Python 3 and 2.6 versions. Python Essential Reference Python Pocket Reference Python Pocket Reference, written by Mark Lutz, is an easy to use reference to the core language, with descriptions of commonly used modules and toolkits. It covers Python 3 and 2.6 versions. Python Pocket Reference Python Cookbook Python Cookbook, written by David Beazley and Brian K. Jones, is packed with practical recipes. This book covers the core Python language as well as tasks common to a wide variety of application domains. Python Cookbook Writing Idiomatic Python Writing Idiomatic Python, written by Jeff Knupp, contains the most common and important Python idioms in a format that maximizes identification and understanding. Each idiom is presented as a recommendation of a way to write some commonly used piece of code, followed by an explanation of why the idiom is important. It also contains two code samples for each idiom: the “Harmful” way to write it and the “Idiomatic” way. For Python 2.7.3+ For Python 3.3+
154
Chapter 6. Additional Notes
Python Guide Documentation, Release 0.0.1
6.4 Documentation
6.4.1 Official Documentation The official Python Language and Library documentation can be found here: • Python 2.x • Python 3.x
6.4.2 Read the Docs Read the Docs is a popular community project that hosts documentation for open source software. It holds documentation for many Python modules, both popular and exotic. Read the Docs
6.4.3 pydoc pydoc is a utility that is installed when you install Python. It allows you to quickly retrieve and search for documentation from your shell. For example, if you needed a quick refresher on the time module, pulling up documentation would be as simple as: $ pydoc time
6.4. Documentation
155
Python Guide Documentation, Release 0.0.1
The above command is essentially equivalent to opening the Python REPL and running: >>> help(time)
6.5 News
6.5.1 PyCoder’s Weekly PyCoder’s Weekly is a free weekly Python newsletter for Python developers by Python developers (Projects, Articles, News, and Jobs). PyCoder’s Weekly
6.5.2 Real Python At Real Python you can learn all things Python from the ground up, with weekly free and in-depth tutorials. Everything from the absolute basics of Python, to web development and web scraping, to data visualization, and beyond. Real Python
6.5.3 Planet Python This is an aggregate of Python news from a growing number of developers. 156
Chapter 6. Additional Notes
Python Guide Documentation, Release 0.0.1
Planet Python
6.5.4 /r/python /r/python is the Reddit Python community where users contribute and vote on Python-related news. /r/python
6.5.5 Talk Python Podcast The #1 Python-focused podcast covering the people and ideas in Python. Talk Python To Me
6.5.6 Python Bytes Podcast A short-form Python podcast covering recent developer headlines. Python Bytes
6.5.7 Python Weekly Python Weekly is a free weekly newsletter featuring curated news, articles, new releases, jobs, etc. related to Python. Python Weekly
6.5.8 Python News Python News is the news section in the official Python web site (www.python.org). It briefly highlights the news from the Python community. Python News
6.5.9 Import Python Weekly Weekly Python Newsletter containing Python Articles, Projects, Videos, and Tweets delivered in your inbox. Keep Your Python Programming Skills Updated. Import Python Weekly Newsletter
6.5.10 Awesome Python Newsletter A weekly overview of the most popular Python news, articles, and packages. Awesome Python Newsletter Note: Notes defined within all diatonic and chromatic musical scales have been intentionally excluded from this list of additional notes. Additionally, this note.
Contribution notes and legal information (for those interested). 6.5. News
157
Python Guide Documentation, Release 0.0.1
6.6 Contribute
Python-guide is under active development, and contributors are welcome. If you have a feature request, suggestion, or bug report, please open a new issue on GitHub. To submit patches, please send a pull request on GitHub. Once your changes get merged back in, you’ll automatically be added to the Contributors List.
6.6.1 Style Guide For all contributions, please follow the The Guide Style Guide.
6.6.2 Todo List If you’d like to contribute, there’s plenty to do. Here’s a short todo list. • Establish “use this” vs “alternatives are. . . .” recommendations Todo: Include code examples of exemplary code from each of the projects listed. Explain why it is excellent code. Use complex examples. (The original entry is located in /home/docs/checkouts/readthedocs.org/user_builds/pythonguide/checkouts/latest/docs/writing/reading.rst, line 50.)
158
Chapter 6. Additional Notes
Python Guide Documentation, Release 0.0.1
Todo: Explain techniques to rapidly identify data structures and algorithms and determine what the code is doing. (The original entry is located in /home/docs/checkouts/readthedocs.org/user_builds/pythonguide/checkouts/latest/docs/writing/reading.rst, line 52.) Todo: Write about Blueprint (The original entry is located in /home/docs/checkouts/readthedocs.org/user_builds/pythonguide/checkouts/latest/docs/scenarios/admin.rst, line 386.) Todo: Fill in “Freezing Your Code” stub (The original entry is located in /home/docs/checkouts/readthedocs.org/user_builds/pythonguide/checkouts/latest/docs/shipping/freezing.rst, line 42.)
6.7 License
The Guide is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported license.
6.7. License
159
Python Guide Documentation, Release 0.0.1
6.8 The Guide Style Guide
As with all documentation, having a consistent format helps make the document more understandable. In order to make The Guide easier to digest, all contributions should fit within the rules of this style guide where appropriate. The Guide is written as reStructuredText. Note: Parts of The Guide may not yet match this style guide. Feel free to update those parts to be in sync with The Guide Style Guide
Note: On any page of the rendered HTML you can click “Show Source” to see how authors have styled the page.
6.8.1 Relevancy Strive to keep any contributions relevant to the purpose of The Guide. • Avoid including too much information on subjects that don’t directly relate to Python development. • Prefer to link to other sources if the information is already out there. Be sure to describe what and why you are linking. • Cite references where needed. • If a subject isn’t directly relevant to Python, but useful in conjunction with Python (e.g., Git, GitHub, Databases), reference by linking to useful resources, and describe why it’s useful to Python. 160
Chapter 6. Additional Notes
Python Guide Documentation, Release 0.0.1
• When in doubt, ask.
6.8.2 Headings Use the following styles for headings. Chapter title: ######### Chapter 1 #########
Page title: =================== Time is an Illusion ===================
Section headings: Lunchtime Doubly So -------------------
Sub section headings: Very Deep ~~~~~~~~~
6.8.3 Prose Wrap text lines at 78 characters. Where necessary, lines may exceed 78 characters, especially if wrapping would make the source text more difficult to read. Use Standard American English, not British English. Use of the serial comma (also known as the Oxford comma) is 100% non-optional. Any attempt to submit content with a missing serial comma will result in permanent banishment from this project, due to complete and total lack of taste. Banishment? Is this a joke? Hopefully we will never have to find out.
6.8.4 Code Examples Wrap all code examples at 70 characters to avoid horizontal scrollbars. Command line examples: .. code-block:: console $ run command --help $ ls ..
Be sure to include the $ prefix before each line. Python interpreter examples:
6.8. The Guide Style Guide
161
Python Guide Documentation, Release 0.0.1
Label the example:: .. code-block:: python >>> import this
Python examples: Descriptive title:: .. code-block:: python def get_answer(): return 42
6.8.5 Externally Linking • Prefer labels for well known subjects (e.g. proper nouns) when linking: Sphinx_ is used to document Python. .. _Sphinx: http://sphinx.pocoo.org
• Prefer to use descriptive labels with inline links instead of leaving bare links: Read the `Sphinx Tutorial `_
• Avoid using labels such as “click here”, “this”, etc., preferring descriptive labels (SEO worthy) instead.
6.8.6 Linking to Sections in The Guide To cross-reference other parts of this documentation, use the :ref: keyword and labels. To make reference labels more clear and unique, always add a -ref suffix: .. _some-section-ref: Some Section ------------
6.8.7 Notes and Warnings Make use of the appropriate admonitions directives when making notes. Notes: .. note:: The Hitchhiker’s Guide to the Galaxy has a few things to say on the subject of towels. A towel, it says, is about the most massively useful thing an interstellar hitch hiker can have.
Warnings:
162
Chapter 6. Additional Notes
Python Guide Documentation, Release 0.0.1
.. warning:: DON'T PANIC
6.8.8 TODOs Please mark any incomplete areas of The Guide with a todo directive. To avoid cluttering the Todo List, use a single todo for stub documents or large incomplete sections. .. todo:: Learn the Ultimate Answer to the Ultimate Question of Life, The Universe, and Everything
6.8. The Guide Style Guide
163
Python Guide Documentation, Release 0.0.1
164
Chapter 6. Additional Notes
Index
E environment variable PATH, 8, 14, 16
P PATH, 8, 14, 16 Python Enhancement Proposals PEP 0257#specification, 62 PEP 1, 146 PEP 20, 53, 147 PEP 249, 96 PEP 257, 63, 147 PEP 282, 69 PEP 3101, 46 PEP 3132, 51 PEP 3333, 81 PEP 391, 71 PEP 8, 26, 53, 147, 152 PEP 8#comments, 62
165