Python打包、安装与发布工具--setuptools

Python中的setuptools工具不仅仅是简化了distutils 的setup.py文件的编写，更重要的是它的包管理能力方面的增强。它可以使用一种更加透明的方法来查找、下载并安装依赖包；并可以在一个包的多个版本中自由进行切换，这些版本都安装在同一个系统上；也可以声明对某个包的特定版本的需求；还可以只使用一个简单的命令就能更新到某个包的最新版本。说白了，这个和java中的Maven，以及在centos中使用的Yum有相似的地方；下面，先对以下几个名词作一个解释：

1.distutils

distutils
(1)Part of the Python standard library since version 1.6(自1.6版本，加入python标准库)
(2)The standard way of building and installing packages（构建、安装包的标准方法）
(3)Primary functionality in distutils.core
(4) Developer or packager creates setup.py

#!/usr/bin/env python
from distutils.core import setup
setup (name = “foo”,
version = “1.0”,
py_modules = [“foo”])
$ python setup.py sdist (create a source distribution)
$ python setup.py bdist (create a build distribution)
$ python setup.py install (install using defaults)
* Other --install-* options (remember to update $PYTHONPATH)

 

setup中的参数说明：

 

setup (

arguments )

The basic do-everything function that does most everything you could ever ask for from a Distutils method. See XXXXX

The setup function takes a large number of arguments. These are laid out in the following table.

 

argument name value type

name	The name of the package	a string
version	The version number of the package	See distutils.version
description	A single line describing the package	a string
long_description	Longer description of the package	a string
author	The name of the package author	a string
author_email	The email address of the package author	a string
maintainer	The name of the current maintainer, if different from the author	a string
maintainer_email	The email address of the current maintainer, if different from the author
url	A URL for the package (homepage)	a URL
download_url	A URL to download the package	a URL
packages	A list of Python packages that distutils will manipulate	a list of strings
py _modules	A list of Python modules that distutils will manipulate	a list of strings
scripts	A list of standalone script files to be built and installed	a list of strings
ext_modules	A list of Python extensions to be built	A list of instances of distutils.core.Extension
classifiers	A list of categories for the package	The list of available categorizations is at http://cheeseshop.python.org/pypi?:action=list_classifiers .
distclass	the Distribution class to use	A subclass of distutils.core.Distribution
script_name	The name of the setup.py script - defaults to sys.argv[0]	a string
script_args	Arguments to supply to the setup script	a list of strings
options	default options for the setup script	a string
license	The license for the package
keywords	Descriptive meta-data. See PEP 314
platforms
cmdclass	A mapping of command names to Command subclasses	a dictionary

2.setuptools
1） A collection of enhancements to the Python distutils package that allow one to more easily build and
distribute python packages（对distutils包的增强，使用setuptools可以更加方便的构建、发布python packages）
2）Additional set of keyword arguments to setup()（在distutils.core.setup的基础上,添加了一些新的参数）
3）Includes easy_install.py （包括一个easy_install.py工具，提供类似centos中的yum安装工具）
4） Creates eggs (.egg)（关于egg,最好的解释还是：‘Eggs are to Pythons as Jars are to Java…‘）
5）Features for developers (e.g. support for data files,MANIFEST, Pyrex, PyPI upload,…)
6）Ability to deploy project in “development mode” via setup.py develop command.

3. egg文件

Eggs

“Eggs are to Pythons as Jars are to Java…” ------------Phillip J. Eby
(1)Single-file importable distribution format .(这句话的意思是说，python中导入挂钩的更改，只要把egg加入到PYTHONPATH或者sys.path中，就可以像往常一样导入)
(2)Eggs are Zipfiles using the .egg extension, that support including data and C extensions as well as Python code

(egg也不一定是zipfile,setup的参数中有一个zip_safe参数,在程序打包时决定是否压缩，如果不指定，bdist_egg会对工程中的每一个文件作检查，具体的解释如下：)

zip_safe

A boolean (True or False) flag specifying whether the project can be safely installed and run from a zip file. If this argument is not supplied, the bdist_egg command will have to analyze all of your project's contents for possible problems each time it buids an egg.

（关于egg中是否会打包.py以外的数据、文件，在setup中有3个参数控制，具体如下：）

include_package_data

If set to True , this tells setuptools to automatically include any data files it finds inside your package directories, that are either under CVS or Subversion control, or which are specified by your MANIFEST.in file. For more information, see the section below on Including Data Files .

exclude_package_data

A dictionary mapping package names to lists of glob patterns that should be excluded from your package directories. You can use this to trim back any excess files included by include_package_data . For a complete description and examples, see the section below on Including Data Files .

package_data

A dictionary mapping package names to lists of glob patterns. For a complete description and examples, see the section below on Including Data Files . You do not need to use this option if you are using include_package_data , unless you need to add e.g. files that are generated by your setup script and build process. (And are therefore not in source control or are files that you don't want to include in your source distribution.)

这里有一个例子：

Including Data Files

The distutils have traditionally allowed installation of "data files", which are placed in a platform-specific location. However, the most common use case for data files distributed with a package is for use by the package, usually by including the data files in the package directory.

Setuptools offers three ways to specify data files to be included in your packages. First, you can simply use the include_package_data keyword, e.g.:

from setuptools import setup, find_packages
setup(
    ...
    include_package_data = True
)

This tells setuptools to install any data files it finds in your packages. The data files must be under CVS or Subversion control, or else they must be specified via the distutils' MANIFEST.in file. (They can also be tracked by another revision control system, using an appropriate plugin. See the section below on Adding Support for Other Revision Control Systems for information on how to write such plugins.)

If you want finer-grained control over what files are included (for example, if you have documentation files in your package directories and want to exclude them from installation), then you can also use the package_data keyword, e.g.:

from setuptools import setup, find_packages
setup(
    ...
    package_data = {
        # If any package contains *.txt or *.rst files, include them:
        '': ['*.txt', '*.rst'],
        # And include any *.msg files found in the 'hello' package, too:
        'hello': ['*.msg'],
    }
)

The package_data argument is a dictionary that maps from package names to lists of glob patterns. The globs may include subdirectory names, if the data files are contained in a subdirectory of the package. For example, if the package tree looks like this:

setup.py
src/
    mypkg/
        __init__.py
        mypkg.txt
        data/
            somefile.dat
            otherdata.dat

The setuptools setup file might look like this:

from setuptools import setup, find_packages
setup(
    ...
    packages = find_packages('src'),  # include all packages under src
    package_dir = {'':'src'},   # tell distutils packages are under src

    package_data = {
        # If any package contains *.txt files, include them:
        '': ['*.txt'],
        # And include any *.dat files found in the 'data' subdirectory
        # of the 'mypkg' package, also:
        'mypkg': ['data/*.dat'],
    }
)

Notice that if you list patterns in package_data under the empty string, these patterns are used to find files in every package, even ones that also have their own patterns listed. Thus, in the above example, the mypkg.txt file gets included even though it's not listed in the patterns for mypkg .

Also notice that if you use paths, you must use a forward slash (/ ) as the path separator, even if you are on Windows. Setuptools automatically converts slashes to appropriate platform-specific separators at build time.

(Note: although the package_data argument was previously only available in setuptools , it was also added to the Python distutils package as of Python 2.4; there is some documentation for the feature available on the python.org website.)

Sometimes, the include_package_data or package_data options alone aren't sufficient to precisely define what files you want included. For example, you may want to include package README files in your revision control system and source distributions, but exclude them from being installed. So, setuptools offers an exclude_package_data option as well, that allows you to do things like this:

from setuptools import setup, find_packages
setup(
    ...
    packages = find_packages('src'),  # include all packages under src
    package_dir = {'':'src'},   # tell distutils packages are under src

    include_package_data = True,    # include everything in source control

    # ...but exclude README.txt from all packages
    exclude_package_data = { '': ['README.txt'] },
)

The exclude_package_data option is a dictionary mapping package names to lists of wildcard patterns, just like the package_data option. And, just as with that option, a key of '' will apply the given pattern(s) to all packages. However, any files that match these patterns will be excluded from installation, even if they were listed in package_data or were included as a result of using include_package_data .

In summary, the three options allow you to:

include_package_data

Accept all data files and directories matched by MANIFEST.in or found in source control.

package_data

Specify additional patterns to match files and directories that may or may not be matched by MANIFEST.in or found in source control.

exclude_package_data

Specify patterns for data files and directories that should not be included when a package is installed, even if they would otherwise have been included due to the use of the preceding options.

NOTE: Due to the way the distutils build process works, a data file that you include in your project and then stop including may be "orphaned" in your project's build directories, requiring you to run setup.py clean --all to fully remove them. This may also be important for your users and contributors if they track intermediate revisions of your project using Subversion; be sure to let them know when you make changes that remove files from inclusion so they can run setup.py clean --all .

(3)Requires Python 2.3 or above

(4)Eggs are built using the setuptools package；eg:

python2.4 setup.py bdist_egg
(5)The published plan is to propose inclusion in the Python 2.5 standard library.(这个计划没有实现)
这个是我在http://www.ibm.com/developerworks/cn/linux/l-cppeak3.html 上节选过来的，说的很好：

 

egg 是一个包含所有包数据的文件包。在理想情况中，egg 是一个使用 zip 压缩的文件，其中包括了所有需要的包文件。但是在某些情况下，setuptools 会决定（或被开关告知）包不应该是 zip 压缩的。在这些情况下，egg 只是一个简单的未曾压缩的子目录，但是里面的内容是相同的。使用单一的版本可以方便地进行转换，并可以节省一点磁盘空间，但是 egg 目录从功能和组织结构上来说都是相同的。一直使用 JAR 文件的 Java™ 技术的用户会发现 egg 非常熟悉。

由于最新的 Python 版本中（需要 2.3.5+ 或 2.4）导入挂钩的更改，可以简单地通过设置 PYTHONPATH 或 sys.path 并像往常一样导入相应的包来使用 egg。如果希望采用这种方法，就不需要使用 setuptools 或 ez_setup.py 了。例如，在本文使用的工作目录中，我就为 PyYAML 包放入了一个 egg。现在我就可以使用这个包了，方法如下：

% export PYTHONPATH=~/work/dW/PyYAML-3.01-py2.4.egg


% python -c 'import yaml; print yaml.dump({"foo":"bar",1:[2,3]})'

1: [2, 3]
foo: bar

(在这个地方，我想说一下，你必须使用过setuptools才会有eazy-install.pth)

不过，PYTHONPATH 的（或者脚本或 Python shell 会话内的 sys.path 的）这种操作有些脆弱。egg 的发现最好是在新一点的 .pth 文件中进行。在 site-packages/ 或 PYTHONPATH 中的任何 .pth 文件都会进行解析来执行其他导入操作，其方法类似于检查可能包含包的那些目录位置一样。如果使用 setuptools 来处理包的管理功能，那么在安装、更新、删除包时，就需要修改一个名为 easy-install.pth 的文件。而且可以按照自己喜欢的方式对这个 .pth 进行命名（只要其扩展名是 .pth 即可）。例如，下面是我的 easy-install.pth 文件的内容：

% cat /sw/lib/python2.4/site-packages/easy-install.pth


import sys; sys.__plen = len(sys.path)
setuptools-0.6b1-py2.4.egg
SQLObject-0.7.0-py2.4.egg
FormEncode-0.5.1-py2.4.egg
Gnosis_Utils-1.2.1-py2.4.egg
import sys; new=sys.path[sys.__plen:]; del sys.path[sys.__plen:];
  p=getattr(sys,'__egginsert',0); sys.path[p:p]=new;
  sys.__egginsert = p+len(new)

这种格式有点特殊：它近似于一个 Python 脚本，但却不完全是。需要说明的是，可以在那里添加额外列出的 egg；更好的情况是，easy_install 会在运行时实现这种功能。也可以在 site-packages/ 下创建任意多个 .pth 文件；每个都可以列出有哪些 egg 是可用的。

Why bother with Eggs?
1）Enable tools like “Easy Install” Python package manager（这个很显然）
 2）They are a “zero installation” format for pure Python packages (put them on PYTHONPATH or sys.path)（上面说的那个.pth）
 3）They can include package metadata (e.g. dependencies)
 4）They allow namespace packages (packages that contain other packages) to be split into separate distributions
 5）They allow applications or libraries to specify the needed version of a library before doing an import (e.g. require(“Twisted-Internet>=2.0”) )（这个涉及到的是setup里install_requires参数）
 6）They provide a framework for plug-ins (similar to Eclipse’s extension point)
 7）Enables one to distribute a project that depends on other software available via PyPI a.k.a. Cheese Shop.(eazy_install或者执行python setup.py install时，会依据install_requires参数自动从http://www.python.org/pypi/ 上寻找规定的版本，并下载安装好这些依赖的包)

4. eazy_install

Easy Install / easy_install.py
1)A Python module bundled with setuptools that lets one automatically build, install, and manage python
packages
2)Part of the setuptools package
3)Installs any distutils-based package
4)Can find packages on PyPI
5)Handles dependencies via arguments to setup() (e.g. install_requires = [‘foo>=1.4’,’Bar’] )

 

 

[1]http://docs.python.org/distutils/index.html

[2]http://peak.telecommunity.com/DevCenter/setuptools

[3]http://peak.telecommunity.com/DevCenter/PythonEggs

[4]http://peak.telecommunity.com/DevCenter/EasyInstall

[5]http://www.ibm.com/developerworks/cn/linux/l-cppeak3.html