Sphinx 0.2 released

After a few weeks' bug-fixing releases, I've now released Sphinx 0.2. The list of changes is here – most are designed to make Sphinx more flexible to meet your documentation needs too.

Download it from Cheese ShopPyPI as usual.

German Python 3000 talk

If someone is interested, here are some slides of a (German) talk about Python 3000 I gave at the Munich User Group.

AST compilation from Python

Thanks to Thomas Lee, we have now in the Python trunk a built-in way to modify the Abstract Syntax Tree compiled from a piece of Python source, and to compile that to an executable code object.

Quick example:

>>> import _ast
>>> # A dumb source snippet
>>> source = "print 5 + 8"
>>> # Compile using the special PyCF_ONLY_AST flag to get an AST
>>> ast = compile(source, "<string>", "exec", _ast.PyCF_ONLY_AST)
>>> # The toplevel node is always a Module for "exec"
>>> ast
<_ast.Module object at 0xb7b00edc>
>>> # Each AST node has different attributes (see the docs)
>>> ast.body
[<_ast.Print object at 0xb7b020a4>]
>>> # The Print node has a list of values -- expressions to print
>>> ast.body[0].values
[<_ast.BinOp object at 0xb7b02114>]
>>> # This is the addition operator between 5 and 8
>>> ast.body[0].values[0].op
<_ast.Add object at 0xb7c98504>
>>> # Now, replace the operator with subtraction ...
>>> ast.body[0].values[0].op = _ast.Sub()
>>> # ... and compile the resulting AST
>>> code = compile(ast, "<string>", "exec")
>>> # Voila:
>>> exec code
-3

From this, you should be able to work out how to write AST manipulation code that actually makes sense :)

The _ast documentation gives you an idea which classes and attributes are available for individual AST elements.

Using Pygments with less

The de-facto standard UNIX pager less supports an environment variable called LESSOPEN that can be set to the name of an input preprocessor. It is normally used to transparently view compressed files etc., but of course it can also colorize your source files using Pygments!

If you use Gentoo Linux, most of the work needed to set this up has already been done for you -- you just need to set

export LESSOPEN="|lesspipe.sh %s"
export LESSCOLORIZER=pygmentize

and make sure your LESS variable contains -r or -R so that the raw ANSI color codes are passed through by less. Gentoo's lesspipe.sh script will then automatically call Pygments for source code files.

On other platforms, you can set up a lesspipe.sh script yourself; it should look roughly like this:

#!/bin/sh
case "$1" in
   # add all extensions you want to handle here
   *.awk|*.groff|*.java|*.js|*.m4|*.php|*.pl|*.pm|*.pod|*.sh|\
   *.ad[asb]|*.asm|*.inc|*.[ch]|*.[ch]pp|*.[ch]xx|*.cc|*.hh|\
   *.lsp|*.l|*.pas|*.p|*.xml|*.xps|*.xsl|*.axp|*.ppd|*.pov|\
   *.diff|*.patch|*.py|*.rb|*.sql|*.ebuild|*.eclass)
      pygmentize "$1" ;;
   *) exit 0;;
esac

Then export LESSOPEN="|lesspipe.sh %s" and enjoy colored viewing!

Interesting facts about the Linux file system

Sometimes you get lucky. (Warning: dramatic events ahead.)

I was playing a large video file with mplayer when I made the mistake to delete it because of a command line accident (note to self: relying on the shell history to do what you mean is evil). First I thought "nice, now I can download the whole 500 MB again", but I realized that the file must still be available somewhere on disk because mplayer still needs to access it.

Helpful people from IRC pointed me to /proc/pid/fd/ which is a directory with symlinks to open files, but didn't believe it would do any good because the symlink's target is already gone.

Not one to give up easily, I looked into there, and the listing looked like that:

$ ls -l /proc/pid/fd
total 0
dr-x------ 2 gbr users  0 Mar 25 21:28 .
dr-xr-xr-x 5 gbr users  0 Mar 25 21:27 ..
lrwx------ 1 gbr users 64 Mar 25 21:28 0 -> /dev/pts/7
lrwx------ 1 gbr users 64 Mar 25 21:28 1 -> /dev/pts/7
lrwx------ 1 gbr users 64 Mar 25 21:28 2 -> /dev/pts/7
lr-x------ 1 gbr users 64 Mar 25 21:28 3 -> /home/gbr/video.mov (deleted)
...

Curious. Not the usual "broken symlink" view. And indeed, I could open the file and copy its contents somewhere safe. Isn't that nice?

Sphinx, take 2

I've just uploaded a new Sphinx release to PyPI. It should now be compatible with docutils SVN snapshots, as opposed to only docutils 0.4. If you still find problems with some SVN version, let me know!

Also new are some improvements in the doctest and autodoc extensions, see the CHANGES file.

Sphinx is released!

I'm delighted to announce that the Sphinx library, used to build the new Python documentation (for 2.6 and 3.0), is now released for general use.

What is it?

Sphinx is a tool that makes it easy to create intelligent and beautiful documentation for Python projects (or other documents consisting of multiple reStructuredText source files).

Its website is here.

What does it do?

Sphinx is not an API documentation generator like Epydoc. Instead, its focus is on hand-written documentation, such as the Python one.

• Main output formats: HTML (including HTML Help) and LaTeX
• Extensive cross-references: semantic markup and automatic links for functions, classes, glossary terms and similar pieces of information
• Hierarchical structure: easy definition of a document tree, with automatic links to siblings, parents and children
• Automatic indices: general index as well as a module index
• Code handling: automatic highlighting using the Pygments highlighter
• Goodies such as changes overview, and external link checking

What else?

Various extensions are available and in development:

• autodoc: pulls in documentation from docstrings that are written in reST, to avoid having to maintain multiple documentation locations
• doctest: automatically tests snippets in the documentation in doctest fashion
• coverage: documentation coverage checker

I'd like to thank everyone who has given it a try so far and provided me with valuable suggestions and patches: Uli Fouquet, Andre Roberge, Armin Ronacher,  Tim Golden and Mark Summerfield.

Now back to documenting new 2.6 features...