Digging into pytest's history

[The-Compiler]

I've always wondered when pytest actually really was born - the first commit in this repository (5992a8e) is from January 2007, and even that commit alone already tells a lot: This repository originally was from the py library (later split off to pytest), and it originally was a SVN revision, migrated to Mercurial, and finally migrated to git.

However, the commit says "create the new development trunk" and is already quite big: 435 files changed, 58640 insertions(+).

So this week, I dug around in various other repositories and mailinglists to find out how it all started. As you might or might not know, pytest originally was born as part of PyPy, to make it easier to write tests for it. Here's what I found it with a bit of digging around:

---

• Late 2002 / early 2003, PyPy was born.
• Like that blog post mentioned, from very early on, there was a big focus on testing. There were various testsupport files on top of unittest.py, and as early as June 2003, Holger Krekel (@hpk42) refactored its test framework to clean things up (pypy.tool.test, but still on top of unittest.py, with nothing pytest-like yet, as far as I can tell).
• In December 2003, there was another iteration at improving their testing situation, by Stefan Schwarzer, called pypy.tool.newtest.
• However, it didn't seem to be around for long, as around June/July 2004, efforts started on a thing called utest, offering plain assertions. This seems like the start of something pytest-like, but unfortunately, I didn't actually find the test runner's code anywhere - the best I could find is this file, but that doesn't seem like a complete test runner at all. What I can see is that there were various efforts by Laura Creighton and Samuele Pedroni (@pedronis) at automatically converting existing tests to the new utest framework.
• Around the same time, for Europython 2004, @hpk42 started a project originally called "std", intended to be a "complementary standard library" - already laying out the principles behind what later became pytest:

• current "batteries included" are very useful, but
  • some of them are written in a pretty much java-like style, especially the unittest-framework
  • [...]
  • the best API is one that doesn't exist

[...]

• a testing package should require as few boilerplate code as possible and offer much flexibility
• it should provide premium quality tracebacks and debugging aid

[...]

• first of all ... forget about limited "assertXYZ APIs"
  and use the real thing, e.g.:

        assert x == y
  

• this works with plain python but you get unhelpful "assertion failed" errors with no information

• std.utest (magic!) actually reinterprets the assertion expression and offers detailed information about underlying values

• In September 2004, the py-dev mailinglist gets born, which is now pytest-dev, but thankfully with all the original archives still intact.

• Around September/October 2004, the std project was renamed to py, and std.utest became py.test. This is also the first time I actually found the entire source code, and I was surprised how much of the API still seems around today:

  • py.path.local, which we're trying to get rid of (in favour of pathlib) some 16-17 years later
  • The idea of the collection tree, including Collector, FSCollector, Directory, PyCollector, Module, Class
  • Arguments like -x / --exitfirst, -l / --showlocals, --fulltrace, --pdb, -S / --nocapture (-s / --capture=off today), --collectonly (--collect-only today)

• In the same month, the py library gets split off from PyPy

• It seemed to get rather quiet for a while, and I can't really find much between October 2004 (removing py from PyPy) and January 2007 (first commit in the now-pytest repository). However, there were various discussions about features/ideas on the mailinglist, and a couple of releases every couple of months:

  • March 2006: py 0.8.0-alpha2
  • May 2007: py 0.9.0
  • March 2008: py 0.9.1 (first release to be found in the pytest changelog!)
  • August 2008: py 0.9.2

• In August 2009, py 1.0.0 was released, introducing a lot of fundamental features:

  • funcargs/fixtures
  • A plugin architecture which still looks very much the same today!
  • Various default plugins, including monkeypatch

• Even back there, the FAQ said:

Clearly, [a second standard library] was ambitious and the naming has maybe haunted the project rather than helping it. There may be a project name change and possibly a split up into different projects sometime.

and that finally happened in November 2010, when pytest 2.0.0 was released as a package separate from py (but still called py.test).

• In August 2016, pytest 3.0.0 was released, which adds pytest (rather than py.test) as the recommended command-line entry point

---

So, do I have an answer to my question "when was pytest started" now? Well, yes and no. It depends what point you really regard as the start of it all - I'd pick Europython 2004, i.e. around June/July 2004 - which means pytest is turning 17 soon, with many of its core concepts still being around! 🎉

If anyone finds something I've missed (e.g. about what happened between mid-2004 and mid-2007), I'd be more than happy to hear about it!

[The-Compiler]

PR for adding this to the docs: #8784