Python 3 is here and the sky is not falling

James Bennett, a long-time Python developer, blogger, and contributor to Django, recently wrote a nice post about the “end” of Python 2.x, entitled “Variations on the Death of Python 2.” It’s a great read for anyone who, like me, has been in the Python community a long time.

I’ve been a Python user since the early 2.x days, first discovering Python in a print copy of Linux Journal in the year 2000, where a well-known open source developer and advocate described his transition from Perl to Python. He wrote:

I was generating working code nearly as fast as I could type. When I realized this, I was quite startled.

An important measure of effort in coding is the frequency with which you write something that doesn’t actually match your mental representation of the problem, and have to backtrack on realizing that what you just typed won’t actually tell the language to do what you’re thinking. An important measure of good language design is how rapidly the percentage of missteps of this kind falls as you gain experience with the language. When you’re writing working code nearly as fast as you can type and your misstep rate is near zero, it generally means you’ve achieved mastery of the language.

But that didn’t make sense, because it was still day one and I was regularly pausing to look up new language and library features!

This was my first clue that, in Python, I was actually dealing with an exceptionally good design.

Python’s wonderful design as a language has always been a source of inspiration for me. I even wrote “The Elements of Python Style”, as an ode to how good Python code, to me, felt like good written prose. And, of course, many of my personal and professional projects are proudly Python Powered.

Thus, I was always a little worried about the Python 2 to 3 transition. I was concerned that this one big risk, taken on by the core team, could imperil the entire language, and thus the entire community. Perl 5 had embarked on a language schism toward Perl 6 (now Raku), and many believe that both communities (Perl 5 and Raku) became weaker as a result.

But, here we are in 2020, and Python 2 is EOL, and Python 3 is here to stay. A lot of the internet debates about Python 2 vs Python 3 (like this flame war on lobste.rs) now seem to boil down to this question: was Python 3 a good idea, in retrospect?

Python 3 caused a split in the Python community. It caused confusion about what Python actually is. It also caused a lot of porting pain for many companies, and a decade-long migration effort among major open source libraries.

If you are a relatively recent Python user and have not heard much about the Python 2 vs 3 community-wide migration effort, you can get pointers to some of the history and technical details in this Python.org wiki page. There’s a nice tl;dr summary in this Python 3 Q&A. To understand some of the porting pain involved, a somewhat common 2-to-3 porting approach is covered in the official Python 2 to 3 porting guide.

Regardless of the amount of pain caused, ultimately, Python 3 is here now. It works, it’s popular, and the Python core team has officially made its final Python 2.7 release. The community has survived the transition. To quote the Python release manager in a recent email announcing the release of Python 2.7.18:

Python 2.7.18 is a special release. I refer, of course, to the fact that “2.7.18” is the closest any Python version number will ever approximate e, Euler’s number. Simply exquisite! A less transcendent property of Python 2.7.18 is that it is the last Python 2.7 release and therefore the last Python 2 release. It’s time for the CPython community to say a fond but firm farewell to Python 2. Users still on Python 2 can use e to compute the instantaneously compounding interest on their technical debt.

Ubuntu 20.04 LTS — the “Long-Term Support” April 2020 release of one of the world’s most popular desktop and server Linux distributions — includes Python 3.8.2 by default, and includes Python 2.7.18 only optionally (via the python2 and python2-dev packages), for compatibility with old scripts.

On Ubuntu 20.04, you can install a package, python-is-python3, to ensure that the python interpreter and associated commands on your Linux machine run Python 3, which means you can then only access Python 2.7 via commands like python2, pydoc2, pdb2, and so on. The default download links on python.org are for Python 3.x. If a Windows 10 user runs python on a stock install, they’re directed to the Windows Store to install Python 3.x. We can only assume this same assumption will be coming to macOS soon.

Support for Python 2.7 and Django 1.11 ends in 2020, according to the Django project FAQ. Major Python open source project maintainers — such as those behind TensorFlow, scikit-learn, pandas, tornado, PyTorch, PySpark, IPython, and NumPy — have signed a pledge to drop Python 2 support from their projects in 2020.

The Python library “wall of shame” became a frozen-in-time “wall of superpowers”. And it is no longer maintained, since its mission has been accomplished.

So, it’s worth asking some questions.

Now that we’re here, is there any point in resisting any longer? Short answer: no. Python 3 is here to stay, and Python 2 is really, truly end-of-life.

Will Python 4.x ever happen, creating a disruptive 3-to-4 transition akin to 2-to-3? Short answer: no. That’s very unlikely. To quote Brett Cannon:

We will never do this kind of backwards-incompatible change again. We have decided as a team that a change as big as unicode/str/bytes will never happen so abruptly again. […] We don’t see any shortcomings in the fundamental design of the language that could warrant the need to make such a major change.

What can we learn from this experience? What has the grand experiment in language evolution taught us? Short answer: that big changes like this always take longer than you think, even when you take into account Hofstadter’s Law. Breaking backwards compatibility in a large inter-connected open source community has real cost that will test the strength of that community.

We are fortunate that Python’s community was very strong indeed at the time of this transition, and it even grew rapidly during the transition, thanks to the explosion of Python-based web programming (via Django, Flask, Tornado, etc.), numerical computing (via PyData libraries, like NumPy, Pandas), machine learning (via scikit-learn, TensorFlow, PyTorch, etc.), distributed computing (via PySpark, Dask, etc.), cloud computing (boto, google-cloud, libcloud). The network effects driven by these popular communities were like a perpetual motion machine that ensured Python’s adoption and the freshness of libraries with regard to Python 3 support.

The community is even learning to evolve beyond the direction of its creator, the BDFL, who resigned in 2018 and laid the groundwork for a smooth transition to a Python Steering Council.

So, here we are. Where do we go from here? Can the Python community continue to evolve in a positive direction atop the foundation of Python 3.x? Short answer: yes!

Python has never been healthier, and the community has learned many lessons.

So, let’s get on with it! If you’ve been holding back on using Python 3 features for some time, you can get a nice summary from this Python 3 feature round-up, which goes up through Python 3.6. Then, you can check out the official Python.org “What’s New” guides for 3.7, 3.8, and 3.9. Some of my favorite new features include:

• the d​ataclasses module (3.7)
• f-string = expressions (3.8)
• dictionary merge | operator (3.9, soon to be released)

So, what are you waiting for? It’s time to get hacking! Here’s to the next Python releases, 3.9 and 4.0 3.10! And to 3.11, 3.12, 3.13, … thereafter!