Guido And Microsoft Want To Make Python x2 Faster
Written by Mike James   
Wednesday, 19 May 2021

A presentation by the father of Python or its BDFL as we used to know him, Guido van Rossum, announced that Microsoft has backed him to create a faster Python.

It's the Python Language Summit at the moment while most of the contributions are interesting, they address very specific language features. Guido's contribution is more wide-reaching and extends news we reported at the beginning of the month - that Microsoft is increasing its existing support for Python, see Microsoft Now Visionary Sponsor Of Python.

Python is a high-level language. In many ways it offers the programmer more abstractions than the average high-level language. This is great because you can write instructions and let the compiler sort out how to implement things that otherwise you would have had to spell out in detail. For example, in most languags to swap two values you would write something like:

temp=a
a=b
b=temp

in Python you can simply write:

a,b=b,a

and let the compiler figure out how to do the swap. But this means the compiler has to allocate a temporary variable and dispose of it. This isn't a particularly sophisticated or exensive example of what is going on, but it does illustrate the idea that the less you do the more the compiler has to do. In the early days of computing the argument used to be that the compilers could be made cleverer and anyway it didn't matter because hardware was always getting faster so inefficiencies in high-level languages would  increasingly become irrelevant. However, things haven't quite panned out like this. Hardware stopped getting much faster a few years ago and resorted to increasing parallelism to use the extra chip area and our algorithms have become increasingly demanding.

Today speed still matters.

It seems to matter to Guido that Python is not as fast as Rust or C. I can testify, most recently with my experience of the Raspberry Pi, that C is up to 100 times faster than Python and often this matters. There are Python implemenations that improve the speed of the language, Pyston for example, but the gold standard compiler is CPython and this is what Guido intends to improve.

pythonfast1

The story seems to be that Guido noticed "the Shannon Plan" a GitHub repo by Mark Shannon proposing  to improve the speed of Python by 1.5x per year to 5x in 4 years - but he was short of funding. The rest can be told by a slide from the presenation:

pythonfast2

Well done Microsoft!

The team includes Mark Shannon as well as Guido and Eric Snow, a Senior Software Engineer at Microsoft who is also a Python Foundation Fellow and has been working on multi-threaded performance.

The problem is once you have a project you have to say how you are going to achieve your ambitious goal. The idea is to improve the speed of CPython by a factor of 2 by Python 3.11 without breaking ABI or API compatibility and without beaking or slowing down extreme cases. And all this with easy-to-maintain code.

With these restrictions the only things that can be changed are the bytecode, the stack frame and the internals of private objects. The big target seems to be the byte code interpreter plus a list of optimizations. With all of these difficulties, it isn't suprising that the team expreses the opinion that they might not reach x2.

Interestingly they speculate that beyond 3.11 there might be machine code generation in their future. Personally I'm not sure why this isn't in their present. After many challenges with the GCC C compiler to improve on its code. I have to admit that a modern compiler out-optimizes the machine code programmer. It is an out-of-fashion opinion but I think it's in Python's future as well.

Is this project necessary?

Probably. Python has really good facilities, better than most languages, for inter-working with C and many Python projects achieve their speedup by using C modules for the speed-critical parts. Even so it would be nice to do it all in Python and it would probably make Python the choice for most technical applications, consolidating it's #1 language status.

mspythonpic

  • Mike James is the author of Programmer's Python: Everything is an Object published by I/O Press as part of the  I Programmer Library. With the subtitle "Something Completely Different" this book is for those who want to understand the deeper logic in the approach that Python 3 takes to classes and objects.

 

 

More Information

PEP 659 -- Specializing Adaptive Interpreter

https://github.com/faster-cpython

Related Articles

Guido van Rossum Joins Microsoft

Microsoft Now Visionary Sponsor Of Python

Google Promises Increased Support For Python

What Makes Python Great & Greater

Guido van Rossum Quits As Python BDFL 

Python Language Summit 2020 - Python Typing

Programmer's Python - Type Annotation

Python Development Trends

What Makes Python Special?

Python 3 For Science - A Survey

Jupyter Receives ACM Award

 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

 

Banner


Stack Overflow Sold for US$1.8 Billion
07/06/2021

Prosus, one of the largest technology investors in the world, has  announced that it is acquiring Stack Overflow for US$1.8 Billion. Stack Overflow CEO Prashanth Chandrasekar described this  [ ... ]



Pyston 2.2 Goes Open Source Again
21/05/2021

Pyston v2.2, the latest version of the high-performance Python implementation originally developed by cloud storage provider Dropbox, has been taken back to an open source model.


More News

square

 



 

Comments




or email your comment to: comments@i-programmer.info

 

 

 

Last Updated ( Thursday, 20 May 2021 )