We’re excited to announce today that it’s easier than ever to use Pyston, our faster Python implementation. We have pre-compiled hundreds of extension modules and are releasing them today, making it easy to use your favorite libraries with Pyston.

For a bit of background: Pyston has always supported third party extension modules, but the practical difficulty of compiling some of them from source has been a roadblock for our users. These modules are difficult to compile on CPython as well, but library authors typically provide pre-compiled versions of their packages specifically for CPython. This means that installing, for example, the “cryptography” library is easy on CPython, but will fail to install in Pyston if one doesn’t have a Rust compiler installed.

Now one can simply install a pre-compiled cryptography, or Tensorflow or PyTorch or pandas (or many others) and run one’s Python code with a speedup.

How to use

We’ve released the packages as conda packages at anaconda.org. The easiest way to get started is to download our custom installer which will install a conda client that is pre-configured to use Pyston and these packages. You can also use the pyston/conda docker image where we’ve done this for you. You can also set up an existing conda environment to use Pyston (wiki).

We’ve chosen to provide conda packages because we can leverage the excellent work of the conda-forge team, who have done the hard work of creating repeatable build scripts for all of the packages. We are currently working on producing wheels for people who use pip, but this is less systematic and we don’t have an estimated timeline for it.

Unfortunately the error messages at this point can be a bit cryptic. If you get an error message while trying to install packages along the lines of “package X requires python_abi 3.8.* *_cp38”, or “package X requires python”, this typically means that one of the libraries you are trying to install (or one of their dependencies) has not yet been built for Pyston. These issues are a bit tricky to debug, so feel free to let us know and we’ll figure it out and get the package built and uploaded.

These packages should work on any Linux distribution that conda supports, though they have only been tested on Ubuntu. For Mac users we provide a docker container (pyston/conda) with our environment pre-installed. Windows users have reported good results by using Pyston inside WSL2.

More technical details can be found on our wiki.

Get in touch

We’d love to hear from you and your experience with Pyston and these new packages. Whether it’s because you want a package that is not available, you run into some sort of difficulty, or if there are any other reasons that you are still unable to try Pyston, let us know in our issue tracker or discord server.

We aim to make it effortless to accelerate your Python code, and if there is anything preventing that please let us know!

We’ve spent some time recently thinking about the future of Pyston, our faster implementation of Python, and wanted to share what’s on our mind. For updates please check out our wiki.

Roadmap

Our primary goal at this point is to get more people using Pyston, and our initial approach to that will be removing some of the reasons that make it difficult to use Pyston. We are currently 30% faster on our benchmarks and 60% faster on commonly-used benchmarks, and while being even faster would entice more people to use Pyston, we believe we will have a bigger impact by reducing obstacles than by working on performance.

The most common issue that our users report is not being able to install the packages they want. This happens because some packages are difficult to compile; CPython users will typically download pre-compiled versions, but there aren’t pre-compiled packages for Pyston yet. So our current focus is building these packages for our users and providing them in a way that’s easy to install.

To do that we’ve decided to provide the packages through conda. There are a few benefits of this: first it lets us provide the packages ourselves instead of waiting for package maintainers to produce Pyston builds, second it makes it easier for users to get Pyston in the first place, and third it will hopefully make us more portable in the process and work on more Linux distros.

After that we have many things we’d like to do, with the exact order to be determined:

• Set up a CI/CD system
• Add support for 64-bit ARM
• Continued performance improvements

And even longer term:

• Add Mac and Windows support
• Integrate with Numba
• Improve multithreading
• Explore “opt-in” features that allow us to break semantics
• Continued performance improvements

Target Python versions

We’ve decided that we only have the resources to target a single version of Python at a time. We would love to be able to provide a version of Pyston regardless of the version of Python you want, but that’s not feasible given our small team.

We currently target Python 3.8.12, and plan to retarget to Python 3.10 some time in early 2022. We intend to backport parts of the “Faster CPython” effort going on in 3.11 depending on the level of compatibility.

We are open to backporting semantic changes made in later versions of Python if we think this means the Python community has decided a certain feature is implementation-dependent. We anticipate that these are largely performance improvements that have small technical semantic differences; we do this on a case-by-case basis, and make a note of it on our wiki.

Supported Pyston versions

While we will try to help with any version of Pyston you are running, because our project moves quickly we won’t be backporting fixes to older versions of Pyston.

If you have any questions/thoughts/suggestions please feel free to join our Discord or file an issue on our GitHub!

We have some very exciting news to announce today: we (Marius and Kevin) are joining Anaconda! Anaconda is a well-known company that produces open-source Python software, and we think that by joining them we can significantly accelerate the trajectory of Pyston, our faster implementation of Python.

[See the corresponding announcement on the Anaconda Blog: https://www.anaconda.com/blog/pyston-team-joins-anaconda]

What will this look like

Things will largely look the same from the outside, except now we will have access to more resources and expertise to move faster. In particular:

• Pyston remains an open-source project with the same license as CPython
• Pyston won’t be tied to using conda
• We still get to set our roadmap, with potentially less time devoted to monetization work. By joining a company with a mature and efficient monetization scheme, we’ll spend more time doing core feature work.
• Once we need it, we’ll have a governance model that is separate from Anaconda
• We may develop integrations with other Anaconda projects in ways that are beneficial to both products
• We’ll continue to work with the community on the other Python performance projects that are underway

Why Anaconda

We talked to a couple of companies about a possible joint future for Pyston, and Anaconda stood out to us in terms of alignment. They’re already doing similar work with Numba and their other projects, and they have a demonstrated open source commitment that means a lot to us.

We also are excited about the possibility of having better integrations with some of their complementary products. We don’t have anything to announce right now, but we already had conda integration on our roadmap, and now that it’s easier, it’s more likely to happen sooner. Together, we are very excited about possibly integrating the features of Numba and Pyston: the two projects target different layers of the stack, and the hope is that by combining features, we will be able to explore more of the space of possible Python optimizations.

And finally, the medium-term roadmap for Pyston mainly involves work to get Pyston into more peoples’ hands. In this sense, we’re finding alternative Python implementations require much more work than simply making them faster, and joining a leading Python distributor will let us short-cut a number of these steps.

The Future

Now that we have Anaconda’s sponsorship, we are planning out a short-term roadmap for the project. We will announce more when it is ready, so stay tuned! In the meantime, give Pyston a try and let us know how it works for you on our Github issue tracker or our Discord channel.

We are proud to announce Pyston v2.2, the latest version of our faster implementation of the Python programming language. This version is significantly faster than previous ones, and importantly is now open source.

We also merged in many changes from CPython and are now based on CPython 3.8.8.

Performance

Pyston v2.2 is 30% faster than stock Python on our web server benchmarks. This is a significant improvement over our previous performance, and if we were feeling cheeky, we would advertise it as “50% more speedup.”

The foundational technology powering Pyston v2.2 is the same as that found in earlier versions, but we have tuned and optimized more areas and found additional speedups, particularly in our JIT and attribute cache mechanisms.

One noteworthy change is that we decided to remove many of the rarely-used debugging features that Python supports because they are expensive even when not needed. Doing so collectively resulted in a 2% speedup, which was remarkable to us: of all the computers in the world running Python, 2% of them are executing debugging checks. We’ve disabled those checks and are positioning ourselves as an “optimized build” similar to binaries without debugging information. Those who still want debugging features can use the “debug build” of stock Python because they are interchangeable. For a full list of the features we removed in Pyston v2.2, please see our wiki.

Open source

As we’ve continued talking to potential customers we now feel convinced that Pyston can thrive on an open-source business model, primarily by starting with support services. This means that we’ve open sourced Pyston v2.2, which you can find at our GitHub here.

We’ve archived our old repository to reduce confusion, but you can still find that here.

We are looking into which of our newest changes can be upstreamed to CPython. Throughout this process, we welcome your contributions. Help with getting Pyston packaged for additional platforms would be especially useful.

Moving forward

We continue to try and make Pyston as compelling and easy to use as possible. Working Pyston into your projects should be as easy as replacing “python” with “pyston.” If that’s not the case, we’d love to hear about it on our GitHub issues tracker or on our Discord channel. We hope you’ll give Pyston a try and see that it really is the easiest way to speed up your Python code.