Quant trading python code

The Future of Backtesting is Python - But 5x Faster

15 July 2021
Matthias Frank, Head of Engineering

Python is set to remain the programming language of choice for backtesting investment strategies, as new research reveals the world’s most popular systematic trading language is set to become even better. Guido Van Rossum, Python’s creator, revealed in a recently published paper that the language can become up to five times faster, which makes backtesting with Python the optimal solution for quants.


Why Python?

Since its inception in the early 90s, Python has taken the finance world by storm. Initially, it was used by only a handful of finance companies and often Python’s usage was reduced to scripting or glue programming, holding different applications together.

Over time though, Python has grown steadily in popularity due its ease of accessibility, and the huge number of quality open source packages available, such as pandas and numpy, as well as its machine learning, data science and AI applications. Today, Python is the number one programming language for modern fintech companies, and will soon surpass C++ as the second most popular language in finance generally.

The most popular programming languages for finance and fintechs
The most popular programming languages for finance and fintechs

Source: https://blog.hackerrank.com/em...

Why not Python?

In one word: speed. Developers and researchers transitioning from more traditional compiled programming languages like Java, C++ and C often criticise Python’s runtime performance - but with careful optimisation, speed need not be an issue.

For SigTech users, speed is particularly relevant. SigTech clients often backtest strategies using vast datasets of over 20 years of data incorporating multiple data points per day, and slow performance would not be acceptable.

Back in 2013, SigTech decided to build its systematic investment strategy platform from the ground up in Python. To ensure optimal runtime of our platform, we continuously monitor strategy runtime performance benchmarks for our codebase. These benchmarks are critical in maintaining the current performance status quo and to protect the framework against sub-optimal code changes.

Since the rise of cloud computing, we see even more compelling reasons to continue to run our framework in Python. Some parts previously coded in Python have been moved to a cloud computing infrastructure or were optimised using Cython. This has already improved SigTech’s backtesting engine performance by 2x across the board, and in some areas, even greater improvements have been seen: for example, in equity universe construction where the cloud compute delivers 10x improvement compared to an equivalent Python implementation.

The evolution of Python in the SigTech platform

The next two years

Historically, the Python core developer community did not focus on improving runtime performance because Python was typically used in situations where ease of writing code outweighed speed. As the applications of Python have grown, speed and performance are becoming more important, and accordingly are now being focussed on by some of Python’s most influential programmers.

Mark Shannon, one of Microsoft’s most senior programmers who works alongside Guido van Rossum has worked on Python performance for some time, with previous projects, such as HotPy for a just-in-time compiler for CPython.

He has his own Faster CPython repository where he wrote that "we want to speed up CPython by a factor of 5 over the next four releases." Although Shannon envisages a JIT compiler eventually, this would not come until Python 3.12 in his plan. Python 3.10, currently in beta, is scheduled for release in October this year. The release schedule is roughly annual, so we might expect 3.11 in October 2022, and 3.12 in October 2023.

You can read more about the Guido van Rossum and Mark Shannon’s work in this recent article published by The Register, as well as van Rossum’s original presentation on his future plans for python published on Github.

Subscribe

Receive our latest blogs, trading strategy ideas and insights

What these performance improvements will mean for SigTech users

SigTech users have already seen a pronounced improvement in terms of speed over the last few years, and with these upcoming changes to Python, we expect that trend to continue.

The chart below shows how the runtime for a 10-year backtest of a systematic strategy using minute bar data and trading has decreased markedly over the last few years - and within another two, we expect the backtest to be complete in under two minutes - an improvement of >95%.

Minutes taken to run backtest in python
Minutes taken to run backtests with Python

The gap between Python’s performance and other compiled languages is narrowing. The unrivalled ecosystem that Python offers continues to make it the best choice for programmers and traders looking to create innovative, data-centric financial models.

That’s why we at SigTech, and many others, firmly believe that Python will remain the number one choice for quants and traders.

Get in touch to find out more about backtesting with Python and SigTech.