Why We're Porting To Python 3

However, after we met our deadlines, we took some time to reconsider our platform decision before committing to the existing code base. We knew Python was the right choice, but we had lingering doubts about our decision to continue avoiding Python 3. Upon a closer look, we found that things had changed drastically since the last time we'd seriously considered this question, and the scale was no longer decidedly tipped against Python 3. We ultimately made the decision to port to it. In this post, we'll go over some of the key factors driving our decision.

Package Availability Isn't What It Used To Be — It's Much Better!

In case any reader is new to the problems facing application developers considering Python 3, they could be summed up thusly: Python 3 fixes a bunch of long-standing Python problems in a way that makes Python much better, but also makes pre-3 code incompatible. This, in turn, means that in the early years of Python 3's lifecycle, in addition to having its own warts, Python 3 was incompatible with a large number of very important packages.

For example, one of the most popular Python packages for web applications, Django, didn't support Python 3 in production until fairly recently. That's almost 5 years to the day from the debut of Python 3 in December 2008. And we don't mean to pick on Django - that package was in good company with respect to this long wait.

However, for the most part, the wait is over. Not only do most of the popular Python packages support Python 3, but contribution of Python 3-compatible packages has accelerated through an inflection point. Therefore, we concluded this concern was no longer as dire as it once was, and the evidence supported the conclusion that the packages we'd need were either available for Python 3 or soon would be.

There is, of course, one notable exception high on the "Wall of Power," namely the boto package. While we use this library extensively in our product, we carefully examined how we use it, the viability of alternatives like boto3 and botocore, and the dedication of Amazon and the community to this project. We decided this single exception was one we could accept right now.

Old Problems Solved Are Solved In New Ways

As we looked more closely at some of the changes in convention and idiom that we would encounter in Python 3, we liked what we saw. It would solve problems we'd worked around and improve upon our existing solutions.

For instance, during our first sprints, we carefully considered the concurrency model we would employ in the component, but only stubbed in implementation of actual concurrency constructs. Concurrency is a sensitive subject in the Python world anyway, but since our component's concurrency needs were largely based on I/O waits and not a need to utilize multiple CPU cores, it was pretty clear we would benefit from parallelizing our component. So, it was a day-one consideration for us.

We started by just spawning thread primitives in-line with our code as needed, basically forming something of a concurrency free-for-all. While we'd made tentative plans to implement thread pooling in our Python 2.7 component, we found when examining Python 3 that this would not be something we had to do; it was already there in a form proven and familiar from the Java world.

It's The Right Thing For The Community

At Luminal, we value the open source community. In countless transactions with computers, FOSS forms the giants on whose shoulders we stand. Therefore, we strive to keep our duties as internet citizens near the front of our minds. Python is a major subset of the FOSS community and, like everyone who values it, we want to see it move forward.

We felt that being one-more-startup to eschew Python 3 did nothing to help this community. If we did run into library shortages and had to port or hand-roll something that was missing, it would be a fine opportunity to give back to the community. We knew that along with the many other package authors and application authors out there using Python 3, our participation would put just a little more wind in Python's sails to propel it towards the future.

In general, it just seemed like the right thing to do and the right time to do it. We think our fellow startups will reach this conclusion in ever-increasing numbers as time goes on.