I'm experimenting with low-barrier sponsoring of some of my work via GitHub Sponsors. In addition to great PyTorch work, you get me to write this little report in return.

As this is the first report, we'll start with a bit of background. Then I'll tell you what I did in the time I allocated to public benefit work in the past month. Finally we look at the priorities you can vote for in the eligible tiers.

I'll close with a bit of a personal note.

Where does PyTorch come from?

The largest corporate sponsor of PyTorch is obviously Facebook. The hardware manufacturers (AMD and NVidia (in alphabetical order) for the GPUs, Intel for CPUs, even Google for TPUs) also have a number of people working on PyTorch, ensuring PyTorch makes the most of their hardware. Then there are some of the large tech names that are most active in specific areas, e.g. Uber has contributed a lot to torch.distributions, which is used by Pyro, and Microsoft worked on things like scriptability and ONNX export in TorchVision. The list is likely incomplete in every sense. There also are independent contributors, some are alumni of Facebook and some who just happen to have come to PyTorch. I'm in the latter group, Vishwak Srinivasan is a very prolific contributor in that group, too. There also are a few people from the Torch history of PyTorch.

The PyTorch ecosystem is, of course, quite bit larger, HuggingFace's (awesome) Transformer library might be the most well-known third party extension.

But what does that mean for PyTorch users?

I have to say that I do believe the PyTorch team does make a point of listening to users. I think that this - in addition to having a vision of how a tool should look like so they want to use it - is a huge part of the success of PyTorch. When Alexander Rush made a very convincing case that Tensor dimension handling is bad and provided a prototype, the PyTorch team at Facebook implemented it in PyTorch proper.

However, as a user, you find yourself in a situation where progress on features or issues that are not a priority of the corporate sponsors is very slow. Take type hints for PyTorch, also enabling autocompletion as an example: It was mentioned on the forums in the summer 2017. In January 2018 issue #4568 was filed. In May 2018, a different approach changing the architecture of the PyTorch extension module was proposed in the issue #7318, but that hit technical roadblocks. Even though many people took the time to complain about the missing support on that issue, it took until January 2019, when Ed Young (of Facebook) put the finishing touches on a PR I had created in September 2018 and very slowly massaged into a mergeable state and we finally got type hints for the core functions.

Another example I have been involved with is PyTorch for Android. In November 2018, there were people interested in having it and I made a small proof of concept, but for Facebook, the AI on mobile seemingly meant caffe2. As a result, no-one was investing into it. I eventually sent my porting as a PR (which was dissected and merged in various pieces) and Facebook has since put quite a bit of work into Java bindings and also iOS support to make PyTorch mobile.

Now, there are a number of issues that - to my mind - are very much desired but there seems little progress: I'm thinking of things like the default initialization of parameters - where the Torch legacy parameters seem quite antiquated - or to make functions and backwards deterministic on CUDA. Many people will try to put in fixes that they need or find interesting to work on and we're making great progress on many things, but there are some things where no-one makes a given fix a priority.

Crowd-Sponsoring of PyTorch work

So here is my idea to fix this: I'm offering to sponsor my development activity on GitHub and sponsors will get to vote on (and also propose) priorities. This way, even if you cannot afford the time to do it yourself or have an end-to-end contract, you do get the opportunity to further the development areas that matter to you.

Achievements in the last month

Without further ado, here are my achievements of this month:

I fixed a few bugs in PyTorch:

• I tracked down and fixed an autograd incompatibility of bfloat16 when a user wondering on slack what was going on.
• I tracked down and fixed a CTC loss bug for large index sizes.
• I improved the error information when loading weights in C++.
• I fixed the C++ torch::nn activation function modules to make them compatible with torch::nn::Sequential.

For a project I'll talk about later, I spent some time with PyTorch Mobile and Quantization. These being new/experimental, I hit some bugs - or opportunities to improve:

• In mobile I found a bug in a network that would work correctly on the (x86) emulator but does not work on an arm device. Apparently it is a memory layout issue for. I'll try to reproduce that using a Raspberry to facilitate easier debugging and fixing on ARM. Prime suspects are strided transpose convolutions.
• I have two usability ideas that I hope to do PRs for soon - converting Tensors back to images and loading models directly from assets.
• I reported some weirdness / bug with quantization semantics that I hope to fix soonish.
• I also wrote a sanic-based mini web-server with request batching that I hope to publish soonish.

I also worked on spreading the word:

• I presented my LibreOffice Translation extension to a great crowd at the Hacking Machine Learning Meetup in Munich - I also enjoyed Ines Montani's talk on spaCy on the same evening. Slides and video are linked from the project page.
• I organized a Munich PyTorch meetup with two great talks by Adam Paszke and Carla Gil with ~150 people in attendance.
• I also helped answering user's quesitons on the PyTorch forums, writing over 120 replies in the past month, making me the (distant) 3rd most active responder on the forums.

Things you can vote on to

See the Sponsorship page for eligibility.

My suggestions for focused work on PyTorch are

• The mobile / quantization issues outlined above,
• improving the default weight initialization in PyTorch,
• working on deterministic GPU kernels,
• a tutorial on PyTorch JIT internals.

LibreOffice translate options:

• A tutorial to train models for LibreOffice Translate,
• train French or Spanish to English translation models,
• distilling models to CTranslate2 for faster inference.

You can also write in with new suggestions via E-Mail.

How this works is that I'll prioritize within each project based on the votes but split time between them.

A personal note on politics

So I'm writing much of this on a Nightjet train from Munich to Milano. I'm trying to take the train whenever I can both for the comfort and environmental reasons. The train has some delay due to severe whether severely damaging the Tauern railway route. Tthanks to the EU, I could book my ticket with the Austrian railway and pay in our common currency, thanks to the EU I have free roaming for the mobile internet, and the friendly ÖBB staff keeps me supplied with coffee. There are lots of things to improve - I look forward to a fast connection through the Brenner Basis Tunnel - but it is neat how seamless it is to travel and work internationally in Europe.