We also made a @3blue1brown style video explaining the paper at a high level. https://t.co/7SspmcceQq

[{"shortUrl":"https://t.co/7SspmcceQq","type":"video"}]

Pluralis has a main tack paper at ICML this week and the team is in Vancouver running several events. The best will be the Open Source Mixer we're running with @gensynai and @akashnet_ Thursday night. For anyone interested in decentralised training it should be a great evening.
https://t.co/7fR7PYGWeS

Our accepted paper (https://t.co/nRazOHBgKJ) has the code is completely open - we’ve found this approach useful in many of our internal experiments, and we think it will be useful to others. https://t.co/X93ovivvEi

The work is the first method to allow fully asynchronous training in the Pipeline Parallel setting with convergence rates comparable to synchronous methods (there is a proof of a sublinear rate in the paper), while removing any idle time on the devices. This means training is faster. It has a large effect in the decentralised case, where idle times are amplified due to the slow communication speeds.

Also a reminder that the live run of the Protocol Models paper is starting soon - we have far more interest than we can accomodate in this first run, however we will prioritise joiners based on completeness of the registered information (see pinned tweet for the registration form).