Research Talks at the Zuse Institute Berlin
Welcome to the ZIB's Research Seminars and Colloquia page. Explore our schedule of research talks in mathematics and computer science. We currently host the following series at our institute:
- The Zuse Research Colloquium, a series of talks by high profile invited speakers from academia and industry.
- The Zuse Research Seminar, a series of talks by researchers from the Zuse Institute Berlin and select external speakers from the Berlin math community.
- The IOL Seminar and Lecture Series, the research seminar of the IOL research lab of Sebastian Pokutta at ZIB and TU Berlin.
For questions relating to an individual seminar or colloquium, please contact the organizers of that series. For question relating to this homepage, please contact Christoph Spiegel. The ZIB also hosts an overview of all mathematical research seminars happening in and around Berlin at seminars.zib.de.
Upcoming Talks
We show that learning-rate schedules for large model training behave surprisingly similar to a performance bound from non-smooth convex optimization theory. We provide a bound for the constant schedule with linear cooldown; in particular, the practical benefit of cooldown is reflected in the bound due to the absence of logarithmic terms. Further, we show that this surprisingly close match between optimization theory and practice can be exploited for learning-rate tuning: we achieve noticeable improvements for training 124M and 210M Llama-type models by (i) extending the schedule for continued training with optimal learning-rate, and (ii) transferring the optimal learning-rate across schedules.