Paramorph Technical Report

Abstract

We introduce Paramorph, a foundation model for training control. It watches a neural network as it trains and, every hundred batches, resets the learning rate, Adam β1 and β2, dropout and weight decay according to what the run is actually doing. It is trained with MARL and is used zero-shot: a new training run gets an adaptive, run-specific schedule without a hyperparameter search.