Paramorph: A foundation model for autonomous neural network optimisation

At frontier scale, every configuration choice compounds. A run is shaped by many interacting decisions - optimiser settings, learning rate schedules, and drop-out rates - most of them fixed before training begins and expensive to revisit once it starts. When they're wrong, the costs are industrial: stalled convergence, wasted epochs, abandoned runs. And the standard remedies don't scale with the models. Sweeps burn compute on runs that exist only to be discarded; expert intuition doesn't transfer cleanly across architectures, workloads, and hardware.

Paramorph replaces configuration with control. Instead of fixing hyperparameters before a run and hoping, it adjusts them while the run unfolds, coordinating them as they interact, in response to the training dynamics it reads in real time. Itself a foundation model trained with multi-agent reinforcement learning, Paramorph brings a trained policy to a problem previously handled by search and intuition: specialised agents steering optimisation across layers and timescales as conditions change.

Paramorph is designed to deliver:

• faster convergence, on less compute
• fewer sweeps
• shorter research cycles
• better final models

Optimisation has always been a craft: defaults, heuristics, and hard-won intuition, re-derived by every team for every new model and every new dataset. Paramorph treats it instead as a sequential decision making problem, and solves it with reinforcement learning. Once learned, its policies carry across architectures and workloads, where intuition never transfers. And unlike heuristics, a learned optimiser keeps improving, generation over generation.