Firstly: The Prostration of Forgetfulness is Legislated for Extraneous Actions Performed Forgetfully
The prostration of forgetfulness is legislated for extraneous actions.  Scholars differ on the details of what necessitates the performance of the prostration of forgetfulness due to extraneous actions in prayer. This is by agreement of the four schools of jurisprudence.
Secondly: Extraneous Bowing and Prostrating Due to Forgetfulness
If one adds an extraneous act of bowing or prostration forgetfully in prayer, this does not invalidate the prayer. Ibn Taymiyyah related consensus on this.
Thirdly: Intentional Performance of Extraneous Actions in Prayer
Intentionally adding to the prayer – such as by praying an extra unit or by performing salams before the prayer is complete – invalidates it. Consensus on this was related by Ibn Hazm, Ibn `Abd al-Barr, and Ibn Taymiyyah.