Ferroelectricity and magnetoelectric coupling in underdoped La$_{2}$CuO$_{4+x}$

La$_{2}$CuO$_{4}$ is an archetypal antiferromagnetic Mott insulator. Upon oxygen doping, a wide range groundstates may be accessed, including glassy magnetic phases and high-T$_{c}$ superconductivity. However, determining the nature of the charge correlations coexistent with magnetic order has remained elusive, particularly in the highly underdoped limit. In this study, we show that the first holes added to La$_{2}$CuO$_{4}$ drive the formation of a ferroelectric phase below 4.5K, with slow charge fluctuations developing below 40K. We invoke the formation of polar nanoregions -- which are a natural consequence of non-stoichiometric oxygen doping -- to explain the emergent ferroelectricity. An anisotropic magnetoelectric coupling is observed and attributed to the Dzyaloshinskii-Moriya interaction. Although this interaction is not responsible for the electronic ordering (unlike in other multiferroic perovskites), the presence of weak magnetoelectricity allows us to confirm charge carrier doping as the cause of ferroelectricity in La$_{2}$CuO$_{4+x}$.