Conjectures of Braverman and Kazhdan, Ngô and Sakellaridis have motivated the development of Schwartz spaces for certain spherical varieties. In this talk, we will prove that under suitable assumptions these Schwartz spaces are naturally a representation of a group that we christen the modulation group. This provides a broad generalization of the defining representation of the metaplectic group. The example of a vector space and the zero locus of a quadric cone in an even number of variables will be discussed. In both of these cases the modulation group is closely related to algebraic groups, and we propose a conjectural method of linking modulation groups to ind-algebraic groups in general.