One of the fundamental assumptions in stochastic control of continuous-time processes is that the dynamics of the underlying process are known. This is, however, usually obviously not fulfilled in practice. On the other hand, a rich theory for nonparametric estimation of the characteristics of continuous-time processes has been developed over the last years. We will discuss how to bring together these two areas for developing purely data-driven strategies for stochastic control, which we explore for ergodic singular control problems associated to continuous diffusions and Lévy processes.