The aim of the present paper is to study the semimartingale property of continuous time moving averages driven by Lévy processes. We provide necessary and sufficient conditions on the kernel for the moving average to be a semimartingale in the natural filtration of the Lévy process, and when this is the case we also provide a useful representation. Assuming that the driving Lévy process is of unbounded variation, we show that the moving average is a semimartingale if and only if the kernel is absolutely continuous with a density satisfying an integrability condition.