We give an introduction into the fundamentals of game theory and games of optimal stopping. We define randomzied Markovian times and give a motivation why these constitute a natural choice of strategies in order to find meaningful equilibria in war of attrition-type Dynkin games. Based on that example, we propose a general scheme to implement existence results for randomized Markovian equilibria in Markovian games of stopping.