According to Michael Pickard and Gurpreet Singh, the creators of "The Windmills of Your Mind" illusion, the dots of the windmill are always at constant speed and direction throughout the video clip.
So why did your mind see both clockwise and counter-clockwise movement? The answer, the two researchers at the University of Sunderland, United Kingdom, said, is that perceived direction is changed when the dot pitch is changed (by removing alternate dots or using alternate light and dark colors).