You might remember it from the showdown at the Tower of Joy wherein Ned Stark duelled with Targaryen forces and later on find his sister Lyanna on her deathbed with a newborn child. The real name of the imposing structure is the Castle of Zafra found in the Spanish province of Guadalajara.
(Image credit: Diego Delso/Wikimedia Commons)