A military fort, out in the ocean, with a moat! Fort Jefferson is a part of Dry Tortugas National Park in the waters off of Key West, Florida. Construction on the "fort in the middle of nowhere" was started in 1846. It was originally meant for the defense of the US, but during the 30 years of construction, some design features became obsolete for that purpose.
During and after the Civil War the fort began to be used as a prison for deserters and other criminals. In 1874 the army completely abandoned the fort after several hurricanes and a yellow fever epidemic, and it wasn't until 1898 that the military returned in the form of the navy, which used the facilities during the Spanish-American War. The fort was also used from 1888 through 1900 as a quarantine station, and was garrisoned again briefly during World War I.
Link -via TYWKIWDBI
(Image credit: National Park Service)
You left out the story of the most notorious prisoners that were held at Fort Jefferson, Dr. Samuel Mudd, and other conspirators in the Lincoln assassination. During the aforementioned yellow fever epidemic, Dr. Mudd provided such selfless care that he was eventually pardoned by President Andrew Johnson and released. You can see Mudd's cell on a tour of the Fort.
Other great sites at the fort are the lighthouse, the moat that was mentioned and there is also very good snorkeling off of the beach and near the walls of the fort. Right next to the Fort are Bush key and Long key, which are closed to the public during nesting season because of the bird sanctuary that is there.
Going north might be a good idea, but you need to think about being able to sustain yourself, and in a choice between tropics and arctic, I'd choose tropic.
Just sayin....