Resting in the Hudson River, on Pollepel Island, is a crumbling hulk of intricate concrete and brickwork that was once Bannerman's Arsenal. In 1900, Francis Bannerman VI bought the island to store ammunition for his prosperous military surplus business; more specifically, when he bought 90% of the US army surplus after the Spanish-American War ended and needed a place outside of New York City to store it all. The arsenal (sometimes referred to as "Bannerman's Castle") was constructed from 1901 to 1908, and was modeled similar to the architecture of an old Scottish castle.

A storm in 1950 sunk the ferryboat that served the island, Pollepel, and the arsenal was left more or less abandoned since then. On August 8th, 1969 a large fire destroyed much of the buildings. Neglect of the already burned and rotting wood roofs caused the deterioration of almost all the buildings except for the remaining stone walls. Submerged bulkheads and other structures make the surrounding area very dangerous for boaters; the island remains private property and is watched closely by preservation groups.

On the night of December 26th 2009, an entire corner of the massive arsenal collapsed due to the weathering of the aging mortar and unprotected bricks.