New York Harbor that contains a museum and former

Military use and Fort Gibson
Ellis Island was also used by the military for almost 80 years.^[69] By the mid-1790s, as a result of the United States' increased military tensions with Britain and France, a U.S. congressional committee drew a map of possible locations for the First System of fortifications to protect major American urban centers such as New York Harbor.^[70][71]:54 A small part of Ellis Island from “the soil from high to low waters mark around Ellis’s Island" was owned by the city. On April 21, 1794, the city deeded that land to the state for public defense purposes.^[67][72] The following year, the state allotted $100,000 for fortifications on Bedloe's, Ellis, and Governors Islands,^[67] as well as the construction of Castle Garden (now Castle Clinton^[73]) along the Battery on Manhattan island.^[67] Batteries and magazines were built on Ellis Island in preparation for a war.^[74] A jetty was added to the northwestern extremity of the island, possibly from soil excavated from an inlet at the northeastern corner; the inlet was infilled by 1813.^[67] Though the military threat never materialized, further preparations were spurred by another possibility of war with France in the late 1790s;^[67][25] these new preparations were supervised by Ebenezer Stevens.^{[71]:153–154[68]} The military conflict also failed to occur, and by 1805, the fort had become rundown.^[26]
Stevens, who observed that the Ellis family still owned most of the island, suggested selling off the land to the federal government.^[68] Samuel Ryerson, one of Samuel Ellis's grandsons, deeded the island to John A. Berry in 1806.^[68][75][72] The remaining portion of the island was acquired by condemnation the next year, and it was ceded to the United States on June 30, 1808, for $10,000.^{[69][68][25][76]} Lieutenant Colonel Jonathan Williams, placed in charge of New York Harbor defenses in the early 1800s, proposed several new fortifications around the harbor as part of the Second System of fortifications. The new fortifications included increased firepower and improved weaponry.^{[71]:55–56[25]} The War Department established a circular stone 14-gun battery, a mortar battery (possibly of six mortars), magazine, and barracks.^[77][78][79] The fort was initially called Crown Fort, but by the end of the War of 1812 the battery was named Fort Gibson, in honor of Colonel James Gibson of the 4th Regiment of Riflemen, who was killed in the war during the Siege of Fort Erie.^[80][69] The fort was not used in combat during the war, and instead served as a barracks for the 11th Regiment, as well as a jail for British prisoners of war.^[26]

Ellis Island buildings circa 1893
Immediately after the end of the War of 1812, Fort Gibson was largely used as a recruiting depot. The fort went into decline due to under-utilization, and it was being jointly administered by the U.S. Army and Navy by the mid-1830s.^[26] Around this time, in 1834, the extant portions of Ellis Island was declared to be an exclave of New York within the waters of New Jersey.^{[24][25][26][27]} The era of joint administration was short-lived: the Army took over the fort's administration in 1841, demoted the fort to an artillery battery, and stopped garrisoning the fort, leaving a small Navy guard outside the magazine. By 1854, Battery Gibson contained an 11-gun battery, three naval magazines, a short railroad line, and several auxiliary structures such as a cookhouse, gun carriage house, and officers' quarters.^[81] The Army continued to maintain the fort until 1860, when it abandoned the weapons at Battery Gibson.^[25][82] The artillery magazine was expanded in 1861, during the American Civil War, and part of the parapet was removed.^[81]
At the end of the Civil War, the fort declined again, this time to an extent that the weaponry was rendered unusable.^[81] Through the 1870s, the Navy built additional buildings for its artillery magazine on Ellis Island,^[83] eventually constructing 11 buildings in total.^[84] Complaints about the island's magazines started to form, and by the 1870s, The New York Sun was publishing "alarming reports" about the magazines.^[68] The guns were ordered removed in 1881, and the island passed under the complete control of the Navy's Bureau of Ordnance.^[82]