Before the arrival of Europeans in Port Jackson., the Aboriginal clans of the Cammeray-gal (Kameraigal) (Gameraigal) lived along Milsons Point foreshores and surrounding bushland. Before settlement the Milsons Point area was a place for Aboriginal corrobories (traditional dances) and gatherings.
The first permanent settlement was established in 1806 when James Milson farmed the area supplying food and water to the growing settlement. He also quarried the sandstone and built a house near the site of the northeast pylon of the Harbour Bridge.
Milsons Point today is a popular tourist spot for panoramic views of Sydney and the harbour.