It is believed that this house has some first period construction elements dating to approximately 1720.

Thomas Franklin Waters wrote that in 1887, William Russell removed a house built in 1772 by James Harris at 12 High Street and built his Victorian house. In “A Walking Tour of Ipswich” by Marge Robie and William Varrell we read that the house was removed to 8 Warren St., in the ownership of David Grady: “In 1887, William Russell, who owned a dry goods store on Market Street, bought the property at 12 High Street and built the beautiful Victorian home still standing. Rather than tear down a serviceable house, he sold it and moved it to the other side of the block.”

Sources:

View from Town Hill in Ipswich by George Dexter, before 1900
The David Grady house is on the right in this closeup from a photo by George Dexter, taken from Town Hill around 1900. The Congregational Church, the Methodist Church and the Agawam house are in the middle of the photo. The Warren St fire house is in the foreground, with a cooling tower. In the background is Heartbreak Hill.