This house was built by William Merrill Ellsworth and his wife Jenny (Lord) Ellsworth. The 1910 Ipswich map shows the house at #38 Summer Street as #36. (Some street addresses were changed in 1991 when the town adopted Enhanced 911). The 1884 map shows this house in the possession of W. Ellsworth. Two similar houses side by side are on the same lot in the 1910 map, owned by Mrs. Jennie Ellsworth. It does not appear in the 1872 Ipswich map.As a 19 year-old Machinist, William M. Ellsworth enlisted in 1864 as a private in the Civil War. He was a member of GAR Post #128 (General James Appleton Post) in Ipswich. He was the brother of Civil War hero Thomas Foulds Ellsworth, and the son of Ipswich lightkeeper Benjamin Ellsworth.
Salem Deeds shows that in 1881, William M. Ellsworth mortgaged a house on Summer Street with the Ipswich Savings Bank. It also shows that in 1930, Jennie Ellsworth sold a house on Summer Street to Marion McGilvary. In the 1940 census, Marion McGilvary was 50 years old and lived in Ipswich, Massachusetts, with her husband, Fred, and daughter Barbara age 13.
The house stayed in the McGilvary family for two or three generations, and was inherited by Barbara Brockelbank (married to Chester Brockelbank Jr). Thanks to Elyse Brockelbank Phillips for providing family information and confirming the identity of this house.