The James Brown house at 50 North Main Street in Ipswich was built about 1700, and altered in 1720 (1st / 2nd Period). In 1721, Stephen Perkins, a shopkeeper and James Brown, a yeoman, bought from Thomas Lovell, currier (39:61) a house on a large lot that extended along North Main St. There had been a dwelling on the larger lot before 1654, when Thomas Lovell acquired the property (1:152), but the oak frame with 1″ chamfers in the southern portion of the present house is of the type constructed in the late First period. The northern section appears to date from the 1720s, and probably was added by Brown after he had acquired the house. The house transferred in 1721 remains at the core of the present house.
At some time before the end of the 18th century the house was extended even further south. This 3/4 acre property had two houses and came to be owned by two families at the same time, Thomas Kimball and Ephraim Kendall. In 1773, Timothy Thornton, husband of Eunice Brown, petitioned the town to divide the property, which was accepted. In 1845 Thomas Morley bought the most southerly portion of the property (355:298), and rotated a portion of the building sitting on it, 90° on the adjacent lot to form a separate dwelling at 48 North Main. James Damon took down the remainder of the old dwelling and built the fine house at 46 North Main. In 1886 Harry K. Dodge took down the home of the widow Margaret Kendall and built the house at 44 North Main St.
In Tales of Olde Ipswich, Volume III, Harold Bowen related that Albert Hills had his grocery store in this building in the early 20th Century. Albert P. Hills was a Civil War veteran, having enlisted at age 18 and served as a drummer in Company I (Captain Hobbs) of the 23 Massachusetts Regiment. It was the home of long-time selectman Brainard Wallace in the 1940s.
Sources:
Historic Ipswich 