The building was built as a school around 1910-11 and was known as New Milford High School. In 1931, a new high school was built at 50 East St. and this building on Main Street became the Main Street School for elementary students. It was building was repurposed as town office space in 1980s. In June 1984, the building was rededicated as the Paul S. Richmond Citizen Center and the town's senior center moved in.