When Father Angelus Baret was named as the first residential pastor in Southbridge on September 11, 1865, the area Catholics numbered 1,673: 850 Irish, 500 French Canadian, 140 in Charlton, 73 in Sturbridge and 110 in Fiskdale. A house for their priest was built next to the church facing Hamilton Street. But by 1870, change was in the air. Pope Pius IX had convened a Vatican Council, the first in over 300 years. There were rumors of a new diocese west of Boston and the Catholic community in Southbridge was asking to be divided into two parishes. When Fr. Baret was transferred in 1869, the new parish of Notre Dame was created on November 27th. On September 25, 1870, the Diocese of Springfield was created separate from that of Boston and Southbridge was one of the towns included. The new Bishop of Springfield appointed the pastor of Saint Peter, Fr. McDermott, as the rector of the new cathedral. Serving less than a year, the parish had now had had 3 pastors in less than 12 months. He was succeeded by Father John M. Kremmen whose lasting impression on the parish would make it his final resting place.
Early on the morning of January 14, 1872, Fr. Kremmen was awakened by the smell of smoke. A fire that started in a shed behind the rectory quickly engulfed it, spread to the church, and threatened the neighborhood. The next day’s newspaper recorded “The absence of a wind, the tin roof of the parsonage, and the fire extinguisher, are all that saved the fire from plowing a furrow of destruction through the business portion of our town.”