Mr. Augustus Elliott came to Pittsford in 1810. A man of some means, he built a store at the corner of what is now Church Street and South Main Street. He also operated a distillery and an ashery at what is now #52 South Main Street. His distillery brought him a great deal of wealth and with it he began building this fine home for his future bride, named Jane Penfield.

The home was begun about 1812 and completed about 1815. Great care was taken with the design and the interior design and trim and much of the interior wood trim was brought from Albany. The large double parlor features beautiful fireplaces copied from 18th century homes. The windows over the front door and the woodwork in the foyer are all original. The archway in the foyer blends beautifully with the curve of the staircase and the floor above.

Everything had to be especially beautiful for Ms. Penfield, but unfortunately she jilted Mr. Elliott and later married a man named Daniel Brown for whom Brown’s race in Rochester is noted. It has been said, (not truthfully, however,) that Augustus Elliott committed suicide. He was devastated by Jane Penfield’s betrayal and he left Pittsford and went to Pennsylvania, where he died a lonely bachelor.

Other owners of the home included Mr. James Guernsey who was a very wealthy man as well as a very innovative one. He established a sort of "running water" system in his fine home. The grounds of this home were extensive, including all land to and including Rand Place, as well as the upper end of Eastview Terrace. From that elevation, Mr. Guernsey funneled water through a complicated series of pipes and troughs to bring water into the cistern in the home and from it would be pumped up to sinks in the kitchen. Not quite the system we have today, however.

Another thing for which Mr. Guernsey was noted was the amazing gingseng plants found on his property. He was widely known for this unusual herb and it was exported for a goodly amount of money to the Orient and to countries which used this spice in preparation of food.

When Mr. Guernsey’s home became the home of the Briggs family from New York City, their gardener, not knowing that this was an expensive plant, destroyed all of the plants and the crop was lost.

The exterior of the house was constructed of bricks which were made in Elihu Doud’s brickyard which was located in "Lusk’s Hollow" near the Milepost. If you haven’t already, do notice the "herring bone" pattern of bricks on the front of the home.

Notice, also, the four chimneys which are very similar to the Phoenix building and we believe were designed by the same hand.

There are many rumors about this house and its connection with the Underground Railroad. It is reported that there was a room in the basement, small and dark, where runaway slaves were hidden. We have no documentation of this and so we can only conjecture what might have happened. It is also rumored that there are caverns under the village which may run from near this home on South Main Street to the four corners and a little way to the west and east. Again this cannot be verified, so it remains as "legend"