He was a member from Suffolk County of the New York State Assembly from 1777 to 1785. During his last term, he was Speaker. As speaker, he took a leading role in reconciling the differences between Tory and Whig factions. He oversaw the repeal of all the laws that had imposed civil and legal penalties on Tories.[4]
In 1787, he removed to New York City, and from 1787 to 1801, was Surrogate of New York County. In 1789, the State Assembly appointed him a delegate to the last session of the Continental Congress. He was a member of the New York State Senate from 1791 to 1794, and from 1798 to 1802.[4]
It was during 1792, that Charles Willson Peale was commissioned by Gelston, a notable American during the time, to paint a "head size" portrait of Gelston as well as a companion piece of his wife and daughter.[7] This painting is currently displayed at the La Salle Art Museum.[8][9]
Gelston died on August 21, 1828, in New York City. He was buried in the First Presbyterian Church Cemetery in New York.[10]