Kellogg founded and was principal (1833–1839, 1847–1851) of the Female Seminary in Clinton. It was set up in part in response to the success of the nearby, all-male Oneida Institute.
Kellogg was a close friend of George Washington Gale, the founder and first President of the Oneida Institute. When Gale left for Illinois, there to found Galesburg and Knox College, originally a manual labor college, Kellogg went too, and became Knox's first president. A trip to England enabled him to purchase what has been preserved as the Knox Manual Labor College Collection. He also attended the 1843 anti-slavery convention in London. However, Gale and the trustees requested his resignation.
Later in life he was the pastor of the Presbyterian Church in Marshalltown, Iowa (1866–1867), and at the Guthrie and Dexter, Iowa, congregations (1870–1875). He helped organize some Iowa churches and would step in to help others.[3]