He was admitted to Trinity College, Cambridge in 1839 and received a B.A. in 1844 and an M.A. in 1847. He was admitted to Lincoln's Inn in 1844, called to the bar in 1847. He became an authorised reporter at the Court of Chancery and was the author of "Kay's Reports" and part of "Kay and Johnson's Reports" during the period from 1853 to 1858.[3][fn 1] He became QC in 1866.
In 1850 he married his wife Mary Valence French, the daughter of William French who was Master of Jesus College from 1820 to 1849.
His wife died in 1889 or 1890 and he founded the Lady Kay Scholarship in her memory, open to students at Jesus College, Cambridge who have the intention of seeking Holy Orders in the Church of England.[5]