Castle was the owner of the Albion Brewery at Brighton. He joined the Royal Philatelic Society in 1879 and joined its Council in 1881. He became Vice President in 1891 after the death of Thomas Tapling. He resigned as Vice President in 1902 and was replaced by the Earl of Crawford. In 1911 he was awarded membership of the Royal Victorian Order by King George V. In 1913 he became President of the Royal Philatelic Society, which office he occupied until his death in 1917.[2]
Castle's collecting interests were in Australian States and Europe of which areas he created large collections. He won a gold medal for his display of New South Wales at the London Philatelic Exhibition 1890.[3]