Gilmour was a lieutenant in the Fifeshire Volunteer Light Horse, and was among the officers of the Fife and Forfar volunteer battalions to volunteer for service in the Second Boer War. He was commissioned a lieutenant in the Imperial Yeomanry on 7 February 1900,[2] and served in South Africa with the 20th (Fife and Forfarshire Light Horse) Company of the 6th Battalion. He left Liverpool for South Africa with the company on the SS Cymric in March 1900.[3] For his service, he was awarded the Queen's medal with 4 clasps and was twice mentioned in despatches (by Lord Roberts dated 4 September 1901[4] and in the final despatch by Lord Kitchener dated 23 June 1902[5]). His letters from the Boer War were published in 1996 under the title "Clearly My Duty" by his son, Sir John Gilmour, 3rd Baronet. He again served in World War I with the Fife and Forfar Yeomanry, where he was again mentioned in despatches and awarded the DSO with bar. His service after the war saw him rise to the rank of Lieutenant-Colonel when he commanded the Fife and Forfar Yeomanry. On 8 May 1931 he was made the Honorary Colonel of the Fife and Forfar Yeomanry.[6]
He was appointed as Secretary for Scotland in 1924, and became the first Secretary of State for Scotland when the post was upgraded in 1926. A member of the Orange Order joining the Pollokshaws Lodge, LOL172, in June 1910. This Lodge is now named after him. Gilmour, as Secretary for Scotland, repudiated the Church of Scotland's report, "The Menace of the Irish Race to our Scottish Nationality". [citation needed]
Later in his career he served as Minister of Shipping during the early months of British involvement in the Second World War but died in office of a heart attack in London on 30 March 1940,[7][8]
Gilmour first married Mary Louise Lambert, daughter of Edward Tiley Lambert, of Telham Court, Battle, Sussex, on 9 April 1902 at St. Mary's Church in Battle, Sussex.[10] The marriage produced two children, Anne Margaret, born in October 1909, and John, born in October 1912. After Mary Louise's death in 1919, he married her younger sister (his former sister-in-law), Violet Agnes Lambert, in 1920. They had one daughter, Daphne, born in January 1922.[11]
Both children from his first marriage both rose to positions of prominence in the civil service. His eldest daughter, Dame Anne Margaret Bryans DBEDStJ, worked for the British Red Cross Society and served as vice-chairman of the executive committee from 1964 to 1976, in addition to holding positions in the governorship of many hospitals in the United Kingdom. He was succeeded in the baronetcy by his son, Sir John Gilmour, 3rd Baronet, who also had a successful political career.