Rayner was elected the same year to the 50th United States Congress. He was an unsuccessful candidate for reelection in 1888 to the 51st Congress, but was victorious in the next two elections to the 52nd and 53rd Congresses. He declined to be a candidate for renomination in 1894.
Five years later in 1899, Rayner was chosen to be the Attorney General of Maryland, serving until 1903. He was elected as a Democrat in 1905 to the U.S. Senate, and was reelected again in 1911. While senator, he served as chairman of the Committee on Indian Depredations (Sixty-second Congress).
In 1912, Rayner died in Washington, D.C., while serving as senator. He is buried at Rock Creek Cemetery.