Rinaker served as delegate to the Republican National Conventions in 1876 and 1884 and then was chairman of the Board of Railroad and Warehouse Commissioners of Illinois 1885-1889. He successfully contested as a Republican the election of Finis E. Downing to the Fifty-fourth Congress and served from June 5, 1896, to March 3, 1897. He was an unsuccessful candidate for reelection in 1896 to the Fifty-fifth Congress.
Rinaker returned to Carlinville, Illinois, and resumed the practice of law. He died in Eustis, Florida, January 15, 1915. He was interred in the City Cemetery, Carlinville, Illinois.