Rutherford was born in North Shields. He started his career in the youth team of Hartlepool United on a two-year scholarship in the summer of 2010.[1] In April 2012, Rutherford was offered his first professional contract.[2] On 14 April 2012, he made his professional debut in a 2–1 defeat to Chesterfield, coming on as a late substitute for Gary Liddle.[3]
Rutherford then signed a short-term deal in the summer of 2012,[4] which was later extended in early January 2013.[5] He scored his first ever professional goal after only being on the pitch for 90 seconds in a 3–1 away win against Portsmouth on 26 January 2013.[6] At the end of the 2013–14 season, Rutherford was one of four players released by Hartlepool.[7]