Griffith's coaching career began at the junior college level at South Georgia College in Douglas, Georgia. He was an assistant there in 1949 before becoming the head coach in 1950. In four years as the head football coach at South Georgia College, Griffith compiled a 32–6 record and took his team to four bowl appearances.
Griffith's coaching career with the Georgia Bulldogs began in 1956, when he became an assistant under head coach Wally Butts. He replaced Butts as head coach in 1961. Except for a Southeastern Conference championship in 1959, the Georgia Bulldogs struggled in the last few years under Butts. Things did not get any better under Griffith and he was only able to compile a 10–16–4 record during his three-year term as head coach. While there were few successes during this time as head coach, he did have two big victories, a 30–21 upset win over Auburn in 1962 and a 31–14 win over heavily favored Miami in 1963. Griffith was replaced after the 1963 season by Vince Dooley.
While coaching at Georgia, Griffith became embroiled in the controversy created when the Saturday Evening Post ran an article alleging that Wally Butts and Bear Bryant had conspired to fix games. The focus of the Post story was Georgia's 35–0 loss to Alabama and Griffith was quoted in the article as "bitterly" saying, "I never had a chance, did I" after the game. Butts sued the Post and Butts, Griffith and Bear Bryant all testified. Griffith denied that he made the statement attributed to him and otherwise called the facts of the Post story into question. Butts and Bryant also vehemently denied the truth of the story. Butts eventually won the case.
