Grayson attended San Diego High School, playing running back and defensive back. As a sophomore, he helped his team achieve an 11-0-1 record, while outscoring opponents 382-65 and being awarded the mythical National championship by the National Sports News Service. After the season he transferred to Lincoln High School.[1]
After graduating from high school he went on to play at San Diego City College. Besides football, in track he was a part of the 4 × 200 metres relay team that set a national junior college record.
Grayson made the team and played four years with the Dallas Texans/Kansas City Chiefs. In 1961, he set the franchise and AFL record for the longest interception return with a 99-yarder against the New York Titans. In 1961, he also led the league in kickoff returns. In 1962 and 1963, he finished second in the league in kickoff returns. In 1965, he was traded to the Oakland Raiders in exchange for cornerbackFred Williamson.
In 1967, he was moved from right cornerback to safety. He made a 48-yard return with the opening kickoff against the Oilers in the 1967 AFL Championship Game, helping his team win the game and reach Super Bowl II. In 1968, he led the AFL with 10 interceptions. His 29 interceptions rank seventh all-time in Raiders history.
Grayson is the all-time AFL leader in interceptions with 47, for a 20-yard return average and 5 touchdowns, and he averaged 25.4 yards on 110 kickoff returns. He is a member of the AFL All-Time Team.
After football he opened different businesses, which included one of the biggest nightclubs in Southeast San Diego during the seventies. He also was involved with organizations like the Boys Club, YMCA and the Committee for Community Involvement of Black Athletes.