Spencer began his college basketball career with the Loyola Greyhounds. He averaged ten points, 3.1 assists, and 3.4 rebounds in 23 games before suffering a hip injury and was named to the Patriot League All-Freshman team.[3] Spencer missed most of his sophomore season while recovering from his hip injury. He returned for the final five games of the season and averaged 10.2 points.[4] Spencer was named first-team All-Patriot League as a junior after averaging 18.9 points, 4.8 rebounds, 3.2 assists, and 2.3 steals per game.[2] Following the end of the season, Spencer entered the NCAA transfer portal.[5][6]
Spencer ultimately transferred to Rutgers after also receiving interest from Minnesota and NC State.[7][8] He entered the 2022–23 as a starting guard for the Scarlet Knights.[9] Spencer scored 14 points on 6-of-8 shooting, including a go-ahead three-point shot with 13.3 seconds left, in Rutgers' 65–64 upset win over top-ranked Purdue.[10] Spencer was named the Big Ten Conference Player of the Week after scored another go-ahead three pointer with 15 seconds left and scored 23 points overall in a 65–62 win against Northwestern on January 11, 2023 and scored 21 points with six rebounds, six assists, and four steals as Rutgers beat Ohio State 68–64.[11] He finished the season averaging 13.2 points, 3.8 rebounds, 3.1 assists, and two steals per game.[12] After the season, Spencer decided to utilize the extra year of eligibility granted to college athletes who played in the 2020 season due to the COVID-19 pandemic and entered the NCAA transfer portal for a second time.[13]
Spencer transferred to UConn.[14] He was named first-team All-Big East Conference after starting all 40 of the Huskies' games and averaging 14.3 points, 4.9 rebounds, 3.6 assists, and 1.5 steals per game as UConn won the national championship[15] Spencer scored 11 points with eight rebounds, two assists, and two steals in UConn's 75–60 win over Purdue in the National Championship game.[16]
Spencer's older brother, Pat, was a four-time All-American and won the Tewaaraton Award as a lacrosse player at Loyola before playing a season of college basketball at Northwestern.[18] He now plays basketball professionally for the Golden State Warriors of the National Basketball Association.[19]