Panayotov began his playing career with Sportist Sofia. In 1949, after merger of Sportist and Sredets, he joined the new formed club DSO Cherveno Zname.[2]
In early 1950, Panayotov joined CSKA Sofia, where he won eleven A Group titles and four Bulgarian Cups.[3] During his fourteen years with CSKA, he scored 68 goals in 294 matches.