In 1921, Agranov was the chief investigator into the "Petrograd military organization", allegedly headed by Vladimir Tagantsev. Tagantsev was arrested and then tricked into giving names 300 "conspirators", who, he was told, would not be executed.[2] in exchange for leniency for himself. The investigation ended with more than 85 persons being sentenced to death, including Tagantsev himself and the poet Nikolay Gumilyov. All concerned were promptly executed. When asked why he was so merciless, Agranov responded: "Seventy percent of Petrograd intellectuals were standing by one leg in the camp of our enemies. We had to burn that leg off".[3]
Agranov was also implicated in suspicions concerning the suicide of poet Vladimir Mayakovsky in 1930. Mayakovsky, a former lover of Lilya Brik, shot himself with a gun given to him as a gift by Agranov, who, himself, had had a later affair with Brik, well known as the muse of Mayakovsky.[4]
Immediately after the assassination of Sergey Kirov in Leningrad on 1 December 1934, Agranov was entrusted with the organization of mass reprisals in the city. Stalin ordered him to fabricate a story that Lev Kamenev, Grigory Zinoviev and other leaders of the opposition were responsible for the murder, but he seems to have resisted, whereupon Stalin entrusted Nikolai Yezhov with the job instead.[5] In 1935, he was ordered by Yezhov to track down and liquidate "an undiscovered centre of Trotskyists" in Moscow, as a preparatory step for the Great Purge that Stalin was planning. When Yezhov took over as head of the NKVD, Agranov remained his First Deputy, and in December 1936 was appointed Head of the Chief Directorate of State Security, which seemingly meant that he was to be trusted to purge the NKVD of officers Yezhov did not trust. In February 1937, he circulated among regional NKVD heads a demand for names of Trotskyists and other oppositionists employed with the state security apparatus. In April, he was demoted to the post of regional NKVD chief in Saratov. He was arrested on 20 July 1937, and appeared on the execution list of 1 November 1937, in which his name was crossed out. He was executed by firing squad as an "enemy of the people" on 1 August 1938.[6]