He was the posthumous child of Prince Pavel Sergeyevich Gagarin (1747–89), the Commander of the Moscow Kremlin. After his mother's death in 1800, Pavel and his elder brother Andrew were brought up at a private boarding school in Moscow.
He started his career as an aide-de-camp to Mikhail Miloradovich and other notable commanders but, citing ill health, moved to the civil service in 1809. He married Maria, the only daughter of Georg Johann von Glasenapp, Vice-Roy of Siberia. She became known as "the Princess Termagant" for her dreadful character and haughty manners.
After Alexander's accession to the throne, Gagarin was involved, together with Yakov Rostovtsev, in the activities of a secret committee that prepared the Emancipation Reform of 1861. Always keen to support the party of wealthy landowners, Gagarin was made Chairman of the Committee of Ministers in 1864.
