He married Lady Mary Howard, daughter of Lord Thomas Howard and Mary Savile, and sister of Thomas Howard, 8th Duke of Norfolk and Edward Howard, 9th Duke of Norfolk, by whom he had five sons and six daughters, most of whom died in infancy. She died in 1723 during the birth of their 11th child, James, who would succeed him as Lord Aston of Forfar. Two daughters also reached adult life: Margaret, who became a nun, and Catherine, who married Edward Weld of Lulworth Castle, Dorset. The Astons, like the Howards, were staunch Roman Catholics: during the outbreak of anti-Catholic hysteria called the Popish Plot, Walter's father and uncle had been charged with treason and imprisoned in the Tower of London, although they were never brought to trial, and in due course were simply released.
The 4th Lord Aston commissioned a painting of himself, his wife, and seven of their children by Richard van Bleeck. The painting, which survives, was not completed until two years after Lady Aston's death.