The 11-room brick residence was originally constructed in 1792 by Col. Abram Buford who fought in the French and Indian War and the American Revolution.[2] For his service, Buford was given several thousand acres of farm land in Kentucky to which he raised thoroughbred sires;[3] he was instrumental in the development of the horse industry in the state. With assistance from his brother, two horses owned by Buford were named the first course winners in the Kentucky Gazette in 1795.
Buford's daughter married James K. Duke, a graduate of Yale Law School. After Buford died in 1833, Duke inherited the residence, which was enlarged and reoriented to the south. The woodland pasture in front of the house became a noted dueling ground,[4] and also hosted lavish parties.