Holst was born in Drammen, Norway. He was a son of bookseller Adolph Theodor Holst and Amalie Fredrikke Bergh.[1] He was a grandson of merchant and politician, member of the Storting, Elling Mathias Holst (1785–1852).[2][3]
Holst enrolled as a student at the University of Christiania (now University of Oslo), his doctoral advisor was Sophus Lie,[4] and he graduated as cand.real. in 1874. He continued his studies in Germany, where Felix Klein was among his teachers. He was appointed teacher at Aars og Voss skole in Christiania (now Oslo). His thesis Et par syntetiske Methoder, især til Brug ved Studiet af metriske Egenskaber was finished in 1882.[5][1]
Holst is particularly known for his children's books Norsk Billedbog for Børn, three collections from 1888, 1890 and 1903 (with illustrations by Eivind Nielsen).[1] The first of these books has been called Norway's first national picture book (although a picture abc had been published previously, in 1876).
[2] Holst started collecting traditional poems for children, several of which were first published in Norwegian writing in these books. These poems, such as "Ride, ride ranke", "Bake kake søte", "Kjerringa med staven", "Hoppe! sa gåsa" and "Du og jeg og vi to", have had a constant popularity over many years, and Norsk Billedbog for Børn has been reissued several times.[1] Also contemporary poetry was included in the books, such as some poems by Henrik Wergeland, "Kom bukken til gutten" by Bjørnstjerne Bjørnson, and "Blaamand" by Aasmund Olavsson Vinje.[2]
Among his other children's books are Julegodter for Børn from 1892, and A.B.C. for Skole og Hjem from 1893 (together with Anna Rogstad).[1] He published the picture book Fra Sæteren in 1899, with illustrations by Lisbeth Bergh.[2]