He was the son of the American sculptor Francis Montague Handley and his Italian wife. He was registered in Rome as an Italian citizen with the baptismal name of Luigi and the surname of his mother, "de Breda".
In 1896 he fled to New York and added to his name the surname of his father. He worked in a small firm imports and devoted himself to his second passion after hunting, swimming. He was also a great water polo player (his style of shooting was called "jumping salmon").
In the 1904 Olympics he won a gold medal in the 4x50 yard freestyle relay, and was a member of New York Athletic Club water polo team, which won a gold medal. He also competed in the one-mile freestyle but did not finish.