Parallel Anthology Track 17
Track 17: The Carter Family - John Hardy was a Desperate Little Man Victor 40190-A; Recorded in Camden, New Jersey, 10.5.1928
The Roud Folksong Index classifies this song as a version of Roud 3262. A Roud number search for this tune returns a number of versions collected prior to the Cater Family's recording, dating back to around 1900 and from various states (N Carolina, Kentucky, W Virginia).
According to the Where Dead Voices Gather blog, "The real John Hardy was an African-American working in the railroad tunnels of West Virginia. He murdered a co-worker during a crap game and was sentenced to hang on January 19, 1894."
This article:  from 1919 claims that this song had been in existence since the 1890s, and that there were several versions. Many of the lyrics of the Anthology version can be found among the various versions given in the article. Although on the Anthology, this track and Track 18, "Gonna Die with a Hammer in My Hand", are separate songs, the 1919 article discusses the possiblity that John Hardy (famous for a murder) and John Henry (famous for his ultimately fatal competition with a machine) were in fact the same person. Some lyrics from Track 18 can be seen in one of the versions of "John Hardy" given in the 1919 article.