Jones was born March 28, 1923, in Pontiac, Mich. He came from a musical family, to say the least; he's the younger brother of pianist Hank Jones and older brother of fellow Down Beat Hall of Famer drummer Elvin Jones. His musical career began in 1939, and he worked with various local bands throughout the 1940s. He spent the early '50s in a quintet with his brother Elvin and pianist Tommy Flanagan and played with Charles Mingus, who called him the best trumpeter he'd ever heard. Jones was hired in 1954 by the Count Basie Orchestra, where he was a key member for nine years.

After leaving the Basie band in 1963, Jones played in bands led by George Russell and Gerry Mulligan, composed for the Basie and Harry James big bands, and launched a quintet with baritone saxophonist Pepper Adams. In 1965, Jones and drummer Mel Lewis formed an all-star big band of top New York jazz musicians. The Jones-Lewis Orchestra recorded a dozen albums, won several Down Beat Readers and Critics polls, and began a renowned streak of Monday night gigs at the Village Vanguard.

Jones moved to Copenhagen in 1978, where he directed the Danish Radio Orchestra, taught jazz at the Royal Conservatory and gigged and recorded throughout Europe. He returned to the United States in 1985 to lead the Basie orchestra after the bandleader's death, then returned to Copenhagen, where he died of cancer on Aug. 20, 1986.

A strong advocate for jazz, Jones said in the August 1985 issue of Down Beat: "Jazz, to me, has been the most vital and progressive music of the last 200 years. A jazz musician of the caliber of a Freddie Hubbard or a Miles Davis can create more in two minutes, spontaneously, than some orchestras can in 25 minutes. That's a marvelous gift that shouldn't be allowed to die or to wither. It should be nourished—especially in America, where jazz was really born."

In 1987, Jones was elected by the Critics into the Down Beat Hall of Fame.