Reinecke was born in what is today the Hamburg district of Altona; technically he was born a Dane, as until 1864 the town was under Danish rule. He received all his musical instruction from his father, (Johann Peter) Rudolf Reinecke (22 November 1795 – 14 August 1883), a music teacher and writer on musical subjects. Carl first devoted himself to violin-playing, but later on turned his attention to the piano. He began to compose at the age of seven, and his first public appearance as a pianist was when he was twelve years old.
At the age of 19, he undertook his first concert tour as a pianist in 1843, through Denmark and Sweden, after which he lived for a long time in Leipzig, where he studied under Felix Mendelssohn, Robert Schumann and Franz Liszt; he entered into friendly relations with the former two. After the stay in Leipzig, Reinecke went on tour with Königslöw and Wilhelm Joseph von Wasielewski (later Schumann's biographer), in North Germany and Denmark. In 1846, Reinecke was appointed Court Pianist for Christian VIII in Copenhagen. There he remained until 1848, when he resigned and went to Paris.
Overall, he wrote four concertos for his instrument (and many cadenzas for others' works, including a large set published as his Opus 87), as well as concertos for violin, cello, harp and flute. In the winter of 1850/51, Carl Schurz reports attending weekly "musical evenings" in Paris where Reinecke was in attendance.
In 1851, Reinecke became a professor at the Cologne Conservatory. In ensuing years he was appointed musical director in Barmen, and became the academic, musical director and conductor of the Singakademie at Breslau.
In 1860, Reinecke was appointed director of the Leipzig Gewandhaus Orchestra and professor of composition and piano at the Leipzig Conservatory. He led the orchestra for more than three decades, until 1895. He conducted premieres such as the full seven-movement version of Brahms's A German Requiem (1869). In 1865 the Gewandhaus-Quartett premiered his piano quintet, and in 1892 his D major string quartet.
Reinecke is best known for his flute sonata "Undine", but he is also remembered as one of the most influential and versatile musicians of his time. He served as a teacher for 35 years, until his retirement in 1902. His students included Edvard Grieg, Basil Harwood, Charles Villiers Stanford, Christian Sinding, Leoš Janáček, Isaac Albéniz, August Max Fiedler, Walter Niemann, Johan Svendsen, Richard Franck, Felix Weingartner, Max Bruch, Mikalojus Konstantinas Čiurlionis, Anna Diller Starbuck, Ernest Hutcheson, Felix Fox, August Winding, Elisabeth Wintzer, Mykola Lysenko, and many others. See: List of music students by teacher: R to S#Carl Reinecke.
After retirement from the conservatory, Reinecke devoted his time to composition, resulting in almost three hundred published works. He wrote several operas (none of which are performed today) including König Manfred. During this time, he frequently made concert tours to England and elsewhere. His piano playing belonged to a school in which grace and neatness were characteristic, and at one time he was probably unrivaled as a Mozart player and an accompanist. In 1904 at the age of 80, he made recordings of seven works playing on piano roll for the Welte-Mignon company, making him the earliest-born pianist to have his playing preserved in any format. He subsequently made a further 14 for the Aeolian Company's "Autograph Metrostyle" piano roll visual marking system and an additional 20 for the Hupfeld DEA reproducing piano roll system.