Jarves was the son of Deming Jarves (1790-1869), an inventor and Anna Smith Stutson (Jarves).
After attending Chauncy Hall School in Boston, Jarves was briefly disabled with a strange disease at age 15. Though recovering, he sought warmer climes. He traveled to South America and Hawaii in 1837. There he tried journalism and silkworm husbandry before returning to Boston in 1840.
Jarves published several travel recollections, including History of the Hawaiian or Sandwich Islands and Scenes and Trip Through Central America. A second try at newspaper publishing in Hawaii in 1844 resulted in a second incarnation of the Polynesian, which the Hawaiian government bought as their official organ.
In 1848 Jarves returned to the United States where he negotiated treaties for the island kingdom. Jarves traveled to Europe in 1851 which captured his interest so profoundly, he seldom returned to the States and never again to Hawaii. A book of Parisian experiences, Parisian Sights and French Principles (1852) preceded his settling permanently in Florence.
Jarves became part of the ex-patriot cultural community there, which included the Brownings and Anna Jameson. In Florence, Jarves began his career as an art writer. His first work was an art primer for American audiences, Art-Hints, Architecture, Sculpture and Painting,1855. A travelogue filled with art description, Italian Sights and Papal Principles followed in 1856. Jarves also began assembling art collections for himself and others, focusing personally on the early Italian masters. In 1859 he approached Charles Eliot Norton hoping to sell his art collection. Norton declined and Jarves spent the next years vainly trying to recoup his investment. Jarves' Art Studies: The "Old Masters" of Italy, Painting was published in 1861.
In 1864, his most important work, The Art Idea: Part Second of Confessions of an Inquirer made press. It had first appeared as part II of his confessional Why and What I Am? The Confessions of an Inquirer (1857). In addition to the text, the book contained an essay on Jarves' personal principles of collection, based upon two earlier books by Alexis Rio and Lord Alexander Lindsay. Jarves' collecting and writing promoted the less-realistic but (as he saw it) more (Christian) spiritual dugento Italian painting over the later periods. Art Thoughts, the Experiences and Observations of an American Amateur in Europe (1869), followed. His Glimpse of the Art of Japan, 1876, was written without a trip to the country.
In the late 1870s, Jarves assembled a collection of drawings (including nine Michelangelos, two Raphaels, nine Rembrandts, and examples by Titian, Tintoretto and Leonardo) which he sold to Cornelius Vanderbilt (1794-1877), a significant enough collection to get Vanderbilt on the board of the Metropolitan Museum of Art. The Met, however, turned down an offer to buy his important Italian paintings. Instead, Yale University took them as collateral on a loan; when Jarves defaulted, a significant portion of his painting collection was acquired by Yale for $22,000 in 1871. A second part was sold in 1884 to Liberty E. Holden (1833-1913) who donated it to the Cleveland Museum of Art. Jarves served vice consul for the United States in Florence between 1880 and1882. Isabella died in 1887. The following year while vacationing in Switzerland, Jarves contracted jaundice and died. He is buried in the Protestant (English) cemetery in Rome.
Jarves is an example of the great 19th-century amateurs who helped develop American art collections, aesthetics, and popular interest in art history. The orderly historic method in his Art-Idea reflects the particular influence of the 1847 Sketches of the History of Christian Art of Alexander Lindsay (Rowland). In Art-Idea, Jarves defines art as the increasingly-naturalistic progression of objects which edify and and re-enforce morality. But Jarves distrusted the (Roman Catholic) church as commissioner of renaissance art. He praised Greek art for having thrown off the bonds of religiosity. Like most historians who defined the progress of art as ever-increasing naturalism, he was dismayed by Manet's Olympia (for example) because of the regression it seemed to make in flat painting. Today, Jarves is most consulted for his opinions, considered typical, on American artists. He disparaged Copley, considered Thomas Cole "greater in idea than action . . . " and praised Elihu Vedder. From John Ruskin, Jarves borrowed the idea of state support for the arts and a predilection for building modern construction in gothic architecture instead of renaissance. The quattrocento sculptor Apollonio di Giovanni (di Tomaso) was known as the "Master of the Jarves Cassoni" before being securely identified, from the Jarves example in Yale's collection.