Fundamentally, basically, as in He's a good fellow at heart. It was first recorded in Alexander Pope's Epistle to a Lady (1735): “But every Woman is at heart a Rake.”
The American Heritage® Idioms Dictionary Copyright © 2002, 2001, 1995 by Houghton Mifflin Harcourt Publishing Company. Published by Houghton Mifflin Harcourt Publishing Company.
How to use at heart in a sentence
With the collection, Lagerfeld offered a young-at-heart interpretation of his high art.
Did each determine to see all the good possible in the other, connections-by-marriage might become kin-at-heart.The Secret of a Happy Home (1896) | Marion Harland
Did the poor mocked-at heart divine what might be then enacting?The Ordeal of Richard Feverel, Complete | George Meredith