back of one's mind
The remote part of one's mind or memory, as in With the idea of quitting in the back of his mind, he turned down the next assignment. [c. 1900]
The American Heritage® Idioms Dictionary Copyright © 2002, 2001, 1995 by Houghton Mifflin Harcourt Publishing Company. Published by Houghton Mifflin Harcourt Publishing Company.