piece by piece
In stages, gradually, as in He took the clock apart piece by piece, or Let's go over your exam paper, piece by piece. Also see bit by bit.
The American Heritage® Idioms Dictionary Copyright © 2002, 2001, 1995 by Houghton Mifflin Harcourt Publishing Company. Published by Houghton Mifflin Harcourt Publishing Company.
Examples from the Web for piece by piece
Nothing on immigration reform (or an incremental, piece-by-piece approach, which will result in nothing).All I Want for Christmas Is a New GOP
December 24, 2012