on the safe side
Avoiding danger, with a margin for error, as in Just to be on the safe side, let's order another hundred chairs. This idiom was first recorded in 1811.
The American Heritage® Idioms Dictionary Copyright © 2002, 2001, 1995 by Houghton Mifflin Harcourt Publishing Company. Published by Houghton Mifflin Harcourt Publishing Company.