lose ground

1. Fail to hold one's position; fall behind, deteriorate. For example, The Democrats were losing ground in this district, or We thought Grandma was getting better, but now she's quickly losing ground. This expression originally referred to territory lost by a retreating army. [Second half of 1700s]