I gain and lose fitness all the time. I've been doing a lot of cross country skiing the past few weeks and I feel like I've gained a lot of fitness I lost. I'm back skiing the whole route that the first few times exhausted me. Now it's no problem. I quickly gained back what I lost.
In as little as two weeks you can start to lose fitness if you take a break. I cross train but that still means I lose some of my fitness while I take a break from running. It's been two weeks since I last ran and I have no idea of how it will feel to run 6 miles when I do start running again. To tell you the truth, I'm kind of happy about my running break. Skiing gives me great cross training so I still feel like I won't lose too much running fitness while I'm taking my break.