Could be that there are still small particles of the smoke being exhaled on your "dry exhale" that your olafactory receptors can then pick up on because they are not being overwhelmed by a large volume of smoke. I don't necessarily do the dry exhale, but I do sometimes retrohale in tiny micro-bursts interspaced with a few seconds between them. This is most often toward the end of the retrohale or just with a small portion of the smoke remaining from the draw.

Think about how after eating something with a very strong flavor or is spicy that your taste buds are muted for a while and need time to recover.