If she is still in there laying and they are superceding, you would most likely have both queens living in harmony for a while, no brood break and likely the maximum number of brood the colony can support.
So I still say stop interfering - they may just make swarm cells instead and be gone in less than a week, and any eventual matings will be that much later in the season. If the queens are failing, better to change now rather than if they go drone layers or peg. Leave them alone, it is only half way through August. Drones will only be kicked out from colonies under threat of starvation at this timeof the season, not from umpteen colonies that are superceding (a good trait for a mated queen?)
Sooo many reasons to leave them alone to get on with doing what bees do best - surviving.