Just because the Blue Fairy said Bae was going to a land without magic, doesn’t mean he went there- first. We all know the BF lies. If Bae/Peter Pan went to Neverland, that would explain his not aging. PP could travel freely between Neverland and our world. If he happened to be in our world when the curse struck, that could have made him be stuck here and start aging. Rumple wrote the curse to take him to where Bae was- at the time of the curse- which was our world.
IDK, for sure, but, it’s plausible 🙂