Also, in The Return, when Rumpel was talking to August (who was pretending to be Bae), he said that Bae had “crossed the barriers of time and space”.
Oh, I hadn’t though about that! He did say time. But how could he know what time period Bae went to? All he knew was that he went to a land without magic. But still, what you said makes sense, that could very well be an explanation.