Whether Bae remained in Neverland or travelled to other worlds before coming here doesn’t matter. The cruel irony remains, had Rumple known Bae didn’t directly come to a land without magic, he would have probably found him without needing the dark curse. Jefferson offered Rumple the use of his hat but Rumple turned it down because the hat only travelled between lands with magic. Bae was in a land with magic. Rumple turned down a means to finding his son without realising it.