In the episode Manhattan, Rumple’s deal with Emma is specifically to get Neal to talk to him – this makes me think that Rumple and Neal had seen each other at least once before they were first separated. Possibly in Neverland.
I could see Bael/neal refusing to talk to Rumple when/if they met in Neverland. Rumple could be the reason that Bae/neal was able to escape Neverland.
Hook and Bae could have left at the same time. Hook went to enchanted forest and got frozen in time with Cora, while Bae/Neal came to our world and met grew up and met Emma.
[adrotate group="5"]