This is what I think happened:
Bae was transported to some realm (our world or another). There, he met Jefferson. Jefferson became a sort of fatherly figure to Bae since he had lost Rumple. Then, Jefferson told Bae about realm jumpers, or whatever he was and how it would be possible to travel through time and space. Bae had nothing except Jefferson, but Jefferson then had Grace and wasn’t going to stay in one dimension forever anyway, so he basically had nothing. He became a realm jumper (or something of the like) and explored other realms.
Maybe something happend with Bae in another realm or something that made Jefferson not want to use the hat anymore, realizing it’s danger. But Bae decided to continue traveling through time and space.
He spent some time in our world, where he met Emma. Thus, stuff happened and Henry was created. But before Bae knew about Henry, he went back to the hat. Maybe there was something left behind that he couldn’t leave unfinished. Maybe he just couldn’t stay away from traveling because it was all he’d ever known and feared staying in our realm for Emma. Maybe he found out about Henry and didn’t want all of the responsibility, just like Emma. Whatever the case, he left.
Maybe he went back to the EF to look for Rumple.
Whatever happened to make him do so, I predict he then went to FTL. But it was post-curse. All of his loved ones, the people he had known and left behind, were gone, and the forest was in ruins.
So, he returns.
And maybe this is when we’ll see him.