My guess? Bae could have spent time in Arendelle after Neverland, and somehow (The Black Fairy?) ended up being sent to our world. Rumple, hearing that his son is there, goes to the kingdom and demands Elsa to return him. Because he is not there anymore, Elsa cannot, enraging Rumple who banishes her to the vault, events happen with Anna and the kingdom, and now it is destroyed and Anna gone.