Graham and Emma will still have a sense of story destiny to them as well. Graham spared Snow and helped Charming escape which lead to the birth of Emma Swan.
Anyway I never started shipping Swanfire right away their story grew on me overtime. So if Neal and Emma were to move on to others I would be perfectly fine with that.
I’m just not okay with the pirate. I don’t think they work, their history is good, nor meld well as a story that ties everything together.
There is gold and then there is fool’s gold.
But that is just my humble not so important opinion.