Could be, or perhaps the night we saw Milah in the tavern wasn’t the first night she’d been there with Killian.
When the woman knocked on Rumple’s door telling him Milah had been kidnapped, she said “the men who came into port last week” had taken Milah. Being pirates, it is likely their first stop would’ve been the local tavern and since Milah seemingly spent most of her time there, it is highly likely Hook and Milah had spent time together there.
Milah may have fallen in love with Hook “that first night” but she probably wanted to make sure it was love and not a crush. After a week, and still feeling the same about him, she probably then decided to leave with him.
Milah isn’t the type to risk losing the roof over her head for something she couldn’t trust was real. Rumple provided for Milah, if she’d left Rumple for a short lived fling then she would’ve ended up homeless or worse, she would’ve had to beg Rumple to take her back. I doubt adultery would’ve been held in high regard so she would’ve been known as the adultering wife of a coward.
All magic comes with a price!
Keeper of Felix