Not only is there the whole “you killed my mother” aspect, but Rumple lied to Bae about where Milah went when Bae was a child, before he was even the Dark One. He told Bae that Milah was dead, though Bae apparently had some doubts even then.
Actually, I think the lie may hurt more than the killing; Bae knows full well what his father was capable of as the Dark One, and since he had already believed his mother dead (and his mother had abandoned him at a fairly young age–5-8 from the kid’s appearance at the time) he may not have been quite as attached to her as he was to his father. It would draw a nice parallel to Henry, too, who was hurt more by the lying that both his mothers did than just about everything else they could have done to him.
Keeper of Belle's Gold magic, sand dollar, cloaks, purple FTL outfit, spell scroll, library key, copy of Romeo and Juliet, and cry-muffling pillow, Rumple's doll, overcoat, and strength, and The Timeline. My spreadsheet: https://drive.google.com/file/d/0B6r8CySCCWd9R0RUNm4xR3RhMEU/view?usp=sharing