They could stick Felix in the sheriff station cell or he could join George in the mines, or he could even live in the woods in August’s old caravan.
I think once Pan is out of the equation, Felix would no longer be a threat. He’s more a thorn in the heroes’ side rather than an actual villain like Pan.