Nope. Adam explained that it’s one Agrabah, but it’s one large region split into different cities. Jasmine’s father rules the capital, while Jafar’s father ruled a different city in Agrabah. That’s why the caption said “Capital of Agrabah”.
I was thinking of realms as parallel dimensions. OUAT already has its own version of DC’s multiverse, with at least one fictional London alongside the real one, so the framework is already there if they want to explain the two Jafars that way.