There will always be some sort of management UI, the scope of that UI seems to change at higher tiers; ie; in a hamlet/village you still have UI's to manage buildings, but you cant manage the town as a whole
https://chroniclesofelyria.gamepedia.com/Settlement
"As a hamlet/village, there's no special UI and they basically just work with their fellow hamlet or village elders to decide where to put plots of food, wells, and other small buildings. That’s it. Beyond that they can play however they want.
As a Town and above, there's a Town Management UI which lets you see your boundaries, zone parcels of land for leasing (residential, commercial, industrial), monitor settlement resources and taxes, and lets you identify where you want different types of structures to be built. This is, as you guessed, a table, much like the Count has, which goes in the Town Hall. Through this table, people can see the "town plan". And can work to bring the mayor’s vision of the town to life."