Unicode is that better encoding. The "small and efficient per locale encoding" that you proposed was the status quo, and was an endless source of mojibake. There is a reason we moved away from that.
I think there is a misunderstanding, which I tried to address but evidentally failed.
UTF-8 is fine for a display encoding. However, not every string encoding need be a display encoding, which the parent post seems to not be considering.
You could also have multiple display encodings, if it makes sense to (a tool only intended for use in a certain part of the world for instance), however that is not what I mean.
