There's no way to understand anything you can't map on to your embodied experience of the world.
You can only even perceive their apparent desires to the extent you can map them onto your own experiences.
You could offer them language for controlling you, and they could learn to use it according to their desires, through trial and error, but without shared bodily experience you cant understand those desires.
And them being able to control you doesn't solve the benevolence problem. Because benevolence is understanding without ceding control. For you to be benevolent you need to actually understand them, which requires shared experience.
This is why I think consent is a much better basis for social coordination than benevolence.
In the case of AIs it is somewhat moot, because the first general/hard AIs will be clones of humans in humanoid bodies.
You can only even perceive their apparent desires to the extent you can map them onto your own experiences.
You could offer them language for controlling you, and they could learn to use it according to their desires, through trial and error, but without shared bodily experience you cant understand those desires.
And them being able to control you doesn't solve the benevolence problem. Because benevolence is understanding without ceding control. For you to be benevolent you need to actually understand them, which requires shared experience.
This is why I think consent is a much better basis for social coordination than benevolence.
In the case of AIs it is somewhat moot, because the first general/hard AIs will be clones of humans in humanoid bodies.