When machines are credited with morals

In a cross-sectional survey of 758 participants, the perceived agency of language-model-based moral advisors was found to be positively associated with the perception of their morality. Anthropomorphism and perceived animacy increase perceived agency, which fully mediated their effect on attributed morality, while likability and perceived intelligence influence morality directly without affecting agency.

Teams weighing whether to bring in chatbots as ethics advisors face the question of whether users will see them as genuine moral agents or simply as persuasive tools. That uncertainty affects decisions on adoption, governance and internal communication, because overestimating a model's moral capacity can lead to delegating complex judgments without adequate oversight.

To clarify the question, the article published in Frontiers in Artificial Intelligence ran a cross-sectional survey with 758 valid participants, who read short moral dilemmas about everyday situations of young adults and imagined seeking advice from an advisor based on a large language model. The authors measured, using Likert scales, the perception of agency (autonomy, intentionality and self-direction) and of morality, as well as four attributes of the model: anthropomorphism, perceived animacy, likability and intelligence. They used structural equation models to examine how those attributes related to the two main perceptions.

Product leads can use these insights to design advisors that avoid inflating the perception of agency: limit anthropomorphic and animacy cues in the interface, stress transparency about the model's statistical workings and add reminders that the system has no intentions or moral responsibility. Working on likability and cognitive clarity also improves acceptance without creating a false sense of ethical authority.

Conclusions

• Users tend to judge systems as more morally capable when they attribute autonomy and intention to them.
• Human appearance and perceived vitality are the main drivers of that attribution of agency.
• Likability and cognitive competence can improve the perception of morality without the system being granted autonomy.
• For designers, it is crucial to calibrate social and cognitive cues to avoid overestimating the moral authority of advisors.