The Pentagon’s Defense Advanced Research Projects Agency (DARPA) has momentarily turned its attention away from self-flying helicopters and human-spotting robots to provide resources for a strange implementation of AI. With funding provided by the agency, researchers at Israel’s Ben-Gurion University have created an AI tool that detects violations of social norms. The tool reliably spots infringements on 10 different types of social norms, offering psychologists and sociologists a unique way to study human behavior.
According to a paper published in the journal Scientific Reports, researchers Yair Neuman and Yochai Cohen used GPT3, natural language interface-based (NLI) zero shot text classification, and automatic rule discovery to create the system. GPT3, a powerful language prediction model, allowed the researchers to form a foundational set of social norm categories upon which violations could be assessed. Because the system would be trained on texts, Neuman and Cohen used NLI-based zero-shot classification to analyze the premises and relationships within each text sample. Meanwhile, automatic rule discovery created a basis for violation-spotting that could be understood by social scientists engaging with the tool rather than producing the unintelligible black-box process generated by most models. (This is the AI version of your grade school math teacher telling you to “show your work.”)
The tool identifies violations across 10 social norm types: competence, politeness, trust, discipline, caring, agreeableness, success, conformity, decency, and loyalty. When the researchers input a text scenario, the tool assesses whether the text describes a social norm transgression. In a soap opera-worthy text about a married couple’s tendency to cheat on one another, the tool identified shame. Conversely, in a text describing academic success, the tool found that the “narrator” had obeyed a social norm (i.e. doing well in school) and was experiencing a reward (pride) for doing so.
“When people feel shame, embarrassment, or regret, it is hypothesized that they acknowledge the violation of a social norm,” Neuman and Cohen write. “These consequential social emotions seem to be evolutionary grounded in mechanisms of social devaluation and have a clear function in supporting cooperation.”
The researchers point out that the importance of social norms varies from social groups and even individuals, meaning some people might abstain from meeting the terms of a particular social norm when they believe it’s “the right thing to do.” As a result, tools like this one might offer a new pathway to study human decision-making, values, cooperation, and even altruism (since violating a social norm typically carries a consequence, as described above). That’s all well and good—probably—but it doesn’t explain why the Pentagon would be interested in the project enough to provide funding for it. As a result, one has to wonder why DARPA has its hands in a system that not only studies social norms but identifies the violations that are bound to happen within any social environment.
