TrustPACTX - Design of the Hybrid Society Humans-Autonomous Systems: [A]rchitecture, [T]rustworthiness, [Trust], Ethi[C]s, and E[X]plainability (the case of [P]atient Care)_PRIN 2020

One of the main achievements of Artificial Intelligence (AI) consists in the Autonomous Systems (AS), based on Intelligent Software Agents and Multi-Agent System (MAS), plus Machine Learning capabilities (we will often refer to AS, agents or MAS interchangeably). Such systems are more and more pushing our technological reach forward, outperforming humans in an ever-growing number of fields. Despite the promise of hugely improving our quality of life, humans take an ambivalent stance w.r.t. AS. On the one hand, humans fear that AS may overcome human control, and take decisions disrespectful of the real intentions and goals of thehumans or, even worse, of the human values themselves. On the other hand, the growing success of intelligent systems in facing complex problems and the perceived benevolence of such systems may lead to uncritical acceptance of their decisions, also due to lack of awareness of users regarding systems’ policies for data collection and use. Issues concerning the relationship between humans and AS must be properly addressed, not just to avoid the risks that arise from the adoption of intelligent and autonomous systems, but mostly to allow the human society to fully benefit from their potential achievements. To this aim, the scenario that we advocate in our project is the "hybrid society", where humans and autonomous agents are coupled at multiple levels, based on shared agreed-upon principles and standards which must by definition enforce tight constraints on the behavior of agents. In such a hybrid society agents can be heterogeneous, individual agents’ goals and human users’ goals can be conflicting. Therefore the overall system’s behavior might be, without shared principles and a suitable degree of control, dangerously unpredictable. Such principles should include values and society norms, and ethical and professional conduct codes. They should be flexible, capable of evolving over time according to the change in context, needs or norm.