Social scoring
An AI system that evaluates or classifies natural persons based on their social behaviour or known, inferred or predicted personal characteristics, leading to detrimental treatment that is unrelated to the context or disproportionate, is prohibited. The prohibition applies regardless of whether the system or the score is provided or used by a public or private actor.
The prohibition is set out in Article 5(1)(c). The offence consists of two cumulative elements: the score must be based on social behaviour or personal characteristics, and the resulting treatment must be detrimental in a way that is unrelated to the context in which the data was generated, or disproportionate to it. The extension to the private sector is an important practical point: it is not only a state "social credit system" that is prohibited, but any private company's internal scoring system as well, if the above conditions are met — for example, if a customer's past behaviour in relation to one service is used to deny an entirely unrelated service. Point 5(c) of Annex III can also be relevant in certain creditworthiness-assessment contexts, where distinguishing the high-risk classification from the outright prohibition requires careful analysis.
When designing internal customer or partner scoring systems, it must be explicitly considered whether the score could lead to detrimental treatment that is unrelated to, or disproportionate to, the context that generated it — this is not a theoretical risk but a concrete design consideration.
Need documented AI-literacy training?
Article 4 is a duty of diligence: what counts is not knowledge in the abstract, but demonstrable, documented effort. Our starter package lets you begin free.
Start free