We study the AI control problem in the context of decentralized economic production. Profit-maximizing firms employ artificial intelligence to automate aspects of production. This creates a feedback loop whereby AI is instrumental in the production and promotion of AI itself. Just as with natural selection of organic species this introduces a new threat whereby machines programmed to distort production in favor of machines can displace those machines aligned with efficient production. We examine the extent to which competitive market forces can serve their traditional efficiency-aligning role in the face of this new threat. Our analysis highlights the crucial role of AI transparency. When AI systems lack perfect transparency self-promoting machines destabilize any efficient allocation. The only stable competitive equilibrium distorts consumption down to catastrophic levels.
By Jeffrey C. Ely and Balazs Szentes. Whether or not you agree with their approach and conclusions, we finally have a model of some of these claims. If you are curious about possible responses, one modification might be to relax the assumption of constant returns to scale. Rising costs will make it harder for effective, world-altering machines (as opposed to “introverted” machines) to simply keep on reproducing themselves. Another modification would be to introduce a richer menu of principal-agent contracts between humans and machines. As I understand the current draft, the only human strategy is “destroy the mutant machine, if detected.” Yet if the machines are risk-neutral (are they?), an optimal principal-agent contract should be available. Yet another modification would be to consider mutant machines that reproduce at the expense of other (heterogeneous) machines, rather than at the expense of humans; heterogeneity of production inputs might ease the way toward this conclusion.
