Who Is Responsible When AI Makes the Decision? The Coming Crisis of Legal Accountability

Artificial intelligence is no longer merely a tool that waits for humans to tell it what to do. AI systems can now analyse enormous quantities of information, identify patterns, generate recommendations, assess risks, classify individuals, produce predictions and increasingly influence decisions that can have significant consequences for people's lives. A bank may use an algorithm to assess a loan application. An insurer may use AI to evaluate risk. A hospital may employ an AI system to assist with diagnosis. An employer may use automated systems to screen applicants. Governments and public authorities may increasingly use algorithmic systems in areas ranging from resource allocation to fraud detection. In many of these situations, AI may not formally make the final decision, but it can materially influence the decision that a human ultimately adopts. This development creates a fundamental legal question that future technology law will have to confront: when an AI system makes, recommends or materially influences a decision that causes harm, who is legally responsible?

Is responsibility with the developer who created the system, the company that deployed it, the organisation that supplied the data, the employee who relied upon its recommendation, or the professional who accepted its output? Could responsibility be distributed among several participants in the AI ecosystem? Or could there eventually be a legal argument for recognising some form of responsibility in the AI system itself? The last question may still belong largely to the realm of legal philosophy, but the first questions are already becoming practical. As artificial intelligence moves from assisting humans towards increasingly autonomous decision-making, the next phase of AI regulation may therefore be less concerned with simply asking what AI should be allowed to do and more concerned with determining who must answer when AI does something wrong.

The Emerging Accountability Gap

Traditional legal systems are largely built around identifiable human or legal actors. A person makes a decision, a company manufactures a product, a professional provides a service or a public authority exercises statutory power. When something goes wrong, the law attempts to identify the relevant actor and apply an appropriate legal principle, whether that involves negligence, breach of duty, contractual responsibility, product liability, consumer protection or another established doctrine. Artificial intelligence complicates this structure because an AI-enabled decision can involve a chain of actors rather than a single identifiable decision-maker. The developer may have designed the model, another company may have supplied the underlying system, a third party may have supplied the data, an organisation may have integrated the system into its operations, and an employee may ultimately have relied upon the output.

Consider a hypothetical automated lending system that rejects an individual's application. The applicant may not know why the application was rejected. The developer designed the model, the financial institution selected the system, the institution may have supplied or approved the relevant data, and an employee may have accepted the system's recommendation without independently examining it. If the decision was discriminatory, inaccurate or otherwise unlawful, identifying the responsible actor may become surprisingly difficult. The problem is not necessarily that there is no one who can be held responsible. The problem is that there may be too many possible points of responsibility, each attempting to attribute the decisive action to another participant. This is the emerging AI accountability gap: the machine produces the outcome, but the legal system still has to identify the human or legal actor who bears responsibility for that outcome.

AI Does Not Eliminate Human Responsibility

One possible response to increasingly autonomous AI would be to argue that AI itself should eventually be treated as a responsible legal actor. But assigning legal responsibility to a machine would create a much larger set of questions. A machine cannot presently be treated simply like a human being or ordinary legal person merely because it can generate sophisticated outputs. It does not ordinarily own assets from which compensation can be recovered, independently enter contracts in the conventional legal sense or bear punishment in the manner contemplated by most legal systems. More importantly, allowing AI itself to become the responsible party could create an undesirable legal escape route for the people and organisations that design, own, deploy and profit from these systems.

The existence of automation should not automatically result in the disappearance of human accountability. If a company deploys an AI system in a high-risk environment, the fact that the final output was generated by software should not, by itself, remove the company's legal obligations. The same principle applies to professionals and public authorities. A doctor cannot simply say that an algorithm made the recommendation if the doctor was legally required to exercise independent professional judgment. A financial institution cannot necessarily avoid responsibility by claiming that an algorithm rejected an application. A public authority cannot automatically transfer constitutional or administrative responsibility to a software system merely because the system assisted in the decision-making process. Automation may change how a decision is reached, but it does not necessarily change who is accountable for the consequences.

The AI Responsibility Chain

Future AI law may therefore have to recognise that responsibility is distributed across an entire technological and organisational chain. The developer may have responsibility for the design and testing of the system. A data provider may have responsibility for the integrity and lawful use of information supplied to the system. The deployer may be responsible for integrating the system into an appropriate environment. The organisation using the system may have duties concerning supervision, risk management and safeguards. A professional user may remain responsible for exercising independent judgment. A public authority may continue to bear responsibility where AI is used in the exercise of public power.

This suggests that future AI liability may need to move away from the simplistic question of who "made" the decision and towards a more sophisticated inquiry into who controlled the relevant part of the process, who knew or should have known about the risk, who had a legal duty to prevent the harm, who had the ability to intervene and who benefited from the deployment of the system. In some cases, responsibility may properly rest with a single actor. In other situations, several participants may bear different forms of responsibility. A future legal framework could therefore develop something resembling a chain of responsibility, where accountability follows control, knowledge, duty and risk rather than simply attaching itself to the person closest to the final output.

The Problem of the Human in the Loop

One commonly proposed safeguard against excessive AI autonomy is the requirement for human oversight. The principle is straightforward: AI may analyse information or make a recommendation, but a human remains responsible for the final decision. The difficulty is that human involvement does not necessarily mean meaningful human control. An employee who receives thousands of algorithmically generated recommendations may technically review every decision while practically having neither the time nor the information necessary to question them. If organisational culture strongly encourages employees to accept AI recommendations unless an obvious error appears, the human being may become little more than a rubber stamp.

Future law may therefore have to distinguish between human presence and meaningful human oversight. A human being should not be considered genuinely in control merely because their name appears at the end of an automated process. Meaningful oversight may require that the person responsible for the final decision has sufficient information, expertise, authority and time to assess the AI output and, importantly, has the practical ability to reject or override it. If the human operator cannot realistically challenge the system, the law may eventually treat the process as substantially automated even if a human technically signs off on the result.

When the AI Becomes a Black Box

Another major challenge concerns explainability. Suppose an AI system produces a decision that has serious consequences for an individual. The individual challenges the decision and asks why it happened. The organisation responds that the algorithm produced the result. Such an answer may become increasingly difficult to reconcile with principles of legal accountability. This does not necessarily mean that every organisation should be required to disclose proprietary source code. Commercial confidentiality, cybersecurity and intellectual-property considerations can legitimately limit disclosure. But protecting proprietary technology is different from preventing meaningful legal scrutiny.

Future AI governance may therefore require organisations to maintain an adequate audit trail surrounding consequential automated decisions. The relevant records could include which system and version were used, what categories of information were considered, what output was generated, what level of human intervention occurred, who authorised the final action and what safeguards were operating at the time. The objective would not necessarily be to make every algorithm completely transparent to every person. Rather, it would be to ensure that when a legally significant decision is challenged, the organisation cannot simply point to the machine and declare that nobody knows how the decision occurred. Accountability requires traceability.

The Question of Foreseeability

Traditional liability law frequently asks whether harm was reasonably foreseeable. Artificial intelligence complicates that question because advanced systems can identify patterns and generate outputs that were not expressly programmed by their developers. Their behaviour may depend upon training data, system architecture, deployment environment, updates, user inputs and interactions with other technological systems. When an unexpected outcome occurs, a developer may argue that the precise behaviour could not have been anticipated.

That defence may have limits as AI becomes more sophisticated. The law may gradually shift from asking whether a particular harmful outcome was specifically foreseeable towards asking whether the category of risk was reasonably foreseeable and whether appropriate measures were taken to identify, test and mitigate it. In other words, developers and deployers may increasingly be expected not to predict every individual action of an AI system but to conduct serious risk assessment and establish safeguards against reasonably foreseeable classes of harm. This would represent an important evolution in technology liability, because the legal focus would move from predicting the exact behaviour of a machine to demonstrating responsible management of technological risk.

AI and Professional Responsibility

The accountability problem becomes even more important when AI enters professions that already operate under established standards of professional responsibility. Lawyers, doctors, engineers, financial advisers and other professionals may increasingly use AI systems to research, analyse, recommend or generate information. The technology may significantly improve efficiency, but it does not automatically remove the professional duties attached to the person using it. A lawyer who relies upon an AI-generated legal proposition may still be expected to verify it. A doctor using an AI diagnostic system may still have to exercise professional judgment. A financial professional may still have duties towards clients even where an automated system has generated the underlying recommendation.

The law may therefore have to distinguish between using AI as an assistive technology and delegating professional judgment to AI. The former may simply represent the modern evolution of professional practice. The latter could raise much more difficult questions about negligence, standard of care and professional accountability. The fact that an AI system was involved may become relevant to determining whether reasonable care was exercised, but it should not automatically become a mechanism through which professional responsibility disappears.

Does Existing Law Need to Change?

Existing areas of law will remain important in dealing with AI-related harm. Contract law can regulate relationships between developers, suppliers and users. Consumer protection law may provide remedies where AI-enabled products or services cause harm. Tort law may address negligence and breach of duty. Product liability principles may become relevant to certain AI-enabled products. Data-protection law may address unlawful processing or misuse of personal information. Constitutional and administrative law may become increasingly important when public authorities employ AI in decision-making.

The difficulty is that AI can cut across all of these legal categories at once. A single AI system can involve software development, data processing, professional decision-making, consumer transactions and the exercise of public or private power. The challenge for future lawmakers may therefore not be to replace existing legal doctrines but to create a framework that allows those doctrines to operate effectively in an AI environment. The central objective should be that technological complexity does not make legal remedies more difficult to obtain.

Should AI Developers Be Strictly Liable?

One of the more difficult policy questions is whether developers of high-risk AI systems should bear a heightened level of liability. There is an argument that those who create powerful technologies should bear greater responsibility for the risks inherent in their systems. There is also a counterargument that organisations deploying AI often have greater knowledge of the particular environment in which the system is being used and therefore should bear significant responsibility for implementation, supervision and safeguards.

A blanket rule may therefore be difficult to design. Different AI systems create different levels of risk, and responsibility may depend heavily upon context. A low-risk generative tool used for drafting a routine document is fundamentally different from an AI system used to determine whether a person receives medical treatment, employment, credit or access to a public service. Future AI liability law may consequently need a risk-based approach in which the level of legal responsibility corresponds to the potential consequences of the system and the degree of control exercised by each participant. The objective should be to create accountability without making technological development legally impossible or concentrating innovation exclusively in organisations capable of absorbing unlimited regulatory risk.

India's Emerging Challenge

For India, these questions are becoming increasingly important as artificial intelligence enters banking, healthcare, education, employment, commerce, public administration and other sectors. India's existing legal framework already contains principles that may become relevant to AI accountability, including constitutional protections, information-technology law, data-protection regulation, consumer law, contract law and established principles of civil and professional liability. Yet the increasing use of automated and semi-autonomous systems raises a broader question: are existing principles sufficient when the decision-making process itself becomes partly or substantially automated?

India does not necessarily need to reproduce another country's approach to AI regulation. A future Indian framework could instead build upon principles that are consistent with India's constitutional and legal traditions, including human accountability, transparency, traceability, proportionality, risk management and effective remedies. The most important question may not ultimately be whether India adopts legislation carrying the title "AI Act." It may be whether an individual affected by an AI-driven decision can identify who was responsible for the relevant process, understand enough about the decision to challenge it and obtain an effective legal remedy when the decision causes unlawful harm.

The Future May Require an AI Chain of Responsibility

Perhaps the most useful way of approaching future AI liability is to stop asking simply, "Who is responsible for AI?" and instead ask, "Who was responsible for each stage of the AI decision?" That change in perspective is significant. A developer may be responsible for designing and testing the system. A data provider may have responsibility for the quality and legality of information supplied to it. A deployer may be responsible for appropriate implementation. An organisation may be responsible for supervision and safeguards. A professional may remain responsible for the exercise of professional judgment. A public authority may remain responsible for decisions made in the exercise of statutory or constitutional power.

This approach would allow responsibility to follow control, knowledge, duty and risk. It would also recognise an important reality of artificial intelligence: the final output may be generated by a machine, but the conditions that produced that output are created by humans and organisations. The law therefore does not necessarily need to identify a single person who "made" the AI decision. It may instead need to identify where legal duties existed along the chain and determine whether those duties were properly discharged. Such a framework could become one of the foundations of future AI accountability.

Can Humans Remain Accountable in an Autonomous Age?

The debate over AI law is often framed around regulation: how much should governments regulate artificial intelligence, which applications should be prohibited, which systems should require certification and what information should developers disclose? These questions are important, but underneath them lies a deeper legal question. Can the law remain fundamentally human-centred when decision-making itself becomes increasingly machine-driven?

For centuries, legal systems have been built around human agency. A person acts, another person suffers harm, and the law determines responsibility. Artificial intelligence introduces a new possibility: a system can process information, reach conclusions and initiate or influence actions with varying degrees of human intervention. The law must therefore determine where human responsibility remains, how it should be distributed and whether entirely new legal categories will eventually be required. This becomes particularly important as AI moves from being an assistant to becoming a decision-maker and potentially, in some contexts, an autonomous actor.

The Law Cannot Outsource Responsibility to the Algorithm

The ultimate challenge of AI law may therefore not be simply controlling artificial intelligence. It may be preventing humans and institutions from using artificial intelligence as a shield against responsibility. An algorithm should not become an answer to the legal question, "Who made this decision?" If an organisation chooses to deploy an AI system, it should remain accountable for understanding and managing the risks associated with that deployment. If a professional relies upon AI, professional responsibility should not simply disappear. If a government authority uses AI to exercise public power, constitutional and administrative accountability cannot simply be delegated to software. And if developers create systems capable of producing consequential outcomes, the law may increasingly demand evidence that reasonable safeguards have been incorporated into their design and deployment.

The future of AI law will therefore not be determined solely by how intelligent machines become. It will also be determined by whether the legal system can preserve human accountability in a world where machines increasingly participate in human decisions. The central principle may ultimately be straightforward: AI may make the decision, but the law will still have to decide who answers for it.

Ambat Legal Insight Perspective

The next generation of AI regulation should move beyond the question of what AI can do and focus increasingly on who bears responsibility when AI acts. The law does not necessarily need to treat AI as a legal person. Instead, it may need to establish a sophisticated framework in which responsibility follows control, duty, risk, knowledge and the ability to prevent harm.

The real test of future AI governance will not simply be whether governments can regulate machines. It will be whether an individual harmed by an AI-driven decision can still turn to the legal system and ask a question that has existed for centuries: Who is responsible?

That may become one of the defining legal questions of the artificial intelligence era.