Skip to content
Book a consult

Insights  /  Operating model

Designing the AI operating model: roles, RACI, and governance

A structure that helps teams ship responsibly — without bureaucracy.

Sopra Labs AI  ·   ·  8 min read
SopraLabs designing hero

Key takeaways

  • When AI belongs to everyone, nobody owns it. The fix is structural: one accountable executive, a small cross-functional working group, a named owner for each use case, and decision rights written down.
  • No function is the natural home for AI. In an IAPP survey of more than 670 professionals, primary responsibility for AI governance sat with privacy (22%), legal and compliance (22%), IT (17%) and data governance (10%) (IAPP).
  • A chief AI officer is a role, not an operating model. IBM’s 2025 study of more than 600 of them found that hub-and-spoke or centralized models “see 36% higher AI ROI” than decentralized ones (IBM Institute for Business Value).
  • A RACI is a one-page table that names, for each recurring decision, who is Responsible, Accountable, Consulted and Informed. Each decision gets exactly one accountable name.

Three executives, one launch, no owner

Monday leadership meeting. A customer service assistant that drafts replies to billing questions goes live on Thursday. The CEO asks who signs off.

The head of customer service looks at the CIO. IT picked the tool and connected it to the billing system. The CIO looks back. IT set up what the business asked for; what the assistant tells a customer is a business decision. The general counsel says she learned about the launch on Friday, from the announcement.

Each of them is right. Customer service owns the customer. IT owns the system. Legal owns the exposure. Nobody was asked to own the whole thing, so nobody does.

The CEO moves the launch two weeks and asks for a name by Wednesday. It is not obvious whose name it should be.

AI arrived everywhere at once

Most business technology arrives as a project, with a budget, a sponsor and a go-live date. AI arrives through every department at the same time: as a feature inside software already licensed, as a pilot, as one manager’s weekend experiment. Each function sees the part that touches it and assumes someone else holds the rest.

Survey data shows how unsettled the question is. The IAPP and Credo AI asked more than 670 professionals in 45 countries which function holds primary responsibility for AI governance. Privacy and legal and compliance each drew 22%, IT 17% and data governance 10% (IAPP). The survey dates from spring 2024, so treat the figures as indicative. The pattern is the point: no function is the default home.

It is no clearer at the top. In McKinsey’s global survey of 1,491 participants, “Twenty-eight percent of respondents whose organizations use AI report that their CEO is responsible for overseeing AI governance” and 17 percent said the board oversees it (McKinsey).

The common reflex is to hire. IBM’s 2026 CEO study, a survey of 2,000 CEOs and equivalent leaders, reports that 76% of surveyed organizations have a chief AI officer (IBM). IBM calls that “up from just 26% in 2025,” though the 26% matches a separate 2025 IBM study with a different sample. IBM sells AI software and consulting, so it is an interested party. The 2025 study, of more than 600 chief AI officers, is candid about the limits of the title: “CAIOs can’t deliver on their broad mandate alone” (IBM Institute for Business Value).

That fits the shape of the work, which spans strategy, data, governance, platforms and tactical delivery at once. One senior hire rarely covers all five. Even one who does still depends on the people who control the budget, the systems and the legal risk.

The same IBM study points at structure instead. “CAIOs spearheading hub-and-spoke or centralized AI operating models see 36% higher AI ROI than those managing decentralized operating models.”

NIST’s AI Risk Management Framework asks for something more modest than a new executive. Roles, responsibilities and lines of communication are to be “documented and are clear to individuals and teams throughout the organization” (NIST AI RMF, GOVERN 2.1). That is a writing task before it is a hiring task.

Give AI a short org chart

The structure a mid-market company needs fits on one page.

Name one accountable executive. The sponsor is someone already on the leadership team, often the COO, who answers for the whole portfolio: what gets funded, how much risk is acceptable, and what stops. Accountable means one person. NACD’s guidance to directors says the board “should ensure that the organization has clearly designated leaders for strategic AI implementation that appropriately balance opportunity and risk” (NACD).

Stand up a small working group. Four or five people are enough: operations, IT or security, legal or compliance, and a business lead. The group sets standards and clears the path for use cases. It does not run them. Its purpose is to bring legal and security into delivery early, integrated and not bolted on at the end. NIST suggests policies that “incentivize AI actors to collaborate with existing legal, oversight, compliance, or enterprise risk functions” (NIST Playbook, GOVERN 2.1).

Give every use case one named owner. The owner sits in the business unit that benefits and answers for results, adoption and what the system does in front of customers. NACD describes the same arrangement: accountability “can remain with the functional, business-unit, or cost-center leader, supported by a centralized AI technology and governance leader.”

Write the decision rights as a RACI. A RACI is a one-page table that says, for each decision, who is Responsible for doing the work, who is Accountable for the outcome, who is Consulted beforehand and who is Informed afterward. List the decisions that recur: approving a use case, choosing a tool, granting data access, going live, handling an incident, retiring a system. Each row gets exactly one accountable name. Legal moves from informed to consulted, which is the difference between reading Friday’s announcement and shaping the design.

Choose the shape that fits your size. A central team that does all the AI work is consistent, and it turns into a queue. Fully distributed ownership is fast, and it produces duplicated tools and uneven controls. The survey evidence describes a mix. McKinsey found that risk, compliance and data governance are often fully centralized, while for adoption respondents most often report “a hybrid or partially centralized model, with some resources handled centrally and others distributed across functions or business units” (McKinsey). In a mid-market company the hub is the working group and the spokes are the use case owners. No new department is required.

The usual objection is that this adds a committee, and committees are where projects wait. PwC’s 2025 Responsible AI survey gives the same warning: committees “can also be a bottleneck if all AI systems require their review” (PwC). The RACI is the answer. The working group is consulted on most decisions and accountable for very few. Owners decide inside agreed limits, and only the exceptions travel upward.

A name by Wednesday

On Wednesday the CEO gets a name: the head of customer service. She owns the assistant and what it tells customers. IT is responsible for the integration. Legal is consulted on the reply templates before launch, not informed after it. The COO is accountable for the portfolio the assistant belongs to.

Nobody changed jobs, and nobody was hired. The company did not lack capable people on Monday. It lacked a sentence saying which of them decides.

The opening scenario is a composite of common patterns, not a client engagement. SopraLabs designs the operating model with leadership teams (roles, governance, intake, delivery and change management) and hands over the governance charter and RACI as client-owned work product; see our approach.

Frequently asked questions

What is an AI operating model?

An AI operating model is the set of roles, decision rights and routines a company uses to choose, build, run and oversee its AI uses. It answers who is accountable, who does the work, who must be asked first, and where AI sits in the organization. A policy states what the company intends. The operating model says who acts on it.

What does RACI stand for, and how do you use it for AI?

RACI stands for Responsible, Accountable, Consulted and Informed. For AI, list the decisions that recur, such as approving a use case, selecting a tool, granting data access and going live, and assign each of the four roles for every decision. The rule that makes it work is one accountable person per decision.

Do we need a chief AI officer?

Not as a first step. When IBM’s researchers contacted more than 2,300 organizations in early 2025, “only 26% said they currently have a CAIO,” and the same study concluded that those officers cannot deliver their mandate alone (IBM Institute for Business Value). A mid-market company can start with an existing executive as the accountable sponsor and add a dedicated role when the portfolio justifies it.

Should AI be centralized or owned by each department?

For most mid-market companies, a hub with owners in the departments works better than either extreme. Standards, risk review and data rules sit with a small central group, and each use case is owned by the business unit that benefits from it. IBM’s 2025 study found that hub-and-spoke or centralized models “see 36% higher AI ROI” than decentralized ones (IBM Institute for Business Value).

Sources

Keep reading

Browse all insights