Making use of the Enterprise Threat Mindset to AI | Mayer Brown

(co-authors: Meghan Milloy, Matt Saidel)*

AT A GLANCE

Synthetic intelligence (AI) and different rising applied sciences have the potential to revolutionize the monetary {industry}. On the similar time, its use introduces new dangers that should be anticipated and addressed. This paper explores these enterprise danger points raised by means of AI instruments.

---

Synthetic intelligence (AI) and different rising applied sciences have the potential to revolutionize the monetary {industry}. In actual fact, many monetary providers companies already use AI, although most organizations are within the early levels of adoption and integration. In our monetary providers report—The Next Organization: Seven Dimensions of a Successful Business Transformation—we notice that greater than seven in 10 leaders of monetary establishments (71%) and over eight in 10 leaders of funding companies (83%) stated that, within the subsequent three years, pervasive AI may have a big influence in the marketplace surroundings. Nevertheless, fewer than a 3rd of those leaders imagine they’ve a sufficiently clear and future-ready technique in place for AI. Most of those leaders (72% of monetary establishment leaders and 73% of funding agency leaders) stated AI is growing so quick that their group is having problem adjusting rapidly sufficient.

In line with a survey by the European Securities and Markets Authority (ESMA), many credit standing businesses and market infrastructures, together with knowledge reporting service suppliers, already use generative AI (GenAI) instruments or plan to begin utilizing them quickly. Banks and monetary establishments might use AI of their lending decision-making processes, and insurers might use AI to generate claims settlement provides.

Whereas AI presents myriad alternatives to spice up effectivity, productiveness, and {industry} developments, it additionally brings with it myriad dangers. Whereas these dangers will not be new, the fast acceleration and proliferation of AI has seen them intensify in distinctive methods:

• Organizations often haven’t developed the AI instruments they use, and will lack perception into how their instruments may match, making it tougher to grasp and sidestep these dangers.
• Persons are reportedly inherently extra trusting of machines. Despite the fact that AI customers don’t have perception into how a device was created, human nature typically compels them to belief AI outcomes no matter their reliability (known as “automation bias”). Organizations want to make sure significant human intervention into AI output, together with critiquing, assessing, refining, and probably overriding outcomes.
• New legal guidelines are increasing present necessities and creating new obligations. As an illustration, some provisions of the EU AI Act will soon take effect. Starting February 2025, corporations can be prohibited from utilizing sure AI features, comparable to:
  • AI techniques that use misleading strategies, exploit vulnerabilities, or use social scoring, create or broaden facial recognition databases by means of untargeted scraping of the web or CCTV footage;
  • or for predictive policing, inferring feelings within the office, utilizing biometric knowledge to categorize people, or operating real-time distant biometric identification techniques.
• The EU AI Act applies to AI techniques used or developed within the European Union (EU), in addition to each time the output of its AI system is used within the EU (even when the group utilizing the device is exterior of the EU).
• In the US, the same complete and risk-based AI legislation goes into impact in Colorado on February 1, 2026. Colorado’s AI legislation gives detailed obligations that builders and deployers are required to implement for high-risk AI techniques with the intention to keep away from algorithmic discrimination. Different states, comparable to Utah, Illinois, New York Metropolis, and California, have additionally handed lighter variations of AI legal guidelines centered on transparency and discrimination in employment dangers. We anticipate extra state-by-state AI legal guidelines to cross within the coming years within the absence of federal laws, together with laws that mirrors the EU AI Act. Even the UK authorities—which has, up till now, not regulated with the intention to take a extra “pro-innovation” strategy—is now contemplating issuing laws protecting using AI following on from the launch of the UK’s AI Alternatives Motion Plan in January 2025, wherein the prime minister introduced that the UK will pursue a practical strategy, testing AI techniques in areas like healthcare and schooling prematurely of adopting new regulation. On the federal stage, in December, the US Division of Treasury launched a complete report on the “makes use of, alternatives, and dangers of synthetic intelligence within the monetary providers sector.” The report concludes with a prolonged dialogue on coverage issues for regulatory frameworks and legislative efforts, noting that there are considerations surrounding conflicting state legal guidelines and uneven necessities on AI builders, customers, and monetary providers companies of various sizes.
• Organizations could also be topic to present knowledge privateness and cybersecurity legal guidelines utilized to AI in new and probably sudden methods. As an illustration, organizations will not be conscious of how their new AI fashions are exposing them to potential knowledge privateness liabilities. Equally, a company might run afoul of long-standing and well-established anti-discrimination employment legal guidelines, if the corporate depends on biased or discriminatory AI-generated outcomes, comparable to a heavy reliance on English-language Giant Language Fashions, age, race, and different demographic options for its employment selections.

Regulatory businesses and consultants all over the world have raised considerations

The regulatory panorama is consistently evolving, and regulatory businesses and consultants all over the world have put organizations on alert relating to potential dangers related to AI.

Gen AI can produce inaccurate outcomes that customers might not be capable of establish as inaccurate

AI is growing with lightning velocity, and these functions depend on possibilities. When AI fashions yield false outcomes, inaccuracies, or hallucinations that aren’t simply recognized as such, the dangers of legal responsibility and reputational harm improve. The Swiss Financial Market Supervisory Authority (FINMA) recognized these considerations in its 2023 Threat Monitor: “Selections can more and more be based mostly on the outcomes of AI functions and even be carried out autonomously by these functions. Mixed with the lowered transparency of the outcomes of AI functions, this makes management and attribution of accountability for the actions of AI functions extra advanced. Because of this, there’s a rising danger that errors go unnoticed and duties turn out to be blurred, significantly for advanced, company-wide processes the place there’s a lack of in-house experience.”

Poor high quality of Gen AI knowledge can current dangers of bias and discrimination

When AI depends on incomplete knowledge units, it may possibly yield biased or discriminatory outcomes, which can be a trigger for concern when AI is used to make consumer-facing selections. Even with full knowledge units, AI utilized in shopper finance has the potential to exacerbate biases, steer customers towards predatory merchandise, or “digitally redline” communities, as highlighted within the December 2024 report from the US Department of the Treasury. Monetary providers companies that use chatbots to interface with clients must be aware about potential legal responsibility and repute dangers because of inaccurate, inconsistent, or incomplete solutions to questions or considerations.

Trendy AI is commonly probabilistic, missing explainability and leading to opaque decision-making

AI could be both deterministic or probabilistic. Deterministic AI features observe strict guidelines to render an explainable end result. Nevertheless, trendy AI is probabilistic, which means that—even for a similar enter—the AI might generate completely different outputs based mostly on possibilities and its weights. This makes the output of probabilistic AI tough to foretell or clarify. As a result of some legal guidelines and pointers require organizations to clarify why an antagonistic resolution was made—comparable to credit score selections or insurance coverage outcomes—if organizations can’t clarify the outcomes of their AI fashions, they could be exposing themselves to important legal responsibility.

Regulatory agencies, including ESMA, have identified concerns in regards to the potential influence on transparency and the standard of shopper interactions, particularly when GenAl is deployed in client-facing instruments, comparable to digital assistants and robo-advisors. As a result of service suppliers stay the proprietor of the algorithm and fashions, customers very often lack entry to the supply of the info used to coach AI. When errors in knowledge yield inaccurate outcomes that are then used to coach the AI system, the output could be inaccurate as properly.

Organizations within the monetary {industry} face focus dangers

Relying on the AI system and the way it’s utilized by the monetary establishment, AI instruments could possibly be thought of Info and Communication Know-how (ICT) belongings. This might convey them into the scope of recent EU cybersecurity guidelines making use of within the finance sector, the EU’s Digital Operational Resilience Act (DORA), which begins making use of on January 17, 2025. To mitigate the potential for industry-wide dangers, DORA establishes sure new cybersecurity administration, reporting, testing, and information-sharing necessities for organizations, which can probably have an effect on AI instruments used within the monetary {industry}. DORA requires monetary establishments to evaluate focus dangers. As a result of AI fashions are concentrated amongst comparatively few suppliers, the rise of third-party AI may have implications for the focus danger of monetary establishments.

AI and rising expertise inherently contain knowledge privateness and cybersecurity dangers

As a result of AI techniques rely in some instances on processing private info, these instruments might already be topic to present knowledge privateness legal guidelines. As an illustration, some US privateness legal guidelines require organizations that use automated expertise to make essential automated selections (e.g., monetary and lending, insurance coverage, housing, schooling, employment, prison justice, or entry to primary requirements) to permit people to decide out of the automated decision-making device. US privateness legal guidelines additionally require organizations to (a) present a transparency discover to people earlier than utilizing private info in reference to the event or deployment of AI, and (b) give people the fitting to entry, delete, appropriate, and opt-out of sure processing if their private info is utilized in AI.

The EU/UK Normal Information Safety Regulation (GDPR) additionally creates strict necessities when people are topic to selections based mostly solely on automated processing, together with profiling, which produce authorized or equally important results, together with comparable transparency and privateness rights like US privateness legal guidelines. The GDPR additional requires corporations to doc a “lawful” foundation for utilizing a person’s private knowledge in reference to AI. Complying with these necessities could also be overly difficult with sure AI techniques, comparable to people who use probabilistic decision-making instruments.

Moreover, widespread use of AI might result in broadened cybersecurity danger. GenAI can be utilized to allow convincing and complicated phishing makes an attempt that lack the same old markers of an unsophisticated try, together with grammatical, translation, and associated language errors. Particularly, password-reset requests and different spoofing and social engineering strategies used to accumulate entry to techniques will probably turn out to be tougher to detect, whatever the stage of sophistication. The advantages of AI-enhanced software program improvement and different cyber operations are additionally prone to accrue to essentially the most subtle of threat actors, together with nation state actors, who’ve the monetary wherewithal to leverage the rapidly altering technological surroundings, rising the danger to the monetary providers sector—already a gorgeous goal.

How an Enterprise Threat Mindset Strategy Can Mitigate AI-Associated Dangers

An enterprise danger mindset strategy to AI and different rising expertise requires sure finest practices.

Enhance consciousness inside the group

Though AI is a fancy expertise, organizations ought to be sure that their workers have a primary understanding of the place and the way AI is used within the group, potential shortcomings, and dangers of AI techniques, how you can spot inaccuracies, and prohibited makes use of of AI. Organizations must also establish these people who can reply AI-related questions and to whom workers can convey considerations.

Create a various, interdisciplinary crew devoted to addressing AI dangers

Managing the dangers and alternatives related to AI is much too monumental for one particular person or division within the group. As a substitute, organizations ought to assemble a devoted AI crew that features stakeholders and workers with skillsets comparable to legislation, knowledge privateness, mental property, info expertise and safety, human assets, advertising and marketing and communications, and procurement. Counting on inside and exterior consultants and assets, this AI crew ought to create, implement, and keep a dependable AI governance program. The AI crew ought to evaluation AI-related instruments (together with these developed by third events), processes, and selections by contemplating danger elements related to opaqueness or a scarcity of readability, bias or discrimination, inaccurate info, privateness, cybersecurity, and mental property—amongst others.

Incorporate governance guardrails

Organizations ought to take steps to implement and talk insurance policies relating to the event or use of AI to all workers inside the group. These guardrails ought to mirror the important thing dangers recognized regarding the event and use of AI. Moreover, specialised or centered coaching guardrails could also be required for particular departments or features inside the group. As an illustration, organizations can instruct workers to not enter private knowledge or delicate enterprise info into AI instruments and/or to solely use company-approved AI techniques, which have applicable contractual protections for the corporate’s knowledge.

Laws set completely different obligations relying on the position of the group and the extent of danger of the AI system (risk-based strategy). Organizations ought to decide the extent of danger posed by the AI system and the group’s position in reference to AI (e.g., developer vs. deployer), after which assess every AI system to make sure they adjust to the group’s role-specific authorized obligations, and that dangers are adequately mitigated. Organizations ought to doc an AI influence evaluation reflecting that the event or deployment of AI is justified, based mostly on the risk-mitigation measures in place.

Organizations stay accountable for the actions taken by AI techniques and AI-generated outcomes.

Apply a strong evaluation course of

Organizations stay accountable for the actions taken by AI techniques and AI-generated outcomes. Ignorance will not be an excuse for legal responsibility, neither is the truth that a 3rd celebration created the AI system. AI techniques must be seen as a supportive device for the group and its professionals; AI is just not an precise decision-maker. Subsequently, the group’s AI crew ought to develop decision-making processes, oversight duties, and implementation standards for AI techniques that take into account parts comparable to anti-money laundering, enterprise continuity, communications, private knowledge safety, cybersecurity, danger administration, regulatory necessities, and vendor administration.

Set up and keep open strains of communication with regulators and stakeholders

Quite a few monetary regulatory businesses—together with the UK’s Monetary Conduct Authority, European Securities and Markets Authority, Swiss Monetary Market Supervisory Authority, Germany’s BaFin, and the US Securities and Alternate Fee and FINRA—have launched steering to assist monetary organizations navigate and mitigate the dangers of AI. Organizations ought to keep abreast of the regulators’ steering and take into account participating with them to raised perceive the altering AI panorama.

In line with a primary research by FTI Consulting, AI observe disclosure in industry-standardized monetary reporting (e.g. Proxy Statements, Company Sustainability Studies, or 10-Ks) must be one other key consideration. For publicly traded monetary organizations with reporting obligations, proactively disclosing AI practices inside the group not solely exhibits good governance, clear and sturdy AI disclosures are additionally an amazing strategic communications device to have interaction with traders and different stakeholders on total AI technique, AI danger mitigation and spotlight group competitiveness in a quickly evolving AI panorama.

AI Threat Administration Requires a Vigilant and Holistic Strategy

AI and different rising applied sciences are quickly evolving, and organizations should regularly stability AI’s dangers with its advantages. This isn’t a one-time resolution, however an ongoing observe. Equally, staying knowledgeable of the expertise, its performance, its dangers, and its advantages is much too expansive for one particular person or division to deal with alone; it requires enter throughout features and departments inside the group, in addition to session with a crew of trusted consultants in IT, authorized and regulatory compliance, communications, and governance. A holistic and ongoing strategy to AI danger administration will allow organizations to harness AI’s advantages whereas minimizing the dangers of legal responsibility, reputational harm, and regulatory scrutiny.

*FTI Consulting

(View source.)