PayPal Increased AI Accuracy From 43% to 95% Without Changing Its AI Model

The AI industry has spent billions trying to build bigger, faster and more intelligent models. Companies compete over parameters, computing power and benchmark scores, assuming that the next generation of AI will automatically solve today's problems.

PayPal has demonstrated a different approach.

The company increased the accuracy of an internal AI assistant from 43% to 95% without changing its underlying large language model. Instead of buying more computing power or rebuilding the model, PayPal gave its AI something it was missing — the company's actual business knowledge.

That distinction could be more important than another breakthrough in model performance.

For businesses struggling to make AI useful, PayPal's experience offers a practical lesson. Having access to information is not the same as understanding what that information means.

PayPal's AI Accuracy Jump Changes the Enterprise AI Conversation

Ren Zhang, PayPal's vice president and global head of AI product, explained that the company's internal data science assistant already had access to high-quality, verified data.

Yet its accuracy was only 43%.

The problem was not necessarily the quality of the underlying information. The assistant lacked the definitions, analytical procedures and business rules employees use to interpret that information.

After PayPal added this knowledge, accuracy increased to 95%.

The underlying language model remained unchanged.

This is significant because many companies assume poor AI performance requires a more powerful model. PayPal's results suggest that the surrounding information architecture can matter just as much.

Access to Company Data Does Not Equal Business Understanding

Consider a simple question about new customer accounts.

An AI assistant might retrieve account records correctly but misunderstand what the company considers a new account.

Does a returning customer opening another account qualify? Does an account created but never activated count?

These distinctions can change the answer completely.

The model may retrieve accurate information and still produce a misleading business conclusion.

PayPal addressed this problem by building a knowledge layer that explains how the company interprets and uses its data.

Two important components are golden queries and analytical playbooks.

Golden queries provide approved instructions for retrieving information for specific business purposes.

Analytical playbooks explain how to investigate questions, including which customer segments, products and behaviors should be examined.

This turns scattered organizational expertise into reusable instructions.

The Knowledge Layer Becomes PayPal's Real AI Asset

PayPal's strategy separates business knowledge from the underlying AI model.

This is a critical architectural decision.

Language models improve rapidly. A model that performs well today may become less attractive when a faster or cheaper alternative arrives.

If business knowledge is deeply embedded in each model through repeated customization, switching providers becomes expensive.

PayPal wants to avoid that problem.

Its knowledge layer can remain intact while the company replaces or upgrades the models connected to it.

In practical terms, the model becomes an interchangeable engine, while the business knowledge becomes the valuable operating system around it.

That creates flexibility and reduces the need to reconstruct organizational expertise whenever a new model appears.

PayPal's Commerce Agent Reveals the Cost of Bigger Models

PayPal has also experimented with different models for its customer-facing commerce assistant.

Zhang described testing four or five models.

Larger models offered strong capabilities but initially introduced approximately 15 seconds of latency.

That delay matters in commerce. Customers expect quick responses, and slow interactions can undermine the experience.

PayPal tested an 8-billion-parameter model to improve response speed.

However, the smaller model initially delivered lower accuracy.

Fine-tuning improved its performance, but the team recognized another problem: maintaining customized models can become expensive.

Every new model release potentially creates another round of training and testing.

PayPal therefore shifted attention toward more sustainable customization rather than repeatedly fine-tuning every new model.

The lesson is straightforward. The best model is not necessarily the biggest. It is the model that delivers the right balance of accuracy, speed and operating cost.

Seven AI Agents Investigate More Than 33,000 Services

PayPal is moving beyond question-answering systems toward agents capable of performing complex operational tasks.

Deepika Khera, the company's senior director of AI, described a seven-agent workflow designed to investigate incidents across more than 33,000 services.

The workflow brings together information from Datadog, ServiceNow and internal logs.

Different agents perform different responsibilities, helping identify meaningful signals, collect relevant context and investigate potential incidents.

For security reviews, PayPal uses workflows involving as many as 18 agents.

This is an important shift.

Instead of asking one AI model to perform every task, companies can distribute responsibilities across specialized agents with different information access and permissions.

However, adding more agents does not automatically improve reliability.

An incorrect diagnosis can still produce an incorrect solution.

Human Approval Remains Essential for High-Risk Decisions

PayPal addresses this risk through authoritative information sources, standard operating procedures and additional policy checks.

Proposed decisions are checked against verified sources before action.

Many decisions still require human approval.

This approach recognizes an important limitation of AI agents. A system can produce convincing explanations without necessarily following the correct company procedure.

PayPal discovered this problem in an internal HR assistant.

The system struggled because instructions had been stored in fragmented pieces rather than as complete sequences of steps.

The company changed how procedures were represented so the assistant could understand the correct order of operations.

PayPal also sometimes forces agents to consult memory at specific stages instead of allowing them to decide independently whether that information is necessary.

That is an important distinction between a helpful chatbot and a dependable enterprise system.

PayPal Is Turning AI Experience Into Reusable Memory

The company is also experimenting with episodic memory.

This records previous sessions and outcomes, including software pull requests.

A nightly process reviews experiences across users and extracts lessons that can improve future interactions.

The objective is to prevent AI agents from repeatedly making mistakes that have already been solved.

Instead of treating every task as a fresh starting point, the system gradually accumulates reusable operational knowledge.

For businesses, this could become a powerful advantage.

An organization that consistently captures its employees' decisions, procedures and lessons can make its AI systems more useful over time.

More Than 1,000 Questions Produced 1,600 Hours of Savings

PayPal reported that its internal assistant handled more than 1,000 questions and generated an estimated 1,600 hours of savings over 10 weeks.

That is a meaningful productivity signal.

Employees can obtain answers without repeatedly asking data scientists to investigate routine business questions.

However, question volume alone does not prove that an AI assistant is delivering genuine economic value.

Businesses must also measure answer accuracy, time saved, employee adoption and the amount of human correction required.

The difference between an impressive demonstration and a profitable enterprise product is measurable operational improvement.

Enterprise AI Testing Still Has a Serious Reliability Problem

PayPal's experience also highlights the gap between controlled testing and real-world performance.

In VentureBeat's October Agentic Reliability and Evals survey, 42% of respondents whose organizations conducted pre-deployment evaluations said an AI agent or LLM feature had passed internal testing but subsequently failed within the previous year.

Among those reporting failures, 49% said their most significant incident reached customers.

Even more concerning, 39% identified no automated checks or monitoring alerts among the ways their organizations would first discover incorrect AI output.

This means many businesses still depend on people noticing when AI gets something wrong.

PayPal is responding with offline simulations, synthetic question generation, single-turn and multi-turn testing, model-based evaluation and human assessment.

The company also uses employee feedback to identify problems after deployment.

The Competitive Advantage Moves From Models to Proprietary Knowledge

The AI market is becoming increasingly competitive.

Companies can access commercial models, open-source alternatives and specialized smaller models.

As model capabilities become more widely available, simply connecting an application to an LLM becomes less defensible.

The harder asset to reproduce is the knowledge accumulated inside an organization.

PayPal understands its customers, financial processes, internal systems and operating procedures.

A competitor cannot automatically replicate that expertise by purchasing the same AI model.

This creates an important strategic opportunity for enterprise software companies.

The value increasingly comes from combining AI capabilities with proprietary data, specialized workflows and trusted operational knowledge.

The Biggest Risk Is Trusting AI With Knowledge It Cannot Verify

From a competition and consumer-protection perspective, PayPal's architecture raises an important governance issue.

Who controls the knowledge layer? Who approves its definitions? How are outdated procedures corrected? And who is responsible when an AI agent makes a decision based on incorrect internal information?

These questions matter because a knowledge layer can become a powerful source of institutional control.

If its information is incomplete or biased, every model connected to it may repeat the same mistakes.

Model independence is useful, but it does not guarantee knowledge independence or accuracy.

Companies need clear ownership, audit trails, access controls and human accountability.

The weakest assumption in many enterprise AI strategies is that giving agents more memory automatically makes them more intelligent.

In reality, memory must be verified, maintained and continuously tested.

Small Businesses Can Build Valuable AI Without Training Their Own Models

PayPal's approach offers a practical opportunity for small and medium-sized businesses.

They do not need to develop their own language models.

They can begin by organizing the knowledge they already possess.

That means documenting customer service procedures, pricing rules, sales processes, product information and frequently asked questions.

These resources can then be connected to an AI assistant that retrieves approved information before generating answers.

Businesses should also establish clear boundaries around actions that require human approval.

For example, an AI assistant might prepare a refund recommendation but require a manager to authorize the payment.

This approach reduces development costs while creating a foundation for more advanced automation.

The commercial opportunity is not selling AI for its own sake. It is using AI to reduce repetitive work, improve service and help employees make better decisions.

PayPal's Biggest AI Breakthrough Is Learning What Not to Change

PayPal's improvement from 43% to 95% accuracy offers a different perspective on the future of artificial intelligence.

The company did not need a new model to achieve a major improvement. It needed to explain its business more effectively to the model it already had.

That is the larger opportunity for enterprise AI.

Models will continue changing. Computing costs will evolve, and new competitors will emerge.

But proprietary business knowledge, reliable procedures and organizational experience are harder to replace.

The companies that win may not be those with the largest AI models.

They may be the companies that know how to turn what they already know into systems that AI can reliably use.

PayPal Turns Internal Business Knowledge Into a Competitive AI Advantage

PayPal AI assistant, enterprise AI, AI accuracy, business knowledge layer, AI agents, model agnostic AI, enterprise automation, proprietary AI data, AI workflow automation, LLM accuracy, AI memory systems, AI reliability testing, PayPal AI strategy, business process automation