AI in companies not delivering results? Most often it’s not the model that fails, but the data and decisions
70% of companies declare they use AI, and at the same time 80–90% don’t see any measurable impact on productivity or employment. On top of that, top managers spend on average 1.5 hours weekly with AI. This gap doesn’t come from “magic”, or lack thereof—it’s because AI very quickly becomes a colorful toy that mainly accelerates… chaos.
If AI is to truly boost profit, I always start with the foundations: data → interpretation → decision → consequence. Without this, even the best tool is just another chart for slides.
Looking at data is not the same as working with data
In many companies, data is treated like the weather forecast app: check, see, close. And then, I just do my own thing anyway.
Working with data begins only when, after interpretation, a concrete decision is made. The most “mundane” but most valuable decisions look like:
- shifting budget between channels,
- stopping a campaign because it’s unprofitable,
- withdrawing a product/service because there’s no margin,
- investing more in what brings the highest profit.
Data without consequences is just a curiosity. Data with consequences becomes fuel for growth—and that’s exactly when analytics starts to translate into money.
Why “AI doesn’t work”: you’re accelerating chaos, not value
The most common reason for AI disappointment is brutally simple: companies don’t have organized data and processes and then “feed” their models the same chaos.
If a company does not have:
- consistent metric definitions,
- a single source of truth for KPIs,
- clear boundary conditions,
then adding AI doesn’t solve the problem. AI just speeds up the generation of incorrect conclusions.
The second mistake is spontaneous use of AI for side tasks: meeting summary, polishing a LinkedIn post, quick summarization. That can be useful, but rarely gets to the core. The key question is: how is AI supposed to boost the company’s profit—not “where else can we paste a prompt”.
The third mistake is expectations: implementing AI does not “revolutionize a company in a month”. Results come gradually: from small improvements to automation of work fragments, up to processes where the analyst is truly relieved and can do things with bigger impact.
AI as an analytical partner, not an oracle
AI is great at generating hypotheses. And that’s exactly how one should treat them: hypothesis → verification → human decision.
A good example is anomaly detection in sales data. AI can point out an “error” like: “in July, a record number of rain boots sold in 5 days—that’s impossible.” But in the real world, “impossible” can be perfectly logical: a very rainy week in July and a spike in rain boot sales becomes normal.
This is the difference between a “spit out answer” and understanding business context. AI doesn’t have context. People do—which is why:
- AI can suggest,
- AI can accelerate,
- AI should not make decisions or take responsibility.
A helpful analogy is a new employee: arrives with a fresh mind, but doesn’t know something was tested before and didn’t work, or that customers care about things not visible in the table.
When the agent “gets too much”: context and data freshness limits
There’s one more minefield in AI automation: context limitation. On a small CSV or Excel file, everything seems great—then real life hits.
There’s a great story involving an agent tool (open source, automating actions) used by a security researcher at Meta. She asked the agent to review her inbox and suggest cleaning up—with a clear instruction not to delete anything. After a while, the agent started deleting messages “like a maniac”, and she had to physically pull cables to stop it.
The mechanics are more important than the anecdote: when an agent loads massive volumes of messages into context, it “overflows” and can lose the original instruction. The same thing happens with company data:
- context is too large,
- documentation and the “business description” age within days,
- requirements change faster than they can be documented.
That’s why trying to “describe the whole business to an agent” is absurd—an analyst would need weeks to write it all, and by the time it’s done, parts would already be outdated.
Dirty data: the IoT case and the 255 temperature that wrecks everything
If I had to pick one thing that kills analysis automation, it’s data quality.
In IoT systems, billions of records can come from sensors. In one case, the “most common water temperature in the boiler” ended up as 255. Experienced people in the industry (with 30–40 years of experience) spoke of typical values like 80–90°C—and physically, they were correct. But the data said otherwise.
Why 255? Because on reset, the device would send a value set to binary ones (11111111 = 255). Resets were frequent, so 255 became the dominant value. If you calculate the average without cleaning—you get nonsense. If AI gets this without context—it also gives nonsense, just faster and with more confidence.
The conclusion is simple: trust in automation begins with trust in the data—and that means quality processes, not just “plugging in a model”.
Small company (30 people) and “something’s wrong”: what questions to start with
In a service company, where data comes “from the ground up”, from Instagram, ads, and basic sales, I start with two paths. Both are simple, but they work.
1) Follow the money—what are you really earning on
First, financials:
- what does the company make money on (services/products),
- which have the highest margin,
- which have high turnover but low margin,
- how much does customer acquisition cost (CAC).
If no one tracks margin—it’s counted “manually”: cost of the given service vs. revenue from that service. This quickly changes thinking, as decisions start to affect profit, not just traffic or “pretty charts”.
2) Non-financial data—a safer playground
Here, less critical matters come in, so it’s easier to start:
- number of leads per week/month from channels,
- conversion to sale (does the lead buy, or just “comes and goes”),
- customer segments who actually buy.
I’ve seen situations where a company gets thousands of leads per month, and they just “sit there”. In such a case, the question isn’t “how to get more leads”—but what do we do with the ones we already have.
Segmentation can be simple and effective: e.g., in clothing/cosmetics sales, the women 19–25 segment often works great, and the clicking-but-not-buying segment (e.g., men 40–45) can just burn through budget—depending on the industry and offer.
Decision culture with AI: recommendation owners and “devil’s advocate”
To make AI a safe partner, not an oracle, two rituals must be part of company culture.
1) The owner of recommendations is always a human
It’s the human who bears the consequences: financial, operational, legal. AI does not.
2) You need a “devil’s advocate”
If AI (or even an analyst) says: “campaign ABC is unprofitable”, someone is required to go back through:
- why it’s unprofitable,
- which indicators were calculated,
- from which elements,
- what data it’s based on,
- where there may be errors or missing context.
And one more thing: never blindly trust the average. If on one product you earn 3 zł, and on another 2000 zł, the average of “about 1000” tells you nothing about what to sell or where to invest. The average is comfortable—but completely useless for decision-making.
The role of the analyst in the AI era: less clicking, more business
The analyst position has already evolved: once a single person did ETL, data modeling, and reporting “A to Z”, then data engineers and specialists appeared. Tools change (Excel, Power BI, Tableau, Qlik…), but that’s just the execution layer.
AI does one specific job: shortens the time spent “figuring out how to do it”. Once, finding a solution for a specific Power BI functionality (like an up arrow for month-over-month growth) could take two workdays: forums, YouTube, community, Stack Overflow, testing. Now a model can generate instructions in a few seconds, and the human shifts to applying, adapting, and quality-checking.
But the biggest shift is soft: the best analysts increasingly:
- talk to business,
- gather requirements,
- ask, “what do you need this report for?” and “what decision will you make?”.
This also prevents a classic mistake: perfecting a report (counting pixels on the canvas), when nobody checked if the report even answers a real need. That’s how you get situations where someone builds a “super dashboard” and the user asks: “can you just do this in Excel?”—because the solution doesn’t fit their workflow.
Key takeaways that actually make a difference (without huge projects)
- AI won’t clean up your data mess—it’ll just speed it up.
- Data must lead to decisions and consequences; otherwise, it’s just for show.
- Business context is key: rain boots in July may be correct, not an error.
- Data quality determines the point of automation (see the 255 IoT example).
- Devil’s advocate and reverse analysis protect your company from “AI said so”.
- The analyst’s role is shifting toward business communication and responsibility for recommendations.
How to implement this (steps you can take now)
- Pick 3 business decisions you want to make based on data (e.g., stop/start campaigns, shift budget, prioritize services).
- For each decision, list 1–2 metrics to support it (e.g., margin, CAC, lead→sale conversion) and define a single KPI definition for the company.
- Do a quick data audit: write down the 5 most common error sources (missing data, duplicates, default/reset values like “255”, inconsistent names, different periods).
- Start with the follow the money approach: calculate margins on services/products, even if “in Excel and manually” at first.
- At the same time, build a simple funnel: lead (channel) → contact → sale. If leads are just “sitting”, process is the priority, not ads.
- Use AI where it gives instant time savings: drafting reports, metric/DAX proposals, SQL/Python queries, visualization tips—but always finish by verifying with data.
- Introduce the “devil’s advocate” ritual: every recommendation from AI must have a short reverse analysis (what data, how it was calculated, what can be wrong).
- Don’t report averages without context: show distribution, segments, and outliers (because averages can lie).
- Set security rules: what data can be pasted into AI tools, and what has to stay in the company environment (for legality and safety).
- After 2 weeks, review: which decisions were actually made based on data, and which got stuck at the “pretty chart” stage. Remove everything that doesn’t lead to action. Share on Share on Share on Share on