How My Own AI Feature Quietly Ate My Gross Margin (2026)

Here's the thing nobody really warns you about when you ship an AI feature, which is that it's basically the first line item on your whole P&L that gets more expensive the better it does its job. Everything else in software gets cheaper the more you sell it, right, that's the entire reason SaaS margins were always sat up around 80% in the first place, one more customer costs you almost nothing to serve. And then you wire a model into the product and suddenly every single thing your customer does, every summary, every classification, every little bit of generation, quietly costs you real money to a provider, and it scales up perfectly in lockstep with your success, which is honestly a slightly cursed thing to build if you think about it too hard.

So let me just answer the obvious question first, plainly, before I tell you how I found this out the embarrassing way. AI gross margin compression is basically what happens when inference stops being a fixed cost you paid once and becomes a variable cost of goods sold that moves with usage, so your lovely 80% software margin drifts down toward something a lot closer to 50 or 60% depending on how heavy your users are, and the annoying part is it doesn't show up as its own scary line, it just hides inside the aggregate COGS you already had, where nobody's really looking until a board meeting. That's the whole problem in one sentence, and everything below is just me explaining how I walked straight into it.

How did I not notice this happening?

Honestly? Because it worked, and working things get ignored. I taught myself to vibe code about six months ago and built Sentrama, our AI sales dialer and CRM, basically from scratch in a couple of months with a teammate called Jay who'd come over from the data side, and there's a sister company, Real Recruitment, running on the same kind of prompts. I wired the latest OpenAI model into 90-odd production prompts, the data was qualifying itself, the call reasons were writing themselves off everything we'd scraped about a person, the reports were going out on their own, all of it just ticking away in the background, and obviously the fact that it all just worked is exactly why I stopped paying any attention whatsoever to what it actually cost. Then we onboarded more and more customers, more and more SDRs, more and more calls, and the bill climbed from about a grand a month to nearly four, and then up into five figures, and one day I'm sat in a finance meeting staring at this number I'd basically been ignoring for months, and the impact of the feature looked incredible, and I'd also somehow been quietly lighting money on fire the whole time and just not clocked it.

Why don't cheaper models just fix this on their own?

Because they genuinely don't, and this is the trap that lets the whole thing keep compounding on you. Per-token prices actually fall really fast, you'll see numbers like models getting roughly ten times cheaper a year thrown around, so you'd think the problem sort of heals itself if you just sit tight and wait, right? But it doesn't, for three reasons that all push the wrong way at once. Your volume explodes, because the cheaper each call gets the more calls everyone makes. Your customers migrate onto the newest, most expensive model basically the day it ships, because they always want the shiniest one, so you never actually get to sit on the cheap old model and enjoy it. And reasoning and agentic workloads have quietly multiplied the tokens per task, so one thing a user does today can cost several times what the same action cost a year ago. So the price per token drops and your actual bill still climbs, which is exactly why waiting for cheaper models didn't fix my bill, and reselling that inference at cost is a zero-margin business anyway, it just makes you a payment rail with extra steps.

So what's the part I could actually fix?

The part you can fix is the cost sitting behind the feature, not the feature and not your prices. Most of your high-volume prompts, your classification, your extraction, your summarisation, your generation off structured data, that workhorse stuff running all day, they don't actually need your most expensive frontier model to hit the same quality bar, they just default to it because you set it once and left it, which is precisely what I did. If you can move those specific prompts onto a cheaper model that actually holds the quality, you claw most of your lost margin straight back with no price change, no feature removed, and no rewrite of your product, you've just changed which model answers the call behind the same credit the customer already bought. On the prompts where that works out, most teams land somewhere in the 30 to 60% range on that inference line. My own bill came down a fair bit harder than that, but I'm not going to sit here and promise you my number as if it's yours, so 30 to 60% is the honest band, and it's a band and not one figure precisely because it genuinely varies prompt by prompt.

But how do you know cheaper doesn't just mean worse?

Right, and this is the exact fear that keeps the margin sat on the table, because there was one thing I was never going to do, which was quietly make my customers' results worse to make my own margins a bit better. The bit everyone gets wrong is switching on vibes and benchmarks, and a benchmark tells you basically nothing about your own traffic, your prompts, your weird edge cases, your JSON schema. So you prove it on your own prompts instead, and you judge it on three things all at once, which is does it come back in the exact format your app expects so nothing downstream breaks, does it make the same categorical calls your current model makes, re-judged blind when the two disagree so a coin-flip doesn't get counted as a real miss, and is it actually saying the same thing in substance. And then the bit almost everyone skips, you measure all of that against how much your own baseline model already disagrees with itself, because if you ask your existing model the same prompt twice you don't get the same answer back, there's a natural spread, and you genuinely cannot call a swap worse until you know what your own model's normal wobble looks like in the first place. That's the whole idea, and after three or four months down a deep, deep rabbit hole it's what I landed on, and it's now patent pending, but the concept underneath is dead simple, you can't measure better until you've measured your own wobble. And if anything ever drifts, your baseline is sat right there as an instant fallback, so you're never worse off than you are today.