Is Your Web Agency Optimising for AI? Ask Them About the Side Gate

Every decent building in Perth has two ways in.

There's the front door. Glass, signage, someone at reception, a spot to park your ute out the front where you hope nobody dings it. That's for clients.

Then there's the side gate. No sign, no reception, no artwork on the walls. Just a way for the sparky to get in, find the switchboard, do the job and leave. Nobody photographs the side gate. It's not there to be admired. It's there because the people using it don't need the tour.

Your website now has both. Most Perth businesses only know about the front door.

The side gate has a name

It's called LLMs.txt, and it's a plain text file sitting at the root of your website, at yourdomain.com/llms.txt.

Inside it is a short, clean summary of your business. Who you are, what you do, where you do it, and links to the pages that matter most. No code, no styling, no cookie banners, no video header, no menu that slides in from the left. Just the facts, written in a format machines find easy to read.

That's it. That's the whole thing. It's the sort of file you could write on the back of an invoice.

The idea came from data scientist Jeremy Howard back in 2024, and the logic is sound.

When an AI system visits your website, it has to chew through the entire page just to find the sentence it actually wanted. The side gate skips all that. It hands over the essentials and gets out of the way.

Now the bit most agencies won't tell you

In June this year, Google updated its official guidance on optimising for AI and said, in plain language, that LLMs.txt does nothing for your Google rankings. Nothing for AI Overviews either. It's now listed by name in Google's own mythbusting section, sitting alongside the tactics that don't work.

The data backs them up. A study of 300,000 domains found only about one in ten sites has the file at all. The same study ran a model to test whether having one correlated with being cited more often by AI, and found no relationship whatsoever. Separate research monitoring more than 500 million AI bot visits over three months found only a few hundred that went looking for an LLMs.txt file directly.

If you've been told that adding this file will get your business cited by ChatGPT, you've been sold something the numbers don't support.

So that's the end of the blog post, right?

Not quite. Because here's where it gets interesting.

Google is arguing with itself

Roughly five weeks before Google's Search team told everyone to skip LLMs.txt, Google's Chrome team started automatically checking for it.

Lighthouse, the performance tool built into Chrome that developers run on websites every day, added an LLMs.txt audit by default. Not under SEO. Under a category about agentic browsing.

Same company. Two teams. Opposite advice. And both of them are correct, because they're answering completely different questions.

The Search team was asked "does this affect Google rankings?" No, it doesn't. The Chrome team was asked "should websites be readable by AI agents?" Yes, they should.

One team is looking after the front door. The other one has noticed people coming through the side gate.

Who's actually using the side gate

Not Google Search. That's settled.

But Perplexity and Claude both retrieve these files. Agentic browsers use them. And most tellingly, the coding tools that developers now live inside, Cursor, GitHub Copilot, Windsurf, Claude Code and others, look for an LLMs.txt as standard when you point them at a website.

Then look at who publishes one. Anthropic. Stripe. Cloudflare. Vercel. Mastercard. These are not companies that add files to their websites out of superstition, and they've all read the same Google guidance you just did. They've kept theirs anyway.

They're not chasing rankings. They're building for a different visitor entirely: software that turns up on behalf of a human, needs an answer, and would rather not read your homepage to get it.

Gartner has reported that the majority of technology executives are planning to deploy agentic AI within the next two years. When a customer's assistant is the one doing the looking, the side gate stops being a curiosity.

What this means for your business, honestly

Let's be straight, because that's more useful than hype.

Adding an LLMs.txt file to your website will not fix your Google rankings. It will not make ChatGPT recommend you tomorrow. If your agency is quoting you for "LLM optimisation" and this file is the deliverable, ask more questions.

What it will do is make your business legible to the growing category of software that visits websites without a human attached, and give you a say in how your business gets described when that happens. Instead of an AI guessing from an outdated directory listing, it reads the version you wrote.

The cost of being wrong about this is close to zero. The cost of being wrong the other way, if agentic traffic keeps growing, is being the business nobody's assistant can find.

That's a fairly easy call.

The good news if you're on Wix

If your website is built on Wix or Wix Studio with a custom domain and indexing switched on, you already have one. Wix generates it automatically and keeps it updated, and you can view or edit it in your dashboard under SEO and GEO. If you edit it, it stops auto updating so your version sticks.

We work in Wix Studio almost exclusively, and this is one of the reasons why. While the rest of the industry was arguing about whether the file mattered, Wix built it, made it editable, and started using it to expose agent facing resources like MCP. That's a platform paying attention to where things are going rather than waiting for permission.

So, is your agency thinking about this?

The question isn't really "do we have an LLMs.txt file". It's a small file. It takes minutes.

The real question is whether the people building your website have noticed that your website now has two audiences, and whether they're designing for both. Clean structure. Clear service descriptions. Content a machine can parse without guessing. A business that reads the same way whether a person or an agent is doing the reading.

We've been implementing LLMs.txt across client sites for a while now, and as far as we can tell we're one of the first agencies in Perth doing it deliberately rather than by accident. We might be wrong about that. If you're a Perth agency already on this, get in touch and we'll shout you a coffee, because we'd genuinely like to compare notes.

But if you ask your current agency about this and get a blank look, that tells you something. Not about a text file. About whether anyone is watching the side gate.

Let's have a chat

If you're wondering how your website looks to the AI systems your customers are starting to ask, that's exactly the kind of thing we like digging into. Book a free discovery call, or pop into the Joondalup office and we'll put the kettle on.

Book a FREE Video Call