How many pages does Kurli have?

The sitemap listed 272 URLs when measured on 1 September 2026, of which 155 sit under a guides path, 56 under a product path, 25 under try-on, 12 under stylists and 10 under brands. The 155 guides are the finding that explains the AI citations.

Does Kurli serve real HTML to crawlers?

Yes. Measured on 1 September 2026 with a Googlebot user agent and no JavaScript executed, the homepage returned 217,700 bytes containing 1,428 readable words. A browser user agent returned the same 1,428 words, so the site does not vary its response by user agent.

What was wrong with Kurli's robots.txt?

It contains both Disallow: / and Allow: / for the same agents, including GPTBot, because two sets of groups written at different times both name them. Under RFC 9309 groups naming the same agent merge and the least restrictive rule wins on a tie, so the intended AI-training block does not reliably apply.

Why did a 26-day-old domain get cited by ChatGPT?

Because it had answered 155 real questions in guide pages while almost nobody else in that niche had, and because no search or citation crawler was blocked from reading them. The domain's age mattered less than the fact that the answers existed and were reachable.

Do you know which pages earned the citations?

No. The referral data records that ChatGPT sent 56 visitors in the first 25 days, not which pages they landed on. That is a gap in our own measurement, and it is being fixed rather than filled in with a guess.

Are these numbers still current?

The referral figures cover roughly 27 July to 21 August 2026 and the live-site measurements were taken on 1 September 2026. Both will be restated with new dates when remeasured, rather than quietly updated in place, since a figure without a date goes stale while still being read.

Teardown: Kurli

This is our own app, which means every number here can be published, including the one that turned out to be a mistake in our own configuration.

See what a crawler gets

Kurli.me was registered on 27 July 2026 and received 56 ChatGPT referrals against 23 from Google Search in its first 25 days. Measured on 1 September 2026, it serves 1,428 readable words to a crawler with no JavaScript executed, publishes a sitemap of 272 URLs of which 155 are guides, and blocks no search crawler. It also has conflicting duplicate robots.txt groups that make its intended AI-training block unreliable.

The app and the domain

Kurli is an AI hairstyle advisor. It reads a selfie, works out the face shape and proportions in the browser, and renders suitable cuts onto the person's own photo. It is free and needs no sign-up.

The domain kurli.me was registered on 27 July 2026, which is verifiable in public WHOIS records. That matters for everything below, because it means the site had no history, no accumulated links, and no track record with any search engine during the window described.

The referral figures we have are from the first 25 days of the domain's life: 56 visitors from ChatGPT and 23 from Google Search. Both numbers are small, and the ratio at that point in a domain's life is the finding rather than the volume.

What a crawler receives

Measured on 1 September 2026, fetching the homepage with a Googlebot user agent and executing no JavaScript.

The response was 200, and the HTML was 217,700 bytes containing 1,428 readable words after markup, scripts and styles were stripped. Fetching the same URL with a browser user agent produced the same 1,428 words, so the site is not varying its response by user agent and there is no verified-crawler arrangement in play.

The title read Kurli: Find Your Best Hairstyle and the first H1 read Find the style that fits your face.

So the rendering problem that dominates this whole category does not apply here. Real content arrives in the HTML, for everybody, without anything needing to run.

The finding that actually explains it

This is the part worth the teardown, and it is not the rendering.

The sitemap at kurli.me/sitemap.xml lists 272 URLs. Of those, 155 sit under a guides path, 56 under a product path, 25 under a try-on path, 12 under a stylist path and 10 under a brands path, with a handful of comparison, tools and legal pages making up the rest.

One hundred and fifty-five guide pages. That is the explanation for the ChatGPT referrals, and it is a far more useful finding than anything about rendering.

The entire argument of this section of our site is that one page means one possible search, and that the way out is a page for each real question. Kurli is what that looks like when it has been done: guides organised by face shape, by hair type, by length, by colour, by thinning, by care. Each of those is a question somebody actually asks, and each is a page an assistant can retrieve and cite when somebody asks it.

A 26-day-old domain did not get cited because of anything clever. It got cited because it had answered 155 questions and almost nobody else in that niche had.

The mistake we found in our own robots.txt

Now the part that is not flattering, and which we found by running our own crawler tool against our own site while writing this page.

The robots.txt carries a Content-Signal declaration of search=yes, ai-train=no, use=reference, which is a clear statement of intent: allow search and citation, decline training. It then lists explicit blocks for the training crawlers, including GPTBot, ClaudeBot and Google-Extended, alongside several bulk scrapers.

Further down the same file, a second set of groups appears, apparently written by a different tool at a different time, which names GPTBot again and this time allows it.

The result is a file containing both Disallow: / and Allow: / for the same agent. Under RFC 9309 groups naming the same user agent are merged and the least restrictive rule wins on an exact tie, so the intended training block does not reliably apply. The declared intent and the actual instruction disagree.

Nothing about the site looks wrong, no tool reported an error, and the file reads as deliberate at a glance. This is precisely the failure mode we describe on the robots.txt page, found on our own domain.

What is set up correctly

For balance, and because a teardown that finds only problems is not an honest one.

No search or citation crawler is blocked. Googlebot, Bingbot, OAI-SearchBot, Claude-SearchBot, PerplexityBot and the user-triggered agents all reach the site, which is the configuration that keeps the AI channel open and is the single most commonly broken thing in this category.

The site was on its own domain from the start rather than a platform subdomain, so nothing has to be moved later.

The sitemap exists, is reachable at the conventional path, and lists a number of URLs consistent with the number of pages that exist.

And the pages answer questions rather than describing the product, which is the thing that is hard to do and impossible to shortcut.

What we cannot tell you

The limits, stated plainly, because a teardown that overreaches is worth less than one that stops.

We do not know which of the 155 guides earned the citations. The referral data we have records that ChatGPT sent 56 visitors, not which page they landed on. That is a gap in our own measurement and it is being fixed rather than filled in with a guess.

We cannot say whether the ratio holds as the domain ages. Twenty-five days is a short window, and Google becomes more willing over time, so the comparison that made this interesting may narrow.

We have not measured what those 56 visitors did after arriving. The number is too small to say anything reliable about behaviour, and a conversion rate calculated from 56 sessions would be arithmetic rather than evidence.

The figures above describe 27 July to about 21 August 2026 for the referrals, and 1 September 2026 for everything measured from the live site. Both will be restated with their new dates rather than quietly updated in place.

What to take from this

Three things, in order of how much they matter.

The pages are the reason. Not the rendering, not the schema, not any configuration detail. A new domain got cited because it answered 155 real questions, and that is the finding that transfers to your app even though the traffic numbers do not.

The technical half is a precondition rather than a strategy. Kurli serves real HTML and blocks no search crawler, and if either of those were untrue the 155 guides would have achieved nothing. Getting it right buys you the chance to compete, not the result.

And read your own robots.txt with your own eyes, today. We write about this constantly, we built a tool that checks it, and we still had a file that contradicted itself. It takes thirty seconds and it is the cheapest thing on this page.

Questions people ask

How many pages does Kurli have?: The sitemap listed 272 URLs when measured on 1 September 2026, of which 155 sit under a guides path, 56 under a product path, 25 under try-on, 12 under stylists and 10 under brands. The 155 guides are the finding that explains the AI citations.
Does Kurli serve real HTML to crawlers?: Yes. Measured on 1 September 2026 with a Googlebot user agent and no JavaScript executed, the homepage returned 217,700 bytes containing 1,428 readable words. A browser user agent returned the same 1,428 words, so the site does not vary its response by user agent.
What was wrong with Kurli's robots.txt?: It contains both Disallow: / and Allow: / for the same agents, including GPTBot, because two sets of groups written at different times both name them. Under RFC 9309 groups naming the same agent merge and the least restrictive rule wins on a tie, so the intended AI-training block does not reliably apply.
Why did a 26-day-old domain get cited by ChatGPT?: Because it had answered 155 real questions in guide pages while almost nobody else in that niche had, and because no search or citation crawler was blocked from reading them. The domain's age mattered less than the fact that the answers existed and were reachable.
Do you know which pages earned the citations?: No. The referral data records that ChatGPT sent 56 visitors in the first 25 days, not which pages they landed on. That is a gap in our own measurement, and it is being fixed rather than filled in with a guess.
Are these numbers still current?: The referral figures cover roughly 27 July to 21 August 2026 and the live-site measurements were taken on 1 September 2026. Both will be restated with new dates when remeasured, rather than quietly updated in place, since a figure without a date goes stale while still being read.

Run the same measurements on your app

The crawler tool used for this teardown is free and public. Paste your URL and see what a crawler receives and which crawlers your robots.txt allows.

See what a crawler gets Run the full check