AI agents are swarming. Does your website need an llms.txt file?

AI agents are swarming. Does your website need an llms.txt file?

https://statescoop.com/llmstxt-government-websites-ai/

Publish Date: 2026-01-12 06:04:00

Source Domain: statescoop.com

• Corrmac McCarthy and Punctuation: Cormac McCarthy discussed his sparse use of punctuation, calling them “weird little marks,” during a 2007 interview with Oprah on his novel The Road. His approach, even from classic favorites like Moby-Dick, sparked significant debate among readers and critics.

• Debates on Technical Details: Similar to the controversy around McCarthy’s punctuation, debates exist among software developers over seemingly trivial choices, such as using tabs or spaces in coding, or the relevance of hosting llms.txt files on websites for AI guidance.

• Maryland’s Use of llms.txt: Maryland’s government website has become the only state website to host an llms.txt file, providing structured information on content usage and policies on accessibility and updates. This move has sparked varied opinions from other states and tech experts.

• Skepticism Towards llms.txt: Several experts, including Luke Fretwell of ScanGov, argue that llms.txt files may be redundant as they provide information already covered by robots.txt and sitemaps. Their adoption remains a debated topic.

• Google’s Stance: Google’s search team expressed indifference towards llms.txt files, with no confirmed use or benefits, though they could still be monitored to observe potential future adoption.

• Impact of llms.txt: Despite initial skepticism, Maryland has seen a significant rise in AI crawler requests since introducing llms.txt, indicating its potential usefulness in guiding AI interactions with government websites.

• Adoption of AI Tools: The shift towards increased AI usage is changing how users access information, with a growing preference for AI-driven searches, prompting a broader push to adapt government websites for AI readiness.

• Future of Government Information Access: In an age with increasing cyberattacks, officials like Crosby Burns advocate for prioritizing public access to information, stressing the importance of foundational information architecture to support evolving technology and user preferences.