Ai
Controlling AI Crawlers with robots.txt - How to Allow Search, Training and Agents Separately
AI crawlers have added more to write in robots.txt. A rundown of per-purpose bot names, Google-Extended and Content-Signal.
Blocking AI Bots with Cloudflare - Managed robots.txt and AI Crawl Control
Cloudflare writes robots.txt on your behalf and blocks AI bots at the edge. It also shows you which bots took how much.
Local AI/LLM Infrastructure Guide
A guide to setting up a local LLM server with Ollama, then connecting Claude Code you already use to run on top of it.
How to Install Ollama and Set Up a Local LLM Server
This explains how to run a local LLM on a Linux server using Ollama. It covers installation, external access configuration, model selection, and HTTP API call examples to outline the process of …
