robots.txt Rules Parser
Fetch robots.txt and test whether specific URLs are allowed or blocked for any user-agent, including wildcard and path rules. Use it to debug crawler access and verify disallow rules.
Tool results are generated in English.
Leathanaigh coibhneasta
Fadhb ar an freastalaí? Suíomh hackáilte? Luchtú mall?
Solúfar fadhbanna criticiúla laistigh de 30 nóiméad — gan stad, 7 lá sa tseachtain.