Reply to post: Re: It's BeOS, it's the OS

Google's Fuchsia OS Flutters into view: We're just trying out some new concepts, claims exec

Cederic Silver badge

Re: It's BeOS, it's the OS

I would expect Bing to comply with robots.txt too.

Interestingly going to https://www.reddit.com/robots.txt returns 'Not found' - looks like they're providing a different response based on who's making the request (user agent or IP address range?)

Based on https://canicrawl.com/reddit.com/ they're not naming any specific forums.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

SUBSCRIBE TO OUR WEEKLY TECH NEWSLETTER

Biting the hand that feeds IT © 1998–2019