Monday, October 8, 2018
On Monday, October 8, 2018 at 12:18:27 AM UTC-4, Hassan Schroeder wrote:
On Fri, Oct 5, 2018 at 11:04 PM, fugee ohu <fuge...@gmail.com> wrote:
> I see bots crawling devise pages in my log:
> Should I disallow those paths in robots.txt ?
1) why?
2) do your logs show these bots *reading* robots.txt?
--
Hassan Schroeder ------------------------ hassan.s...@gmail.com
twitter: @hassan
Consulting Availability : Silicon Valley or remote
Yes
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe@googlegroups.com.
To post to this group, send email to rubyonrails-talk@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/1fe76e02-c222-450b-8eca-16e4df491f18%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment