I wondered about the robots.txt
I can see the case for it, I could also see the case for allowing at least Google to index the site.
Has there been some discussion about this previously?
I wondered about the robots.txt
I can see the case for it, I could also see the case for allowing at least Google to index the site.
Has there been some discussion about this previously?
It’s any day now I think, EU pension funds are moving out https://www.removepaywall.com/search?url=https%3A%2F%2Fwww.ft.com%2Fcontent%2F9d90d557-48e5-4f4b-a927-88071cef8ea9
Would you be up for re-enabling Google indexing? It is crappy, but still…
Not very motivated, but I can look into it.
I think it would just be
User-agent: * Disallow: / User-agent: Googlebot Allow: /