How to resolve Google “Indexed, though blocked by robots.txt” warning for a user login protected page?

How to resolve Google “Indexed, though blocked by robots.txt” warning for a user login protected page? - Google Search Console is a free application that allows you to identify, troubleshoot, and resolve any issues that Google may encounter as it crawls and attempts to index your website in search results. If you’re not the most technical person in the world, some of the errors you’re likely to encounter there may leave you scratching your head. We wanted to make it a bit easier, so we put together this handy set of tips about seo, google-search-console, google-search, googlebot to guide you along the way. Read the discuss below, we share some tips to fix the issue about How to resolve Google “Indexed, though blocked by robots.txt” warning for a user login protected page?.Problem :

Or, in other words, how can I tell google this is a user login protected page?

The website is basically a single page app with server side rendering. That said, if the visitor is asking for a web page by text/html, the server will do server side rendering to show the html, and download all the app. After that, the app will has its own routing and navigate to the page inside the app. The app only ask for data if necessary.

Now, google show me a warning Indexed, though blocked by robots.txt, the url is actually one url of the user's account pages. It's protected, of course.

If google visit the url, it will see 200 http code, then download the html, but it's actually an empty container. Then, since this visitor is not logged in. The app will immediately navigate to the sign-in route.

I guess this warning will bring negative impact to our SEO. I'm wondering what should I do for Google SEO, because our seo is quite bad now.

1.Respond with 403 and all the html/app, and navigate to sign in after the app is mounted?

2.Just like traditional website, respond with 302 to redirect google to sign-in page, and request the server again?

Appreciated for your time to let me know what's the correct way to tell google this is an user login required page. It seems it's so hard to please google.

Solution :

I think you'll want to go for the 403 - which Google shouldn't attempt to index and will remove from it's index. If it's a page that only has content behind a login, it should just be removed.

A 302 suggests that the content has temporarily moved, which isn't the case, and you might find Google indexing the login page and treating it as the URL for the protected user account page, which isn't what you want.

If the issue about seo, google-search-console, google-search, googlebot is resolved, there’s a good chance that your content will get indexed and you’ll start to show up in Google search results. This means a greater chance to drive organic search traffic to your site.