How to allow Googlebot to crawl paywall content?

How to allow Googlebot to crawl paywall content? - Google Search Console is a free application that allows you to identify, troubleshoot, and resolve any issues that Google may encounter as it crawls and attempts to index your website in search results. If you’re not the most technical person in the world, some of the errors you’re likely to encounter there may leave you scratching your head. We wanted to make it a bit easier, so we put together this handy set of tips about google-search, googlebot, googlebot-mobile, to guide you along the way. Read the discuss below, we share some tips to fix the issue about How to allow Googlebot to crawl paywall content?.Problem :

My website has pages for around 1000 companies. Without an account, I only allow users to view 5 company pages a month (I track them by cookie). And Googlebot can crawl my site freely since it does not use cookie.

To avoid Googlebot thinking I am doing cloaking, I am specify the content of my company pages as paywall content (following this guide). The problem is now when I search the company pages of my website on Google, the search result only shows the page title and my website's default description. It does not show content of each company page.

I think the problem is I list the content of my company pages as paywall. My question is: What is the best way to let Googlebot crawl and display the content of my website in the search results without making Googlebot think that I am cloaking? (when the users click on the search result link and visit my website, they are still subject to the monthly quota if they don't have an account).

Solution :

Google says in their flexible sampling documentation:

Enclose paywalled content with structured data in order to help Google differentiate paywalled content from the practice of cloaking, where the content served to Googlebot is different from the content served to users.

If you are using structured data to tell how the content is limited for users, then you may give Googlebot full access based on either the User-Agent header or its IP addresses. You won't be penalized for cloaking when implementing flexible sampling despite basing decisions about access partly on whether or not it is Googlebot.

If the issue about google-search, googlebot, googlebot-mobile, is resolved, there’s a good chance that your content will get indexed and you’ll start to show up in Google search results. This means a greater chance to drive organic search traffic to your site.