Skip to main content

Website pages cannot be crawled properly

Jp Martinez

Hi!  I've been looking at a few support tickets in the open forum regarding the sitemap.xml file having a negative affect on Google Search rankings.  I also saw that you responded with a breakdown of the default robots.txt files which are created  properly.

But I would like you to take a look at the first site I built with Brizy:  williamgibbsmd.com

The robots.txt are not being blocked, as the site is ranking on Google search, but I did see the pages meta information is blocking crawlers with messages such as "The page is blocked by "noindex" meta tag" and "The page is blocked by "nofollow". See the below extracted from the first line of the source code for the home page.

 

<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"><link rel="canonical" href="https://www.williamgibbsmd.com/" /><link rel="icon" href="https://a-cloud.b-cdn.net/images/cropped-favicon-192x192.png" sizes="16x16"><title>Dr. William Gibbs, MD - NYC Sports Physician &amp; Wellness Specialist </title><meta name="description" content="Dr. William Gibbs, MD provides expert care with non-narcotic pain management therapy, physician-assisted weight Loss management, &amp; anti-aging treatments."><meta property="og:title" content="Dr. William Gibbs, MD - NYC Sports Physician &amp; Wellness Specialist "><meta property="og:description" content="Dr. William Gibbs, MD provides expert care with non-narcotic pain management therapy, physician-assisted weight Loss management, &amp; anti-aging treatments."><meta property="og:url" content="https://www.williamgibbsmd.com/"><meta content="summary_large_image" name="twitter:card"><meta name="robots" content="noindex, nofollow">

Based on Google specs of the meta name attribute (https://developers.google.com/search/docs/advanced/robots/robots_meta_tag#directives)  the value  of content="noindex, nofollow" means:

  • noindex =  Do not show this page, media, or resource in search results. If you don't specify this directive, the page, media, or resource may be indexed and shown in search results.
  • nofollow = Do not follow the links on this page. If you don't specify this directive, Google may use the links on the page to discover those linked pages.

This is a big issue. For my white label service, my clients, and potential seo issues for everyone, especially your customer base. Please rectify this as soon as possible.  Your may want to add checkboxes on the page settings to allow users to make this choice.

Please feel free to contact me. We can zoom anytime to discuss a solution.  Thank you.  JP
0

Comments

1 comment

Date Votes
  • Jp Martinez

    Apologies. It seems that the SEO visibility toggle was not active. Doing so cleared the problem.

    0

Please sign in to leave a comment.

Didn't find what you were looking for?

Ask a Question