We’re with you. JoomShaper is offering a 20% Discount on all plans due to the impact of COVID-19.

Support Offline : Mon - Fri / 10am - 6pm (GMT +6)

Your Time: Our Time:

access denied to googlebot

Featured Lock Resolved Issue
Hi,

I just put online a website with 2 content extensions including SPPB.
After optimizing the images, I cannot pass the performance test of developers.google.com/speed/pagespeed/insights/?url=https%3A%2F%2Fwww%2Fferronnerie-57.com (see image1)

I ended up finding that files would block the passage of googlebot (see image2)
Google can not understand this page fully because some page resources are blocked by robots.txt

Blocked page resources:

- https:/components/com_sppagebuilder/assets/css/font-awesome-5.min.css
- https:/components/com_sppagebuilder/assets/css/font-awesome-v4-shims.css
- https:/components/com_sppagebuilder/assets/css/animate.min.css
- https:/components/com_sppagebuilder/assets/css/sppagebuilder.css
- https:/components/com_sppagebuilder/assets/css/sppagecontainer.css
- https:/components/com_sppagebuilder/assets/css/magnific-popup.css
- https:/components/com_sppagebuilder/assets/css/jquery.bxslider.min.css
- https:/components/com_sppagebuilder/assets/js/jquery.parallax.js
- https:/components/com_sppagebuilder/assets/js/sppagebuilder.js
- https:/components/com_sppagebuilder/assets/js/jquery.magnific-popup.min.js
- https:/components/com_sppagebuilder/assets/js/jquery.bxslider.min.js

I can remove Disallow: /components/
but I find this to be a problem in general. Is it possible that you are moving the files or a temporary solution?

Thanks for your help

Attachments (2)

  • ferr_pb-perf.png
    ferr_pb-perf.png 27.2 KB
  • ferr_pb-google-access.png
    ferr_pb-google-access.png 227.1 KB

11 Answers

Paul Frankowski - Staff

More than a month ago #Permalink
Hi,
we are not going to move those files anywhere.

Just unlock this folder inside robots.txt

Paul Frankowski - Staff

More than a month ago #Permalink
Besides, as you may know, css & js files don't have website content only styles.

HDcms

More than a month ago #Permalink
HI,
I will do the manual solution in robots.txt but I am not very happy that you do not understand the situation.
I use the standard robots.txt file from joomla which seems interesting to me so as not to see indexed urls (notably /component/).
I made on 2 sites a home page with SPPB and I put months before to realize that the home page was not well indexed because of SPPB files (in particular) !!
Please talk to the developers
Regards

Paul Frankowski - Staff

More than a month ago #Permalink
To be honest, but I don't understand two things:
1) Since when CSS & JS files may help SEO?
2) All components keep CSS & JS in this same way. We used standard method.

Chris

More than a month ago #Permalink
The standard Joomla robots.txt is set as it is for a reason - there are ways to allow Google to see the files it needs and this is not a JoomShaper issue in any way, shape or form.

I manually edit the robots.txt file of every Joomla website I create to make sure the relevant files can be scanned/checked as you mention, including those websites which do not use any JoomShaper products.

Nothing to do with JoomShaper...

Paul Frankowski - Staff

More than a month ago #Permalink
Thank you Chris,
I am ironically waiting for someday someone will blame us for burned pixels on the screen or broken keyboard because he used it when working with our products.
If Google would offer a support forum then it would be much easier to solve such cases.

HDcms

More than a month ago #Permalink
HI,
I sincerely believe that there is a responsibility for SPPB. I try one last time to explain it differently.
Sorry for my bad english
By default, I use the robots.txt file or I leave the settings on the directories like /componnents/
There are good reasons for this

1/ I create SPPB pages
2/ I realize that I am poorly referenced
3/ I spend weeks and a lot of time looking for the source with online tools or in SEO forums.
4/ I discover one day that googlebot does not pass on certain pages because certain SPPB files are compressed in their original directory (see image at the beginning)
5/ Finally I find these 2 extensions which cause problems.
6/ The other extension recognizes and will no longer put these files under /component/

I will manually modify the robots.txt file but I find that it remains a problem

Paul Frankowski - Staff

More than a month ago #Permalink
On my private website about my books, I kept original robots.txt + JCH_Optimzer and ... my site is on the top10 google.
I think Google is "blind" and indexes how it wants and whenever it wants

Paul Frankowski - Staff

More than a month ago #Permalink
I think you should talk with former or current Goolge employee :p
to be 99% sure.

Chris

More than a month ago #Permalink
https://moz.com/blog/why-all-seos-should-unblock-js-css

You don't, so far as I know, have to allow the full components directory.

Chris

More than a month ago #Permalink
I use JCH Optimise on some of my sites, this lets you sort this out entirely with their Pro version.

It is commonly enough documented online for Joomla that it is certainly not a 'JoomShaper' thing.


There are no replies made for this post yet.
Be one of the first to reply to this post!

126

Templates

333946

Community Users

Newsletter

Don’t miss any updates of our new templates and extensions and all the astonishing offers we bring for you.
We never spam

Joomla! ® name is used under a limited license from Open Source Matters in the United States and other countries. JoomShaper.com is not affiliated with or endorsed by Open Source Matters or the Joomla! Project.

Connect Us