We’re with you. JoomShaper is offering a 20% Discount on all plans due to the impact of COVID-19.

Support Offline : Mon - Fri / 10am - 6pm (GMT +6)

Your Time: Our Time:

access denied to googlebot

Featured Lock Resolved Issue

I just put online a website with 2 content extensions including SPPB.
After optimizing the images, I cannot pass the performance test of developers.google.com/speed/pagespeed/insights/?url=https%3A%2F%2Fwww%2Fferronnerie-57.com (see image1)

I ended up finding that files would block the passage of googlebot (see image2)
Google can not understand this page fully because some page resources are blocked by robots.txt

Blocked page resources:

- https:/components/com_sppagebuilder/assets/css/font-awesome-5.min.css
- https:/components/com_sppagebuilder/assets/css/font-awesome-v4-shims.css
- https:/components/com_sppagebuilder/assets/css/animate.min.css
- https:/components/com_sppagebuilder/assets/css/sppagebuilder.css
- https:/components/com_sppagebuilder/assets/css/sppagecontainer.css
- https:/components/com_sppagebuilder/assets/css/magnific-popup.css
- https:/components/com_sppagebuilder/assets/css/jquery.bxslider.min.css
- https:/components/com_sppagebuilder/assets/js/jquery.parallax.js
- https:/components/com_sppagebuilder/assets/js/sppagebuilder.js
- https:/components/com_sppagebuilder/assets/js/jquery.magnific-popup.min.js
- https:/components/com_sppagebuilder/assets/js/jquery.bxslider.min.js

I can remove Disallow: /components/
but I find this to be a problem in general. Is it possible that you are moving the files or a temporary solution?

Thanks for your help

Attachments (2)

  • ferr_pb-perf.png
    ferr_pb-perf.png 27.2 KB
  • ferr_pb-google-access.png
    ferr_pb-google-access.png 227.1 KB

11 Answers

Paul Frankowski - Staff

More than a month ago #Permalink
we are not going to move those files anywhere.

Just unlock this folder inside robots.txt

Paul Frankowski - Staff

More than a month ago #Permalink
Besides, as you may know, css & js files don't have website content only styles.


More than a month ago #Permalink
I will do the manual solution in robots.txt but I am not very happy that you do not understand the situation.
I use the standard robots.txt file from joomla which seems interesting to me so as not to see indexed urls (notably /component/).
I made on 2 sites a home page with SPPB and I put months before to realize that the home page was not well indexed because of SPPB files (in particular) !!
Please talk to the developers

Paul Frankowski - Staff

More than a month ago #Permalink
To be honest, but I don't understand two things:
1) Since when CSS & JS files may help SEO?
2) All components keep CSS & JS in this same way. We used standard method.


More than a month ago #Permalink
The standard Joomla robots.txt is set as it is for a reason - there are ways to allow Google to see the files it needs and this is not a JoomShaper issue in any way, shape or form.

I manually edit the robots.txt file of every Joomla website I create to make sure the relevant files can be scanned/checked as you mention, including those websites which do not use any JoomShaper products.

Nothing to do with JoomShaper...

Paul Frankowski - Staff

More than a month ago #Permalink
Thank you Chris,
I am ironically waiting for someday someone will blame us for burned pixels on the screen or broken keyboard because he used it when working with our products.
If Google would offer a support forum then it would be much easier to solve such cases.


More than a month ago #Permalink
I sincerely believe that there is a responsibility for SPPB. I try one last time to explain it differently.
Sorry for my bad english
By default, I use the robots.txt file or I leave the settings on the directories like /componnents/
There are good reasons for this

1/ I create SPPB pages
2/ I realize that I am poorly referenced
3/ I spend weeks and a lot of time looking for the source with online tools or in SEO forums.
4/ I discover one day that googlebot does not pass on certain pages because certain SPPB files are compressed in their original directory (see image at the beginning)
5/ Finally I find these 2 extensions which cause problems.
6/ The other extension recognizes and will no longer put these files under /component/

I will manually modify the robots.txt file but I find that it remains a problem

Paul Frankowski - Staff

More than a month ago #Permalink
On my private website about my books, I kept original robots.txt + JCH_Optimzer and ... my site is on the top10 google.
I think Google is "blind" and indexes how it wants and whenever it wants

Paul Frankowski - Staff

More than a month ago #Permalink
I think you should talk with former or current Goolge employee :p
to be 99% sure.


More than a month ago #Permalink

You don't, so far as I know, have to allow the full components directory.


More than a month ago #Permalink
I use JCH Optimise on some of my sites, this lets you sort this out entirely with their Pro version.

It is commonly enough documented online for Joomla that it is certainly not a 'JoomShaper' thing.

There are no replies made for this post yet.
Be one of the first to reply to this post!




Community Users


Don’t miss any updates of our new templates and extensions and all the astonishing offers we bring for you.
We never spam

Joomla! ® name is used under a limited license from Open Source Matters in the United States and other countries. JoomShaper.com is not affiliated with or endorsed by Open Source Matters or the Joomla! Project.

Connect Us