How to know if your website is in Google's mobile-first index ?
Watch your server logs easily with Unix command-line tools.
By Julien on December 15, 2017
In a recent hangout, Google’s John Mueller told webmasters to watch their server logs, in order to know if a website has been moved to the mobile first index.
This kind of monitoring can easily be done with a few Unix command-line tools, as long as you know where to find your log files.
First, we’ll filter our log file to keep only the lines corresponding to any Googlebot:
grep Googlebot access.log
You should see loads of lines stream on your screen. Stop the process by typing ctrl
+c
.
We’ll now use awk
to feep only the User-agent from each line. The -F
argument sets the field separator, while '{print $6}'
prints only the 6th field of each line (in our case, the User-agent). Let’s try this:
grep Googlebot access.log | awk -F\" '{print $6}'
Loads of lines streaming again, but only with User-agents. Everything is working fine :-)
Finally, let’s count how many visits we got from each UA. We’ll just sort
the stream, then count duplicate lines:
grep Googlebot access.log | awk -F\" '{print $6}' | sort | uniq -c
This displays the count of visits per User-agent: simply compare volumes for desktop and mobile Googlebots.
Here is what I get on one of my sites:
10405 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
779 Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Still in desktop index !