What does it mean for a site to be indexed by Google?
Google’s index acts like a giant library of all the web pages it has crawled and stored. When someone types a query into Google, they’re tapping into Google’s vast index to discover the most relevant web pages on that topic.
For your page to be a part of those search results, it first needs to be indexed. If your page isn’t indexed, it’s as good as invisible to those trying to find it.
How Do You Check If Google Has Indexed Your Site?
Method 1: Using Google Search Operators
Process
The easiest way to check if your website has been indexed by Google is to do the following;
-
Open Up Google.
-
Use the site search operator (e.g.site:example.com.au) for the domain of your website.
Click Tools on the top right and it will display the number of search results.
What to Expect
If no results show, that means Google has not indexed any of the pages on your site.
Limitations
This method lacks the capacity to show all of the pages that are indexed by Google, especially if your site has a significant number of pages.
Method 2 - Leveraging Google Search Console for Index Checking
Process
A more reliable method is to use Google Search Console, a free website diagnostics tool Google provides to monitor a wide range of factors relating to your site’s health. There is a dedicated section to check indexing performance, so how do we do this?
- Navigate to Google Search Console.
- Make sure your website’s Google Search Console profile is verified.
- Enter the URL of any of your pages, in the search bar at the top of the page.
- Google will fetch and return the index status of the page.
There is also the option to use the page indexing report which highlights the number of pages that are indexed and not indexed, as well as providing a historical view for the past few months as seen below;
You also have the ability to filter by sitemaps you’ve submitted, allowing you to assess how different sections of your site are performing.
What to Expect
When Google fetches and returns the index status it will produce one of the following;
- URL is on Google
- URL is on Google, but has issues
- URL is not on Google
- Coverage issues
Limitations
While Google Search Console is a sure-fire way to get the index status of a page, this method isn’t very efficient, as checking each page individually is quite time-consuming.
In contrast the page indexing report provides details about the state of the sites indexed pages, however this data is lagged and may not accurately depict the status of all current live pages.
Method 3 - Streamlining Index Checks with Google Search Console API and Screaming Frog
Process
For those wanting a more accurate and up-to-date representation of indexing performance, integrating Screaming Frog with the Google Search Console API offers a dynamic solution.
Here’s how to set it up:
-
Open up Screaming Frog
-
Go to Config > API Access > Google Search Console
-
Choose your property e.g. your domain
-
Under URL Inspection tick ‘Enable URL Inspection’
-
Then crawl your site
What to Expect
This will tell us the same information as Google Search Console but in an automated fashion and will output the following;
-
URL Is on Google – The URL is indexed by Google
-
URL is on Google, But Has Issues – The URL has been indexed and can appear in Google Search results, but there are some problems with mobile usability.
-
URL Is Not on Google – The URL is not indexed by Google and won’t appear in the search results.
-
Indexable URL Not Indexed – Indexable URLs found in the crawl that are not indexed by Google and won’t appear in the search results.
Limitations
- Limited API Calls: The API can handle up to 2000 URL checks per day per property.
- Crawl Coverage: If a URL isn’t captured during the crawl, its index status remains unknown.
- Immediate Needs: Pinpointing whether a specific page is indexed can be challenging without a targeted search.
Method 4 - Automating Index Checks with Google Search Console API, Screaming Frog, and Google Sheets Integration
Process
In order to once again improve on this process we can integrate Screaming Frog with Google Search Console API and Google Sheets to offer a more effective solution. This setup allows for scheduled, automated index checks that can streamline your workflow. Here’s how to get started:
- Open up Screaming Frog
- Go to File > Scheduling > Add
- General – Name your Task and Project and set a Date/Time and regularity for how often you would like this task to be completed
- Start Options – Choose your desired crawl settings (Ensure you selected a saved crawl with the URL Inspection setting turned on)
- API – Google Search Console should be ticked.
- Export
- Tick headless
- Choose what tabs you would like to export
- Format = gsheet
What to Expect
Once configured, this method automates the indexing status checks of your site, producing the same options but sending the output straight to a Google Sheet that is automatically uploaded into your Google Drive.
Benefits;
Accessibility: The results are accessible through Google Drive, which allows the whole team to access and review the indexing status of the site.
Customisation – Users can format the Google Sheet making the data easier to analyse.
Alerts – An integration with Google Apps Script or other automation tools can enable alerts to notify you when important pages are unexpectedly deindexed, ensuring immediate access and awareness.
So what method is best for you?
At the end of the day, all of these methods can be useful but if you’re looking to take your SEO to the next level, we recommend ongoing and comprehensive monitoring of your site’s indexing in Google. Our method 4 may provide such a solution – especially in the case of larger sites with thousands of pages.
However, for those times when you need to quickly check the indexing status of a few URLs, the earlier methods may be more practical and faster. They offer simplicity and immediacy, which can be advantageous for spot checks.
If you’re aiming to enhance your business’s online presence, consider reaching out to nimbl. As a boutique digital marketing agency, nimbl specialises in SEO and paid digital advertising, offering tailored strategies that align with your specific needs to drive growth and visibility. Contact us today to learn how we can help your business grow.
