The Google Search Console API offers a gold mine of free search data for those who are able to get at it. The first step in being able to get at that data is to connect to the API with Python.

Why is the Search Console API so great?

If you’re an SEO’er or even a webmaster, then you’ve probably used Google Search Console at some time to see what search terms people are using to find your website on Google. SC also offers vast amount of historic data for each page of your website that details things like CTR, impressions, search rankings and even what devices your visitors are using.

By accessing Search Console through your browser, you are limited in the amount of actual data you are able to extract due to a general limitation. For example, you are only able to see about 1000 keywords for your site over a time period.

The Search Console API has a much bigger limit and with some clever programming, you can basically remove the limit all together.

What are the steps needed to connect to Search Console with Python?

  • Verifying a domain in Google Search Console
  • Setting up the python scripts and environment
  • Activating your Google Search Console API
  • Authenticating your Python app to access your search data

Before we begin, I am assuming you have a basic understanding of how to use Python. If not there are tons of online courses like the one over at Codecademy where you can learn the basics and even take it further if you feel daring.

Verifying a domain in Google Search Console

Before you can do anything, you’ll need to make sure that the Google account you use is a verified owner of the domain property in Search Console. You can learn more about this here

There are two main types of verification offered currently by Google. The easiest way is to verify by updating your DNS records with a TXT file that Google gives you. However, I find that this way only verifies a property without the full URL E.G digitalhop.co.uk as opposed to https://digitalhop.co.uk

You will need to be verified for a full URL such as https://yourdomain.com for your scripts to work. So if you don’t have that yet or you only have a property without the https part verified, then go ahead and verify a full domain property using the URL prefix option.

Google Search Console Domain Authentication

There are two ways to verify your website in Search Console. You will need to have URL prefix option for the Python scripts to work properly. Sometimes it is easier to do the domain verification first and then the url prefix verification

Setting up the python scripts and environment

  • You’ll need some Python scripts to do the heavy lifting. So head over to my Github Search Console repository.
  • Clone the repository and store it on your local machine.
  • Open up your command prompt and navigate to the directory where you stored your cloned repository
  • We’re going to install all the modules you need to run these Python scripts

Activating your Google Search Console API

Google let’s you set up free API’s for its tools like Search Console in the Google API Dashboard. We’re going to setup a Search Console API and also give it the credentials it needs to connect your python scripts to Search Console API.

Step1 – Follow this link to create your Google API project.

Create Google API Project

Here you create your API Project in Google.

Step2 – Click the go to credentials button

Click the go to credentials button

Step 3 – don’t worry about too much here, just click the OAuth consent screen link in the left hand navigation

Add credentials to your project screen

We will not be worrying about this now. Click the OAuth consent screen in the left hand navigation

Step 4 – We are going to select External here

OAuth Consent Screen

Select External

Step 6 – Enter a name for the App and click save

Give your App a name

Step 7 – On left hand navigation click Credentials and then – click create credentials at the top and then click Oath client ID

Click create credentials

Select OAuth client ID from the drop down menu

Select OAuth client ID

Step 8 – Select Other for the application type and give it a unique short name – (if ‘other’ is not an option it could be because you already have an Oath credential in your project using the ‘other’ property, you can always make a new project)

Create OAuth client ID

The client id and secret should appear, but we are going to use the json file

Step 9 – Click the down arrow to download the json file with your client id and your client secret. Put this file into the config folder found in the repository you downloaded from Github

Download the JSON Secret Key

Authenticating your Python app to access your search data

Now that we have set up our Google Search Console API we need to authenticate it.

  • Run the check_verified_sites.py file in the folder you cloned from the repository
  • The terminal will print out a URL for you to visit to verify that you are accepting your new Python app to connect to your Google personal account. Visit the URL and login to your personal Google account and accept the conditions
  • Google then gives you an authentication code. Copy this code and paste it into the terminal and hit enter

If Everything has worked, then the terminal will print out a list of the sites that are verified for you to access the search console data.

What next?

Now that you’re set up with the Google API and basic Search Console Python script, it’s a good time to see how many keywords you have stored in Search Console. It’s possible that you have over 16 months worth of Keyword data waiting for you.

Every time a visitor has used Google to find your website on Google, this has been recorded in your Search Console webmaster account. Using the Top keywords extractor script we can get up to 25 000 keywords if a few seconds.