Uncategorized

SEO Analytics for Free – Combining Google Search with the Moz API

Posted by Purple-Toolz

I’m a self-funded start-up business owner. As such, I want to get as much as I can for free before convincing our finance director to spend our hard-earned bootstrapping funds. I’m also an analyst with a background in data and computer science, so a bit of a geek by any definition.

What I try to do, with my SEO analyst hat on, is hunt down great sources of free data and wrangle it into something insightful. Why? Because there’s no value in basing client advice on conjecture. It’s far better to combine quality data with good analysis and help our clients better understand what’s important for them to focus on.

In this article, I will tell you how to get started using a few free resources and illustrate how to pull together unique analytics that provide useful insights for your blog articles if you’re a writer, your agency if you’re an SEO, or your website if you’re a client or owner doing SEO yourself.

The scenario I’m going to use is that I want analyze some SEO attributes (e.g. backlinks, Page Authority etc.) and look at their effect on Google ranking. I want to answer questions like “Do backlinks really matter in getting to Page 1 of SERPs?” and “What kind of Page Authority score do I really need to be in the top 10 results?” To do this, I will need to combine data from a number of Google searches with data on each result that has the SEO attributes in that I want to measure.

Let’s get started and work through how to combine the following tasks to achieve this, which can all be setup for free:

  • Querying with Google Custom Search Engine
  • Using the free Moz API account
  • Harvesting data with PHP and MySQL
  • Analyzing data with SQL and R

Querying with Google Custom Search Engine

We first need to query Google and get some results stored. To stay on the right side of Google’s terms of service, we’ll not be scraping Google.com directly but will instead use Google’s Custom Search feature. Google’s Custom Search is designed mainly to let website owners provide a Google like search widget on their website. However, there is also a REST based Google Search API that is free and lets you query Google and retrieve results in the popular JSON format. There are quota limits but these can be configured and extended to provide a good sample of data to work with.

When configured correctly to search the entire web, you can send queries to your Custom Search Engine, in our case using PHP, and treat them like Google responses, albeit with some caveats. The main limitations of using a Custom Search Engine are: (i) it doesn’t use some Google Web Search features such as personalized results and; (ii) it may have a subset of results from the Google index if you include more than ten sites.

Notwithstanding these limitations, there are many search options that can be passed to the Custom Search Engine to proxy what you might expect Google.com to return. In our scenario, we passed the following when making a call:

https://www.googleapis.com/customsearch/v1?key=<google_api_id>&userIp=
<ip_address>&cx<custom_search_engine_id>&q=iPhone+X&cr=countryUS&start=
1</custom_search_engine_id></ip_address></google_api_id>

Where:

  • https://www.googleapis.com/customsearch/v1 – is the URL for the Google Custom Search API
  • key=<GOOGLE_API_ID> – Your Google Developer API Key
  • userIp=<IP_ADDRESS> – The IP address of the local machine making the call
  • cx=<CUSTOM_SEARCH_ENGINE_ID> – Your Google Custom Search Engine ID
  • q=iPhone+X – The Google query string (‘+’ replaces ‘ ‘)
  • cr=countryUS – Country restriction (from Goolge’s Country Collection Name list)
  • start=1 – The index of the first result to return – e.g. SERP page 1. Successive calls would increment this to get pages 2–5.

Google has said that the Google Custom Search engine differs from Google .com, but in my limited prod testing comparing results between the two, I was encouraged by the similarities and so continued with the analysis. That said, keep in mind that the data and results below come from Google Custom Search (using ‘whole web’ queries), not Google.com.

Using the free Moz API account

Moz provide an Application Programming Interface (API). To use it you will need to register for a Mozscape API key, which is free but limited to 2,500 rows per month and one query every ten seconds. Current paid plans give you increased quotas and start at $250/month. Having a free account and API key, you can then query the Links API and analyze the following metrics:

Moz data field

Moz API code

Description

ueid

32

The number of external equity links to the URL

uid

2048

The number of links (external, equity or nonequity or not,) to the URL

umrp**

16384

The MozRank of the URL, as a normalized 10-point score

umrr**

16384

The MozRank of the URL, as a raw score

fmrp**

32768

The MozRank of the URL’s subdomain, as a normalized 10-point score

fmrr**

32768

The MozRank of the URL’s subdomain, as a raw score

us

536870912

The HTTP status code recorded for this URL, if available

upa

34359738368

A normalized 100-point score representing the likelihood of a page to rank well in search engine results

pda

68719476736

A normalized 100-point score representing the likelihood of a domain to rank well in search engine results

NOTE: Since this analysis was captured, Moz documented that they have deprecated these fields. However, in testing this (15-06-2019), the fields were still present.

Moz API Codes are added together before calling the Links API with something that looks like the following:

www.apple.com%2F?Cols=103616137253&AccessID=MOZ_ACCESS_ID&
Expires=1560586149&Signature=<MOZ_SECRET_KEY>

Where:

  • https://ift.tt/1bbWaai&#8221; class=”redactor-autoparser-object”>https://ift.tt/2oVcks4&#8230; – Is the URL for the Moz API
  • http%3A%2F%2Fwww.apple.com%2F – An encoded URL that we want to get data on
  • Cols=103616137253 – The sum of the Moz API codes from the table above
  • AccessID=MOZ_ACCESS_ID – An encoded version of the Moz Access ID (found in your API account)
  • Expires=1560586149 – A timeout for the query – set a few minutes into the future
  • Signature=<MOZ_SECRET_KEY> – An encoded version of the Moz Access ID (found in your API account)

Moz will return with something like the following JSON:

Array
(
    [ut] => Apple
    [uu] => <a href="http://www.apple.com/" class="redactor-autoparser-object">www.apple.com/</a>
    [ueid] => 13078035
    [uid] => 14632963
    [uu] => www.apple.com/
    [ueid] => 13078035
    [uid] => 14632963
    [umrp] => 9
    [umrr] => 0.8999999762
    [fmrp] => 2.602215052
    [fmrr] => 0.2602215111
    [us] => 200
    [upa] => 90
    [pda] => 100
)

For a great starting point on querying Moz with PHP, Perl, Python, Ruby and Javascript, see this repository on Github. I chose to use PHP.

Harvesting data with PHP and MySQL

Now we have a Google Custom Search Engine and our Moz API, we’re almost ready to capture data. Google and Moz respond to requests via the JSON format and so can be queried by many popular programming languages. In addition to my chosen language, PHP, I wrote the results of both Google and Moz to a database and chose MySQL Community Edition for this. Other databases could be also used, e.g. Postgres, Oracle, Microsoft SQL Server etc. Doing so enables persistence of the data and ad-hoc analysis using SQL (Structured Query Language) as well as other languages (like R, which I will go over later). After creating database tables to hold the Google search results (with fields for rank, URL etc.) and a table to hold Moz data fields (ueid, upa, uda etc.), we’re ready to design our data harvesting plan.

Google provide a generous quota with the Custom Search Engine (up to 100M queries per day with the same Google developer console key) but the Moz free API is limited to 2,500. Though for Moz, paid for options provide between 120k and 40M rows per month depending on plans and range in cost from $250–$10,000/month. Therefore, as I’m just exploring the free option, I designed my code to harvest 125 Google queries over 2 pages of SERPs (10 results per page) allowing me to stay within the Moz 2,500 row quota. As for which searches to fire at Google, there are numerous resources to use from. I chose to use Mondovo as they provide numerous lists by category and up to 500 words per list which is ample for the experiment.

I also rolled in a few PHP helper classes alongside my own code for database I/O and HTTP.

In summary, the main PHP building blocks and sources used were:

One factor to be aware of is the 10 second interval between Moz API calls. This is to prevent Moz being overloaded by free API users. To handle this in software, I wrote a “query throttler” which blocked access to the Moz API between successive calls within a timeframe. However, whilst working perfectly it meant that calling Moz 2,500 times in succession took just under 7 hours to complete.

Analyzing data with SQL and R

Data harvested. Now the fun begins!

It’s time to have a look at what we’ve got. This is sometimes called data wrangling. I use a free statistical programming language called R along with a development environment (editor) called R Studio. There are other languages such as Stata and more graphical data science tools like Tableau, but these cost and the finance director at Purple Toolz isn’t someone to cross!

I have been using R for a number of years because it’s open source and it has many third-party libraries, making it extremely versatile and appropriate for this kind of work.

Let’s roll up our sleeves.

I now have a couple of database tables with the results of my 125 search term queries across 2 pages of SERPS (i.e. 20 ranked URLs per search term). Two database tables hold the Google results and another table holds the Moz data results. To access these, we’ll need to do a database INNER JOIN which we can easily accomplish by using the RMySQL package with R. This is loaded by typing “install.packages(‘RMySQL’)” into R’s console and including the line “library(RMySQL)” at the top of our R script.

We can then do the following to connect and get the data into an R data frame variable called “theResults.”

library(RMySQL)
# INNER JOIN the two tables
theQuery <- "
    SELECT A.*, B.*, C.*
    FROM
    (
        SELECT 
            cseq_search_id
        FROM cse_query
    ) A -- Custom Search Query
    INNER JOIN
    (
        SELECT 
            cser_cseq_id,
            cser_rank,
            cser_url
        FROM cse_results
    ) B -- Custom Search Results
    ON A.cseq_search_id = B.cser_cseq_id
    INNER JOIN
    (
        SELECT *
        FROM moz
    ) C -- Moz Data Fields
    ON B.cser_url = C.moz_url
    ;
"
# [1] Connect to the database
# Replace USER_NAME with your database username
# Replace PASSWORD with your database password
# Replace MY_DB with your database name
theConn <- dbConnect(dbDriver("MySQL"), user = "USER_NAME", password = "PASSWORD", dbname = "MY_DB")
# [2] Query the database and hold the results
theResults <- dbGetQuery(theConn, theQuery)
# [3] Disconnect from the database
dbDisconnect(theConn)

NOTE: I have two tables to hold the Google Custom Search Engine data. One holds data on the Google query (cse_query) and one holds results (cse_results).

We can now use R’s full range of statistical functions to begin wrangling.

Let’s start with some summaries to get a feel for the data. The process I go through is basically the same for each of the fields, so let’s illustrate and use Moz’s ‘UEID’ field (the number of external equity links to a URL). By typing the following into R I get the this:

> summary(theResults$moz_ueid)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
      0       1      20   14709     182 2755274 
> quantile(theResults$moz_ueid,  probs = c(1, 5, 10, 25, 50, 75, 80, 90, 95, 99, 100)/100)
       1%        5%       10%       25%       50%       75%       80%       90%       95%       99%      100% 
      0.0       0.0       0.0       1.0      20.0     182.0     337.2    1715.2    7873.4  412283.4 2755274.0 

Looking at this, you can see that the data is skewed (a lot) by the relationship of the median to the mean, which is being pulled by values in the upper quartile range (values beyond 75% of the observations). We can however, plot this as a box and whisker plot in R where each X value is the distribution of UEIDs by rank from Google Custom Search position 1-20.

Note we are using a log scale on the y-axis so that we can display the full range of values as they vary a lot!

A box and whisker plot in R of Moz’s UEID by Google rank (note: log scale)

Box and whisker plots are great as they show a lot of information in them (see the geom_boxplot function in R). The purple boxed area represents the Inter-Quartile Range (IQR) which are the values between 25% and 75% of observations. The horizontal line in each ‘box’ represents the median value (the one in the middle when ordered), whilst the lines extending from the box (called the ‘whiskers’) represent 1.5x IQR. Dots outside the whiskers are called ‘outliers’ and show where the extents of each rank’s set of observations are. Despite the log scale, we can see a noticeable pull-up from rank #10 to rank #1 in median values, indicating that the number of equity links might be a Google ranking factor. Let’s explore this further with density plots.

Density plots are a lot like distributions (histograms) but show smooth lines rather than bars for the data. Much like a histogram, a density plot’s peak shows where the data values are concentrated and can help when comparing two distributions. In the density plot below, I have split the data into two categories: (i) results that appeared on Page 1 of SERPs ranked 1-10 are in pink and; (ii) results that appeared on SERP Page 2 are in blue. I have also plotted the medians of both distributions to help illustrate the difference in results between Page 1 and Page 2.

The inference from these two density plots is that Page 1 SERP results had more external equity backlinks (UEIDs) on than Page 2 results. You can also see the median values for these two categories below which clearly shows how the value for Page 1 (38) is far greater than Page 2 (11). So we now have some numbers to base our SEO strategy for backlinks on.

# Create a factor in R according to which SERP page a result (cser_rank) is on
> theResults$rankBin <- paste("Page", ceiling(theResults$cser_rank / 10))
> theResults$rankBin <- factor(theResults$rankBin)
# Now report the medians by SERP page by calling ‘tapply’
> tapply(theResults$moz_ueid, theResults$rankBin, median) 
Page 1 Page 2 
    38     11 

From this, we can deduce that equity backlinks (UEID) matter and if I were advising a client based on this data, I would say they should be looking to get over 38 equity-based backlinks to help them get to Page 1 of SERPs. Of course, this is a limited sample and more research, a bigger sample and other ranking factors would need to be considered, but you get the idea.

Now let’s investigate another metric that has less of a range on it than UEID and look at Moz’s UPA measure, which is the likelihood that a page will rank well in search engine results.

> summary(theResults$moz_upa)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   1.00   33.00   41.00   41.22   50.00   81.00 
> quantile(theResults$moz_upa,  probs = c(1, 5, 10, 25, 50, 75, 80, 90, 95, 99, 100)/100)
  1%   5%  10%  25%  50%  75%  80%  90%  95%  99% 100% 
  12   20   25   33   41   50   53   58   62   75   81 

UPA is a number given to a URL and ranges between 0–100. The data is better behaved than the previous UEID unbounded variable having its mean and median close together making for a more ‘normal’ distribution as we can see below by plotting a histogram in R.

A histogram of Moz’s UPA score

We’ll do the same Page 1 : Page 2 split and density plot that we did before and look at the UPA score distributions when we divide the UPA data into two groups.

# Report the medians by SERP page by calling ‘tapply’
> tapply(theResults$moz_upa, theResults$rankBin, median) 
Page 1 Page 2 
    43     39 

In summary, two very different distributions from two Moz API variables. But both showed differences in their scores between SERP pages and provide you with tangible values (medians) to work with and ultimately advise clients on or apply to your own SEO.

Of course, this is just a small sample and shouldn’t be taken literally. But with free resources from both Google and Moz, you can now see how you can begin to develop analytical capabilities of your own to base your assumptions on rather than accepting the norm. SEO ranking factors change all the time and having your own analytical tools to conduct your own tests and experiments on will help give you credibility and perhaps even a unique insight on something hitherto unknown.

Google provide you with a healthy free quota to obtain search results from. If you need more than the 2,500 rows/month Moz provide for free there are numerous paid-for plans you can purchase. MySQL is a free download and R is also a free package for statistical analysis (and much more).

Go explore!

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Uncategorized

5 Things You Should Know About “People Also Ask” & How to Take Advantage

Posted by SamuelMangialavori

It’s undeniable that the SERPs have changed considerably in the last year or so. Elements like featured snippets, Knowledge Graphs, local packs, and People Also Ask have really taken over the SEO world — and left some of us a bit confused.

In particular, the People Also Ask (PAA) feature caught my attention in the last few months. For many of the clients I’ve worked with, PAAs have really had an impact on their SERPs.

If you are anything like me, you might be asking yourself the same questions:

  • How important are these SERP features?
  • How many clicks do they “steal” from SEO?
  • And most importantly: who are these people that also ask SO MANY questions? Somehow, I always imagine the hipster-looking man from Answer the Public being the leader of such a group of people…

The first part of the post focuses on five things I’ve learned about People Also Ask, while the second part outlines some ideas on how to take advantage of such features.

Let’s get started! Here are five things you should know about PAAs.

1. PAA can occupy different positions on the SERP

I don’t know about you all, but I wasn’t fully aware of the above until a few months ago; I just assumed that most of the time PAAs appeared in the same location, IF and only IF it was actually triggered by Google. I didn’t really pay attention to this featured until I started digging into it.

Distinct from featured snippets (which appear always at the top of the SERP), PAAs can be located in several different parts of the page.

Let’s look at some examples:

Keyword example: [dj software]

Example of SERP where PAA is at the top of the page

For the keyword [dj software], this is what the SERP looks like:

  • 3 PPC ads
  • Related videos
  • 4 PAA listings at the top of the page
  • 10 organic results

Keyword example: [cocktail dresses under 50 pounds]

Example of SERP where PAA is in the middle of the page

For the keyword [cocktail dresses under 50 pounds], this is what the SERP looks like:

  • Shopping ads
  • 1 PPC ad
  • Image carousel
  • 3 organic results
  • 4 PAA listings in the middle of the page

Keyword example: [tv unit]

Example of SERP where PAA is at the bottom of the page

For the keyword [tv unit], this is what the SERP looks like:

  • Shopping ads
  • 1 PPC ad
  • 10 organic results
  • 3 PAA listings at the bottom of the page

Why does this matter to you?

Understanding the implications of the different positions of PAA in the SERPs impacts organic results’ CTR, especially on mobile, where space is very precious.

2. Do PAAs have a limit?

I’m just giving away the answer now: No-ish.

This feature has the ability to trigger a potentially infinite number of questions on the topic of interest. As Britney Muller researched in this Moz post, the initial 3–4 listing could continue into the hundreds once clicked on, in some cases.

With one simple click, the 4 PAA questions can trigger three more listings, and so on and so forth.

Has the situation changed at all since the original 2016 Moz article?

Yes, it has! What I’m seeing now is actually very mixed: PAAs can vary extensively, from a fixed number of 3–4 listings to a plethora of results.

Let’s look at an example of a query that’s showing a large number of PAAs:

Keyword example: [featured snippets]

Example of SERP where the number of PAA expands when clicked upon, and is not fixed

For the query [featured snippets], the PAA listings can be expanded if clicked on, which process generates a large number of new PAA listings that appear at the bottom of such SERP feature.

For other queries, Google will only show you 4 PAA listings and such number will not change even if the listings get clicked on:

Keyword example: [best italian wine]

Example of SERP where the number of PAA listings is fixed and does not expand

For the query [best italian wine], the PAA listings cannot be expanded, no matter how many times you hover or click on them.

Interestingly, it also appears that Google does not keep this feature consistent: a few days after I took the above screenshots, the fixed number of PAAs was gone. On the other hand, I’ve recently seen instances where the keywords have a fixed amount of only 3 PAAs instead of 4.

Now, the real question for Google would be:

“What methodology are they using to decide which keywords trigger an infinite amount of PAAs and which keywords cannot?”

As you might have guessed by now, I don’t have an answer today. I’ll continue to work on uncovering it and keep you folks posted when/if I get an answer from Google or discover further insights.

My two cents on the above:

The number of PAAs does not relate to particular verticals or keywords patterns at the moment, though this may change in the future (e.g. comparative keywords more or less inclined to a fixed amount of PAAs.)

Google’s experiments will continue, and they may change PAAs quite a bit in the next one to two years. I wouldn’t be surprised if we saw questions being answered in different ways. Read the next point to know more!

Why does this matter to you?

From an opportunity standpoint, the number of questions you can scrape to take advantage of will vary.

From a user standpoint, it impacts your search journey and offers a different number of answers to your questions.

3. PAAs can trigger video results

I came across this by reading an article on Search Engine Roundtable.

Example PAA with video results

I wasn’t able to replicate the above result myself in London — but that doesn’t matter, as we’re used to seeing Google experimenting with new features in the US first.

Answering a PAA listing with a video makes a lot of sense, especially if you consider the nature of many of the queries listed:

  • What is…
  • How to…
  • Why is/are…

And so on.

I expect this to be tested more and more by Google, to a point where most of the keywords that are currently showing video results in the SERPs will trigger video results in the PAA listings, too.

Keyword example: [how to clean suede shoes diy]

Example of SERP for keywords that often trigger video results

Video results will matter more and more in the near future. Why is that?

Just examine how hard Google is working on the interpretation and simplification of video results. Google has added key moments for videos in search results (read this article to know more). This new feature allows us to jump to the portion of the video that answers our specific query.

Why does this matter to you?

From an opportunity standpoint, you can optimize your YouTube and video results to be eligible to appear in PAAs.

From a user standpoint, it enriches your search journey for PAA queries that are better answered with videos.

4. PAA questions are frequently repeated for the same search topic and also trigger featured snippets

This might be obvious, but it’s important to understand these three points:

  1. Most PAA questions also trigger featured snippets
  2. The same PAA question (& answer) can be triggered for different keywords
  3. The same answer/listing that appears for a certain question in a PAA can also appear for different questions triggered by PAAs

Let’s look at some examples to better visualize what I mean:

1. PAA questions also trigger Featured Snippets

Keyword 1: [business card ideas]

Keyword 2: [what is on a good business card?]

Example of PAA listings for case n.1

The keyword [business card ideas] triggers some PAA listings, whose questions, if used as the main query, trigger a featured snippet.

2. Different keywords can trigger the same PAA question and show the same result. 

The same listing that appears for a PAA question for keyword X can also appear for the same question, triggered by a different keyword Y.

Keyword 1: [quality business cards]

Keyword 2: [business cards quality design]

Example of PAA listings for case n.2

To summarize: Different keywords, same question in the PAA and same listing in the PAA.

3. Different questions listed in a PAA triggered by different keywords can show the same result. 

The same listing that appears for a PAA question for keyword X can also appear for the same question, triggered by a different keyword Y.

Keyword 1: [quality business cards]

Keyword 2: [best business cards online]

Example of PAA listings for case n.3

To summarize: Different keywords, different question in the PAA but same listing in the PAA.

The above keywords are clearly different, but they show the same intent:

“I’m looking for a business card by using terms that highlight certain defining attributes — best & quality.”

Small Biz Trends in the above screenshot has created a page that matches that particular intent. Keyword intent is a crucial topic that the SEO community has been talking about for a few years by now.

Why does this matter to you?

From an opportunity standpoint, your PAA listings can trigger featured snippets and also have the possibility to cover a portfolio of different keyword permutations.

5. PAAs have a feedback feature

Most of you have probably glanced over this feature but never really paid attention to it: at the bottom of the last PAA listing, there is often a little hyperlink with the word Feedback.

By clicking on it, you’re shown the following pop-up:

Example of feedback for PAA

Google states that this option is available “on some search results” and it allows users to send feedback or suggest a translation. Even if you do go through the effort, Google says they will not reply to you directly, but rather collect the info submitted and work on the accuracy of the listings.

Does this mean they’ll actually change the PAA listing based off of feedback?

Unfortunately, I don’t have an answer for this (I’ve tried to submit feedback manually and nothing really happened) but I think it’s very unlikely.

The only for-sure thing you get from Google is the following response:

Google’s response after feedback submission

Why does this matter to you?

From an opportunity standpoint, if you notice that PAA listings (for questions you are trying to appear for) are not accurate, you can flag it to Google and hope they’ll change it.

Now that we’ve covered some interesting facts, how can we take advantage of PAA?

Determine how deeply your SERP is being affected by PAA (and other SERP features)

This task is fairly straightforward, but I guarantee you very few people actually pay much attention to it. When monitoring your rankings, you should really try to dig deeply into which other elements are affecting your overall organic traffic & organic CTR.

Start by asking yourself the following questions:

  • What elements affect the SERP for my core keywords?
  • How often do these SERP elements appear?
  • How deeply are they affecting my organic results?

You might spot an increasing amount of paid results (in the form of shopping ads for products or text ads for services) appearing for many of your key terms.

Established tools like SEMrush, Sistrix, and Ahrefs can show you the number of ads, overall spending, & how the ads look at a keyword level.

Kw: [hr software]

SEMrush ads history graph by keyword

Or it may be the case that organic SERP elements, such as video results, are being triggered in the SERP for many of your informational queries, or that featured snippets appear for a high percentage of your navigational & transactional terms, and so on.

Recently, I came across a client where over 90% of their primary keywords triggered PAAs at the top of the SERP. 90%!

Which tools can help?

At Distilled we use STAT, which reports on such insights in a really comprehensive manner with a great overview of all the SERP elements.

This is what the STAT SERP features interface looks like:

STAT SERP features

Ahrefs also does a great job of allowing you to download the SERP features of the top twenty results for any of the keywords you’re interested in.

Understanding where you stand in the current SERP landscape & how your SEO has been affected by it is a crucial step prior to implementing any SERP strategy.

Tactics to take advantage of PAAs

There are several ways to incorporate PAAs into your SEO strategy. It’s already been written about many times online, so I’m going to keep it simple and focus on a few easy tactics that I think will really improve your workflow:

1. Extract PAA listings

This one’s pretty straightforward: how can we take advantage of PAAs if we cannot find a way to extract those questions in the first place?

There are several ways to “scrape” PAAs, more or less compliant with Google’s Terms & Conditions (such as using Screaming Frog).

Personally, I like STAT’s report, so I’ll talk about how easy it is to extract PAA listings using this tool:

  • One of the features of STAT’s reporting is called “People also ask (Google),” which is pretty self-explanatory: for the keywords you’ve decided to track in the tool, this report will provide the PAA questions they trigger and the URLs appearing for those listings, along with their exact rankings within the PAA box.

This is an example of how the report will look like after you’ve downloaded the “People also ask (Google)” report:

STAT PAA report

2. Address questions in your content

Once you have a list of all PAA questions and you are able to see which URLs rank for such results, what should you do next?

This is the more complicated part: think how your content strategy can incorporate PAA findings and start experimenting. Similarly to featured snippets, PAAs should be included in your content plan. If that’s not yet the case, well, I hope this blog post can convince you to give it a go!

Since I am not focusing (sadly, for some) on content strategy with this article, I will not dwell on the topic too much. Instead, I’ll share a few tips on what you could do with the data gathered so far:

Understand what type of results such PAA questions are triggering: are they informational, navigational, transactional?

Many people think featured snippets and PAA questions are triggered by heavily informational or Q&A pages: trust me, do NOT assume anything. heck your data and behave accordingly. Keyword intent should never be taken for granted.

Create or re-optimize your content

Depending on the findings in the previous point, it may be a matter of creating new content that can address PAA questions or re-optimizing the existing content on your site.

If you discover that you have a chance at ranking in a PAA with your current transactional/editorial pages, it might be best to re-optimize what you have.

It may also be the case that one of the following options can be enough to rank in PAAs:

  • Adding questions and answers to your content (don’t limit yourself to just the bottom of the page)
  • Using the right headings to mark up such elements (h1, h2, h3, whatever works for your page)
  • Copying the formatting of results that are currently appearing in PAA
  • Simply changing the language used on your site

If you do not have any content to cover a certain keyword theme, think about creating new ones that would match the keyword intent that Google is favoring. Editorial content with SEO in mind (don’t limit yourself to PAA, but look at the overall SERP spectrum) or simple FAQs pages could really help win PAA or featured snippets.

Depending on your KPIs (traffic, leads, signups, etc), tailor your newly optimized content and be ready to retain users on your site

Once users land on your site after clicking on a PAA listing, what do you want them to see/do? Don’t do half the job, worry about the entire user journey from the start!

3. Test schema on your page

The SEO community has gone a bit cray-cray over the new FAQs schema — my colleague Emily Potter wrote a great post on it.

FAQs and how-to schema represent an interesting opportunity for SERP features such as featured snippets and PAAs, so why not give it a go? Having the right content & testing the right type of schema may help you win precious snippets or PAAs. In the future, I expect Google to increase the amount of markup that refers to informational queries, so stay tuned — and test, test, and test some more!

Think of the extended search volume opportunity

Without digging too much into this topic (it deserves a post on its own), I’ve been thinking about the following idea quite a lot recently:

What if we started looking at PAAs as organic listings, hence counting the search volume for the keywords that trigger such PAAs?

Since PAAs and other elements have been redefining the SERPs as we know them, maybe it’s time for us marketers to redefine how these features are impacting our organic results. Maybe it’s time for us to consider the extended search opportunity that such features bring to the table and not limit ourselves at the tactics mentioned above.

Just something to think about!

PAA can be your friend

By now, I hope you’ve learned a bit more about People Also Ask and how it can help your SEO strategy moving forward.

PAA can be your friend indeed if you’re willing to spend time understanding how your organic visibility can be influenced by such features. The fact that PAAs are now popular for a large portfolio of queries makes me think Google considers them a new, key part of the user journey.

With voice search on the rise, I expect Google to pay even more attention to elements like featured snippets and People Also Ask. I don’t think they’re going anywhere soon — so my dear fellow SEOs, you should start optimizing for the SERPs starting today!

Feel free to get in touch with us at Distilled or on Twitter at @SamuelMng to discuss this further, or just have a chat about who these people who also ask so many questions actually are…

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Uncategorized

Intro to SEO Competitive Analysis 101 – Whiteboard Friday

Posted by Cyrus-Shepard

A good, solid competitive analysis can provide you with priceless insights into what’s working for other folks in your industry, but it’s not always easy to do right. In this week’s edition of Whiteboard Friday, Cyrus walks you through how to perform a full competitive analysis, including:

  • How to identify your true competitors
  • Keyword gap analysis
  • Link gap analysis
  • Top content analysis

Plus, don’t miss the handy tips on which tools can help with this process and our brand-new guide (with free template) on SEO competitive analysis. Give it a watch and let us know your own favorite tips for performing a competitive analysis in the comments!



Click on the whiteboard image above to open a high-resolution version in a new tab!

Video Transcription

Howdy, Moz fans. Welcome to another edition of Whiteboard Friday. I’m Cyrus Shepard. Today we’re talking about a really cool topic — competitive analysis. This is an introduction to competitive analysis.

What is competitive analysis for SEO?

It’s basically stealing your competitors’ traffic. If you’re new to SEO or you’ve been around awhile, this is a very valuable tactic to earn more traffic and rankings for your site.

Instead of researching blindly what to go after, competitive analysis can tell you certain things with a high degree of accuracy that you won’t find other ways, such as:

  • what keywords to target, 
  • what content to create, 
  • how to optimize that content, and 
  • where to get links.

How to do an SEO competitive analysis

How does it do this?

Well, instead of researching just in a keyword tool or a link tool, with competitive analysis you look at what’s actually working for your competitors and use those tactics for yourself.

This often works so much better than the old-style ways of research, because you can actually improve upon what other people are actually doing and make those tactics work for you.

1. Identify your top competitors

So to get started with competitive analysis, the first challenge is to actually identify your top competitors.

This sounds easy. You probably think you know who your competitors are because you type a keyword into Google and you see who’s ranking for your desired keyword. This does work, to certain degree.

Another way to do it is to look at the keywords you rank for, because the challenge is you probably rank for far more keywords than you believe you do.

Moz, for instance, ranks for hundreds of thousands or possibly even millions of keywords, and we want to know at scale who are all the competitors ranking for all those different queries. This is very hard to do manually.

Fortunately there are a lot of SEO tools out there — Ahrefs, SEMrush — many tools that can tell you look at all the keywords that you rank for across thousands of SERPs and then calculate, using advanced metrics, exactly who your true competitors are.

I’m happy to announce that Moz just released a tool that does exactly this. We’re going to link to it in the transcript below.

It’s called Domain Analysis. It’s a free tool. Anybody can use it.

You simply type in your domain, and we look through all the keywords that your site ranks for in our database, we look at all the competitors, and we use some advanced heuristics and we match those up and we tell you who your true competitors are. Once you know your true competitors, you can continue with the rest of the analysis.

2. Perform a keyword gap analysis

The first step that most people take in doing an SEO competitive analysis is identifying the keyword gap. Now for a long time, when I was new to SEO, I heard this term “keyword gap” and I didn’t really know what it meant. But it’s actually really simple.

It’s simply what keywords do my competitors rank for that I don’t rank for, and that’s the gap. The idea is that we want to close that gap if the keyword is valuable or high volume. The trick is you can do this on your own manually. You can see all the keywords you rank for using an advanced keyword tool and then list all the keywords your competitors rank for and then combine those lists in Excel. It’s a long, tedious process.

Fortunately, again, major SEO tools, such as Moz, can do this at scale for you within seconds. If you go to Moz Keyword Explorer, you simply enter your domain, enter your top competitor’s domain that we found in this first step, and it will list all the keywords that your competitors rank for that you don’t rank for. 

You can then pull this into a spreadsheet and find keywords with high volume or keywords that are valuable and relevant to your business.

This is an important point. You don’t just want to go willy-nilly after any keyword your competitor ranks for. You want to actually find the keywords that are relevant to your business.

3. Perform a link gap analysis

So after you do that, we also have the cousin of a keyword gap analysis — link gap analysis.

This is a very similar concept, because you need links to rank. But where do you find the links? So you want to ask, “Who links to my competitors but does not link to me?”

The theory here is that if someone is linking to your competitor on a similar topic, they are more likely to link to you because they are in that business of linking out to that type of content.

An advanced tip is you often want to look at two or more competitors. The idea is that if someone is linking to multiple sources but not to you, it’s more likely they’ll link to you if you have superior content.

Again, SEO tools can provide something like this. You can list all the backlinks to yourself or your competitors and combine them in a spreadsheet. But the tools make it much easier.

In Moz’s Link Explorer, you simply enter your competitor, you enter another competitor and yours, and you can find all the people who are linking to those competitors but not to you.

An advanced tip that I like to use is do it at the page level. Don’t look for domains that are linking to your competitors. Look for specific pages and you can do this in Link Explorer. We’re going to show you in a little more detail in a guide I’m going to link to at the bottom of this post.

4. Perform a top content analysis

So we understand links, we understand the keywords. But what content do we want to create?

Top content analysis, this is very easy to do these days. You’re basically looking for content that earns your competitors a lot of traffic or a lot of links.

The idea is if other people are linking to these things, then it’s highly probable that you can earn links with similar but better content. So the idea is you go to a tool like Link Explorer. You can sort by top pages, and you pick out the content that has the most links for your competitor. Then don’t just re-create the content, but make it better. This is called the skyscraper technique, the idea of finding content that does really well and then making it better.

Then once you have this, you go back to your link gap analysis and you reach out to those people who are linking to that content and you ask them for links, showing them the better content.


So that’s it in a nutshell. When we put it all together, we have a very valuable process. We can go back to our individual pages, look at those pages that are ranking for our competitors. When you’re all done, you can actually take your page, plug it into your keyword gap, and see all the keywords the page is ranking for.

Our original keyword gap analysis looked at the domain, but now we just want to know what the page is ranking for. We can add that into our own page and make the page even better. We can again reach out to the same people who are linking to this page, show them our better content, and that is the process.

New Guide & Free Template

Whew, I’m exhausted. This is a huge process. I went over it really quickly. Fortunately, if it went by a little fast for you, we just released a guide, “An Introduction to SEO Competitive Analysis.” We’re going to link to it.

Get the Guide + Free Template

Guide to SEO Competitive Analysis

It explains all these processes in much more detail. It’s free to use. I hope you enjoy it.

Hey, I really enjoyed making this video. If you found value in it, give it a thumbs up. Please share on social media and we’ll talk to you next time. Thanks, everybody.

Video transcription by Speechpad.com

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Uncategorized

Franchise Marketing: How People Buy Now

Posted by MiriamEllis

This post contains an excerpt from our new primer: The Practical Guide to Franchise Marketing.

Planet Fitness, Great Clips, Ace Hardware… you can imagine the sense of achievement the leadership of these famous franchises must enjoy in making it to the top of lists like Entrepreneur’s 500. Behind the scenes of success, all competitive franchisors and franchisees have had to manage a major shift — one that centers on customers and their radically altered consumer journeys.

Research online, buy offline. Always-on laptops and constant companion smartphones are where fingers do the walking now, before feet cross the franchise threshold. Statistics tell the story of a public that searches online prior to the 90% of purchases they still make in physical stores.

And while opportunity abounds, “being there” for the customers wherever they are in their journey has presented unique challenges for franchises. Who manages which stage of the journey? Franchisor or franchisee? Getting it right means meeting new shopping habits head-on, and re-establishing clear sight-lines and guidelines for all contributors to the franchise’s ultimate success.

Over the next few weeks, we’ll be publishing a series of articles dedicated to franchises. Want all the info now? Download The Practical Guide to Franchise Marketing:

Download now



Seeing the Shift

Whoever your franchise’s customers are, demographically, we can tell you one thing: they aren’t buying the same way they were ten, or even five years ago. For one thing, they used to decide to buy at your business as they browsed shelves or a menu. Now, 82% of smartphone users consult their devices before making an in-store purchase. Thank you, digital marketing!

Traditionally, online marketing wasn’t something that franchisees had to think much about. And that was sort of a good thing because everyone knew their lane.

  • Franchisors handled national or regional marketing through broadcast, print, and other media. They also handled digital marketing — which, within recent recall, consisted mainly of a website, social media accounts, and paid search.
  • Franchisees managed the local beat with coupons, flyers, direct mail, and other community and word-of-mouth marketing efforts.

Then people started shopping differently and traditional lanes began merging. Customers started using online directories to get information. They started using online listings for discovering local businesses “near me” on a map. They started reading online reviews to make choices. They started browsing online inventories or menus in advance. They started using cell phones to make reservations, click to call you, or to get a digital voice assistant like Siri or Alexa to give them directions to the nearest and best local option.

Suddenly, what used to be a “worldwide” resource — the internet — began to be a local resource, too. And a really powerful one. People were finding, choosing, and building relationships online not just with the national brand, but with local shops, services and restaurants, often making choices in advance and showing up merely to purchase the products or services they want.

Stats State the Case

Consider how these statistics are impacting every franchise:

  • 76% of people who search for something nearby on their smartphone visit a related business within a day, and 28% of those searches result in a purchase. – Google
  • 88% of shoppers regularly or occasionally browse products online before purchasing them in a store. – Adweek
  • 45% of brick-and-mortar sales in 2018 started with an online review — a 15% year-over-year increase from 2017. – Bazaarvoice
  • According to Google, “near me” mobile searches that contain a variant of “can I buy” or “to buy” have grown over 500% in the past two years, and we’ve seen a 900% growth in mobile search for “___near me today/tonight.” – Google
  • Search interest in ”open now” has increased 300% in the past two years. – Google

These are huge changes — and not ones the franchise model was entirely ready for.

There used to be a clear geographic split between a franchise’s corporate awareness marketing and franchisee local sales marketing that was easy to understand. But the above statistics tell new tales. Now there is an immediacy and urgency to the way customers search and shop that’s blurring old lines.



Ace is the place with the helpful hardware folks

Even a memorable jingle like this one goes nowhere unless the franchisor/franchisee partnership is solid. How do customers know a brand like Ace stands by its slogan when they see a national TV campaign like this one which strives to distinguish the franchise from understaffed big box home improvement stores?

Customers feel the nation-wide promise come true as soon as they walk into an Ace location:

  • Place located where the internet said it was? Check!
  • Abundance of staff? Check!
  • Friendly? Check!
  • Online purchase ready for pickup? Check!
  • Trust earned? Check!

A brand promo only works when all sides are equally committed to making each location of the business visible, accessible, and trusted. This joint effort applies to every aspect of how the business is marketed. From leadership to door greeter, everyone has a role to play. It’s defining those roles that can make or break the brand in the new consumer environment.

We’ll be exploring the nuts and bolts of building ideal partnerships in future installments of this series. Up next is The Unique World of Franchise Marketing. Keep an eye out for it on the blog at the end of the month!

Don’t want to wait for the blog posts to come out? Download your copy now of our comprehensive look at unique franchise challenges and benefits: 

Get my copy

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

Uncategorized

New SEO Experiments: A/B Split Testing Google’s UGC Attribute

Posted by Cyrus-Shepard

When Craig Bradford of Distilled reached out and asked if we’d like to run some SEO experiments on Moz using DistilledODN, our reply was an immediate “Yes please!”

If you’re not familiar with DistilledODN, it’s a sophisticated platform that allows you to do a number of cool things in the SEO space:

  1. Make almost any change to your website through the ODN dashboard. Since the ODN is a cloud platform that sits in front of your website (like a CDN) it doesn’t matter how your website is built or what CMS it uses. You can change a single page — or more likely — entire sections.
  2. The ODN allows you to A/B split test these changes and both measure and predict their impact on organic traffic. They also have a feature called full-funnel testing allowing you to measure impact on both SEO and CRO at the same time.

When you find something that works, you see a positive result like this:

DistilledODN Positive Result

SEO experimentation is great, but almost nobody does it right because it’s impossible to control for other factors. Yes, you updated your title tags, but did Google roll out an update today? Sure, you sped up your site, but did a bunch of spam just link to you?

A/B split testing solves this problem by applying your changes to only a portion of your pages — typically 50% — and measuring the difference between the two groups. Fortunately, the ODN can deploy these changes near-instantly, up to thousands of pages at a time.

It then crunches the numbers and tells you what’s working, or not.

Testing Google’s UGC link attribute

For our first test, we decided to tackle something simple and fast. Craig suggested looking at Google’s new link attributes, and we were off!

To summarize: Google recently introduced new link attributes for webmasters/SEOs to label links. Those attributes are:

  • rel=”sponsored” – For paid and sponsored links
  • rel=”ugc” – For links in user-generated content (UGC)
  • rel=”nofollow” – Remains a catch-all for all followed links

On the Moz blog, all comments links are currently marked “nofollow” — following years of SEO best practices. Google has stated that using the new attributes won’t give you a rankings boost. That said, we wanted to test for ourselves if changing these links to “ugc” would impact the rankings/traffic of our blog pages.

To be clear: We are not testing if the pages we link to change rankings, but instead the source page that hosts the link — in this case, the blog pages with comments.

Here’s an example of a comment the ODN modified.

UGC Comment

After we set the test running, 50% of blog posts had comments with “ugc” links, while 50% kept their original “nofollow” attributes.

Experiment results

We expected a “null” test — meaning we wouldn’t see a significant impact.

In fact, that’s exactly what happened.

DistilledODN Null Results

If we detected a significant change, the probability cone at the bottom right would have pointed more dramatically up or down.

In fact, at a 95% confidence interval, the test predicted traffic would either fall 26,000 visits/month or gain 9,300 visits/month.

Hence, a null result.

This validates Google’s statements that using the “ugc” attribute won’t give you a ranking boost.

What should Moz test next?

While “null” tests aren’t as fun as a positive result, we have a lot of cool A/B SEO testing ahead of us.

The great thing is we can now test out changes with the ODN, and when we find one that works, pass that to our developers to make the changes permanently. This cuts down on needless development work and stops the guessing game.

We have a Trello board set up for test ideas, and we’d love to add some community ideas to the mix. The ODN is currently running on the Moz Blog and Q&A, so anything in these site sections is fair game.

We’re also looking at experiments where we use Moz data to inform these decisions. For example, a Moz Pro crawl identified that the Moz Blog titles currently use H2 tags instead of H1. Google recently indicated this likely shouldn’t impact rankings, but wouldn’t it be good to test?

Missing H1 Tags

What wild/clever/ridiculous/obvious SEO things should we test? With each good test, we’ll publish the results. Leave your ideas in the comments below.

Big thanks to the Distilled Team, including Will Critchlow and Tom Anthony, for embarking on this journey with us.

And if you’d like to learn more about DistilledODN and SEO split testing in general, this post is highly recommended.

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!