Category: Online Marketing

What are redirects? What do redirects and SEO have to do with each other? We’ll go over the various types of redirects, which one is best for SEO, and how to create and test your redirects.

What is a Redirect?

A redirect is when one page URL forwards the visitor to a different page URL. A redirect can be in place because the original page is no longer available and there is a different relevant page that the visitor will likely be interested in.

A redirect can solve a 404 error page. If the page no longer exists, instead of giving the visitor an error page, redirect them to a relevant existing page.

Types of Redirects

301

The 301 redirect means that the page has been permanently removed. It means the original page is gone forever, and instead you should view this other page.

302

The 302 redirect means that the page has been temporarily moved. This type of redirect is not as common as a 301 redirect. With a 302 redirect you are telling search engines that you only have this in place short-term. This type of redirect can be used to perform a test.

Meta Refresh

A meta refresh is a type of redirect that is controlled on the page instead of on the server. This type of redirect is slower and you will often see a message that says “You will be redirected to…”.

Which Type of Redirect is Best for SEO?

For SEO purposes you want to use 301 redirects. A 301 redirect actually passes link equity from the original page to the new page.

Google has stated that both 301 and 302 redirects can pass link equity. However, it can take longer for a 302 since it has to first be determined that it is being used permanently instead of temporarily.

When Do You Add a Redirect?

When a Page Has Been Removed

If you remove a page and you have a different relevant page on your site you can add a redirect. The key here is relevance. You don’t want to add a redirect if the new page is not a good replacement for the missing page.

If a page is removed and there isn’t a good replacement page it’s okay to leave it as a 404 error page. Or, you can change it to a 410 page to show that it has been intentionally removed.

To Streamline URL Entry

Another scenario where a redirect could be beneficial is to confirm your website visitors are accessing the preferred URL. If you have a secure site using SSL and your domain can be accessed via https, make sure the http version redirects.

Similarly, you should redirect the www subdomain to the non-www subdomain or vice versa. Either way is fine, just choose a preference for your site and stick to it.

How to Add a Redirect

Understanding how redirects are implemented at least from a high-level perspective is beneficial. We’ll try and describe the different methods that can be used but not dig into every detail.

.htaccess File

If you have an Apache server (which you probably do unless your website uses ASP.NET) you can add redirects in your .htaccess file. This file should be found in the root directory of your web server.

If you don’t have the file already, all you need to do is make a blank file in a text editor such as Notepad and name it .htaccess. Make sure it doesn’t add the .txt file extension when you save it.

To add a redirect just add a line to the .htaccess file like this:

Redirect 301 /old-page.html /new-page.html

Configure Through Web Host

Depending on who your hosting provider is they may allow you to add a redirect through their system. Sometimes the hosts will have an interface that allows you to enter in old URLs and choose the type of redirect and the new URL you’d like to send the traffic to.

Add Redirects in WordPress

If you have a WordPress site there are plugins you can download to make the process as simple as possible. I’d recommend using the Redirection plugin.

The Redirection plugin allows you to manually add in redirects through a simple to use interface. It also has an option to monitor changes to post and page URLs. When a URL changes, it automatically adds a redirect from the old URL to the new URL. This is a very convenient feature because it saves you from having to worry about it.

Test Your Redirects

Fix Redirect Chains

A redirect chain is when an internal link on your website directs to an old URL that is now redirected to a new URL. This is fairly harmless but it can increase load time. It’s better for search engine spiders and users to clean up your URLs when you notice a redirect chain.

To find them, you can use Screaming Frog. Screaming Frog is a downloadable desktop tool that can scan your website to help uncover technical website details. It’s free for up to 500 URLs.

To determine if you have redirect chains that need to be fixed:

1. Run a Screaming Frog scan on your website.
2. Under the first Internal tab change the filter dropdown to HTML.
3. Click the column header to sort by Status Code.
4. If any 301 redirects appear in the list it’s because a page somewhere on your site is linking to it and causing a redirect chain. Click on a row to get more details.
5. At the very bottom click the tab called Inlinks. This will list the pages on your website that link to this old URL.
6. Go through and update the links on your website where necessary.

Fix Protocol Errors (http vs https)

Another common redirect problem you want to test for is when you link to the wrong version of the web page. This is similar to a redirect chain. It isn’t a real problem but it isn’t a great experience for the user so it should be fixed.

To identify protocol errors:

1. Run a Screaming Frog scan on your website.
2. Under the first Internal tab click the column header to sort by Address.
3. If any URLs are using the wrong protocol on your website they should now be grouped together at the top. Click on a row to get more details.
4. At the very bottom click the tab called Inlinks. This will list the pages on your website that link to this old URL.
5. Go through and update the links on your website where necessary.

Redirects can be very helpful when you’re trying to keep your website organized and easy to navigate. However, implementing them in a manner that causes no issues could sometimes be tricky. Just remember to keep the user in mind and do what creates the best user experience and you’ll be in a good situation.

Do you have any SEO related redirect tips? Let us know in the comments!

Understanding some basic HTML tags for SEO purposes will be beneficial when you’re adding content to your website. Don’t be scared, HTML doesn’t have to be overwhelming and complicated. There are some simple tags that a beginner can learn. These will go a long way when it comes to optimizing your content for SEO.

What is HTML?

HTML stands for Hypertext Markup Language and it is the code used to build web pages. It is the basic building blocks of any website.

Why do I need to use HTML?

If you’re adding content to a website it will be added using HTML. If you don’t understand HTML, that doesn’t mean you’re out of luck and can’t build a website. If you’re using a content management system, such as WordPress, you can build out website content by filling out pre-built elements. But, to make sure you’re building content in an SEO friendly manner, you still want to have at least a basic understanding of HTML tags and components.

Ensuring your content is formatted in HTML properly will benefit both the readers and search engines. When you add in headings, bullets, bold text, and hyperlinks you are making it easier for the reader to scan the article to identify what it is about. That structure is also having the same impact for search engine spiders.

Formatting your content with the proper HTML tags will make your article make more sense to your readers and search engines. It’s a win-win.

Let’s go over some of the most important, and simplest, HTML tags to include in your content.

HTML Elements You Should Know

Title Tag

Why the Title Tag Matters

You want to make sure each page on your website has a unique title tag. The title tag is one of the elements search engine spiders can use to identify what the page is about. The title tag is also the text that is used to display as the main headline in search results.

Make sure to include your targeted keyword in your title tag. Also, keep it under 60 characters long. That way it doesn’t get truncated in the search results.

What the Title Tag Code Looks Like

The title tag is found in the <head> section of the HTML code. You will find the title wrapped in a <title> tag. Here is an example:

<title>This is an Example Title</title>

How to Add a Title Tag in WordPress

To add a title tag in WordPress, all you need to do is insert a title in the page title field on your new post or page.

Now, this does depend on your website theme. It’s possible this has been modified on your theme. To confirm that the page title worked as intended, save your work and view the new page or post. If the title tag updated properly the title will be displayed in the tab at the top of your browser. Or, you can view your source code and search for <title> to confirm it is accurate.

Meta Tags

There are a few important meta tags you should be aware of, specifically the meta description. These tags help communicate details about the page content to search engines.

If you’re using a content management system such as WordPress, you won’t need to know the HTML to implement these tags. You still want to be aware of them so you remember to insert them and optimize them. Here is a full guide on what meta tags are and how to optimize them for SEO.

Header Tags

Why Header Tags Matter

The header tags create the hierarchy of your article. The most important header is called the h1 tag.

There should be only one h1 tag on each page. And, you want to include your targeted keyword in your h1 tag.

The rest of your header tags will be h2, h3, h4, h5, or h6. The higher the number, the less important and lower in the hierarchy the header is.

It’s common for an article to have one h1 tag and every other heading uses an h2 tag. I often will have h3 tags and sometimes even h4 tags if that’s how the article is naturally structured.

It’s uncommon to go all the way to h5 and h6. Normally using bullets or numbered lists at the point is more effective. But, there is no rule against it. Use the level of headings you need for the article to be beneficial to the reader.

What the Code Looks Like for Header Tags

Header tags wrap the header text with the level of heading you want to assign to the text. Here is an example for each header level:

<h1>This is an h1 tag</h1>
<h2>This is an h2 tag</h2>
<h3>This is an h3 tag</h3>
<h4>This is an h4 tag</h4>
<h5>This is an h5 tag</h5>
<h6>This is an h6 tag</h6>

How to Add Header Tags in WordPress

Adding header tags in WordPress is simple, but the process varies depending on which editor you use.

I like to use the code editor. So, to add a header I just surround the header text with the code from the example above.

If you use the visual editor (which is the default) you can click the plus icon to Add Block. Then select Heading. You can then insert your heading text and select the level such as h2, h3, or h4.

If you already have entered the text, just select it. You will see an editor appear with a dropdown and can select a new heading level from the dropdown menu.

Image and Alt Tags

Why do Image and Alt Tags Matter?

Images are important because they provide a better experience for the reader. They also are important from an SEO perspective because they can allow your page to rank in image searches.

Alt tags provide a text description of the image. These are important because they provide accessibility for vision impaired visitors to be able to understand what the image is about. They also communicate the relevance and meaning of the image to search engine spiders.

What Does the Code Look Like for Image and Alt Tags?

The alt tag is an attribute of the image tag. Unlike a header tag that has an opening and closing tag, the image tag does not need to be closed. Here is an example:

<img src="image.jpg" alt="example image" />

How to Add Image and Alt Tags in WordPress

To add an image to your WordPress page you can go to Media > Add New. Upload your image. In the fields provided, make sure to include a description under Alt Text. Then, just copy the image URL to paste in your article.

If you’re using the visual editor you can also add an image by clicking the plus icon to Add Block. Then select Image and follow the prompts.

Links

Why Links Matter

Including links in your articles will help both readers and search engine spiders. Both readers and search engines will use the links to navigate from one article to another.

What the Code Looks Like for Links

To add a link you use an anchor tag. The anchor tag surrounds the item you want to link. This can be text or an image. When it surrounds text, this text is referred to as anchor text.

Links can vary depending on if they open another webpage, send an email, or jump to a different section of the same webpage.

Here is a link that opens another webpage:

<a href="https://example.com">example website</a>

Here is a link that sends an email:

<a href="mailto:test@example.com">email example</a>

To add a link that jumps to another part of the same page you need the link as well as an ID on the section you want the link to jump to. You can add in ID to an existing paragraph tag or create a new div tag. Here is a an example:

<a href="#top">Top</a>
<p id="top">This is the top of the page.</p>

How to Add Links in WordPress

In WordPress you can select the text you want to add a link to and click the insert / edit link button.

Lists

Why Lists Matter

Lists make it easier for readers to digest complicated or process oriented content. Lists make it easier for people to scan the article and quickly understand what it’s about.

In HTML, you can create either an ordered list or unordered list. An unordered list would be a bulleted list. An ordered list would be a numbered list.

What the Code Looks Like for Unordered Lists

Here is an example to create an unordered (bulleted) list:

<ul>
<li>Here is a list item.</li>
<li>This is another item on the list.</li>
<li>This is the third and final item.</li>
</ul>

What the Code Looks Like for Ordered Lists

The HTML code for ordered lists is very similar to unordered lists. You still list out the list items (<li> tags) but instead of wrapping it in <ul> tags you use <ol>.

Here is an example:

<ol>
<li>This is the first item in the ordered list.</li>
<li>This is the next item.</li>
<li>And the last one.</li>
</ol>

How to Add Lists in WordPress

To add a list in WordPress you can click the plus icon to Add Block. The select List and follow the prompts.

Or, you can select your text and click the Bulleted List or Numbered List icon in the formatting menu at the top.

Bold and Italics

Why Bold and Italics Matter

Similar to lists, bold and italics help the reader scan the article and quickly understand the key topics. Emphasized text does also help with SEO. It’s good to bold or italicize text with your targeted keyword if it comes across as natural and beneficial to the reader.

What the Code Looks Like for Bold and Italic

You can use <b> to bold text and <i> to italicize text but these tags are antiquated. It’s best to use <strong> for bold and <em> for italics. Using <strong> and <em> does help with SEO.

Here is an example:

This word will be <strong>bold</strong> and this one will be <em>italic</em>.

How to Add Bold and Italic Text in WordPress

To bold and italicize in WordPress simply select the text and click the Bold or Italic buttons.

How to Review Your HTML Code

If you ever want to check the HTML on your page (or anyone else’s page) the best way to do it is to check the source code.

To view the HTML source code of any website right click on the page and select View page source. From there you can press ctrl (cmd) + F to search for any specific tags you want to review.

These are just a few of the important HTML tags that can help you optimize your SEO efforts.

Are there any important tags that I left off this list? Share your thoughts in the comments!

Do you want to listen to this article? Here’s the podcast episode:

Let’s dig in to every detail you need to know about internal links for SEO. We’ll start by going over what internal links are. Then we’ll review why they matter for SEO. And finally we’ll walk through the method you can use to build out internal links on your website.

Internal links are hyperlinks where the source domain and target domain are the same. They are links on a website that point to different parts of that same website.

Internal links keep you on the same website, whereas external links take you to a different website.

Types of Internal Links

There are a variety of elements that can be used to create an internal link. The most common internal links are in the main navigation, text links, image links, and footer links.

Main Navigation

The main navigation menu is usually found at the top of a website near the logo.

The navigation menu on a website will, for the most part, be filled with internal links. Sometimes you’ll find external links included in a navigation but the majority of navigation links will be internal.

Text Links

Text links are words or phrases within your content that have a hyperlink. Often the text link will be formatted with blue text and sometimes includes an underline.

Here is an example of what a text link looks like:

This is the most common type of link for an internal linking SEO strategy.

Image Links

Images can be used as links as well. This can include button images, infographics, or charts. This type of link can be helpful for readers if there is a page that would be an obvious benefit to someone who wants more information related to that image.

Image links don’t carry as high of an SEO benefit as text links. Use them when it makes sense for the reader, don’t focus on them for your SEO strategy.

Footer Links

At the bottom of a website you sometimes can see secondary menu systems. These are footer links.

Many years ago stuffing links in your footer was a good SEO tactic. That is no longer the case.

If it makes sense for the flow of your website to include links in your footer, than do it. But, don’t waste your time by including them for SEO benefit.

Why are internal links useful?

They Provide Navigation

Internal links create the pathways both readers and search engines will use to navigate your website.

Whether the link is in your navigation and specifically intended to guide the reader or a text link within an article, these links are how your community will access the various parts of your website.

Like your readers, search engine spiders will also use links to look around your website. Following links will inform the spider that the page exists.

If you create a page and there are no internal links from any existing pages it will be lost. It would be an orphaned page. It wouldn’t be impossible for people or crawlers to find it, but it wouldn’t be easy.

If you want your page to be viewed, make it easy for people and search engines to find it by adding internal links.

They Create Your Site Hierarchy

Internal links create the structure of your website. Depending on the path you need to take to access a page it helps identify where it is in your site hierarchy.

A website hierarchy should look like a pyramid. The homepage is at the top, then the category pages, then the specific article pages at the base.

Your internal links create this structure. This not only is important for the ease of navigation, but also helps distribute authority. Which brings us to the next reason.

They Distribute Authority

Internal links can help distribute the value of one page to another page. This is sometimes referred to as link juice or PageRank. This makes any backlinks you’ve earned have a greater impact.

The value given to one page trickles down to the links on that page.

It’s common for your homepage to have the majority of total backlinks for your website. Adding internal links on your homepage to deeper pages on your site will share some of the value your homepage is receiving.

They Increase Dwell Time

Adding internal links can increase the time users spend on your website. When they visit the page and see a text link about a relevant topic they may want to click to dig deeper into the subject.

Viewing another page on your site and staying on the site longer tells Google that your page was a relevant, useful result. These are metrics Google cares about and can have a positive impact on search rank.

Always Keep the Reader in Mind

When creating internal links remember to keep the reader in mind.

This is true even if you think SEO is your only goal. The only long term SEO strategy is to keep users in mind. Search engines are constantly updating their algorithms to make results better for users. You should have the same approach.

Add links that provide value to the reader. The link should be important enough for the reader to leave the current page and read that page instead.

Where to Add Internal Links

When creating internal links for SEO you will most commonly be adding text links between blog articles.

You don’t need to add several links to the About page and Contact page. Those are already in your main navigation.

You want to make it easy for readers to navigate the more complicated pieces of your website. This normally means the deeper pages such as blog articles.

How to Format a Link for SEO

Add links to the most relevant piece of text in a sentence. Don’t add text such as click here to use as your hyperlink. Create the link using the existing sentence.

Don’t over optimize your link text. Don’t try and make your link use the exact keyword you want to target. If that happens sometimes, it’s not a big deal. But, don’t make it happen every time.

You want the link to be natural and the sentence to still flow and make sense in the article.

Here is an example of what your text link would look like in a sentence:

<p>This is an example sentence to demonstrate <a href="/internal-linking">internal linking</a>.</p>

The text in between the <a> and </a> is referred to as anchor text and is the section that is used as the link on the page.

Relative versus Absolute Internal Links

In the example above the link I used was a relative link. I left off the domain name and just included the specific page path.

When you include an internal link you can either create an absolute or relative link. An absolute link includes the domain name whereas it is left off in a relative link.

When using a relative link the current domain is the same domain that is used for the link. That means if the reader is on http instead of https, they will continue to view the http version. Similarly, if they are on the www version instead of non-www they will remain there.

Don’t Use NoFollow for Internal Links

You can tell search engines not to follow a link by adding rel=nofollow to the hyperlink.

This is helpful for external links in blog comments and links in guest posts.

This is not something you want to add to internal links. Years ago you were able to add nofollow to internal links to send more value (link juice) to the links you want to rank. If you try this now you’ll just lose that extra link juice, it won’t be distributed to the remaining links.

Keep Internal Links Unique

When adding internal links to text add them to the first relevant text on the page. If the topic comes up again there is no need to add another link.

If you add the same link multiple times on the page Google will only pay attention to the anchor text used in the first instance.

You should keep in mind that this applies to your navigation as well. If you’re including a text link but the same link appears in your navigation, the anchor text used in your nav (if that appears first on your page) will be what Google pays attention to.

How many links should you include?

Include as many internal links as necessary and useful. That normally means add anywhere from 2-5 links to a 1200 word article.

Don’t over do it. If your page includes more than 100 links search engines may start to ignore the links. That might sound like a lot but when you include all of the navigation links they add up fast.

Of course, depending on the page and the site search engines could be okay with closer to 250 links on a page.

There isn’t a solid number to aim for when it comes to number of internal links. Instead, go with what is best for the user experience.

Foolproof Method to Regularly Add Internal Links

Each time you add a new blog post you want to review your internal links. You’ll want to add links to old articles in your new article. You’ll also want to add links to your new article in older articles.

There are plugins and tools that you can use for this. However, I find the process works better if you don’t automate it. Some things are just better to do manually.

Step 1: Create a Spreadsheet

Start a two column spreadsheet. One column will be the page URL, the other column will list keywords you are targeting on the page.

Only include URLs for pages you want to focus on for your SEO efforts. This will primarily be your blog pages.

I wouldn’t include your homepage, contact page, about page, or category pages. Those will naturally have several internal links and there won’t be as many relevant opportunities to build those up during your regular blog updates.

Keep this spreadsheet simple. Don’t stress about putting exact keywords. Instead, put the main idea. For example, if you have a page that you would want to rank for what is a meta tag, meta tag, and metadata; just add meta under the keyword column.

You’ll be reviewing this manually so you just need enough to identify if the article has any relevant content.

Step 2: Add Old Article Links on Your New Article

Each time you publish a new article you’ll want to review it to identify where you can add links to existing articles on your website. Including links to your older articles in your new posts signals to search engines that the old content is still relevant and fresh.

1. Navigate to the article you just published.
2. Refer to the existing spreadsheet you have with your existing posts and keywords.
3. Use ctrl (cmd) + F to find keywords from old articles in your new article.
4. When you find a keyword match review the text to see if it would make sense to add a link to the old article.
5. If it would be a benefit to the reader, add the link to the old article in the new article.
6. Repeat steps 3-5 for each keyword in your spreadsheet.

Step 3: Add Links to Your New Post on Old Articles

Depending on the size of your website there are two different ways you can go about this. Manually searching each page will yield more accurate results. But, the second option is more realistic when the content on your website has grown and you need a solution that scales.

Option 1: Manually search for keyword in old articles

1. Open every URL in your spreadsheet. You can open them in new tabs by holding ctrl (cmd) while clicking on each link.
2. Go to the first page in your spreadsheet.
3. Search the page (ctrl + F) for the keyword you are targeting for your new post.
4. If you find a keyword match, review the content it see if a link to your new article makes sense.
5. If it does, add a link.
6. Repeat steps 3-5 for each subsequent page.

Option 2: Perform a search in Google to find linking opportunities

Go to Google and perform a site search for your website with the keyword you want to target. The site attribute restricts results to the URL (your site) you input. Then, you can put your keyword in quotes. Here is an example search:

site:https://example.com "keyword"

This will bring up results from your website that rank for that keyword. This will give you the top pages that you should look at to see if it makes sense to add a link to your new article.

Step 4: Add the New Article to Your Spreadsheet

You want to keep your spreadsheet current. So, when you’re done make sure to add your newest post to the list.

Tools that Can Help with Internal Linking

Google Search Console

In the Google Search Console you can view your top linked internal pages. To access the report go to Links and then click More under Internal links.

This report shows you which pages have the most internal links. The top pages will likely be the more popular pages on your website. Those pages would be good spots to add links to your new article.

Conversely, the bottom pages with the fewest links are articles that could benefit from getting a link in a new article.

Yoast SEO WordPress Plugin

If you’re using WordPress and have the Yoast SEO plugin installed you can see this info right on your posts or pages menu.

On the right hand side you will see two columns with numbers. There is a column for how many internal links are in the article, and another column for how many internal links in other articles link to this one.

This gives you an at-a-glance idea of what pages could benefit from internal linking.

Do you have an internal linking process that has saved you time or a helpful tool? Share it with us in the comments!

Do you want to listen to this article? Here’s the podcast episode:

As you start digging into the technical side of SEO you may hear about robots.txt. What is robots.txt? The robots.txt file is a set of instructions to guide search engine spiders while they crawl your website.

This is part of the Robots Exclusion Protocol which is a standard used by websites to communicate with web crawlers.

The robots.txt file is located in the root directory of your website. It always has to be called robots.txt and there can be only one.

How to find out if your website has a robots.txt file

To find out if your website has a robots.txt file (and view it) simply add the filename to the end of your URL in a web browser.

So, if your website URL is https://example.com, open Google Chrome and type in https://example.com/robots.txt.

If the file exists on your website it will load and you will be able to read it directly in the web browser.

How to make a robots.txt file

To make a robots.txt file you can use any plain text editor.

Do not use a program such as Microsoft Word or Google Docs. Those programs will try to auto-correct your text. They also format characters into a version that will not translate properly when uploaded to your website.

Use Notepad on a Windows computer or TextEdit on a Mac. If you’d like the editor to look a little prettier, you can download a text editor such as Sublime Text.

Before we show the exact code you’ll want to use, let’s discuss the common pieces of the robots.txt file.

Components of a robots.txt file

User-agent

This specifies the web crawler you are targeting for the following rule. If you use an asterisk (*) the rule applies to all.

The Web Robots Database has a large list of the various user agents. For Google specific user agents, Google provides a full list of crawlers.

Disallow

This specifies the URL or directory you would like to block from web crawlers. You could use this to block logged in administrative pages or thank you pages that should only be displayed after a successful form submission.

It’s important to note that disallowing the crawlers to view parts of your website is meant to optimize the crawl rate for the spiders. It helps guide them to the pages that matter.

It’s no guarantee that those pages won’t appear in search engine results pages. If you’re looking to exclude pages from search results, look into adding a noindex meta tag.

Noindex

You can add a noindex tag to robots.txt with the hope that the page will not be indexed in search engines. It’s common to add a rule to remove a page from search engines by including both a disallow and a noindex assignment.

But, Google has stated they do not recognize the noindex directive in robots.txt and this should instead be included as a meta tag. If this page is linked to somewhere else it is still possible for it to appear in search results.

Allow

If you block a directory, you can use this to allow access to a specific file within a blocked directory.

Sitemap

You can specify the location(s) of your sitemap file(s).

Now, let’s go over some of the common scenarios you’d want to use in your robots.txt file.

The most common configurations for robots.txt files

Block Nothing

This is the most common (and safest) setup for your robots.txt file. Allow all web crawlers to view everything on your website.

This is what the robots.txt file would look like:

User-agent: *
Disallow:

This is saying for all web crawlers, disallow nothing.

Block Everything

If you have a robots.txt file to block everything this is most likely a mistake. If you truly want to block everything so the site doesn’t appear in search results you’ll want to password protect the website instead.

This is what the robots.txt file looks like when everything is blocked:

User-agent: *
Disallow: /

You can see it’s very similar to the code you’d provide to block nothing. But, instead of leaving disallow blank, it now includes a / which indicates every single page after the root domain is disallowed.

Block Directory

You can use robots.txt to block a specific directory. This is useful to block administration pages or pages that require the user to be logged in.

If you use WordPress you’ll want to block the wp-admin directory.

Here is what the robots.txt file looks like when you block a directory:

User-agent: *
Disallow: /wp-admin/

This tells all web crawlers to ignore the /wp-admin/ directory. Since all of those pages are password protected this helps optimize the pages that are crawled. It guides the crawler toward the pages you want indexed and ranking.

Block Page

You can also create your robots.txt file to disallow web crawlers on a particular page.

This can be used to deter spiders from accessing pages that are accessed only when an action has been performed. For example, a thank you page after a payment has been processed.

The configuration is very similar to blocking a directory. You input the page URL after the initial domain into the disallow attribute. It would look like this:

User-agent: *
Disallow: /thank-you.html

This would tell all search engine spiders to ignore the page located at /thank-you.html.

The robots.txt file can be ignored

The robots.txt file will help search engine spiders crawl your website, if they choose to read it.

This file will do nothing to deter spam bots from crawling your website. Spam bots will ignore the file completely so even if you have it set to block everything they can still crawl your site.

The data in robots.txt is publicly available

Anyone can view your robots.txt file. Don’t use it to try and block private data.

The same easy method you use to access your robots.txt file can be used by anyone. Since the file always has to be in the same location and use the same name anyone can view it.

Of course, this also means you can view anyone else’s as well if you are curious what rules they’ve applied.

Test your robots.txt file

Once you have a robots.txt file in place remember to test it. You can do this in the Google Search Console. For now, it has to be done in the old version. Soon it will be moved to the new version.

To access the robots.txt file tester:

1. Navigate to Google Search Console.
2. If using the new version, scroll down the the bottom of the left menu and click Go to old version.
3. Navigate to Crawl > robots.txt Tester.

This will show you your robots.txt file and if there are any errors or warnings.

Do you have any robots.txt tips you’d like to share? Let us know in the comments!

So, you want to optimize your web pages to perform well in search engines and you’ve heard meta tags are an important part. But, what are meta tags? And how do they help SEO?

Meta tags are snippets of HTML code that provide information about the contents of the page. They are located in the <head> section of a web page. They are not displayed to the user, but instead appear in code for search engine spiders.

Adding meta tags to your web pages makes it easier for search engines to understand what the page is about. This won’t impact your search rankings but it can affect the text displayed in the search engine results.

Making sure the text displayed in search results is relevant and persuasive can increase click-through rate and decrease bounce rate. Improving those metrics can in turn increase your search rankings.

Now, let’s go over some of the most important type of meta tags.

Types of Meta Tags

Meta Description

The meta description is arguably the most important meta tag for SEO purposes. It does not impact search ranking but it is commonly used as the description that appears on the search results page.

Search engines do not always use the meta description but they do for most cases.

The search engine can display any text on your web page as the description in the search results. If you don’t have a meta description on your web page most often the search engine displays the first text that appears on the page.

The first text on the page is very rarely an appropriate description of the page. Often times it includes the text in your menu and tagline. So, it’s in your best interest (and the user’s best interest) to include a meta description to help clarify what the page is about.

To add a meta description place the following tag in your <head> section:

<meta name="description" content="Put your meta description here.">

If you’re using WordPress with the Yoast SEO plugin you can add your meta description by following these steps:

1. Scroll down to the Yoast SEO panel
2. Click Edit snippet under the Snippet Preview
3. Edit text in the Meta description field

Ideally you want your meta description to be around (but not over) 156 characters. Keep it at a minimum of 100 characters but try not to go over 156 or it may get truncated on Google.

Though the meta description doesn’t directly impact search rankings it can help. If your meta description is persuasive it can help increase click-through rate. And, if the meta description is relevant and an accurate description of the page then it should help reduce bounce rate as well by providing realistic expectations to users.

Both high click-through rate and low bounce rate can help increase your search rankings.

So, keep your meta description concise, relevant, and make it persuasive by including a call-to-action. Also, keep these unique. Each page should have its own unique description.

Meta Keywords

This meta tag lists out the keywords you are targeting on the page. This tag is no longer used for search rankings.

These days, the search engine spiders are smart enough to figure out what the page is about. You no longer need to add the keywords.

Adding the keywords in a meta tag only benefits your competitors since it makes it easy to reverse engineer your search marketing strategy.

Meta Robots

The meta robots tag allows you to instruct search engine spiders what they should do with the data they find on the page.

A meta robots tag looks like this:

<meta name="robots" content="noindex,follow">

In the content attribute you can include:

• index or noindex
  • index tells the search engine spiders to include this page in the search engine results page; noindex tells them to not include it.
• follow or nofollow
  • follow tells the search engine spiders to follow the links found on this page; nofollow tells them to not follow the links.

If a meta robots tag is not included, by default the search engine spiders will index the page and follow the links.

If you add a noindex attribute to the meta robots tag on a page do not disallow it in your robots.txt file. If the page is being blocked by search engine spiders in robots.txt then it will never see your meta tag with the noindex assignment.

Title Tag

The title tag is not technically a meta tag but it is often talked about during the meta tag conversation. Like meta tags it is in the <head> section. And, like meta tags it helps search engines understand the contents of the page.

This is what a title tag looks like:

<title>Your Title Here</title>

The title tag is displayed as the heading in the search engine results.

It’s best practice to keep your title tag under 60 characters. Google has a fixed width of 600 pixels for the headline so the character limit varies depending on the width.

If you have enough characters remaining, it’s common practice to include a separator such as a bar (|) and add your brand name after your page title.

Include the keyword you are targeting in your title. Try and keep your title enticing to encourage users to click. And, like the meta description, you should have a unique title for each page.

Even More Meta Tags

There are many other meta tags. But, these four (description, keywords, robots, and title tag) are the most common and impactful for SEO.

The other meta tags, such as the viewport meta tag, are technical and should be handled automatically by your content management system or website theme.

Make sure each page on your website includes a unique title tag and meta description. That will be enough for the search engines to provide a clear description to users. Don’t stress about the other meta tags.

Have you added meta tags to your web pages? Do you have an important tip you’d like to share? Please let us know in the comments!

Do you want to listen to this article? Here’s the podcast episode: