🪴 Adithya Jayasankar

Search

SearchSearch

SEO

Jan 12, 20258 min read

f:: Next.js

What is SEO? - Search Engine Optimization | Learn Next.js

SEO stands for Search Engine Optimization. The goal of SEO is to create a strategy that will increase your rankings position in search engine results. The higher the ranking, the more organic traffic to your site, which ultimately leads to more business for you!

Why is SEO so important?

SEO is the key to increased conversion and confidence in your brand. Higher search ranking placement equates to more organic visitors. Search engine organic traffic – visitors who come to your site through clicking a result in a search engine – is key to many businesses for three reasons:

  1. Qualitative – Increased chance that visitors turn into customers.
  2. Trustable – Higher confidence in your brand or mission.
  3. Low-Cost – Aside from the time and effort spent, having good SEO practices that result in higher search engine ranking is free. There is no direct cost to appear in top organic search results positions.

Search engine optimization is different from Search Engine Marketing (SEM), where the content at the top of search results is 100% paid and distinguished from organic results with a Sponsored label.

Three Pillars of Optimization

The process of optimizing a website can be divided into three main pillars:

  1. Technical – Optimize your website for crawling and web performance.
  2. Creation – Create a content strategy to target specific keywords.
  3. Popularity – Boost your site’s presence online so search engines know you are a trusted source. This is done through the use of backlinks – third-party sites that link back to your site.

The SEO discipline is broad and has many facets, but as a Next.js developer, the first step is to understand how you can make your web app SEO-ready with some best practices.

Search Systems

Search Systems are what you typically refer to as Search Engines (Google, Bing, DuckDuckGo, etc.). They are massively complex systems that tackle some of the biggest challenges in technology history.

Search Systems have four main responsibilities:

  1. Crawling – the process of going through the Web and parsing the content in all websites. This is a massive task as there are over 350 million domains available.
  2. Indexing – finding places to store all of the data gathered during the crawling stage so it can be accessed.
  3. Rendering – executing any resources on the page such as JavaScript that might enhance the features and enrich content on the site. This process doesn’t happen for all pages that are crawled and sometimes it happens before the content is actually indexed. Rendering might happen after indexing if there are no available resources to perform the task at the time.
  4. Ranking – querying data to craft relevant results pages based on user input. This is where the different ranking criteria are applied in Search engines to give users the best answer to fulfill their intent.

In the next section, we will learn more specifically how Googlebot works. Googlebot is Google’s internet crawler, the part of the search system that gathers all the information needed to create the massive database of content to serve search results.

What are Web Crawlers?

In order for your website to appear in search results, Google (as well as other search engines such as Bing, Yandex, Baidu, Naver, Yahoo or DuckDuckGo) use web crawlers to navigate the website to discover websites and its web pages.

Different search engines have different market shares in each country.

In this guide we cover Google, which is the biggest search engine in most countries. That being said, you might want to check other search engines and their guidelines, especially if your target customers are in China, Russia, Japan or South Korea.

While there are some differences when it comes to Ranking and Rendering, most search engines work in a very similar way when it comes to Crawling and Indexing.

Web crawlers are a type of bot that emulate users and navigate through links found on the websites to index the pages. Web crawlers identify themselves using custom user-agents. Google has several web crawlers, but the ones that are used more often are Googlebot Desktop and Googlebot Smartphone.

How Does Googlebot Work?

The journey the Googlebot makes to index webpages

A general overview of the process can be the following:

  1. Find URLs: Google sources URLs from many places, including Google Search Console, links between websites, or XML sitemaps.
  2. Add to Crawl Queue: These URLs are added to the Crawl Queue for the Googlebot to process. URLs in the Crawl Queue usually last seconds there, but it can be up to a few days depending on the case, especially if the pages need to be rendered, indexed, or – if the URL is already indexed – refreshed. The pages will then enter the Render Queue.
  3. HTTP Request: The crawler makes an HTTP request to get the headers and acts according to the returned status code:
    • 200 - it crawls and parses the HTML.
    • 30X - it follows the redirects.
    • 40X - it will note the error and not load the HTML
    • 50X - it may come back later to check if the status code has changed.
  4. Render Queue: The different services and components of the search system process the HTML and parse the content. If the page has some JavaScript client-side based content, the URLs might be added to a Render Queue. Render Queue is more costly for Google as it needs to use more resources to render JavaScript and therefore URLs rendered are a smaller percentage over the total pages out there on the internet. Some other search engines might not have the same rendering capacity as Google, and this is where Next.js can help with your rendering strategy.
  5. Ready to be indexed: If all criteria are met, the pages may be eligible to be indexed and shown in search results.

Further Reading

Crawling and Indexing | Learn Next.js

What are HTTP Status Codes?

HTTP response status codes indicate whether a specific HTTP request has been successfully completed. There are many status codes, but only a handful are meaningful in an SEO context.

200

The HTTP 200 OK success status response code indicates that the request has succeeded.

In order for a page to be indexed on Google it must return status code 200. This is what you typically want to look for in your pages in order for them to receive organic traffic.

This is the default code that will be set when Next.js renders a page successfully.

301/308

The HTTP 301 Moved Permanently redirect status response code indicates that the resource requested has been definitively moved to the destination URL.

This is a permanent redirect. In general, this is the most widely used redirect type.

Redirects can be used for a variety of reasons, but the main one is to indicate that a URL has been moved from point A to point B.

Redirects are needed to ensure that, if a content is moved from one place to the other, you do not lose current and potential clients and that the bots can continue to index your site.

Note: Next.js permanent redirects use 308 by default instead of 301 as it is the newer version and considered the better option.

The main difference between the two status codes is that a 301 allows for changing the request method from POST to GET, whereas a 308 does not.

302

TheHTTP 302 Found redirect status response code indicates that the resource requested has been temporarily moved to the destination URL.

In most cases, 302 redirects should be 301 redirects. This may not be the case if you are redirecting users temporarily to a certain page (e.g. a promotion page) or if you are redirecting users based on location.

404

The HTTP 404 Not Found client error response code indicates that the server can’t find the requested resource.

As noted above, pages that are moved should be redirected with a HTTP 301 status code to the new location. When this doesn’t happen, URLs may return a 404 status code.

404 Status Codes are not necessarily bad by default, as it’s the desired outcome if a user happens to visit a URL that doesn’t exist, but they shouldn’t be a frequent error within your pages as it could lead to lackluster search rankings.

Next.js will automatically return a 404 status code for URLs that do not exist in your application.

410

The HTTP 410 Gone client error response code indicates that access to the target resource is no longer available at the origin server and that this condition is likely to be permanent.

This is not used very often, but you might want to look for this status code if you are deleting content on your website that won’t exist any more.

Examples where a HTTP 410 Gone might be wise to use include:

  • E-Commerce: Products permanently removed from inventory.
  • Forum: Threads that have been deleted by the user.
  • Blog: Blog post removed from site.

This status code informs bots that they should never return to crawl this content.

500

The HTTP 500 Internal Server Error response code indicates that the server encountered an unexpected condition that prevented it from fulfilling the request.

Next.js will automatically return a 500 status code for an unexpected application error. You can create a custom 500 error page that is statically generated at build time by creating pages/500.js.

503

The HTTP 503 Service Unavailable server error response code indicates that the server is not ready to handle the request.

It’s recommended to return this status code when your website is down and you predict that the website will be down by an extended period of time. This prevents losing rankings on a long-term basis.

Graph View

Backlinks