Technical SEO ensures search engines can effectively crawl, index, and understand your website. This checklist covers the essential elements based on Google’s official documentation and industry best practices.

Crawlability

Crawlability refers to search engines’ ability to access and navigate your site.

Robots.txt

Your robots.txt file controls which pages crawlers can access.1

  • Robots.txt exists at yourdomain.com/robots.txt
  • Not blocking important pages - Check for accidental Disallow rules
  • Not blocking CSS/JS - Google needs these to render pages properly
  • Sitemap reference included - Add Sitemap: https://yourdomain.com/sitemap.xml
  • Syntax is correct - Use Google’s robots.txt tester in Search Console

Example robots.txt:

User-agent: *
Allow: /
Disallow: /admin/
Disallow: /private/

Sitemap: https://yourdomain.com/sitemap.xml

XML Sitemap

Sitemaps help search engines discover your pages.2

  • Sitemap exists at standard location (/sitemap.xml or /sitemap_index.xml)
  • Submitted to Google Search Console
  • Only includes indexable pages (200 status, no noindex)
  • Under 50MB uncompressed and under 50,000 URLs per file
  • Uses lastmod dates accurately (when content actually changed)
  • Automatically updated when content changes

Internal Linking

Internal links help crawlers discover pages and understand site structure.3

  • All important pages linked from at least one other page
  • No orphan pages (pages with no internal links pointing to them)
  • Logical hierarchy - Important pages closer to homepage
  • Descriptive anchor text - Tells users and crawlers what the linked page is about
  • Working links - No broken internal links (404 errors)

Crawl Budget

For large sites (100,000+ pages), crawl budget optimization matters.4

  • Block low-value pages from crawling (faceted navigation, internal search results)
  • Fix crawl errors promptly
  • Avoid redirect chains (more than 2-3 hops)
  • Keep server response time low

Indexability

Indexability determines which pages appear in search results.

Index Status

  • Check indexed pages in Google Search Console (Index > Pages)
  • Review “Not indexed” reasons and fix issues
  • Site: search shows expected pages (site:yourdomain.com in Google)

Canonical Tags

Canonical tags tell search engines which URL is the “main” version.5

  • Self-referencing canonicals on all pages
  • Consistent canonicals - Same URL format (www vs non-www, trailing slash)
  • HTTPS canonicals (not HTTP)
  • No conflicting signals - Canonical, hreflang, and sitemap should agree

Example:

<link rel="canonical" href="https://yourdomain.com/page/" />

Noindex Directives

Control what gets indexed.6

  • Important pages not noindexed accidentally
  • Thin/duplicate pages noindexed appropriately
  • Noindex via meta tag or HTTP header (not robots.txt - that blocks crawling, not indexing)

Example:

<meta name="robots" content="noindex, follow" />

Duplicate Content

Duplicate content confuses search engines about which page to rank.7

  • No duplicate pages without canonical tags
  • URL parameters handled (Search Console > Settings > URL Parameters)
  • Pagination handled correctly (rel=“next/prev” deprecated, use canonical to page 1 or view-all)
  • WWW and non-WWW resolve to same version
  • HTTP redirects to HTTPS
  • Trailing slash consistency

Site Architecture

URL Structure

Clean URLs help users and search engines.8

  • URLs are readable - /products/blue-widget/ not /p?id=123&cat=4
  • Use hyphens between words (not underscores)
  • Lowercase URLs - Avoid mixed case
  • Keep URLs short but descriptive
  • Include relevant keywords naturally
  • Avoid URL parameters when possible

Site Hierarchy

  • Flat architecture - Most pages within 3 clicks of homepage
  • Logical categories - Related content grouped together
  • Breadcrumb navigation - Helps users and search engines understand hierarchy
  • Clear navigation menus - All main sections accessible

Breadcrumbs improve UX and can appear in search results.9

  • Breadcrumbs on all pages (except homepage)
  • BreadcrumbList schema markup implemented
  • Links are functional
  • Hierarchy matches URL structure

Example schema:

{
  "@context": "https://schema.org",
  "@type": "BreadcrumbList",
  "itemListElement": [
    {"@type": "ListItem", "position": 1, "name": "Home", "item": "https://example.com/"},
    {"@type": "ListItem", "position": 2, "name": "Articles", "item": "https://example.com/articles/"},
    {"@type": "ListItem", "position": 3, "name": "SEO Guide"}
  ]
}

Page Speed & Core Web Vitals

Page speed is a confirmed ranking factor.10 Core Web Vitals measure real-world user experience.

Core Web Vitals Thresholds

MetricGoodNeeds ImprovementPoor
LCP (Largest Contentful Paint)≤2.5s2.5s - 4s>4s
INP (Interaction to Next Paint)≤200ms200ms - 500ms>500ms
CLS (Cumulative Layout Shift)≤0.10.1 - 0.25>0.25

Performance Checklist

  • Test with PageSpeed Insights - https://pagespeed.web.dev/
  • Check Core Web Vitals in Search Console (Experience > Core Web Vitals)
  • Test on real devices (not just emulators)

LCP Optimization

  • Optimize images - Compress, use modern formats (WebP, AVIF)
  • Lazy load below-fold images - But NOT the LCP element
  • Preload critical resources - <link rel="preload">
  • Use a CDN for static assets
  • Minimize render-blocking resources
  • Server response time under 200ms (TTFB)

INP Optimization

  • Minimize JavaScript execution time
  • Break up long tasks (over 50ms)
  • Use web workers for heavy computation
  • Debounce input handlers

CLS Optimization

  • Set dimensions on images and videos - width and height attributes
  • Reserve space for ads and dynamic content
  • Avoid inserting content above existing content
  • Use CSS transforms for animations (not properties that trigger layout)

Additional Performance

  • Enable compression (Gzip or Brotli)
  • Minify CSS and JavaScript
  • Remove unused code
  • Enable browser caching (appropriate Cache-Control headers)
  • Reduce server response time

Mobile SEO

Google uses mobile-first indexing.11

Mobile-Friendly Checklist

<meta name="viewport" content="width=device-width, initial-scale=1">
  • Text readable without zooming - 16px minimum font size
  • Tap targets large enough - At least 48x48 pixels with spacing
  • No horizontal scrolling required
  • No intrusive interstitials - Avoid popups that block content12

Mobile Content Parity

  • Same content on mobile and desktop - Don’t hide content on mobile
  • Same structured data on both versions
  • Same meta tags (title, description, robots)
  • Images accessible on mobile version

Security

HTTPS

HTTPS is a ranking signal.13

  • SSL certificate installed and valid
  • All pages served over HTTPS
  • HTTP redirects to HTTPS (301 redirect)
  • No mixed content - All resources loaded over HTTPS
  • HSTS enabled (HTTP Strict Transport Security)

Security Headers

  • X-Content-Type-Options: nosniff
  • X-Frame-Options or Content-Security-Policy frame-ancestors
  • Content-Security-Policy (if applicable)

Structured Data

Structured data helps search engines understand your content and can enable rich results.14

Implementation Checklist

Common Schema Types

  • Organization - For your business
  • WebSite - With SearchAction for sitelinks search
  • Article - For blog posts and articles
  • BreadcrumbList - For breadcrumb navigation
  • FAQ - For FAQ pages
  • Product - For e-commerce products
  • LocalBusiness - For local businesses
  • HowTo - For instructional content

International SEO

For multi-language or multi-region sites.15

Hreflang Implementation

  • Hreflang tags on all language versions
  • Return links - If page A links to page B with hreflang, page B must link back
  • Self-referencing hreflang included
  • Valid language codes (ISO 639-1)
  • x-default for fallback page

Example:

<link rel="alternate" hreflang="en" href="https://example.com/page/" />
<link rel="alternate" hreflang="es" href="https://example.com/es/page/" />
<link rel="alternate" hreflang="x-default" href="https://example.com/page/" />

Monitoring & Maintenance

Regular Checks

  • Monitor Search Console weekly for errors
  • Check crawl stats for anomalies
  • Review Core Web Vitals monthly
  • Audit for broken links quarterly
  • Update sitemap when content changes
  • Monitor index coverage for drops

Tools for Technical SEO

Free:

Paid:

  • Screaming Frog SEO Spider - Crawls sites to find technical issues
  • Ahrefs Site Audit - Comprehensive technical SEO auditing
  • Semrush Site Audit - Similar to Ahrefs

Further Reading


References

Footnotes

  1. Google Search Central. “Introduction to robots.txt.” https://developers.google.com/search/docs/crawling-indexing/robots/intro

  2. Google Search Central. “Build and submit a sitemap.” https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap

  3. Google Search Central. “Links best practices.” https://developers.google.com/search/docs/crawling-indexing/links-crawlable

  4. Google Search Central. “Large site owner’s guide.” https://developers.google.com/search/docs/crawling-indexing/large-site-managing-crawl-budget

  5. Google Search Central. “Consolidate duplicate URLs.” https://developers.google.com/search/docs/crawling-indexing/consolidate-duplicate-urls

  6. Google Search Central. “Block search indexing with noindex.” https://developers.google.com/search/docs/crawling-indexing/block-indexing

  7. Google Search Central. “Avoid creating duplicate content.” https://developers.google.com/search/docs/crawling-indexing/consolidate-duplicate-urls

  8. Google Search Central. “URL structure guidelines.” https://developers.google.com/search/docs/crawling-indexing/url-structure

  9. Google Search Central. “Breadcrumb structured data.” https://developers.google.com/search/docs/appearance/structured-data/breadcrumb

  10. Google Search Central. “Page experience documentation.” https://developers.google.com/search/docs/appearance/page-experience

  11. Google Search Central. “Mobile-first indexing best practices.” https://developers.google.com/search/docs/crawling-indexing/mobile/mobile-sites-mobile-first-indexing

  12. Google Search Central. “Avoid intrusive interstitials.” https://developers.google.com/search/docs/appearance/avoid-intrusive-interstitials

  13. Google Security Blog. “HTTPS as a ranking signal.” https://security.googleblog.com/2014/08/https-as-ranking-signal_6.html

  14. Google Search Central. “Understand how structured data works.” https://developers.google.com/search/docs/appearance/structured-data/intro-structured-data

  15. Google Search Central. “Managing multi-regional and multilingual sites.” https://developers.google.com/search/docs/specialty/international