Django - Sitemap Tutorial

How to create XML sitemaps with Django sitemap framework. Jun 7, 2020

Quickstart

Make these changes to the project settings file:

INSTALLED_APPS = [
    # Add these at the bottom:
    'django.contrib.sites',
    'django.contrib.sitemaps',
]

# Define SITE_ID:
SITE_ID = 1

Add a field called modified and a method called get_absolute_url() to a model:

class Post(models.Model):
    # Add "modified" field:
    modified = models.DateTimeField(auto_now=True)
    # Add "get_absolute_url()" method:
    def get_absolute_url(self):
        return reverse('blog:detail',
                       args=[self.slug])

Make these changes to your URL configuration file:

from django.contrib import admin
# Import these
from django.urls import include, path
from blog.models import Post
from django.contrib.sitemaps.views import sitemap
from django.contrib.sitemaps import GenericSitemap
from mysite.sitemaps import StaticViewSitemap

# Add `sitemaps` dictionary:
sitemaps = {
    'blog': GenericSitemap({
        'queryset': Post.objects.all(),
        'date_field': 'modified',
    }, priority=0.9),
    'static': StaticViewSitemap,
}

urlpatterns = [
    # ...
    # Pass the `sitemaps` dictionary to the `sitemap` view:
    path('sitemap.xml', sitemap,
         {'sitemaps': sitemaps},
         name='django.contrib.sitemaps.views.sitemap'),
]

Create a file called sitemaps.py somewhere (in the mysite directory in this case) and add these lines to it:

from django.contrib.sitemaps import Sitemap
from django.urls import reverse

class StaticViewSitemap(Sitemap):

    def items(self):
        return ['about']

    def location(self, item):
        return reverse(item)

Visit /sitemap.xml:

Full tutorial

Sitemap is an XML file that lists URLs for a site. Each URL can be associated with metadata. For example, we can tell the crawler how important a URL is related to other URLs in the site, how often it usually changes and when it was last updated. This helps search engines to index your site.

Setup

Run these commands to setup a project:

mkdir mysite && cd mysite
python3 -m venv venv
source venv/bin/activate 
pip install django
django-admin startproject mysite .
python manage.py startapp blog
python manage.py startapp pages

Configure settings.py

Add these lines to the project settings.py file:

INSTALLED_APPS = [
    # START
    'blog.apps.BlogConfig',
    'pages.apps.PagesConfig',
    # END
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',    
    # START
    'django.contrib.sites',
    'django.contrib.sitemaps',
    # END
]

# HERE
SITE_ID = 1

TEMPLATES = [
    {
        # ...
        # HERE
        'DIRS': [os.path.join(BASE_DIR, 'templates')],
        # ...
    },
]

django.contrib.sites enables the "sites" framework. The sitemap framework needs it to work properly. It gets the domain for the URLs using the current Site object.
django.contrib.sitemaps enables the sitemap framework.
We wire our apps into the project by adding blog.apps.BlogConfig and pages.apps.PagesConfig to the INSTALLED_APPS list.
SITE_ID = 1 associates a Site object from the database (with an ID of 1) to this settings file.
'DIRS': [os.path.join(BASE_DIR, 'templates')] makes Django look for template files in the root templates directory.

Add Post model

Add a class called Post to the blog app models.py file:

from django.db import models
from django.utils.text import slugify
from django.urls import reverse


class Post(models.Model):

    title = models.CharField(max_length=255,
                             unique=True)
    slug = models.SlugField(max_length=50,
                            unique=True,
                            default='')
    # HERE
    modified = models.DateTimeField(auto_now=True)

    def __str__(self):
        return self.title

    def save(self, *args, **kwargs):
        self.slug = slugify(self.title)
        super().save(*args, **kwargs)

    # HERE
    def get_absolute_url(self):
        return reverse('blog:detail',
                       args=[self.slug])

The modified field is used for the sitemap lastmod attribute. It tells when the item was last modified.
The get_absolute_url() method is used for the sitemap loc attribute. This is the page URL.

An example from the generated sitemap:

<loc>http://example.com/blog/my-post-title/</loc>
<lastmod>2020-05-30</lastmod>

Run migrations

python manage.py makemigrations && \
python manage.py migrate

This makes the necessary changes to the database.

Create posts

python manage.py shell
>>> from blog.models import Post
>>> import uuid
>>> for x in range(10):
...     post = Post.objects.create(title='%d' % uuid.uuid4())
...
>>>

python manage.py shell starts the Python interactive interpreter.
for x in range(10) loops 10 times.
uuid4 is used to create a random title for each object. The title has to be unique because we are using it to create a slug for the object. That slug goes to the generated sitemap through the get_absolute_url method.

Update URLs

Edit mysite/urls.py and add these lines to it:

from django.contrib import admin
from django.urls import include, path
from blog.models import Post
from django.contrib.sitemaps.views import sitemap
from django.contrib.sitemaps import GenericSitemap
from sitemaps.sitemaps import StaticViewSitemap
from pages.views import PostListView
from django.views.generic import TemplateView

sitemaps = {
    'blog': GenericSitemap({
        'queryset': Post.objects.all(),
        'date_field': 'modified',
    }, priority=0.9),
    'static': StaticViewSitemap,
}

urlpatterns = [
    path('', PostListView.as_view(), name='home'),
    path('blog/', include('blog.urls')),
    path('about/', TemplateView.as_view(template_name='about.html'), name='about'),
    path('sitemap.xml', sitemap,
         {'sitemaps': sitemaps},
         name='django.contrib.sitemaps.views.sitemap'),
    path('admin/', admin.site.urls),
]

The sitemaps dictionary holds information about the items we want to add to the sitemap. In this example it holds two labels. blog maps to a GenericSitemap instance that adds post URLs to the sitemap. static adds static pages (an about page in this case) using a custom sitemap class called StaticViewSitemap. The labels can map to Sitemap class instances (GenericSitemap(..)) or Sitemap classes (StaticViewSitemap).
The GenericSitemap convenience class allows you to create basic sitemaps by passing it a queryset entry. 'queryset': Post.objects.all() means that all posts will be added to the generated sitemap. 'date_field': 'modified' adds a lastmod attribute to each item using the Post model modified field. The date_field entry is optional.

Additionally, you can add the following keyword arguments:

priority informs the crawler how important an item is relative to other items in your site. Search engines may use this information when crawling your site.
changefreq specifies how often the content is likely to change. For example: changefreq="weekly". Search engines may use this information when crawling your site.
protocol defines the protocol for the sitemap URLs (http or https). By default, the request protocol is used, or http if the sitemap is generated without a request.

Finally we pass in the sitemaps dictionary to the sitemap view: {'sitemaps': sitemaps}. This generates the sitemap when a client accesses /sitemap.xml. Note: the sitemap is generated dynamically. A file called sitemap.xml is not added to your codebase.

Add StaticViewSitemap

Create a file called sitemaps.py in the project package directory and add these lines to it:

from django.contrib.sitemaps import Sitemap
from django.urls import reverse

class StaticViewSitemap(Sitemap):

    def items(self):
        return ['about']

    def location(self, item):
        return reverse(item)

Use the items method to return the pages you want to add to the sitemap. You should return a QuerySet or sequence. In this case we provide the about URL pattern name in a list.
The list items that we return from the items method are passed to the location method. For static pages we can't use the model get_absolute_url() method so we use the location method to return the URL for each item in the list. The reverse function gets the URL using the URL pattern name (about).

Add blog URL configuration file

Create a file called urls.py in the blog app directory:

from django.urls import path
from .views import PostDetailView

app_name = 'blog'

urlpatterns = [
    path('<slug:slug>/', PostDetailView.as_view(), name='detail')
]

Edit blog app views

Edit the blog app views.py file and add the following lines to it:

from django.shortcuts import render
from django.views.generic.detail import DetailView
from .models import Post

class PostDetailView(DetailView):

    model = Post
    template_name = 'blog/detail.html'
    context_object_name = 'post'

This allows us to get information about one post and display it on the detail page.

Add PostListView

Edit the pages app views.py file and add the following lines to it:

from django.shortcuts import render
from django.views.generic.list import ListView
from blog.models import Post

class PostListView(ListView):

    model = Post
    template_name = 'pages/home.html'
    context_object_name = 'posts'

This allows us to list all posts on the homepage.

Update home template

Create a directory called templates in the project root. Create a directory called pages inside it. Add a file called home.html to the templates/pages directory:

<h1>Home</h1>

<ul>
{% for post in posts %}
<li>
    <a href="{{ post.get_absolute_url }}">{{ post }}</a>
</li>
{% endfor %}
</ul>

This displays post links on the homepage.

Add about.html

Add a file called about.html in the templates/pages directory:

<h1>About</h1>

Add detail.html

Create a directory called blog in the templates directory. Add a file called detail.html in it:

<h1>{{ post.title }}</h1>

This displays the post title.

Result

Run the development server:

python manage.py runserver

Visit /sitemap.xml and you should see the generated sitemap:

<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>http://example.com/blog/84513265289461491745040709504185504735/</loc>
<lastmod>2020-05-30</lastmod>
<priority>0.9</priority>
</url>
<url>
<loc>http://example.com/about/</loc>
</url>
</urlset>