Pywikibot Tutorial: Automate Wikipedia Edits and Maintenance

Editing Wikipedia manually is tedious. If you need to fix thousands of broken links or update citations across hundreds of pages, doing it by hand will take years. Pywikibot is a Python library designed for automating tasks on MediaWiki-based sites like Wikipedia. It lets you write simple scripts that handle repetitive maintenance work, freeing you up for complex editing.

This guide walks you through setting up Pywikibot, understanding its core components, and writing your first useful script. We will focus on practical examples, such as fixing references and checking page categories. By the end, you will know how to automate common Wikipedia chores without breaking the site’s rules.

Key Takeaways

• Pywikibot uses the MediaWiki API to interact with Wikipedia programmatically.
• You need a Python environment (version 3.8+) and basic knowledge of command-line tools.
• Always use a dedicated bot account to avoid conflicts with human editors.
• The library handles rate limiting automatically, but you should still monitor your edit speed.
• Start with read-only operations before attempting mass edits.

Setting Up Your Environment

First, install the necessary packages. You can do this via pip. Open your terminal and run the following command:

pip install pywikibot

Once installed, create a configuration file. This file tells Pywikibot which site to connect to and where to store your login credentials. The default location is ~/.pywikibot/user-config.py. Here is a minimal example:

# user-config.py
mylang = 'en'
family = 'wikipedia'
userfile = 'user-passwords.txt'
revertthreshold = 10

The user-passwords.txt file stores your bot account credentials in a specific format. Make sure this file has restricted permissions so only you can read it. Your bot account should have the "bot" flag enabled if you plan to make high-volume edits. Without this flag, you might get throttled or blocked by administrators.

Understanding Core Components

Pywikibot revolves around three main classes: Page, Site, and Bot.

• Site: Represents the connection to a specific Wikipedia project (e.g., English Wikipedia). It manages authentication and API calls.
• Page: Represents a single article. It allows you to read content, check revisions, and save changes.
• Bot: A base class for interactive bots. It provides a loop structure for processing pages one by one.

Most scripts start by initializing a Site object. For example:

import pywikibot
site = pywikibot.Site('en', 'wikipedia')

This line creates an object representing the English Wikipedia. You can then use this site object to fetch pages or perform searches.

Your First Script: Checking Page Status

Before making changes, let’s write a script that simply checks if a page exists. This is a safe way to test your setup.

import pywikibot

site = pywikibot.Site('en', 'wikipedia')
page = pywikibot.Page(site, 'Madison, Wisconsin')

if page.exists():
    print(f"Page '{page.title()}' exists.")
else:
    print(f"Page '{page.title()}' does not exist.")

Run this script, and it will output whether the specified page is live. Notice how we pass the site object and the page title to the Page constructor. This pattern repeats throughout Pywikibot usage.

Automating Reference Fixes

One of the most common uses for Pywikibot is fixing broken references. Let’s say you have a list of pages where the citation template is missing a required parameter. You can write a script to detect and fix these issues.

Here is a simplified example that checks for a specific string in the page text:

import pywikibot

site = pywikibot.Site('en', 'wikipedia')
pages_to_check = ['Example Article 1', 'Example Article 2']

for title in pages_to_check:
    page = pywikibot.Page(site, title)
    if page.exists():
        text = page.get()
        if '{{cite web|' in text and '|url=' not in text:
            print(f"Missing URL in {title}")
            # Logic to fix the text would go here

In a real-world scenario, you would replace the print statement with code that modifies the text variable and then calls page.save(text). Always include a comment explaining why you are making the change. Wikipedia editors appreciate transparency.

Best Practices for Bot Operations

Running a bot on Wikipedia requires care. Here are some essential rules to follow:

1. Use a Dedicated Account: Never run your bot from your main editor account. Create a separate bot account. This makes it easy for admins to track your actions and revert mistakes if needed.
2. Respect Rate Limits: Pywikibot handles delays between requests, but don’t push too hard. Aim for no more than 5-10 edits per minute unless you have admin approval for faster speeds.
3. Log Your Actions: Keep a log of every page your bot touches. If something goes wrong, you need to know exactly what happened and when.
4. Test on a Sandbox: Before running a script on live articles, test it on your user talk page or a sandbox namespace. This catches bugs without risking damage to good-faith content.
5. Communicate with the Community: Post on the relevant WikiProject talk page to announce your bot’s activity. Transparency builds trust and reduces the chance of being blocked.

Common Pitfalls and How to Avoid Them

Even experienced developers make mistakes with Pywikibot. Here are a few common traps:

• Ignoring Redlinks: A redlink is a link to a non-existent page. If your script assumes all pages exist, it will crash. Always check page.exists() before trying to read or modify content.
• Encoding Issues: Wikipedia supports multiple languages. Ensure your script handles Unicode correctly. Use UTF-8 encoding when reading or writing files.
• Outdated Templates: Citation templates change over time. Hardcoding template parameters in your script can lead to errors. Instead, parse the template dynamically or use the Template class provided by Pywikibot.
• Forgetting to Update Dependencies: The MediaWiki API evolves. Occasionally, Pywikibot releases updates to match new API endpoints. Keep your installation current by running pip install --upgrade pywikibot regularly.

Advanced Techniques: Using Generators

For large-scale tasks, you need to process pages efficiently. Pywikibot provides generators that yield pages in batches. For example, you can iterate through all pages in a category:

cat = pywikibot.Category(site, 'Category:Wisconsin cities')
for page in cat.articles():
    print(page.title())

This approach is memory-efficient because it loads one page at a time. You can combine generators with filters to target only pages that meet specific criteria, such as those lacking a certain category or containing outdated data.

Frequently Asked Questions