How to Monitor AbuseFilter Logs via the Wikipedia API

Imagine you are running a bot that edits thousands of pages on Wikipedia every day. Suddenly, your edits start failing silently, or worse, getting reverted by an automated filter because of a minor syntax error in your code. You don't see an error message in your terminal; you just see the edit disappear. How do you know what went wrong? You have to look at the AbuseFilter logs. But scrolling through human-readable web interfaces is slow and painful when you need to debug hundreds of entries. The solution lies in querying the Wikipedia API directly for structured data.

This guide walks you through extracting AbuseFilter log entries using the MediaWiki Action API. Whether you are building a monitoring dashboard, debugging a new bot, or analyzing editor behavior, knowing how to pull this data programmatically saves hours of manual work. We will cover the specific parameters you need, how to handle pagination, and common pitfalls that trip up even experienced developers.

Why Query AbuseFilter Logs Programmatically?

The standard Wikipedia interface displays abuse filter hits in a neat table. It looks nice, but it is terrible for automation. If you want to count how many times a specific filter triggered in the last hour, you cannot just "count" the HTML page easily. You end up scraping HTML, which breaks whenever the site design changes. Using the API gives you clean JSON output that you can parse with any programming language.

More importantly, the API provides access to historical data that might not be immediately visible in the current UI views. For bot operators, this is critical. If your bot triggers Filter 12345 (a hypothetical example), you need to know if it was a false positive or a genuine violation. By pulling the log, you can inspect the `filter_result` field, which tells you exactly why the filter fired. Was it a regex match? A size limit? Knowing this lets you adjust your bot’s logic without guessing.

Consider a scenario where you update your bot’s script to add citations. You deploy it, and within ten minutes, fifty edits get flagged. Instead of panicking, you run a quick script to fetch the last 50 AbuseFilter log entries associated with your bot account. You see they all hit the same filter ID. You check the filter definition, realize your citation format missed a closing bracket, fix it, and redeploy. This loop takes minutes instead of days.

Understanding the Logevent Endpoint

To get these logs, you use the `logevents` module in the MediaWiki API. This module retrieves various types of log events, including user contributions, deletions, protections, and yes, abuse filter hits. The key parameter here is `type=abusefilter`. When you specify this type, the API returns only entries related to the Anti-Spam Extension, commonly known as AbuseFilter.

The basic structure of your request looks like this:

GET https://en.wikipedia.org/w/api.php?action=query&list=logevents&type=abusefilter&format=json

However, this simple request has limits. By default, it returns only 10 entries. You need to control how many results you get and which ones you retrieve. This is where parameters like `lelimit`, `letimeend`, and `leuser` come into play. Let's break down the most useful attributes for filtering your query.

• lelimit: Controls the number of results returned per request. Maximum value is usually 500 for bots and 50 for anonymous users. Always set this to 500 if you are authenticated to minimize requests.
• leuser: Filters logs by a specific username. Essential for monitoring your own bot’s activity.
• letimeend: Sets the timestamp for the end of your search range. Use ISO 8601 format (e.g., `2026-09-16T12:00:00Z`).
• letitle: Filters logs by a specific page title. Useful if you suspect a particular article is causing issues.

Notice that there is no direct `lefilterid` parameter in older versions of the API, but newer extensions often support filtering by filter ID via custom parameters or post-processing. For now, we focus on retrieving the raw logs and parsing them client-side.

Constructing Your First API Request

Let’s build a concrete example. Suppose you want to find all AbuseFilter hits for a bot named `ExampleBot` in the last 24 hours. You need to calculate the timestamp for 24 hours ago. In Python, using the `requests` library, this looks straightforward.

You must also include a `User-Agent` header. Wikipedia blocks requests without one. This is a common mistake that leads to HTTP 403 errors. Your User-Agent should identify your tool and contact information.

When you send this request, the API returns a JSON object containing a `query` array. Each element in this array represents a single log entry. You iterate through this array to process each hit. The data structure includes fields like `timestamp`, `user`, `page`, and `params`. The `params` field is particularly interesting because it contains serialized data about the filter execution, such as the filter ID and the reason string.

Parsing the JSON Response

The response from the API is nested JSON. Here is a simplified view of what a single log entry looks like:

{
  "logid": 12345678,
  "ns": 0,
  "title": "Main_Page",
  "timestamp": "2026-09-16T10:30:00Z",
  "user": "ExampleBot",
  "type": "abusefilter",
  "action": "hit",
  "comment": "Filter 123: Test edit",
  "params": {
    "filter_id": 123,
    "filter_result": "warn"
  }
}

Your code needs to extract the `filter_id` and `filter_result` from the `params` object. This allows you to categorize the hits. For instance, if `filter_result` is "disallow", your edit was blocked. If it is "warn", the edit went through but the user received a warning. If it is "tag", a tag was added to the revision. Understanding these outcomes helps you prioritize fixes. Disallowed edits are urgent; warnings might be acceptable noise.

A common pitfall is assuming the `params` field is always present or always in the same format. Sometimes, depending on the wiki configuration or the specific filter version, the structure might vary slightly. Always write defensive code. Check if `params` exists before trying to access `filter_id`. Use try-except blocks in Python or optional chaining in JavaScript to prevent crashes when unexpected data appears.

Handling Pagination and Large Datasets

If you need more than 500 entries, you cannot just increase `lelimit` indefinitely. The API enforces a hard cap. To get more data, you must use pagination. The API returns a `continue` object in the JSON response if there are more results available. This object contains a `lecontinue` token.

You take this token and add it to your next request as a parameter called `lecontinue`. Repeat this process until the `continue` object is missing from the response. This indicates you have reached the end of the dataset for your specified time range.

Here is the logical flow for handling pagination:

1. Send initial request with desired filters.
2. Parse the JSON response.
3. Check for the presence of a `continue` object.
4. If present, append the `lecontinue` value to the next request URL.
5. Repeat steps 1-4 until no `continue` object is returned.

Be mindful of rate limits. Wikipedia allows roughly 200 requests per minute for bots. If you are paginating through thousands of logs, ensure you respect this limit. Sleeping between requests or using exponential backoff when you receive HTTP 429 errors keeps your bot polite and prevents temporary bans.

Real-World Application: Building a Monitoring Dashboard

Why go through all this trouble? Imagine you maintain a suite of five bots. You want a simple web dashboard that shows red alerts when a bot’s error rate spikes. You can write a cron job that runs every ten minutes. This job queries the AbuseFilter logs for each bot, counts the number of "disallow" results, and compares it to the previous interval.

If the count jumps from 5 to 50, you trigger an alert. You could even integrate this with Slack or Discord. The message would say: "Warning: BotA had 50 disallowed edits in the last 10 minutes. Most common filter: 452." This level of insight transforms reactive debugging into proactive maintenance.

Another use case is research. Academics studying vandalism patterns can pull AbuseFilter logs to analyze which types of edits are most frequently flagged. They can correlate filter hits with time of day, user registration age, or topic area. Since the API provides structured timestamps and user IDs, joining this data with other datasets becomes trivial.

Troubleshooting Common Issues

Even with a solid plan, things break. Here are frequent issues and how to solve them.

Empty Results: If you get an empty array, check your timestamps. Timezone confusion is rampant. Ensure you are using UTC (`Z` suffix) in your ISO 8601 strings. Also, verify the username spelling. Wikipedia usernames are case-sensitive in some contexts, though generally, the API handles capitalization gracefully. Double-check that the bot actually made edits during the queried period.

HTTP 403 Forbidden: This usually means your User-Agent is missing or malformed. Wikipedia requires a descriptive User-Agent. Something like `MyBot/1.0 ([email protected])` works well. Avoid generic agents like `curl/7.68.0`.

Missing Params Field: As mentioned earlier, some older log entries or specific configurations might lack detailed params. Your parser should handle null values gracefully. Don’t assume every log entry has a filter ID.

Rate Limiting: If you start seeing HTTP 429 errors, slow down. Implement a retry mechanism with delays. Do not hammer the API. Remember, Wikipedia is a volunteer-run project; being a good citizen matters.

Final Thoughts on Data Hygiene

Monitoring AbuseFilter logs via the API is a powerful skill for anyone working with Wikimedia projects. It moves you from guesswork to data-driven decisions. You stop wondering why your edits failed and start knowing exactly which rule you broke. This clarity speeds up development cycles and reduces frustration.

Start small. Write a script that fetches the last 50 logs for your account. Print them out. Look at the structure. Then expand. Add filters. Add pagination. Build your dashboard. The complexity grows naturally as your needs evolve. Keep your code modular so you can swap out the logging source later if you decide to monitor other log types like deletions or protections.