Planet Python

Last update: June 10, 2026 04:43 PM UTC

June 10, 2026

Python Morsels

Stacks and queues in Python

Use a Python list for stack operations (last-in, first-out) and a deque from the collections module for queue operations (first-in, first-out).

Read the full article: https://www.pythonmorsels.com/stacks-and-queues/

June 10, 2026 04:30 PM UTC

Real Python

Cursor vs Windsurf: Which AI Code Editor Is Best for Python?

AI-powered code editors have moved beyond novelty to become everyday tools for many Python developers. Instead of having to switch between your editor and a separate AI chat, you can use tools like Cursor and Windsurf that bring AI directly into your workflow. As a result, the Cursor vs Windsurf question is a common one for developers deciding which to adopt.

Both Cursor and Windsurf are VS Code forks that import your keybindings, themes, and Python extensions, and both run the same frontier models. They look similar at first but diverge in how they handle changes as you build.

Cursor focuses on control, surfacing AI-generated edits as reviewable diffs and relying on explicit rules to guide agent behavior. Windsurf focuses on flow, applying edits directly in the editor while using broader workspace context, including terminal output, recent edits, and conversation history, to shape its behavior.

In this tutorial, you’ll compare both editors across:

• AI code completion: How each editor’s completion system behaves and what context it draws on
• Agentic multi-file editing: How each editor handles tasks involving multiple files, directories, and the terminal
• Debugging and error correction: How each editor reviews generated code and integrates with your linter

By the end, you’ll have a clear picture of which editor fits your Python workflow. If you’re coming from VS Code, the Python Development in Visual Studio Code tutorial covers the baseline configuration that carries over to both forks.

The table below helps you choose the right editor at a glance:

Use case	Cursor	Windsurf
You want AI-generated changes shown as reviewable diffs before they’re written to your files, guided by explicit rules	✅	—
You want edits applied directly as the agent works, using a broader workspace context (terminal output, recent edits, conversation history, and memory)	—	✅

Cursor is the better fit if you want to review changes before they’re applied. Windsurf is the better fit if you prefer the agent to apply edits directly in your files as it works, drawing on the broader workspace context. To see how this plays out in completion, context management, and debugging, read on.

Get Your Code: Click here to download the free sample code for the resilient HTTP client you’ll build with Cursor and Windsurf in this tutorial.

Take the Quiz: Test your knowledge with our interactive “Cursor vs Windsurf: Which AI Code Editor Is Best for Python?” quiz. You’ll receive a score upon completion to help you track your learning progress:

---

Interactive Quiz

Cursor vs Windsurf: Which AI Code Editor Is Best for Python?

Test your understanding of how Cursor and Windsurf compare for Python across AI completion, agentic edits, and debugging workflows.

Metrics Comparison: Cursor vs Windsurf

As you work through the hands-on sections and eventually bring either editor into your own Python projects, the table below gives you a quick reference for some key differences you might expect from each tool:

Metric	Cursor	Windsurf
IDE support	Standalone VS Code fork plus a JetBrains plugin	Standalone VS Code fork plus plugins for JetBrains IDEs, Vim, Neovim, Xcode, Visual Studio, and more
AI code completion	Fast, line-by-line prediction; strong on single-file typed structures	Slower but more structurally aware across interconnected files
Startup performance	Faster. Uses lightweight text search that requires no upfront project indexing.	Slower initial response. Builds a semantic map of your project structure before it begins.
Debugging performance	Identifies and fixes the root cause in one pass	Reaches passing tests by working around the root cause over multiple iterations
Resource impact	Light. Low background CPU and RAM usage.	Heavy. Background indexing can spike local CPU during initial project load.
Billing model	Monthly credit pool with unlimited Tab and Auto mode on Pro	Daily and weekly usage quotas that refresh automatically on a schedule
Pro plan pricing	$20/month	$20/month
Ideal project size	Small to medium codebases where you already know the structure and can target files manually	Large, highly interconnected codebases that benefit from its RAG-based context engine and automatic semantic indexing

In the next sections, you’ll build a resilient HTTP client in Python from scratch and then send the same prompts to both editors to compare their responses.

Getting Started: Installation

Both editors ship as standalone desktop applications that closely match the VS Code experience. On first launch, they offer to import your local VS Code configuration, copying your keybindings, extensions, themes, and settings so your environment carries over with minimal setup.

To follow the hands-on project later in this tutorial, you’ll also want Python 3.12 or later installed on your system. Beyond that, if you need a full VS Code baseline before starting, the Python Development in Visual Studio Code (Setup Guide) course covers the editor setup from scratch.

Both Cursor and Windsurf offer free plans with enough model access to work through this comparison, though keep in mind that free-tier usage is limited and may run out under heavy use.

Installing Cursor

Head to the Cursor download page and download the correct version for your system. During setup, Cursor offers to import your VS Code configuration, including extensions, keybindings, and themes, so your environment carries over with minimal setup.

Once the editor opens, you’re ready to go. You don’t need to configure anything else yet.

If Cursor is new to you, Real Python’s video course on Tips for Using the AI Coding Editor Cursor covers setup, Agent mode, Plan mode, and model selection in a practical context, making the comparisons later in this tutorial easier to follow.

Installing Windsurf

Download Windsurf from the Windsurf download page and run the installer. The VS Code profile import works identically to Cursor’s.

Read the full article at https://realpython.com/cursor-vs-windsurf-python/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

June 10, 2026 02:00 PM UTC

Python GUIs

How to Set Row Background Colors in a QTableView — Use Qt's BackgroundRole to color entire rows based on your data

I have a QTableView table showing some data about connected devices. How can I highlight rows to give a visual indicator of the current status of the device?

When you're working with a QTableView and a custom model, it's common to want to highlight entire rows based on some condition in your data. For example, you might want to color a row blue when a device has a connected status, or red when something has gone wrong.

Understanding How data() Works

In Qt's Model/View architecture, the view calls your model's data() method for every cell in the table — and for each cell, it asks about multiple roles. One of those roles is Qt.BackgroundRole, which tells the view what background color to use for that cell.

The view asks for Qt.BackgroundRole on every single cell, not just one column. So if your data() method returns a color for Qt.BackgroundRole based on the row data (ignoring the column), the color will be applied to every cell in that row.

Let's build a working example.

A Complete Working Example

Here's a full example you can run directly. It creates a QTableView with colored rows based on the PRESENT_STATUS field in each row of data:

import sys
from typing import Union

from PyQt6.QtCore import QAbstractTableModel, QModelIndex, Qt
from PyQt6.QtGui import QColor
from PyQt6.QtWidgets import QApplication, QMainWindow, QTableView


class TableModel(QAbstractTableModel):

    def __init__(self, data: Union[list, None] = None):
        super().__init__()
        self._data = data or []
        self._hdr = self._gen_hdr_data() if data else []
        self._base_color = {
            "NewConnection": QColor("blue"),
            "Registered": QColor("green"),
        }

    def _gen_hdr_data(self):
        """Build a sorted list of all unique keys across all row dicts."""
        all_keys = set()
        for d in self._data:
            all_keys.update(d.keys())
        return sorted(all_keys)

    def rowCount(self, parent=QModelIndex()):
        return len(self._data)

    def columnCount(self, parent=QModelIndex()):
        return len(self._hdr)

    def headerData(self, section, orientation, role):
        if role == Qt.DisplayRole and orientation == Qt.Horizontal:
            return self._hdr[section]

    def data(self, index: QModelIndex, role: int):
        if not index.isValid():
            return None

        row_dict = self._data[index.row()]
        state = row_dict.get("PRESENT_STATUS", "")

        if role == Qt.DisplayRole:
            col_key = self._hdr[index.column()]
            value = row_dict.get(col_key, "")
            return str(value) if value else ""

        if role == Qt.BackgroundRole:
            color = self._base_color.get(state)
            if color:
                return color

        return None


class MainWindow(QMainWindow):
    def __init__(self):
        super().__init__()
        self.setWindowTitle("Row Background Colors in QTableView")

        data = [
            {"IP": "192.168.1.10", "PRESENT_STATUS": "NewConnection"},
            {"IP": "192.168.1.108", "FORMER_STATUS": "NewConnection",
             "PRESENT_STATUS": "Registered"},
            {"IP": "192.168.1.50", "PRESENT_STATUS": "Unknown"},
        ]

        self.table = QTableView()
        model = TableModel(data)
        self.table.setModel(model)
        self.setCentralWidget(self.table)


app = QApplication(sys.argv)
window = MainWindow()
window.show()
app.exec()

The method that Qt calls on the model is called data, so in the example above, the list is stored as self._data (with a leading underscore) to avoid this.

Run this and you'll see three rows. The first row ("NewConnection") has a blue background, the second row ("Registered") has a green background, and the third row ("Unknown") has no special coloring because it isn't in the _base_color dictionary.

How Colors are Set on Rows

To understand how the color is being set to the entire row, take a look at the Qt.BackgroundRole section of data():

if role == Qt.BackgroundRole:
    color = self._base_color.get(state)
    if color:
        return color

Notice that index.column() isn't used here at all. The color decision is based entirely on the row's PRESENT_STATUS value. Since the view calls data() for every cell in the row — column 0, column 1, column 2, etc. — and each call gets the same color back, the entire row ends up painted.

If you only wanted to color a specific column (say, just the status column), you would add a column check:

if role == Qt.BackgroundRole:
    # Only color the PRESENT_STATUS column
    if self._hdr[index.column()] == "PRESENT_STATUS":
        color = self._base_color.get(state)
        if color:
            return color

Making the Text Readable

One thing you'll notice with a dark background color like blue is that the default black text becomes hard to read. You can fix this by also handling Qt.ForegroundRole and returning a light text color when the background is dark:

def data(self, index: QModelIndex, role: int):
    if not index.isValid():
        return None

    row_dict = self._data[index.row()]
    state = row_dict.get("PRESENT_STATUS", "")

    if role == Qt.DisplayRole:
        col_key = self._hdr[index.column()]
        value = row_dict.get(col_key, "")
        return str(value) if value else ""

    if role == Qt.BackgroundRole:
        color = self._base_color.get(state)
        if color:
            return color

    if role == Qt.ForegroundRole:
        # If this row has a background color, use white text.
        if state in self._base_color:
            return QColor("white")

    return None

Now blue and green rows will have white text, making everything easy to read.

Updating Colors Dynamically

If your data changes at runtime — for example, a device's status changes from "NewConnection" to "Registered" — you need to tell the view that something has changed so it repaints. You do this by emitting the dataChanged signal:

def update_status(self, row, new_status):
    self._data[row]["PRESENT_STATUS"] = new_status
    # Emit dataChanged for the entire row.
    top_left = self.index(row, 0)
    bottom_right = self.index(row, self.columnCount() - 1)
    self.dataChanged.emit(top_left, bottom_right)

This tells the view to re-query data() for every cell in that row, which picks up both the new display text and the new background color. For a deeper look at how signals work to keep your model and view in sync, see Signals, Slots & Events.

Summary

Once you understand how the model's data() method works, coloring entire rows in a QTableView is relatively straightforward. The view asks for each role on every cell, so returning a color from Qt.BackgroundRole based on row-level data — without filtering by column — naturally paints the whole row. Pair that with Qt.ForegroundRole for readable text, and you've got a clean, data-driven way to highlight rows in your table.

To learn more about using QTableView with custom models and data from numpy or pandas, see the QTableView with numpy and pandas tutorial. If you want to add sorting and filtering to your table, take a look at Sorting and Filtering Tables.

For an in-depth guide to building Python GUIs with PyQt6 see my book, Create GUI Applications with Python & Qt6.

June 10, 2026 06:00 AM UTC

Python Insider

Python 3.14.6 and 3.13.14 are now available!

A pair of bug fix releases await your upgrade.

June 10, 2026 12:00 AM UTC

Seth Michael Larson

Are insecure code completions a vulnerability?

Three months ago I saw that PyCharm shipped with a “Full Line Completion” plugin that “uses a local deep learning model to suggest entire lines of code”. These suggestions manifest as whole-line suggestions after you start typing and can be accepted with Tab. Essentially auto-complete for entire lines.

I decide to test this functionality. I started by writing import urllib3, created a new line, and then typed u and received a suggested completion for the line marked below with a dashed border. I was not impressed by the result:

import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

Accepting this line would mean that any insecure requests made with urllib3 would not result in a user-visible warning. I didn't accept this suggestion and then began to instantiate a urllib3.PoolManager and what I feared would come next was confirmed:

import urllib3

urllib3.PoolManager(
    cert_reqs='CERT_NONE',

The suggestion offered to disable certificate verification (CERT_NONE) which would make every request made by the PoolManager susceptible to monster-in-the-middle (MITM) attacks. Accepting this code as-is would mean the program I am writing has a severe vulnerability. If I had accepted the prior suggestion too, then urllib3 would have no chance to warn the user about this mistake prior to productionizing this code.

Clearly something insecure is going on here, but for a CVE to be assigned we have to decide which software component is vulnerable. Does this behavior warrant a CVE at all? I am not sure which is unfortunate, without a security-angle to a bug report companies are less likely to prioritize reports.

I reported this behavior to JetBrains for “Full Line Code Completion” v253.29346.142 and clearly their support staff weren't certain whether this defect was a security vulnerability or not either. When I asked to publish a blog post about this behavior after they confirmed this report wasn’t a “direct security vulnerability” (which I agree with) but then was asked not to publicize my report and referred to PyCharm’s Coordinated Disclosure Policy so... which is it? Security vulnerability or not?

I ended up waiting the 90 days anyway and I didn't hear back with any substantive update from the development team. I double-checked again today using “Full Line Code Completion” v261.24374.152 and the behavior is identical, suggesting the same insecure code for both contexts.

This isn’t meant to be a specific dig at PyCharm or JetBrains, I have no-doubt that examples like this exist in every code generation model available. I don’t think using CVEs for this purpose is appropriate or helpful for users, either. But not prioritizing and addressing this behavior at the source means more work to mitigate the potential for insecure code to be accepted by users who are trusting what is offered to them by their IDE.

What do you think? I am interested in knowing your thoughts about this specific class of issue with code generation models.

Thanks for reading ♥ I would love to hear your thoughts! Contact me via Mastodon, Bluesky, or email. Browse the blog archive. Check out my blogroll.

June 10, 2026 12:00 AM UTC

Armin Ronacher

Gaslighting Openness

I have been a staunch supporter of Open Source for a long time, including experiments in funding it. I’m a true believer in the idea that Open Source always wins in the long run, but not automatically and not quickly. Right now it is being stressed by AI slop, shifting contributor dynamics, the falling cost of producing code, and large companies learning to close doors behind them.

A lot of that battle today is manipulation of the narrative. Opinion makers on social media and in business circles increasingly frame access as irresponsibility. That is why the EU’s DMA matters, even if many people (including myself) reflexively hate EU regulation. Apple’s fight over delayed AI features in Europe is not about Brussels being annoying: it is about whether users can access their own devices and data. The phone is yours, the data is yours, yet Apple decides who may reach it and takes the agency away from you and then tries to make that sound like it is in your interest (supposedly it’s for your safety and security).

The closer you get to the core of AI, the more this shows up. Anthropic has every financial incentive to restrict what people can do with Mythos and Fable, and they wrap those restrictions in safety and (national) security language. Some restrictions may be defensible, but not all of them are. They trained their models on public works, then block Open Source attempts to learn from and distill these systems.

Disliking the EU, China, or any other large government should not make us forget that true democratized access to technology including AI is in all our interest. Some temporary product pain, including delayed Apple AI features, will be worth paying if it keeps gates open. We should not let companies own the narrative that preventing access is in our interest, particularly not as Europeans where the odds are already stacked against us by our underdeveloped capital markets, brain drain and internal fighting.

June 10, 2026 12:00 AM UTC

June 09, 2026

PyCoder’s Weekly

Issue #738: sleep(), Polars Workflows, Iterators, and More (2026-06-09)

#738 – JUNE 9, 2026
View in Browser »

B2B AI Agent Auth Support

Your users are asking if they can connect their AI agent to your product, but you want to make sure they can do it safely and securely. PropelAuth makes that possible →
PROPELAUTH sponsor

Articles & Tutorials

Projects & Code

Events

Happy Pythoning!
This was PyCoder’s Weekly Issue #738.
View in Browser »

[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]

June 09, 2026 07:30 PM UTC

Python Docs Editorial Board

Meeting Minutes: Jun 9, 2026

Meeting Minutes from Python Docs Editorial Board: Jun 9, 2026

June 09, 2026 07:00 PM UTC

Real Python

Accessing Multiple AI Models With the OpenRouter API

One of the quickest ways to call multiple AI models from a single Python script is to use OpenRouter’s API, which acts as a unified routing layer between your code and multiple AI providers. By the end of this course, you’ll be able to access models from several providers through one unified API.

This convenience matters because the AI ecosystem is highly fragmented: each provider exposes its own API, authentication scheme, rate limits, and model lineup. Working with multiple providers often requires additional setup and integration effort, especially when you want to experiment with different models, compare outputs, or evaluate trade-offs for a specific task.

OpenRouter gives you access to thousands of models from leading providers like OpenAI, Anthropic, Mistral, Google, and Meta. You can switch between them without changing your application code.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

June 09, 2026 02:00 PM UTC

Quiz: Embeddings and Vector Databases With ChromaDB

In this quiz, you’ll test your understanding of Embeddings and Vector Databases With ChromaDB.

By working through this quiz, you’ll revisit key concepts like vectors, cosine similarity, word and text embeddings, ChromaDB collections, metadata filtering, and retrieval-augmented generation (RAG).

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

June 09, 2026 12:00 PM UTC

Quiz: Accessing Multiple AI Models With the OpenRouter API

In this quiz, you’ll test your understanding of Accessing Multiple AI Models With the OpenRouter API.

By working through this quiz, you’ll revisit how OpenRouter provides a unified routing layer, how to call AI models from a single Python script, how to switch between intelligent routing and a specific model, how to prioritize providers, and how to add model fallbacks for reliability.

It also reinforces how to weigh trade-offs like cost, latency, and quality when you choose a model for your use case.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

June 09, 2026 12:00 PM UTC

Python Bytes

#483 Thanks Brian

<strong>Topics covered in this episode:</strong><br> <ul> <li><strong>Vulnerability and malware checks in uv</strong></li> <li><strong><a href="https://alexwlchan.net/2026/python-http-with-the-stdlib/?featured_on=pythonbytes">HTTP GET requests with the Python standard library</a></strong></li> <li><strong>Millions of AI agents imperiled by critical vulnerability in open source package</strong></li> <li><strong><a href="https://github.com/Mergifyio/alembic-git-revisions?featured_on=pythonbytes">alembic-git-revisions</a></strong></li> <li><strong>Extras</strong></li> <li><strong>Joke</strong></li> </ul><a href='https://www.youtube.com/watch?v=WIykgbceuVg' style='font-weight: bold;'data-umami-event="Livestream-Past" data-umami-event-episode="483">Watch on YouTube</a><br> <p><strong>About the show</strong></p> <p><strong>Goodbye and Thanks Brian</strong></p> <p>Thanks Calvin for being part of this and future episodes! Also new time for the live show. Thanks Brian for all the hard work over the years.</p> <p><strong>Calvin #1: Vulnerability and malware checks in uv</strong></p> <ul> <li>release just yesterday by Astral https://astral.sh/blog/uv-audit</li> <li><strong><code>uv audit</code></strong> scans dependencies for known vulnerabilities and abandoned packages via the OSV database — runs 4–10x faster than <code>pip-audit</code></li> <li><strong>Malware check</strong> runs on every install/sync, catching actively malicious packages (credential stealers, etc.) before they execute — including ones PyPI quarantined but lockfiles can still reference</li> <li>Enable malware scanning with <code>UV_MALWARE_CHECK=1</code> — it's opt-in and in preview</li> <li>Future roadmap includes a resolver that steers toward vulnerability-free versions and install-time warnings scoped to newly added deps only</li> </ul> <p><strong>Michael #2: <a href="https://alexwlchan.net/2026/python-http-with-the-stdlib/?featured_on=pythonbytes">HTTP GET requests with the Python standard library</a></strong></p> <ul> <li>If you’re doing HTTP in Python, you’re probably using one of three popular libraries: <a href="https://requests.readthedocs.io/en/latest/?featured_on=pythonbytes">requests</a>, <a href="https://github.com/encode/httpx?featured_on=pythonbytes">httpx</a>, or <a href="https://github.com/urllib3/urllib3?featured_on=pythonbytes">urllib3</a>.</li> <li>There have been <a href="https://pythonbytes.fm/episodes/show/476/common-themes">issues with httpx lately</a>.</li> <li><a href="https://github.com/jawah/niquests?featured_on=pythonbytes">Niquest</a> is another option: Drop-in replacement for Requests. Automatic HTTP/1.1, HTTP/2, and HTTP/3. WebSocket, and SSE included.</li> <li>But maybe less is more, especially in the age of agentic AI</li> <li>A good candidate needs two things to be true at once, not one: the <em>used surface</em> is small, and the <em>behavior behind that surface</em> is shallow.</li> </ul> <p><strong>Calvin #3: Millions of AI agents imperiled by critical vulnerability in open source package</strong></p> <ul> <li><strong>"BadHost" (CVE-2026-48710)</strong> is a critical vulnerability in Starlette — the ASGI framework underlying FastAPI — with 325 million weekly downloads; also affects vLLM, LiteLLM, and most MCP server tooling</li> <li><strong>The exploit is trivial</strong>: injecting a single character into an HTTP Host header bypasses path-based authentication, and can lead to credential theft, SSRF, and in some cases remote code execution</li> <li><strong>MCP servers are a prime target</strong> since they store credentials for external services (email, databases, cloud accounts) — exposed data in the wild includes biopharma clinical trial DBs, full mailboxes, HR/PII pipelines, and AWS topology</li> <li><strong>Fix is available</strong> — patch to Starlette 1.0.1 immediately; use the free scanner at mcp-scan.nemesis.services to check if your servers are still running a vulnerable version</li> <li><strong>Open source sustainability footnote</strong>: the maintainer triages near-daily security reports solo, in his free time — most are AI-generated noise, and real ones like this still compete for the same evenings and weekends</li> </ul> <p><strong>Michael #4: <a href="https://github.com/Mergifyio/alembic-git-revisions?featured_on=pythonbytes">alembic-git-revisions</a></strong></p> <ul> <li>By Julien Danjou from <a href="https://mergify.com/?featured_on=pythonbytes">Mergify</a></li> <li>Automatic <a href="https://alembic.sqlalchemy.org/?featured_on=pythonbytes">Alembic</a> migration chaining based on git commit history. No more <code>Multiple head revisions are present for given argument 'head'</code>.</li> <li>See <a href="https://julien.danjou.info/blog/fixing-alembics-multiple-heads-problem-with-git/?featured_on=pythonbytes">the introductory article</a></li> <li>Caused by two migrations landed with the same <code>down_revision</code>, and Alembic doesn’t know which one comes first. The fix is always the same: someone manually edits the migration file to re-chain the revisions.</li> <li>The insight: git already knows the order</li> </ul> <p><strong>Extras</strong></p> <p>Calvin:</p> <ul> <li>GNU <code>make</code> can do pattern matching in the target. Not new at all, mentioned in the 1994-era docs. <code>just</code> and <code>task</code> don’t have this super power on the target name yet. <pre><code>train-%: uv run ./train.py $* --save-hyper-params --overwrite $(TRAIN_ARGS) </code></pre></li> </ul> <p>Michael:</p> <ul> <li>Updated my HTTP client using packages from httpx to <a href="https://github.com/pydantic/httpx2?featured_on=pythonbytes">httpx2</a>: <a href="https://pypi.org/project/listmonk/?featured_on=pythonbytes">listmonk</a>, <a href="https://pypi.org/project/umami-analytics/?featured_on=pythonbytes">umami</a>, and <a href="https://pypi.org/project/memberful/?featured_on=pythonbytes">memberful</a>. For motivation, see <a href="https://www.reddit.com/r/Python/comments/1rl5kuq/anyone_know_whats_up_with_httpx/?featured_on=pythonbytes">this reddit thread</a>.</li> </ul> <p><strong>Joke: <a href="https://x.com/PR0GRAMMERHUM0R/status/2061508112083714478?featured_on=pythonbytes">Accurate</a></strong></p>

June 09, 2026 08:00 AM UTC

Hynek Schlawack

How to Ditch Codecov for Python Projects

Codecov’s unreliability breaking CI on my open source projects has been a constant source of frustration for me for years. I have found a way to enforce coverage over a whole GitHub Actions build matrix that doesn’t rely on third-party services.

June 09, 2026 12:00 AM UTC

June 08, 2026

Real Python

Python 3.15 Hits Feature Freeze and Other News for June 2026

_MISSING = object()

def update(value=_MISSING):
    if value is _MISSING:
        ...  # No value was provided

MISSING = sentinel("MISSING")

def update(value: int | MISSING = MISSING) -> None:
    if value is MISSING:
        ...  # No value was provided

Read the full article at https://realpython.com/python-news-june-2026/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

June 08, 2026 02:00 PM UTC

death and gravity

Ordered key sharding in DynamoDB

So, you want to keep a sorted index in DynamoDB, but for whatever reason – usually throughput-related – it won't fit on a single partition. What do you do?

Today, we look at the available solutions, do the math, and find out which is best.

Tip

This worked example is part of my DynamoDB crash course series.

• Requirements
• A sparse index is almost enough
• But scan results are not ordered
• But a single partition key causes throttling
• But random suffixes are random
• But hash suffixes are not ordered
• But there are a lot of first characters
• But some first bytes need multiple shards
• But tries and prefix ranges are complicated
• But the prefix distribution can change

Requirements #

Say you're using single table design with a table of artists, albums, and songs.¹

You keep an artist's items in a single collection (aka same partition key), and use sort keys artist, album#{Album}, and song#{Album}#{Song}, depending on their type:

# table Music (partition key: Artist, sort key: sk)
Solar Fields: !btree
  'album#Leaving Home': { Album: Leaving Home, ... }
  'artist': { ... }
  'song#Leaving Home#Air Song': { ... }
  'song#Leaving Home#Monogram': { ... }

To list albums without doing a full table scan, you need a global secondary index.

Let's come up with some reasonable requirements; the GSI should support:

1. items up 500 bytes (we project additional attributes besides the keys)
2. 10,000 queries/second, max 100 items/query, sorted alphabetically
   • list all albums
   • list albums by title
3. 10,000 writes/second (to avoid write throttling during imports)

A sparse index is almost enough #

One way to do it is to use a dedicated sparse index, taking advantage of the fact that items with missing index keys don't appear in the index.

If only albums have an Album attribute, we just create a new GSI:

# GSI Albums (partition key: Album, sort key: Artist)
Leaving Home: !btree
  International Pony: { sk: 'album#Leaving Home', ... }
  Solar Fields: { sk: 'album#Leaving Home', ... }
Heavy Migration: !btree
  Dday One: { sk: 'album#Heavy Migration', ...}

If songs have an Album too, we add a dedicated AlbumsPK attribute instead.

In many ways, this is the ideal solution. To list all albums, we scan the index. To list albums by title, we query an index partition key. We have lots of unique partition keys with items spread pretty evenly across them, which should prevent throttling.

But scan results are not ordered #

...except scan results are not ordered, so we're missing the sorted alphabetically part.

What is ordered are sort keys, so we can use a single index collection instead:

# GSI GSI1 (partition key: gsi1pk, sort key: gsi1sk)
'albums': !btree
  Heavy Migration: { Artist: Dday One, sk: 'album#Heavy Migration', ... }
  Leaving Home: { Artist: Solar Fields, sk: 'album#Leaving Home', ... }
  Leaving Home: { Artist: International Pony, sk: 'album#Leaving Home', ... }

This is also seemingly ideal. To list all albums, we query the entire index partition key. To list albums by title, we use a sort key. The results are sorted as required, and there's no limit on the number of items in a collection.

But a single partition key causes throttling #

However, there are per-partition limits of 24 MB/s for reads and 1 MB/s for writes.

Let's see how they compare to our requirements:

• reads: 500 bytes/item * 10k queries/s * 100 items/query = 500 MB/s (~21x)
• writes: 500 bytes/item * 10k items/s = 5 MB/s (5x)

Uh-oh, turns out we need 21 times the throughput one partition can deliver.

One way to spread the load is sharding, using multiple synthetic partition keys of the form album#{shard_id}. A common option for the shard id is a random number from a known range, e.g. album#{randrange(21)}:

# GSI GSI1 (partition key: gsi1pk, sort key: gsi1sk)
'album#1': !btree
  Leaving Home: { Artist: Solar Fields, ... }
'album#12': !btree
  Heavy Migration: { Artist: Dday One, ... }
'album#20': !btree
  Leaving Home: { Artist: International Pony, ... }

To list all albums, query each shard in turn:

for shard in range(21):
    for item in dynamodb.query(f"album#{shard}"):
        yield item

But random suffixes are random #

There's a problem, though – with random shard ids we can't easily list albums by title, since albums with the same title may end up on any shard.

A better option is to calculate the shard id from the album title using a hash function:

def hash(s):
    return int.from_bytes(sha256(s.encode()).digest())

def album_shard_id(album_title):
    return hash(album_title) % 21

# GSI GSI1 (partition key: gsi1pk, sort key: gsi1sk)
'album#6': !btree
  Leaving Home: { Artist: Solar Fields, ... }
  Leaving Home: { Artist: International Pony, ... }
'album#8': !btree
  Heavy Migration: { Artist: Dday One, ... }

To list albums by title:

dynamodb.query(f"album#{album_shard_id(album_title)}", sk=album_title, index='GSI1')

But hash suffixes are not ordered #

That takes care of throughput, but now results aren't sorted alphabetically anymore. We can sort items within each shard using the sort key, but they are spread uniformly across shards, and there's no order between shards.

Maybe we could use the first letter as shard id instead?

Of course, we have to account for some first letters being more frequent than others. In this case, we can approximate the actual distribution by using MusicBrainz data.

There are 5.5 million albums:

>>> import polars as pl
>>> titles = pl.read_csv(
...     'mbdump/release',
...     has_header=False,
...     separator='\t',
...     quote_char=None,
...     columns=[2],
...     new_columns=['title'],
... )[:,0]
>>> titles.count()
5535986

...but only 3.3 million unique titles, partly due to different releases of the same album, partly due to some titles being more popular – a few of them, really popular:

>>> titles.value_counts(sort=True)
shape: (3_370_505, 2)
 title            count
 Greatest Hits    4638
 Demo             3140
 …                …
 Salsa salsa      1
 Glamour: Deluxe  1
>>> titles.unique_counts().quantile([.9, .99, .999, .9999, 1])
[2.0, 11.0, 58.0, 221.0, 4638.0]

Let's look at first characters:

>>> normalized = titles.str.to_lowercase().str.normalize('NFKD').sort()
>>> normalized.str.slice(0, 1).value_counts(sort=True)
shape: (5_402, 2)
 title  count
 t      584760
 s      509065
 a      317513
 l      298757
 …      …
 🫀     1
 🫂     1
 🫧     1
 󠀼       1

But there are a lot of first characters #

5402?! Indeed, there's more to Unicode than the Latin alphabet:

>>> normalized.str.slice(0, 1).unique().sample(10).to_list()
['學', '舒', 'і', '进', '੦', '潮', '向', '妳', '陳', '🍅']

And it's actually worse than that – there are five thousand characters in our dataset, but there are hundreds of thousands of possible Unicode characters.

This is not a problem when adding the albums, but it is a problem when listing them, since we need to enumerate all the shards in a reasonable amount of time (and most shards being empty doesn't help, either).

As a very bad compromise, we could use the first byte of the UTF-8 encoding instead; this caps the number of shard ids at 256, and at least Latin titles would be sorted (I did say it's a bad compromise). There:

>>> firstbytes = normalized.map_elements(str.encode).bin.slice(0, 1)
>>> firstbytes.value_counts(sort=True)
shape: (136, 2)
 title    count
 b"t"     584760
 b"s"     509065
 b"a"     317513
 b"l"     298757
 …        …
 b"\xd4"  2
 b"U"     1
 b"\xee"  1
 b"\xf3"  1

But some first bytes need multiple shards #

We knew the first byte distribution would be skewed, but some of them don't even fit on a single shard (and it gets worse the more shards we need):

>>> shard_count = 21
>>> firstbytes.value_counts(sort=True).with_columns(
...     pl.col('count') / (len(firstbytes) / shard_count)
... ).head()
shape: (5, 2)
 title  count
 b"t"   2.218206
 b"s"   1.931068
 b"a"   1.204442
 b"l"   1.133294
 b"b"   1.106949

We're back to where we started: how do we sort between shards with the same prefix?

We don't – we find longer prefixes that fit in one shard.

That sounds like the perfect job for a trie (aka prefix tree). This would also allow us to switch back to characters, and merge small prefixes into ranges until each range fits one shard. But that's complicated, and as often the case, there must be a better way.²

But tries and prefix ranges are complicated #

We're looking for contiguous ranges, each of a certain size. Tries are good for finding the shortest prefix, but we don't really care about prefix length.

Why not just split the sorted titles into N equal ranges instead? This takes care of the uneven distribution:

>>> boundaries = normalized.gather_every(2000).str.slice(0, 4)
>>> boundaries.value_counts(sort=True)
shape: (2_103, 2)
 title     count
 the       161
 live      23
 …         …
 風吹けは  1
 魔法少女  1

...provided a long enough prefix:

>>> boundaries = normalized.gather_every(2000).str.slice(0, 16)
>>> boundaries.value_counts(sort=True).filter(pl.col('count') > 1)
shape: (3, 2)
 title             count
 greatest hits     3
 demo              2
 the very best of  2

...almost there:

>>> boundaries = normalized.gather_every(2000).str.slice(0, 20)
>>> boundaries.value_counts(sort=True).filter(pl.col('count') > 1)
shape: (2, 2)
 title          count
 greatest hits  3
 demo           2

This highlights another problem – if the shard size is too small, there may be more than a shard's worth of albums with identical titles; we can fix this by using another, random suffix (ordering doesn't matter anymore, since they have the same title).

Thankfully, our shards are huge, so it's not an issue:

>>> shard_size = int(math.ceil(len(normalized) / shard_count))
>>> shard_size
263619
>>> boundaries = normalized.gather_every(shard_size).str.slice(0, 20)

To use this, save the list of boundaries in code, and find the index of the biggest boundary smaller than a given album title:

import bisect
import unicodedata

ALBUM_TITLE_BOUNDARIES = [
    '',  # replaced with the smallest possible string
    'agartha',
    'barstow / crazy',
    'can you feel it',
    'cyan rot',
    'dreams take over eve',
    'feud semiotics (rb. ',
    'grave poetry',
    'i live',
    'kannaval',
    'live in florence',
    "mir ist's gleich / i",
    'notice',
    'platforms ep',
    'rituals',
    'skylten',
    'surtr / absorbed',
    'the human touch',
    'tonttujen jouluyö: ',
    'walking away',
    'голос',
]

def album_shard_id(album_title):
    normalized = unicodedata.normalize('NFKD', album_title.lower())
    return bisect.bisect(ALBUM_TITLE_BOUNDARIES, normalized) - 1

>>> album_shard_id('2 Pie Island')
0
>>> album_shard_id('Heavy Migration')
7
>>> album_shard_id('Leaving Home')
9
>>> album_shard_id('Space Cadet')
15

But the prefix distribution can change #

We were lucky to have data on the prefix distribution, but that's not always the case, and even if it is, the distribution can change over time.

For example, the last of the 21 shards above starts in the Cyrillic Unicode block, which means most existing scripts go into a single shard. What if we import 1 million Chinese and Japanese albums at some point?

One way to deal with this is to give more weight to known gaps in the data. Another is to have more shards from the start to account for unknown gaps – 210 shards instead of 21 sounds pretty reasonable.

If all else fails, you can move to a new index with more shards, but that comes with its own complications, so it's best to get it roughly right from the start.

Anyway, that's it for now.

Learned something new today? Share it with others, it really helps! PyCoder's Weekly HN Bluesky linkedin Twitter

Want to know when new articles come out? Subscribe here to get new stuff straight to your inbox!

1. This is a simplified example; as the MusicBrainz database shows, the schema for this kind of thing would be way more complicated in practice. ^[return]

2. You're welcome to try, though, especially if you're preparing for an interview. ^[return]

June 08, 2026 01:43 PM UTC

Wingware

Wing Python IDE 12 Early Access - June 8, 2026

Wing 12 is now available as an early access release that focuses on AI agent driven development. Wing 12 introduces deep integration with Claude Code, including a dedicated Claude Code tool, a new Tasks tool for planning, executing, and reviewing AI agent work, and a set of MCP servers that allow agents to work more efficiently by giving them access to Wing's source code analysis, unit testing, and debugger features.

For those using Claude Code, Wing 12 broadens the focus from code-centric direct development to also include managing multiple concurrent AI agents. Of course all of Wing's classic IDE features are still available -- the powerful debugger, deep code analysis and warnings, full-featured editor, project & package management, and much more.

Wing 12 also adds true pseudo-terminal support to OS Commands and Debug I/O, reenvisions the OS Commands tool so that commands can be dragged to tool or editor splits, reorganizes the Tools menu, adds search and back/forward navigation to the Preferences dialog, supports automatic test discovery, speeds up detection of externally modified files, and makes many other improvements.

For more information, see the Wingware Early Access Program. Anyone can participate just by downloading the release.

If you have questions, please don't hesitate to contact us at support@wingware.com.

June 08, 2026 01:00 AM UTC

June 07, 2026

Eli Bendersky

Plugins case study: mdBook preprocessors

mdBook is a tool for easily creating books out of Markdown files. It's very popular in the Rust ecosystem, where it's used (among other things) to publish the official Rust book.

mdBook has a simple yet effective plugin mechanism that can be used to modify the book output in arbitrary ways, using any programming language or tool. This post describes the mechanism and how it aligns with the fundamental concepts of plugin infrastructures.

mdBook preprocessors

mdBook's architecture is pretty simple: your contents go into a directory tree of Markdown files. mdBook then renders these into a book, with one file per chapter. The book's output is HTML by default, but mdBook supports other outputs like PDF.

The preprocessor mechanism lets us register an arbitrary program that runs on the book's source after it's loaded from Markdown files; this program can modify the book's contents in any way it wishes before it all gets sent to the renderer for generating output.

The official documentation explains this process very well.

[preprocessor.narcissistpy]
command = "python3 ../preprocessor-python-narcissist/narcissist.py"

June 07, 2026 07:38 AM UTC

June 06, 2026

Armin Ronacher

Communities of Not

There is a strange thing that happens in communities that gather around abstinence from something: identity from opposition. At their best these communities are not just negative: childfree spaces can be about autonomy, choice and acceptance, anti-car spaces about safer streets and transit, and LLM-skeptical developer spaces about the future of labor, code quality and slop¹. But the thing being refused often does not go away and instead becomes the main subject of the community’s identity.

That would be fine if it stayed at criticism, maybe even angry criticism, but more often than not it turns into policing and hatred towards others. An influencer without children becomes a parent, an urban bike commuter by choice buys a Porsche, a respected developer tries LLMs, and the community feels betrayed because it assumed they were members of the same tribe. The expulsion of that person (who never signed up to be a community member) is entirely imaginary but the punishment that the community unleashes is not: people pile on and shame them, quote them out of context and turn their weakest moments into proof that the person was always unserious, a sharlatan or should not be listened to.

I do not think the answer is to tell people to stop paying attention. Cars shape cities even for people who cycle, children influence politics, workplaces and taxes even for people who do not have them. For us developers, LLMs show up in editors, issue trackers, hiring conversations, management pressure and code reviews whether we asked for them or not. Resisting that can be legitimate but that is no excuse for using one’s rejection to justify shitty mob behavior.

I understand the thinking all too well, because I have done versions of this myself in the past. It took me a while to become more accepting of other people’s worldviews that diverge from mine. Whatever insecurities we have, finding a group of others sharing them can be comforting. The danger is that being part of a crowd of negativity can easily make us part of collective harassment.

I can only encourage you to breathe, slow down, de-escalate when given the chance, and resist the temptation to always assume the most catastrophic reading. Default to being open to new things. Being negative towards something, and making that ones identity, is an easy trap to fall into.

June 06, 2026 12:00 AM UTC

June 05, 2026

Kay Hayen

Nuitka Release 4.1

This is to inform you about the new stable release of Nuitka. It is the extremely compatible Python compiler, “download now”.

This release adds many new features and corrections with a focus on async code compatibility, missing generics features, and Python 3.14 compatibility and Python compilation scalability yet again.

Bug Fixes

• Python 3.14: Fix, decorators were breaking when disabling deferred annotations. (Fixed in 4.0.1 already.)

• Fix, nested loops could have wrong traces lead to mis-optimization. (Fixed in 4.0.1 already.)

• Plugins: Fix, run-time check of package configuration was incorrect. (Fixed in 4.0.1 already.)

• Compatibility: Fix, __builtins__ lacked necessary compatibility in compiled functions. (Fixed in 4.0.1 already.)

• Distutils: Fix, incorrect UTF-8 decoding was used for TOML input file parsing. (Fixed in 4.0.1 already.)

• Fix, multiple hard value assignments could cause compile time crashes. (Fixed in 4.0.1 already.)

• Fix, string concatenation was not properly annotating exception exits. (Fixed in 4.0.2 already.)

• Windows: Fix, --verbose-output and --show-modules-output did not work with forward slashes. (Fixed in 4.0.2 already.)

• Python 3.14: Fix, there were various compatibility issues including dictionary watchers and inline values. (Fixed in 4.0.2 already.)

• Python 3.14: Fix, stack pointer initialization to localsplus was incorrect to avoid garbage collection issues. (Fixed in 4.0.2 already.)

• Python 3.12+: Fix, generic type variable scoping in classes was incorrect. (Fixed in 4.0.2 already.)

• Python 3.12+: Fix, there were various issues with function generics. (Fixed in 4.0.2 already.)

• Python 3.8+: Fix, names in named expressions were not mangled. (Fixed in 4.0.2 already.)

• Plugins: Fix, module checksums were not robust against quoting style of module-name entry in YAML configurations. (Fixed in 4.0.2 already.)

• Plugins: Fix, doing imports in queried expressions caused corruption. (Fixed in 4.0.2 already.)

• UI: Fix, support for uv_build in the --project option was broken. (Fixed in 4.0.2 already.)

• Compatibility: Fix, names assigned in assignment expressions were not mangled. (Fixed in 4.0.2 already.)

• Python 3.12+: Fix, there were still various issues with function generics. (Fixed in 4.0.3 already.)

• Clang: Fix, debug mode was disabled for clang generally, but only ClangCL and macOS Clang didn’t want it. (Fixed in 4.0.3 already.)

• Zig: Fix, --windows-console-mode=attach|disable was not working when using Zig. (Fixed in 4.0.3 already.)

• macOS: Fix, yet another way self dependencies can look like, needed to have support added. (Fixed in 4.0.3 already.)

• Python 3.12+: Fix, generic types in classes had bugs with multiple type variables. (Fixed in 4.0.3 already.)

• Scons: Fix, repeated builds were not producing binary identical results. (Fixed in 4.0.3 already.)

• Scons: Fix, compiling with newer Python versions did not fall back to Zig when the developer prompt MSVC was unusable, and error reporting could crash. (Fixed in 4.0.4 already.)

• Zig: Fix, the workaround for Windows console mode attach or disable was incorrectly applied on non-Windows platforms. (Fixed in 4.0.4 already.)

• Standalone: Fix, linking with Python Build Standalone failed because libHacl_Hash_SHA2 was not filtered out unconditionally. (Fixed in 4.0.4 already.)

• Python 3.6+: Fix, exceptions like CancelledError thrown into an async generator awaiting an inner awaitable could be swallowed, causing crashes. (Fixed in 4.0.4 already.)

• Fix, not all ordered set modules accepted generators for update. (Fixed in 4.0.5 already.)

• Plugins: Disabled warning about rebuilding the pytokens extension module. (Fixed in 4.0.5 already.)

• Standalone: Filtered libHacl_Hash_SHA2 from link libs unconditionally. (Fixed in 4.0.5 already.)

• Debugging: Disabled unusable unicode consistency checks for Python versions 3.4 to 3.6. (Fixed in 4.0.5 already.)

• Python3.12+ Avoided cloning call nodes on class level which caused issues with generic functions in combination with decorators. (Added in 4.0.5 already.)

• Python 3.12+: Added support for generic type variables in async def functions. (Added in 4.0.5 already.)

• UI: Fix, flushing outputs for prompts was not working in all cases when progress bars were enabled. (Fixed in 4.0.6 already.)

• UI: Fix, unused variable warnings were missing at C compile time when using zig as a C compiler. (Fixed in 4.0.6 already.)

• Scons: Fix, forced stdout and stderr paths as a feature was broken. (Fixed in 4.0.6 already.)

• Fix, replacing a branch did not accurately track shared active variables causing optimization crashes. (Fixed in 4.0.7 already.)

• macOS: Fix, failed to remove extended attributes because files need to be made writable first. (Fixed in 4.0.7 already.)

• Fix, dict pop and setdefault using with := rewrites lacked exception-exit annotations for un-hashable keys. (Fixed in 4.0.8 already.)

• Python 3.13: Fix, the __parameters__ attribute of generic classes was not working. (Fixed in 4.0.8 already.)

• Python 3.11+: Fix, starred arguments were not working as type variables. (Fixed in 4.0.8 already.)

• Python2: Fix, FileNotFoundError compatibility fallback handling was not working properly. (Fixed in 4.0.8 already.)

• Compatibility: Fix, loop ownership check in value traces was missing, causing issues with nested loops.

• Windows: Improved --windows-console-mode=attach to properly handle console handles, enabling cases like os.system to work nicely.

• Python2: Fix, there was a compatibility issue where providing default values to the mkdtemp function was failing.

• Windows: Fix, there were spurious issues with C23 embedding in 32-bit MinGW64 by switching to coff_obj resource mode for it as well.

• Plugins: Fix, the post-import-code execution could fail because the triggering sub-package was not yet available in sys.modules.

• UI: Fix, listing package DLLs with --list-package-dlls was broken due to recent plugin lifecycle changes.

• UI: Fix, --list-package-exe was not working properly on non-Windows platforms failing to detect executable files correctly.

• UI: Handled paths starting with {PROGRAM_DIR} the same as a relative path when parsing the --onefile-tempdir-spec option.

• Plugins: Followed multiprocessing forkserver changes for newer Python versions.

• Python 3.12+: Fix, generic class type parameters handling was incorrect.

• Python 3.12: Fix, deferred evaluation of type aliases was failing.

• Python 3.12+: Aligned sum built-in float summation with CPython’s compensated sum for better accuracy.

• Python 3.10+: Fix, uncompiled coroutine throw() return handling was incorrect, restoring completed coroutine results via StopIteration.value rather than exposing them as ordinary return values to the outer await chain.

• Python 3.13+: Fix, uncompiled coroutine cancel()/await suspension handling was incorrect, improved to ensure integration compatibility.

• macOS: Made finding create-dmg more robustly by also checking the Homebrew path for Intel and from PATH properly.

• Compatibility: Fix, class frames were not exposing frame locals.

• UI: Detected static-libpython problems, which affected some forms of Anaconda.

• Distutils: Rejected --project mixed with --main arguments as it is not useful.

• macOS: Fix, zig from PATH or from ziglang was not being used.

• Distutils: Fix, the wrong module-root config value was being checked for uv build backend.

• macOS: Fix, was attempting to change removed (rejected) DLLs, which of course failed and errored out.

• Python 3.14: Fix, tuple reuse was not fully compatible, potentially causing crashes due to outdated hash caches.

• Fix, fake modules were still being attempted to located when imported by other code, which could conflict with existing modules.

• Python 3.5+: Fix, failed to send uncompiled coroutines the sent in value in yield from.

• Fix, older gcc compilers lacking newer intrinsic methods had compilation issues that needed to be addressed.

• Standalone: Fix, multiphase module extension modules with post-load code were not working properly.

• Fix, Avoid using the non-inline copy of pkg_resources with the inline copy of Jinja2. These could mismatch and cause errors.

• Fix, loops could make releasing of previous values very unclear, causing optimization errors.

• Fix, incbin resource mode was not working with old gcc C++ fallback.

• Python 3.4 to 3.6: Fix, bytecode demotion was not working properly for these versions, also bytecode only files not working.

• Plugins: Added a check for the broken patchelf versions 0.10 and 0.11 to prevent breaking Qt plugins.

• Android: Allowed patchelf version 0.18 on Android.

• Windows: Fix, the header path for self uninstalled Python was not detected correctly.

• Release: Fix, inclusion of the pkg_resources inline copy for Python 2 to source distributions was missing.

• UI: Detected the OBS versions of SUSE Linux better.

• Suse: Allowed using patchelf 0.18.0 there too.

• Python 3.11: Fix, package and module dicts were not aligned close enough to avoid a CPython bug.

• Fix, unbound compiled methods could crash when called without an object passed.

• Standalone: Fix, multiphase module extension modules with postload. (Fixed in 4.0.8 already.)

• Onefile: Fix, while waiting for the child, it may already be terminated.

• macOS: Removed existing absolute rpaths for Homebrew and MacPorts.

• Python 3.14: Avoided warning in CPython headers.

• Python 3.14: Followed allocator changes more closely.

• Compatibility: Avoided using pkg_resources for Jinja2 template location for loading.

• No-GIL: Applied some bug fixes to get basic things to work.

Package Support

• Standalone: Add support for newer paddle version. (Added in 4.0.1 already.)

• Standalone: Add workaround for refcount checks of pandas. (Fixed in 4.0.1 already.)

• Standalone: Add support for newer h5py version. (Added in 4.0.2 already.)

• Standalone: Add support for newer scipy package. (Added in 4.0.2 already.)

• Plugins: Revert accidental os.getenv over os.environ.get changes in anti-bloat configurations that stopped them from working. Affected packages are networkx, persistent, and tensorflow. (Fixed in 4.0.5 already.)

• Standalone: Added missing DLLs for openvino. (Added in 4.0.7 already.)

• Enhanced the package configuration YAML schema by adding the relative_to parameter for from_filenames DLL specification, avoiding error-prone purely relative paths.

• Standalone: Fix, flet_desktop app assets were missing, now preserving the packaged runtime and sidecar DLLs.

• Standalone: Added support for the tyro package.

• Standalone: Added data files for the perfetto package.

• Standalone: Added support for anyio process forking.

• Standalone: Added support for the plotly.graph package.

• Anaconda: Fix, dependencies for the numpy conda package on Windows were incorrect.

• Plugins: Enhanced the auto-icon hack in PySide6 to use compatible class names.

• Standalone: Fix, Qt libraries were duplicated with PySide6 WebEngine framework support on macOS.

• Plugins: Fix, automatic detection of mypyc runtime dependencies was including all top level modules of the containing package by accident. (Fixed in 4.0.5 already.)

• Anaconda: Fix, delvewheel plugin was not working with Python 3.8+. This enhances compatibility with installed PyPI packages that use it for their DLLs. (Fixed in 4.0.6 already.)

• Plugins: Fix, our protection workaround could confuse methods used with PySide6.

New Features

• UI: Added the --recommended-python-version option to display recommended Python versions for supported, working, or commercial usage.

• UI: Add message to inform users about Nuitka[onefile] if compression is not installed. (Added in 4.0.1 already.)

• UI: Add support for uv_build in the --project option. (Added in 4.0.1 already.)

• Onefile: Allow extra includes as well. (Added in 4.0.2 already.)

• UI: Add nuitka-project-set feature to define project variables, checking for collisions with reserved runtime variables. (Added in 4.0.2 already.)

• Scons: Added new option to select --reproducible builds or not. (Added in 4.0.6 already.)

• Python 3.10+: Added support for importlib.metadata.package_distributions(). (Added in 4.0.8 already.)

• Plugins: Added support for the multiprocessing forkserver context. (Added in 4.0.8 already, for 4.1 Python 3.6 and earlier, as well as 3.14 support were added too.)

• Reports: Added structured resource usage (rusage) performance information to compilation reports.

• Reports: Included individual module-level C compiler caching (ccache/clcache) statistics in compilation reports.

• Added support for detecting and correctly resolving the Python prefix for the PyEnv on Homebrew Python flavor.

• macOS: Added support for rusage information for Scons.

• UI: Added the __compiled__.extension_filename attribute to give the real filename of the containing extension module.

• Windows: Added support for --clang or ARM. (Added in 4.0.8 already.)

• Windows: Added support for resources names as not just integers, important when we copy them from template files.

• MacPorts: Added basic support for this Python flavor. More work will be needed to get it to work fully though.

Optimization

• Avoid including importlib._bootstrap and importlib._bootstrap_external. (Added in 4.0.1 already.)

• Linux: Cached the syscall used for time keeping during compilation to avoid loading libc for each trace. (Added in 4.0.8 already.)

• UI: Output a warning for modules that remain unfinished after the third optimization pass.

• Added an extra micro pass trigger when new variables are introduced or variable usage changes severely, ensuring optimizations are fully propagated, avoiding unnecessary extra full passes.

• Provided scripts to compile Python statically with PGO tailored for Nuitka on Linux, Windows, and macOS.

• Added support for running the Data Composer tool from a compiled Nuitka binary without spawning an uncompiled Python process.

• Enhanced the usage of vectorcall for PyCFunction objects by directly checking for its presence instead of relying purely on flags, allowing more frequent use of this faster execution path.

• Cached frequently used declarations for top-level variables to speed up C code generation.

• Sped up trace collection merging by avoiding unnecessary set creation and using a set instead of a list for escaped traces.

• Optimized plugin hook execution by tracking overloaded methods and added an option to show plugin usage statistics.

• Improved performance of module location by avoiding unnecessary module name reconstruction and redundant filesystem checks for pre-loaded packages.

• Improved the caching of distribution name lookups to effectively avoid repeated IO operations across all package types.

• Plugins: Cached callback plugin dispatch for onFunctionBodyParsing and onClassBodyParsing to skip argument computation when no plugin overrides them.

• Python 3.13: Handled sub-packages of pathlib as hard modules.

• Handled hard attributes through merge traces as well.

• Made constant blobs more compact by avoiding repeated identifiers and unnecessary fields.

• Enhanced Python compilation scripts further. (Fixed in 4.0.8 already.)

• Recognized late incomplete variables better. (Fixed in 4.0.8 already.)

• Made constant blobs more compact. (Fixed in 4.0.8 already.)

• Optimized calls with only constant keywords and variable posargs too.

Anti-Bloat

• Fix, memory bloat occurred when C compiling sqlalchemy. (Fixed in 4.0.2 already.)

• Avoid using pydoc in PySimpleGUI. (Added in 4.0.2 already.)

• Avoided using doctest from zodbpickle. (Added in 4.0.5 already.)

• Avoided inclusion of cython when using pyav. (Added in 4.0.7 already.)

• Avoided including typing_extensions when using numpy. (Added in 4.0.7 already.)

Organizational

• UI: Relocated the warning about the available source code of extension modules to be evaluated at a more appropriate time.

• Debian: Remove recommendation for libfuse2 package as it is no longer useful.

• Debian: Used platformdirs instead of appdirs.

• Debugging: Removed Python 3.11+ restriction for clang-format as it is available everywhere, even Python 2.7, and we still want nicely formatted code when we read things. (Added in 4.0.6 already.)

• Removed no longer useful inline copy of wax_off. We have our own stubs generator project.

• Release: Added missing package to the CI container for building Nuitka Debian packages.

• Developer: Updated AI instructions for creating Minimal Reproducible Examples (MRE) to skip unneeded C compilation.

• Debugging: Added an internal function for checking if a string is a valid Python identifier.

• AI: Added a task in Visual Studio Code to export the currently selected Python interpreter path to a file, making it available as “python” and “pip” matching the selected interpreter. This makes it easier to use a specific version with no instructions needed.

• AI: Updated the rules to instruct AI to only generate useful comments that add context not present in the code.

• Containers: Added template rendering support for Jinja2 (.j2) container files in our internal Podman tools.

• Projects: Clarified the current status and rationale of Python 2.6 support in the developer manual.

• Debugging: Added experimental flag --experimental=ignore-extra-micro-pass to allow ignoring extra micro pass detection.

• Visual Code: Added integration scripts for bash and zsh autocompletion of Nuitka CLI options. These are now also integrated into Visual Studio Code terminal profiles and the Debian package.

• RPM: Included the Python compile script for Linux.

• RPM: Removed the requirement for distutils in the spec.

Tests

• Install only necessary build tools for test cases.

• Avoided spurious failures in reference counting tests due to Python internal caching differences. (Fixed in 4.0.3 already.)

• Fix, the parsing of the compilation report for reflected tests was incorrect.

• Python 3.14: Ignored a syntax error message change.

• Python 3.14: Added test execution support options to the main test runner to use this version as well.

• Fix, the runner binary path was mishandled for the third pass of reflected compilations.

• Removed the usage of obsolete plugins in reflected compilation tests.

• Debugging: Prevented boolean testing of namedtuples to avoid unexpected bugs.

• Added the Test suffix to syntax test files and disabled “python” mode and spell checking for them to resolve issues reported in IDEs.

• Fix, newline handling in diff outputs from the output comparison tool was incorrect.

• Covered post-import-code functionality with a new subpackage test case.

• Prevented the program test suite from running an unnecessary variant to save execution time.

• macOS: Ignored differences from GUI framework error traces in headless runs in output comparisons.

• Reflected test for Nuitka, where it compiles itself and compares its operation has been restored to functional state.

• Used the new method to clear internal caches if available for reference counts.

• Disabled running nested loops test with Python 2.6.

• Containers: Detected Python 2 defaulting containers in Podman tooling.

Cleanups

• UI: Fix, there was a double space in the Windows Runtime DLLs inclusion message. (Fixed in 4.0.1 already.)

• Onefile: Separated files and defines for extra includes for onefile boot and Python build.

• Scons: Provided nicer errors in case of “unset” variables being used, so we can tell it.

• Refactored the process execution results to correctly utilize our namedtuples variant, that makes it easier to understand what code does with the results.

• Quality: Enabled automatic conversion of em-dashes and en-dashes in code comments to the autoformat tool. AI won’t stop producing them and they can cause SyntaxError for older Python versions, nor is unnecessarily using UTF-8 welcome.

• Ensured that cloned outline nodes are assigned their correct names immediately upon creation, that avoids inconsistencies during their creation.

• Quality: Updated to the latest versions of black and adopted a faster isort execution by caching results.

• Quality: Modified the PyLint wrapper to exit gracefully instead of raising an error when no matching files require checking.

• Quality: Avoided checking YAML package configuration files twice, since autoformat already handles them.

• Quality: Ensured that YAML package configuration checks output the original filename instead of the temporary one when a failure occurs.

• Quality: Prevented pushing of tags from triggering git pre-push quality checks.

• Quality: Silenced the output of optipng and jpegoptim during image optimization auto-formatting.

• Visual Code: Added the generated Python alias path file to the ignore list.

• Quality: Enabled auto-formatting for the Nuitka devcontainer configuration file.

• Watch: Avoided absolute paths in compilation to make reports more comparable across machines.

• Quality: Changed mdformat checks to run only once and silently.

• Scons: Disabled format security errors in debug mode and moved Python-related warning disables into common build setup code.

• Quality: Updated to the latest deepdiff version.

• Scons: Avoided MSVC telemetry since it can produce outputs that break CI.

• Debugging: Enhanced non-deployment handler for importing excluded modules.

• Split import module finding functionality into more pieces for enhanced readability.

• Debugging: Added more assertions for constants loading and checking.

• macOS: Dropped the universal target arch.

• Debugging: Added more traces for deep hash verification.

Summary

This release builds on the scalability improvements established in 4.0, with enhanced Python 3.14 support, expanded package compatibility, and significant optimization work.

The --project option seems usable now.

Python 3.14 support remains experimental, but only barely made the cut, and probably will get there in hotfixes. Some of the corrections came in so late before the release, that it was just not possible to feel good about declaring it fully supported just yet.

June 05, 2026 10:00 PM UTC

Will Kahn-Greene

Bleach 6.4.0 releases -- final release

What is it?

Bleach is a Python library for sanitizing and linkifying text from untrusted sources for safe usage in HTML.

Bleach v6.4.0 released!

Bleach 6.4.0 includes two security fixes, a fix to tinycss2 dependency requirements, and some other things.

See the changes here:

https://bleach.readthedocs.io/en/latest/changes.html#version-6-4-0-june-5th-2026

Bleach v6.4.0 is the final release

I haven't used Bleach on a project in years, but I still had some time to maintain it. That changed about a year ago when I got re-orged into a new role and I haven't had time to do any Bleach work since then.

To recap, Bleach sits on top of html5lib which hasn't been actively maintained in years. It is dangerous to maintain Bleach in that context.

We vendored html5lib so we could make adjustments to the library to keep Bleach going. This is not a sustainable approach, but it was ok for the short term.

Over the years, we've talked about other options:

1. find another library to switch to

2. take over html5lib development

3. fork html5lib and vendor and maintain our fork

4. write a new HTML parser

5. etc

None of those are feasible for me.

Bleach has been a solo-maintained project for a while now. The world is crazy and it's much harder to build a team of trusted maintainers now than it was (or at least, it sure feels that way). I don't see any possibility of increasing the maintenance team or passing it to someone else responsibly.

Switching contexts from my regular work to Bleach is really hard. Bleach is complicated, the problem domain is complicated, and there's a lot of nuanced context. I can't just switch gears, spend 15 minutes on Bleach to do something, and then switch back to the rest of my day. I periodically get nag messages about this which are entirely valid, but there's nothing I can do about it. It doesn't feel great.

Then in 2025, Emil, a long-time Bleach contributor, built justhtml which gives us an easy migration path off of Bleach. He even took the time to write a migration guide.

Thoughts and statistics

In 2019, when I stepped down the first time, I wrote a post on stepping down.

In 2023, when I deprecated the project, I wrote a post on Bleach 6.0.0 and deprecation.

• From the first commit on 2010-02-18 to today's final commit on 2026-06-05, the Bleach project lasted 16 years, 3 months — 5,951 days, or about 16.29 years.

• There were 64 releases.

• There were roughly 960 commits.

  • From 80 roughly contributors

  • Top 3:

    • Will Kahn-Greene: 462

    • James Socol: 182

    • Greg Guthe: 133

• Roughly 5,040 lines of Python code excluding the vendored html5lib.

• I was maintainer from October 2015 to now--that's a little under 11 years.

It feels weird to end a project that's outlived many of the Mozilla sites and Python web frameworks it was designed to protect.

What happens now?

This is the end of the project.

Bleach. Last release.

If you're still using Bleach, I think you have three options:

1. End your project. Maybe you don't need to be maintaining your thing anymore? Use Bleach as your reason to exit and do something different with your time on Earth.

2. Switch to the sanitizer API. Rework your project to use the sanitizer API.

   • Spec: https://wicg.github.io/sanitizer-api/

   • Docs: https://developer.mozilla.org/en-US/docs/Web/API/Element/setHTML

3. Swap Bleach out for justhtml. Emil provided a migration guide for switching from Bleach to justhtml.

Good luck with whatever option you choose!

Thanks!

Many thanks to James who created Bleach and gave it a set of first principles that guided our choices for 16 years.

Many thanks to Greg who I worked with on Bleach for a long while and maintained Bleach for several years. Working with Greg was always easy and his reviews were thoughtful and spot-on.

Many thanks to Emil who was a contributor to Bleach for a long while and created justhtml providing Bleach users a migration path.

Many thanks to Jonathan who, over the years, provided a lot of insight into how best to solve some of Bleach's more squirrely problems.

Many thanks to Sam who was an indispensible resource on HTML parsing and sanitizing text in the context of HTML.

Many thanks to all the users and contributors of Bleach!

Where to go for more

For more specifics on this release, see here: https://bleach.readthedocs.io/en/latest/changes.html#version-6-4-0-june-5th-2026

Documentation and quickstart here: https://bleach.readthedocs.io/en/latest/

Source code and issue tracker here: https://github.com/mozilla/bleach/

June 05, 2026 01:00 PM UTC

Real Python

The Real Python Podcast – Episode #298: Reducing the Size of Python Docker Containers

How can you easily reduce the size of a Python Docker container? What are the exceptions you should catch in your code? Christopher Trudeau is back on the show this week with another batch of PyCoder's Weekly articles and projects.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

June 05, 2026 12:00 PM UTC

EuroPython Society

EuroPython Society at PyCon US 2026

This year we were back at PyCon US, and this time in sunny Long Beach, California.

We had a booth again, which has quickly become one of our favourite parts of the trip. It&aposs such a great chance to meet folks from other Python communities, catch up with old friends, and put faces to names we&aposve only seen online. People stopped by to chat about EuroPython, pick up stickers, ask about our grants programme, and share what their own local communities are up to. We loved every minute of it.

We also filmed some shorts at the booth, which will be up on our YouTube channel soon! Keep an eye out, there are some lovely conversations in there.

Since EuroPython is celebrating its 25th anniversary this year, we took the chance to talk to community members who have been to many, many EuroPythons over the years. Hearing their stories, the editions they remember most, the friendships that started at one of our conferences, was genuinely moving. It&aposs a good reminder of how much history this community carries with it, and how much of it has been built by people simply showing up year after year.

PyCon US was also where some wonderful people from our community received well-deserved recognition. A huge congratulations to Maria Jose Montreas-Colina, who received an Outstanding PyLady Award for her work with PyLadies and the wider community. Maria is part of our team and helps look after PyLadies and community matters at EuroPython. Congratulations also to Rodrigo Girão Serrão for receiving the Community Service Award for his contributions to the community. Rodrigo works on our programme and sprints.

Thank you both for everything you do. 💛

Long Beach itself was a lovely city. Palm trees, warm weather, the ocean nearby. A very different vibe from the usual conference cities, and a really lovely backdrop for a week of Python.

A big thank you to the PyCon US organisers for having us, and for making space for the wider Python world to come together. And a thank you to everyone who stopped by the booth to say hello, it was a pleasure meeting you.

See you next year, and we hope to see many of you in Kraków for EuroPython 2026!

June 05, 2026 09:28 AM UTC

Bob Belderbos

How to Update Multiple Page Elements from One htmx Request

A button submits code, tests run, feedback appears. Standard htmx. But the submissions dropdown stays stale; the new submission is in the database, just not in the dropdown. One request, two elements to update.

The problem: one request, two things to update

On our Rust platform, each exercise page has a "Run Tests" button. It posts the editor code to a Django view, which compiles and runs the tests, then swaps a pass/fail panel into a #feedback div.

Next to the editor there is a dropdown of your past submissions. Run the tests, and a new submission gets saved server-side. But the dropdown stayed stale until you reloaded the page. The new submission was there in the database, just not in the <select>.

So now I have one request that needs to update two unrelated parts of the page: the feedback panel (the htmx target) and the submissions dropdown (somewhere else entirely in the DOM).

First instinct: write JavaScript to read the response, build a new <option>, prepend it to the select. That works until you remember the dropdown also enforces a max number of submissions, drops duplicates, and orders newest-first. Replicate that logic in the browser and you now have two sources of truth that drift apart the first time you change the server rule.

Htmx out-of-band swaps

htmx has a feature for exactly this: out-of-band swaps.

The hx-swap-oob attribute allows you to specify that some content in a response should be swapped into the DOM somewhere other than the target, that is "Out of Band". This allows you to piggyback updates to other element updates on a response.

htmx swaps any element carrying hx-swap-oob into its matching target on the page, separately from the main swap. One response, many updates.

First I pulled the submissions dropdown options into a partial so the page and the view render them identically:

<!-- _submission_options.html -->
<option disabled selected>Submissions / Reset</option>
{% for submission in submissions %}
  <option value="{{ submission.unique_hash }}">{{ submission.created_at|date:"Y-m-d H:i" }} {% if submission.ok %}(OK){% else %}(Failed){% endif %}</option>
{% endfor %}
<option value="reset">Reset</option>

The page includes it inside the <select id="submissions">. The view renders the same partial and tags it for an out-of-band swap:

from django.template.loader import render_to_string
from django.utils.html import escape

def validate(request):
    # ... run the tests, save the submission ...

    submissions = Submission.objects.filter(
        exercise=exercise, user=user
    ).order_by("-created_at")
    options = render_to_string(
        "_submission_options.html", {"submissions": submissions}
    )

    return HttpResponse(
        f"""
        <div class="...">{message}</div>
        <pre>{escape(output)}</pre>
        <div hx-swap-oob="innerHTML:#submissions">{options}</div>
        """
    )

The first part of the response swaps into #feedback as usual. htmx spots the hx-swap-oob element, pulls it out, and applies it to #submissions instead. The button HTML only knows about #feedback. The view decides what else to update. Whoever owns the data controls how it renders.

A second example: progress bars that update themselves

On the Python platform the same trick drives the learning-path progress widget. Passing an exercise recomputes your progress along every path it belongs to and swaps the bars into a sidebar, from the same request that renders the pass/fail panel:

if ok:
    paths_html = ""
    for path in bite.bite_paths.prefetch_related("bites"):
        # ... compute completed / total / pct for this path ...
        paths_html += render_progress_bar(path, completed, total, pct)
    extra_html += (
        f'<div id="learning-paths-progress" hx-swap-oob="innerHTML">{paths_html}</div>'
    )

The two examples aim at their targets differently. The progress widget uses a bare hx-swap-oob="innerHTML": htmx swaps the fragment into whatever element already shares its id. The dropdown uses the selector form, hx-swap-oob="innerHTML:#submissions", so the carrier <div> can target the <select id="submissions"> without needing to share its id.

Use innerHTML instead of the default hx-swap-oob="true". true replaces the whole element (its outerHTML), which for the <select> throws away the htmx listener attached to it. innerHTML keeps the element and swaps only its children, so the listener survives and the options refresh underneath it.

Here's the progress bars before and after passing an exercise. The bars update via out-of-band swap from the same response that renders the pass/fail feedback:

You might spot the submissions dropdown in these shots and wonder why it is not OOB-swapped here too, like on the Rust platform above. It is the boundary of the technique: OOB swap is solid when you replace an element's whole inner content (innerHTML:#submissions), but inserting a single new <option> means wrapping it in a <template> so the browser does not strip it, and htmx 2.x handled that case unreliably. So here the one-option insert runs through a small htmx:afterSwap listener instead. The view writes the new submission into a hidden div, and the listener reads it and prepends the <option>:

document.addEventListener("htmx:afterSwap", () => {
  const newSub = document.getElementById("new-submission");
  if (!newSub) return;
  const select = document.getElementById("submissions");
  const opt = document.createElement("option");
  opt.value = newSub.dataset.hash;
  opt.textContent = newSub.dataset.label;
  select.insertBefore(opt, select.children[1]); // after the placeholder
  select.value = opt.value;
});

One honest cost: this adds a query. After saving, the view re-fetches the submissions to render the partial. The alternative is re-implementing state changes in pure JavaScript, creating behavior in two places. With out-of-band swaps you drive the logic from the view, all in one place. One more query, but less code and a more maintainable solution.

Whoever fetches the data should render it. Keep the query and the template together.

For more on hypermedia-driven applications, see this great book: Hypermedia Systems.

June 05, 2026 12:00 AM UTC

Seth Michael Larson

Is the Super Smash Bros. Brawl donut from Mister Donut?

Happy Donut Day (and #FediDonutFriday) to those who celebrate! 🍩 Present and Correct shared a link to the Mister Donut museum on Bluesky and upon clicking through I was greeted with a familiar face: a chocolate ring donut.

Strangely, I've seen this chocolate ring donut before: from the hours staring at sprite-sheets from the Super Smash Bros. and Kirby Air Riders franchises. That donut looked just like the one from Super Smash Bros. Brawl.

“But Seth”, I hear you say, “chocolate ring donuts all look the same anyway!” Maybe... and yet...

Funnily enough, the Render96 wiki, which collects origins for artwork for many games like Super Smash Bros., lists the donut from Super Smash Bros. Melee as one of the few foods where the origin is not known. Could this donut also be a Mister Donut? We'll probably never know!

Thanks for reading ♥ I would love to hear your thoughts! Contact me via Mastodon, Bluesky, or email. Browse the blog archive. Check out my blogroll.

June 05, 2026 12:00 AM UTC

June 04, 2026

The Python Coding Stack

Down The Iterator Rabbit Hole

You know that street game where the performer (con artist?) has three opaque cups and a small ball. He places the cups upside down on the table, with the ball under one of the cups. He quickly shuffles the cups around and then asks the player to guess which cup has the ball. You’ve seen the game on TV, even if you’ve not seen it in real life.

Following what’s happening when you have a chain of iterators in Python can feel like playing that game. But, unlike the street game, there are no scams when you’re playing the iterator game. Let’s make sure you’ll always win.

I’ll keep this article short. I wrote many articles about iterables and iterators. If you need to refresh your memory, have a look at The Anatomy of a for Loop and A One-Way Stream of Data • Iterators in Python (Data Structure Categories #6).

Follow The Data in a Chain of Iterators

Let’s keep the example simple. Start with this list in a REPL session:

All code blocks are available in text format at the end of this article • #1

A list is iterable. You can create an iterator from any iterable. Let’s create an iterator from this list:

#2

The built-in function iter() creates an iterator from an iterable. Iterators don’t contain data. They don’t create copies of the data. They’re lightweight objects that create a stream. They’ll fetch data from the original source, which is the list boring_numbers in this case, as and when needed.

Iterators can only fetch an item once. So, they’re a one-way stream. Once you use an item, it’s gone from the iterator – but not from the original list, which remains unchanged.

Therefore, first_iter is an iterator that relies on data from the list boring_numbers. But let’s not fetch any items from the first_iter iterator. Not yet, anyway.

Create a second iterator. This time, you’ll use a generator expression. Generators are iterators, so you create a second iterator with this code:

#3

Note that the expression on the right-hand side of the equals sign is enclosed in parentheses – the round ones, to be clear. This is a generator expression, which creates a generator iterator. Read Pay As You Go • Generate Data Using Generators (Data Structure Categories #7) for more on generators.

As we said, generators are iterators.

The second_iter iterator generates data from first_iter, which is itself an iterator. Iterators are also iterable, which is why you can use them directly in a for clause or anywhere else you’d generally use an iterable. The second_iter iterator will yield the values as floats. But you’ve not yielded any value from this iterator either. Not yet.

Let’s go a step further and create a third iterator, which is also a generator in this case. You build this third iterator from the second one, second_iter:

#4

The generator iterator third_iter yields the sum of 0.5 and the value yielded by second_iter.

Incidentally, I used a “standard” iterator and two generator iterators in this example. However, for the journey we’re following in this article, it doesn’t matter whether we’re using a basic iterator or a generator iterator. If you prefer, you can repeat this exercise with iterators you get from iter() directly.

Support The Python Coding Stack

Don’t Blink • Follow the Data

You started with a list called boring_numbers. This data structure contains* the data. It’s where the data lives. We’ll be following the data in this section. So it’s important to know where it’s stored!

*Note: Lists, like all data structures, don’t really contain data in the purest sense of the word. See What’s In A List—Yes, But What’s Really In A Python List for more on this. But in general, it’s fine to talk about a list ‘containing’ items of data.

You then create three iterators. The first uses data from boring_numbers. The second iterator uses data from the first. And the third iterator uses data from the second.

But you haven’t tried to fetch any value from any of the iterators yet.

Let’s look at what each iterator is doing at the moment before you fetch any values. The first iterator, first_iter, is pointing at the first item in boring_numbers. It’s ready to read this value and yield it.

The second iterator, second_iter, is pointing at the first item in first_iter. But first_iter doesn’t have any data. Iterators don’t have their own data. But that’s OK. Whenever second_iter needs to fetch the value, it will ask first_iter to fetch and yield its “first” value. I put “first” in quotation marks because you’ll see later that this may or may not be the first value.

Finally, third_iter is pointing at the first item in second_iter. The same logic applies. When third_iter needs the first item, it will ask second_iter for its “first” item, and second_iter will need to ask first_iter for its “first” item. And first_iter is pointing at the first item in the list boring_numbers.

Are you with me? Let’s complicate things a bit…

Note how your code so far includes the following lines:

#5

None of the iterators has yielded any value. For now.

Let’s jumble things up and start by fetching the first value from second_iter:

#6

You ask for the next value in second_iter, which is the first one since you haven’t yielded any values yet.

As you’ve seen earlier, second_iter needs the first value from first_iter. So, behind the scenes, Python calls next(first_iter), which yields the first item from boring_numbers.

So, first_iter reads the first value from boring_numbers, which is the integer 1, and it yields it to second_iter, which then yields the transformed version to the REPL as the return value of next(second_iter). That’s why the output is the float 1.0. The first iterator, first_iter, now moves to point at the second item in boring_numbers, ready for when it’s needed.

Note that boring_numbers doesn’t change in this process. The first item in boring_numbers remains there. It doesn’t disappear.

So far, so good?

Continue in the same REPL session and try the following:

#7

You ask third_iter to give you its “next” value. You haven’t used third_iter anywhere so far. So, you might expect it to yield the “first” value.

And it does.

But its interpretation of what’s the “first” item may be different to what you expect.

Let’s follow the data. When you call next(third_iter), the third iterator asks second_iter for its next item. The second iterator, second_iter, relies on first_iter, so it asks first_iter for its next item. And first_iter, as you may recall, is currently pointing at the second item in boring_numbers, which is the integer 2.

So:

1. The first iterator first_iter gets the integer 2 from boring_numbers and yields it to second_iter. And first_iter now points at the third item in boring_numbers.

2. Then, second_iter transforms this value into a float and yields 2.0 to third_iter.

3. Finally, third_iter adds 0.5 to this value and yields 2.5, which is what you see displayed in the REPL.

When you called next(second_iter) earlier in the code, you used up the first item in second_iter, which in turn used up the first item in first_iter. Since this first value is gone and since third_iter depends on the data yielded by second_iter and first_iter, the earlier call to next(second_iter) also affected the iterator that’s downstream, third_iter.

What will happen if you call next(first_iter) now? Try to follow the data in your head before trying it out or reading on.

.

.

Have you worked it out?

.

.

Let’s run the code:

#8

Although it’s the first time you explicitly use first_iter in your code, you already used two of its values when your code yielded values from iterators downstream. Therefore, the next item in first_iter is the third item in boring_numbers, the integer 3.

Let’s finish with one more expression, still running in the same REPL session:

#9

You call next(third_iter), which asks second_iter for its next item. And second_iter asks first_iter for its next item. At this stage in the process, first_iter is pointing at the fourth item in the original source of data, which is the list boring_numbers. That’s why the output is 4.5.

Independent Iterators

Consider the following code, which is similar to the one you wrote above but has one extra line:

#11

The iterators first_iter and another_first_iter both use the same source of data, boring_numbers. However, they are independent iterators. Note that when you use up some of the elements in first_iter, the independent another_first_iter is not affected. The first time you ask for the first item in another_first_iter, you get the integer 1.

Final Words

Iterators don’t contain data. They rely on data that’s stored elsewhere. But you can have a chain of iterators, each asking the previous one to yield a value. Weird things can happen if you’re not careful. But now you know how to follow the data when you have a chain of iterators.

As a rule of thumb, if you create an iterator that depends on another iterator, you should only use the final iterator to avoid these issues. So, in the example above, you should only yield values from third_iter.

Have a play with this example and make your own chains of iterators, too. And once you’re comfortable with this, get ready to be confused again with my next article, which will discuss itertools.tee()!

And next time you pass by someone in the street offering to let you play the three-cups-and-ball game, don’t feel overconfident because of your iterator knowledge – it won’t help you find the ball.

Code in this article uses Python 3.14

The code images used in this article are created using Snappify. [Affiliate link]

Join The Club, the exclusive area for paid subscribers for more Python posts, videos, a members’ forum, and more.

Subscribe now

For more Python resources, you can also visit Real Python—you may even stumble on one of my own articles or courses there!

Also, are you interested in technical writing? You’d like to make your own writing more narrative, more engaging, more memorable? Have a look at Breaking the Rules.

And you can find out more about me at stephengruppetta.com

Further reading related to this article’s topic:

• The Anatomy of a for Loop

• A One-Way Stream of Data • Iterators in Python (Data Structure Categories #6)

• Pay As You Go • Generate Data Using Generators (Data Structure Categories #7)

• What’s In A List—Yes, But What’s Really In A Python List

Appendix: Code Blocks

Code Block #1

boring_numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

Code Block #2

# ...
first_iter = iter(boring_numbers)

Code Block #3

# ...
second_iter = (float(number) for number in first_iter)

Code Block #4

# ...
third_iter = (num + 0.5 for num in second_iter)

Code Block #5

boring_numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
first_iter = iter(boring_numbers)
second_iter = (float(number) for number in first_iter)
third_iter = (num + 0.5 for num in second_iter)

Code Block #6

# ...
next(second_iter)
# 1.0

Code Block #7

# ...
next(third_iter)
# 2.5

Code Block #8

# ...
next(first_iter)
# 3

Code Block #9

# ...
next(third_iter)
# 4.5

Code Block #10

boring_numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
first_iter = iter(boring_numbers)
another_first_iter = iter(boring_numbers)
second_iter = (float(number) for number in first_iter)
third_iter = (num + 0.5 for num in second_iter)
next(second_iter)
# 1.0
next(third_iter)
# 2.5
next(first_iter)
# 3
next(third_iter)
# 4.5
next(another_first_iter)
# 1

For more Python resources, you can also visit Real Python—you may even stumble on one of my own articles or courses there!

Also, are you interested in technical writing? You’d like to make your own writing more narrative, more engaging, more memorable? Have a look at Breaking the Rules.

And you can find out more about me at stephengruppetta.com

June 04, 2026 12:50 PM UTC

• • RSS feed
  • Titles Only
  • Powered by Planet!
• Other Python Planets
  • Python Summer of Code
  • Planet Python Francophone
  • Planet Python Argentina
  • Planet Python Brasil
  • Planet Python Poland
• Python Libraries
  • OLPC
  • PySoy
  • SciPy
  • SymPy
  • Twisted
• Python/Web Planets
  • CherryPy
  • Django Community
  • Plone
  • Turbogears
• Other Languages
  • Haskell
  • Lisp
  • Parrot
  • Perl
  • Ruby
• Databases
  • MySQL
  • PostgreSQL
• Subscriptions
  • [OPML feed]
  • "Menno's Musings"
  • "Michael J.T. O'Kelly"
  • "Michael Kennedy's Thoughts on Technology"
  • "Michiel's Blog"
  • "Morphex's Blogologue"
  • "Speno's Pythonic Avocado"
  • "William's Journal"
  • 2degrees
  • A. Jesse Jiryu Davis
  • ABlog for Sphinx
  • Aahz
  • Abhijeet Pal
  • Abu Ashraf Masnun
  • Adam Pletcher
  • Adrarsh Divakaran
  • Agendaless Consulting
  • Ahmed Bouchefra
  • Al-Ahmadgaid Asaad
  • Alec Munro
  • Alex Grönholm
  • Alex Morozov
  • Alexander Limi
  • Alexandre Conrad
  • Alexandre Vassalotti
  • Alexey Evseev
  • Allison Kaptur
  • Amjith Ramanujam
  • AmvTek
  • Anarcat
  • Anatoly Techtonik
  • Andre Roberge
  • Andrea Grandi
  • Andrew Dalke
  • Andriy Kornatskyy
  • Andy Dustman
  • Andy R. Terrel
  • Anna Martelli Ravenscroft
  • Anthony Baxter
  • Anton Belyaev
  • Anton Bobrov
  • Antonio Cuni
  • Anwesha Das
  • Ari Lamstein
  • Armin Ronacher
  • Artem Golubin
  • Artem Rys
  • Ashish Vidyarthi
  • Astro Code School
  • Audrey M. Roy Greenfeld
  • Automating OSINT
  • Awesome Python Applications
  • Baiju Muthukadan
  • Bajusz Tamás
  • BeDjango
  • Ben Bass
  • Ben Cook
  • Ben Rousch
  • Benjamin Peterson
  • Benji York
  • Bertrand Mathieu
  • Bhavin Gandhi
  • Bhishan Bhandari
  • BioPython News
  • Bit of Cheese
  • Bob Belderbos
  • Bojan Mihelac
  • Brandon Rhodes
  • BreadcrumbsCollector
  • Brendan Scott
  • Brett Cannon
  • Brian Okken
  • Bruno Oliveira
  • Bruno Ponne / Coding The Past
  • Bruno Rocha
  • Caktus Consulting Group
  • Calvin Cheng
  • Calvin Spealman
  • Carl Chenet
  • Carl Düvel
  • Carl Trachte
  • Carlos Eduardo de Paula
  • Casey Duncan
  • Catherine Devlin
  • Ching-Hwa Yu
  • Chris Hager
  • Chris Miles
  • Chris Mitchell
  • Chris Moffitt
  • Chris Rose
  • Chris Warrick
  • Christian Heimes
  • Christian Ledermann
  • Christian Scholz
  • Christoph Schiessl
  • Christoph Zwerschke
  • CodeGrades
  • CodeSnipers
  • CodersLegacy
  • Coding Diet
  • Core Dispatch
  • Corey Gallon
  • Corey Goldberg
  • Cormoran Project
  • Cross-Platform Command Line Tools
  • CubicWeb
  • Curtis Miller
  • DSPIllustrations.com
  • DaPythonista
  • Daily Tech Video (Python)
  • Dallas Fort Worth Pythoneers
  • Dan Crosta
  • Dan Stromberg
  • Dan Yeaw
  • Daniel Bader
  • Daniel Nouri
  • Daniel Roy Greenfeld
  • Data Community DC
  • Data School
  • DataWars.io
  • Dave Beazley
  • David Amos
  • David Caron
  • David Goodger
  • David Lindelof
  • David MacIver
  • David Malcolm
  • David Szotten
  • Davide Moro
  • Davy Mitchell
  • Davy Wybiral
  • Declassed Art
  • Denis Kurov
  • Django Weblog
  • Djangostars
  • Doing Math with Python
  • Doug Hellmann
  • Dougal Matthews
  • Duncan McGreggor
  • Ed Crewe
  • Edward K. Ream
  • Eli Bendersky
  • Eniram Ltd.
  • Eray Özkural (examachine)
  • Erik Marsja
  • Etienne Desautels
  • EuroPython
  • EuroPython Society
  • Evennia
  • Everyday Superpowers
  • Fabio Zadrozny
  • Filip Wasilewski
  • Filipe Saraiva
  • First Institute of Reliable Software
  • Flavio Coelho
  • Flavio Percoco
  • Floris Bruynooghe
  • Frank Wierzbicki
  • François Dion
  • François Marier
  • Frederik Rietdijk
  • From Python Import Podcast
  • Full Stack Python
  • Gaël Varoquaux
  • Georges Dubus
  • Ghaandee on IT
  • Giampaolo Rodola
  • Giulio Fidente
  • Glenn Franxman
  • Glyph Lefkowitz
  • Go Deh
  • Gocept Weblog
  • Godson Gera
  • Graeme Cross
  • Graham Dumpleton
  • Graham Wheeler
  • Grant Baillie
  • Grant Rettke
  • Greg Taylor
  • Grig Gheorghiu
  • Grzegorz Śliwiński
  • Guido van Rossum
  • Gustavo Narea
  • Gustavo Niemeyer
  • Gökhan Sever
  • Hernan Grecco
  • Hilary Mason
  • Holger Peters
  • HoloViz
  • Hugo van Kemenade
  • Humberto Rocha
  • Hynek Schlawack
  • Ian Ozsvald
  • Ilian Iliev
  • Import Python
  • Inspired Python
  • Ionel Cristian Maries
  • IronPython-URLs
  • IslandT
  • Israel Fruchter
  • Itamar Turner Trauring
  • Ivan Velichko
  • J. Pablo Fernández
  • Jack Diederich
  • Jacob Perkins
  • Jahongir Rahmonov
  • Jaime Buelta
  • Jamal Moir
  • James Bennett
  • Janusworx
  • Jarrod Millman
  • Jean-Paul Calderone
  • Jeff Bisbee
  • Jeff Bradberry
  • Jeff Hinrichs
  • Jeff Shell
  • Jeremy Epstein
  • Jeremy Hylton
  • Jim Fulton
  • Joe Abbate
  • Joe Pitz
  • Johan Dahlin
  • John Burns
  • John Cook
  • John Jacobsen
  • John Ludhi/nbshare.io
  • Jon Parise
  • Jonathan Ellis
  • Jonathan Harrington
  • Jonathan Hartley
  • Jonathan Street
  • Jorgen Schäfer
  • Juan Manuel Contreras
  • Julien Danjou
  • Julien Tayon
  • Juri Pakaste
  • Just a little Python
  • Justin Mayer
  • Kai Lautaportti
  • Karim Elghamrawy
  • Kay Hayen
  • Kay Schluehr
  • Kelly Yancey
  • Kevin Renskers
  • Kodnito
  • Kogan Dev
  • Konrad Delong
  • Koodaamo
  • Kristján Valur Jónsson
  • Kriti Godey
  • Kulbir Saini
  • Kumar McMillan
  • Kumar Vipin Yadav
  • Kushal Das
  • Károly Nagy
  • LAAC Technology
  • Laurent Szyster
  • Leigh Honeywell
  • Lennart Regebro
  • Leonhard Vogt
  • Lintel Technologies
  • Linux Stans
  • ListenData
  • Logilab
  • Low Kian Seong
  • Luca Botti
  • Lucas Cimon
  • Ludovic Gasc
  • Ludvig Ericson
  • Luke Plant
  • Maciej Fijalkowsk
  • Made With Mu
  • Mahmoud Hashemi
  • Malthe Borch
  • Marc Richter
  • Marc-André Lemburg
  • Marcos Dione
  • Mariatta
  • Marius Gedminas
  • Mark Dufour
  • Mark McLoughlin
  • Mark McMahon
  • Martijn Pieters
  • Martin Fitzpatrick
  • Mats Kindahl
  • Matt Layman
  • Matthew Rocklin
  • Matthew Rollings
  • Matthew Wilson
  • Matthew Wright
  • Mattias Brändström
  • Mauveweb
  • Michael Becker
  • Michael Droettboom
  • Michael Foord
  • Michael Nelson
  • Michal Kwiatkowski
  • Michał Bultrowicz
  • Michele Simionato
  • Michy Alice
  • Mike C. Fletcher
  • Mike Driscoll
  • Mike Müller
  • Mikhail Korobov
  • Mikko Ohtamaa
  • Mirek Długosz
  • Mitchell Garnaat
  • Mitya Sirenef
  • Montreal Python User Group
  • Moshe Zadka
  • Moya Project
  • Mozilla Web Development
  • Muharem Hrnjadovic
  • Mycli
  • Nadav Samet
  • Naomi Ceder
  • Natan Zabkar
  • Ned Batchelder
  • Neil Schemenauer
  • Nick Coghlan
  • Nick Craig-Wood
  • Nick Efford
  • Nick Janetakis
  • Nicola Iarocci
  • Nicolas Dumazet
  • Nicolas Paris
  • Nigel Babu
  • Nikola
  • Nikolaos Diamantis
  • Not Invented Here
  • Nsukami Patrick
  • Obey the Testing Goat
  • Ofosos
  • Omaha Python Users Group
  • Ondřej Čertík
  • Paolo Melchiorre
  • Pathwright
  • Patrice Neff
  • Patrick Altman
  • Patrick Kennedy
  • Patrick Müller
  • Paul Everitt
  • Paul Harrison
  • Paul Redman
  • Paweł Fertyk
  • Pedro Lima
  • Peter Bengtsson
  • Peter Eisentraut
  • Peter Fankhänel
  • Peter Harkins
  • Peter Hoffmann
  • Phil Hassey
  • Philip Jenvey
  • Philipp von Weitershausen
  • Philippe Normand
  • Phillip J. Eby
  • Podcast.__init__
  • Polyglot.Ninja()
  • Possbility and Probability
  • Pradeep Gowda
  • Pranav Pandey
  • Praveen Gollakota
  • Programiz
  • Programming Ideas With Jake
  • Przemysław Kołodziejczyk
  • PyATL Bytecode
  • PyBites
  • PyCharm
  • PyCoder’s Weekly
  • PyCon
  • PyCon Ireland
  • PyPodcats
  • PyPy
  • PyTennessee
  • Python 4 Kids
  • Python 411 Podcast
  • Python Advocacy
  • Python Anywhere
  • Python Bytes
  • Python Celery - Weekly Celery Tutorials and How-tos
  • Python Circle
  • Python Data
  • Python Diary
  • Python Docs Editorial Board
  • Python Does What?!
  • Python Engineering at Microsoft
  • Python GUIs
  • Python Insider
  • Python Koans
  • Python Morsels
  • Python People
  • Python Piedmont Triad User Group
  • Python Pool
  • Python Software Foundation
  • Python Sweetness
  • Python User Groups
  • Python for Beginners
  • Python on Karan
  • Python with Myo
  • Python(x,y) News
  • PythonClub - A Brazilian collaborative blog about Python
  • PythonDebugging.com
  • Pythonicity
  • Pythonology
  • Python⇒Speed
  • Péter Szabó
  • Péter Zsoldos
  • Quansight Labs Blog
  • R David Murray
  • RMOTR
  • Ralph Bean
  • Ram Rachum
  • Randell Benavidez
  • Randle Taylor
  • Randy Zwitch
  • Raymond Hettinger
  • Read the Docs
  • Real Python
  • Red Hat Developers
  • Rene Dudfield
  • Reuven Lerner
  • Richard Tew
  • Richard Wall
  • Rickard Lindberg
  • Rishi Maker
  • Rob Galanakis
  • Robert Collins
  • Robert Zaremba
  • Robin Parmar
  • Robin Wilson
  • Rodrigo Araúj
  • Rodrigo Girão Serrão
  • RoseHosting Blog
  • Ruslan Spivak
  • Ryan Cox
  • S. Lott
  • S. R. Krishnan
  • SDJournal
  • SPE Weblog
  • STX Next
  • Salim Fadhley
  • Sandipan Dey
  • Sandro Tosi
  • Sean McGrath
  • Sebastian Pölsterl
  • Sebastian Witowski
  • Selena Deckelmann
  • Senthil Kumaran
  • Seth Michael Larson
  • Shannon -jj Behrens
  • ShiningPanda
  • Simeon Franklin
  • Simeon Visser
  • Simon
  • Simon Brunning
  • Simon Wittber
  • Simple is Better Than Complex
  • SoftFormance
  • Speed Matters
  • Spike ekipS
  • Spyder IDE
  • Stack Abuse
  • Stanislas Morbieu
  • Starzel.de
  • Stefan Behnel
  • Stefan Scherfke
  • Stefanie Molin
  • Stein Magnus Jodal
  • Stephen Ferg
  • Steve Holden
  • Steven Klass
  • Stuart Gordon Reid
  • Stéphane Wirtel
  • Sumana Harihareswara - Cogito, Ergo Sumana
  • Suresh Dasari/Tutlane.com
  • Swisscom ICT
  • Talk Python Blog
  • Talk Python to Me
  • Taylor Edmiston
  • TechBeamers Python
  • Techiediaries - Django
  • Terri Oda
  • Terry Jones
  • Test and Code
  • TestDriven.io
  • The Data Scientist
  • The Digital Cat
  • The Lunar Cowboy
  • The No Title® Tech Blog
  • The Occasional Occurrence
  • The Open Sourcerer
  • The Parcon Blog
  • The Python Coding Blog
  • The Python Coding Stack
  • The Python Papers
  • The Python Show
  • The Three of Wands
  • Thibauld Nion
  • ThisHosting.Rocks
  • Thomas Guest
  • Thomas Vander Stichele
  • Tibo Beijen
  • Tim Arnold / reachtim
  • Tim Gilbert
  • Tim Knapp
  • Tim Lesher
  • Tobias Ivarsson
  • Tom Christie
  • Tomasz Ducin
  • Tomaž Muraus
  • Tomer Filiba
  • Tony Breyal
  • Toshio Kuratomi
  • Travis Oliphant
  • Trey Hunner
  • Tryton News
  • Turnkey Linux
  • Twisted Matrix Labs
  • TypeThePipe
  • V.S. Babu
  • Varun Nischal
  • Vasudev Ram
  • Vinay Sajip
  • Vinay Sajip (Logging)
  • Vinayak Mehta
  • Virgil Dupras
  • Vladimir Perić
  • Wayne Witzel
  • Weekly Python Chat
  • Weekly Python StackOverflow Report
  • Wes Mason
  • Wesley Chun
  • Will Kahn-Greene
  • Will McGugan
  • Will Pierce
  • William Minchin
  • William Reade
  • Wing Tips
  • Wingware
  • Wyatt Baldwin
  • Yair Lenga
  • Yaniv Aknin
  • Yann Larrivée
  • Yuval Greenfield
  • Zaki Akhmad
  • Zato Blog
  • Zero to Mastery
  • Zero-with-Dot (Oleg Żero)
  • ZeroDB
  • bottlepy-dev
  • codeboje
  • death and gravity
  • eGenix.com
  • hypothesis.works articles
  • kdev-python
  • meejah.ca
  • nl-project
  • pgcli
  • py.CheckIO
  • pygame
  • pythonwise
  • qutebrowser development blog
  • saaj/recollection
  • scikit-learn
  • testmon
  • tryexceptpass
  • Éric Araujo
  • Łukasz Langa
  • To request addition or removal, open a PR or issue