Planet Python

Last update: June 01, 2026 04:44 PM UTC

June 01, 2026

Real Python

Python sleep(): How to Add Time Delays to Your Code

Sometimes you need to make Python sleep, wait, or pause before running the next line of code. Whether you’re spacing out API requests, pacing a thread, or adding a delay to terminal output, Python’s time.sleep() function is the standard tool:

Language: Python

from time import sleep
sleep(3)  # Pause execution for 3 seconds

Beyond time.sleep(), Python provides different ways to add time delays depending on the context, including threads, async code, and GUI applications.

By the end of this tutorial, you’ll understand that:

• time.sleep() suspends execution for a given number of seconds, including fractional values like milliseconds.
• Retry decorators use time.sleep() to add a delay between failed attempts.
• Event.wait() is the preferred way to add delays in threads because it can be interrupted cleanly.
• asyncio.sleep() pauses a single coroutine without blocking the rest of your async code.
• GUI frameworks like Tkinter provide scheduling methods such as .after() to avoid freezing the event loop.

The following sections cover each of these approaches with working code examples.

Get Your Code: Click here to download the free sample code you’ll use to add time delays to scripts, threads, async code, and GUI apps.

Take the Quiz: Test your knowledge with our interactive “Python time.sleep()” quiz. You’ll receive a score upon completion to help you track your learning progress:

---

Interactive Quiz

Python time.sleep()

In this quiz, you'll revisit how to add time delays to your Python programs.

Pause Execution With Python sleep()

Python has built-in support for making your program wait. The time module has a sleep() function that you can use to add a delay by suspending execution of the calling thread for the number of seconds you specify:

Language: Python

>>> import time
>>> time.sleep(3)  # Sleep for 3 seconds

Here’s a quick example of time.sleep() in action:

Language: Python Filename: coffee.py

import time

print("Brewing coffee...")
print("This would take like 3 secs...")
time.sleep(3)
print("Done! Your coffee is ready!")

If you run this script, you’ll see a three-second pause between the messages while time.sleep() suspends execution.

You can also pass fractional seconds to time.sleep() for finer-grained durations. Here are some common values:

Language: Python

import time

time.sleep(0.5)  # Wait 500 milliseconds
time.sleep(0.001)  # Wait 1 millisecond
time.sleep(1.5)  # Wait 1.5 seconds
time.sleep(60)  # Wait 1 minute

The time.sleep() function isn’t perfectly precise. The specified value acts as a minimum delay. The actual pause will almost always be slightly longer in practice due to operating system scheduler overhead and current system load.

You can test how long the sleep lasts by using Python’s timeit module:

Language: Shell

$ python -m timeit -n 3 "import time; time.sleep(3)"
3 loops, best of 5: 3 sec per loop

Here, you run the timeit module with the -n parameter, which tells timeit how many times to run the statement per repeat. With the default of five repeats, the statement runs 15 times in total (3 × 5). timeit then reports the best time across all repeats, which is three seconds per loop, as expected.

For a more realistic example, say you need to monitor whether a website is up. You want to check its status code periodically, but querying the server too often could overload it or get you rate-limited. You can use time.sleep() to space out the checks:

Language: Python Filename: uptime_bot.py

import time
import urllib.request
import urllib.error

CHECK_INTERVAL = 60  # Seconds between checks

def uptime_bot(url):
    while True:
        try:
            urllib.request.urlopen(url)
        except urllib.error.HTTPError as e:
            # Email admin or log
            print(f"HTTPError: {e.code} for {url}")
        except urllib.error.URLError as e:
            # Email admin or log
            print(f"URLError: {e.reason} for {url}")
        else:
            # Website is up
            print(f"{url} is up")
        time.sleep(CHECK_INTERVAL)

if __name__ == "__main__":
    url = "https://www.google.com/py"
    uptime_bot(url)

Read the full article at https://realpython.com/python-sleep/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

June 01, 2026 02:00 PM UTC

death and gravity

DynamoDB crash course: part 3 – design patterns

Previously

This is the last part of a series covering core DynamoDB concepts. The goal is to help you understand idiomatic usage and trade-offs in under an hour.

In the first part, I summarized DynamoDB's main proposition to its users like so:

data modeling complexity is always preferable to complexity coming from infrastructure maintenance, availability, and scalability

Today, we're looking at the design patterns that help manage this complexity, making the most of its data model and features and working around its limits.

• Composite keys
• Single table design
• GSI overloading
• Partition key sharding
• Sparse indexes
• Base table indexes
• Optimistic locking

Composite keys #

Composite (aka synthetic) keys underpin most other patterns.

The idea is simple: keys don't have to be natural attributes of your data, they can be composed of other attributes that enable specific access patterns. This works both with table and index keys.

How do you compose keys? By string concatenation, of course! Careful with numbers though, they need padding to be useful in sort keys.

Example

To sort lexicographically by more than one attribute, you group them in a sort key, e.g. {Album}#{Song}.

Or, in single table design, you distinguish between item types by prefixing keys with the type, e.g. album#{Album}.

Or, in partition key sharding, you spread the load on a GSI partition by splitting one partition key into multiple ones, e.g. {Genre}#{shard}.

But denormalization has its trade-offs. For sort key {Album}#{Song}, should Album and Song also be separate attributes? If yes, you need to ensure they never change, but you can use them in indexes (e.g. a GSI with Album as primary key). If no, items can't become inconsistent, but you need to parse the key to get them.

This was inconvenient enough that DynamoDB finally added multi-attribute keys support to GSIs in 2025 (although not inconvenient enough to also add it to tables).

See also

• Data modeling building blocks # Composite sort key
• Multi-attribute keys pattern

Single table design #

The AWS guidance is to use as few tables as possible:

As a general rule, you should maintain as few tables as possible in a DynamoDB application. [...] A single table with inverted indexes can usually enable simple queries to create and retrieve the complex hierarchical data structures required by your application.

This culminates in single table design, where you put all entities in the same table, and tell them apart based on the key format, usually using a prefix. With this pattern, one DynamoDB table corresponds to a whole relational database.

The easiest way is to put items related to a top-level entity on the same partition. The main benefit is that joins with the top-level entity become trivial. A second one is that you can sometimes get different entity types in a single query, which can be both faster and cheaper (fewer queries; small items pack into fewer capacity units).

Example

You can group items related to an Artist on the same partition, with sort keys like artist, album#{Album}, and song#{Album}#{Song}.

# table Music (partition key: Artist, sort key: sk)
Solar Fields: !btree
  'album#Leaving Home': { Genre: Electronic }
  'artist': { Variations: [ Solarfields ] }
  'song#Leaving Home#Air Song': { Duration: 741 }
  'song#Leaving Home#Monogram': { Duration: 944 }

Besides getting items of a single type, you can also get artist details and albums in a single query (sk BETWEEN "album#" AND "artist").

But choose wisely – queries can have only one sort key condition, so you can't also get album details and songs in a single query with this schema; sort keys {Album} and {Album}#{Song} would do it, at the expense of the first query.

Sometimes, it can be useful to put some sub-entities on dedicated partitions, accepting that joins will have to be done in code.

Example

In the example above, a popular artist with lots of songs can lead to:

• throttling due to partition throughput limits
• slow list songs for artist due to sequential paginated queries

Perhaps it's better to put the songs in each album on separate partitions:

• partition key artist#{Artist}, sort key artist or album#{Album}
• partition key song#{Artist}#{Album}, sort key {Song}

# table Music (partition key: pk, sort key: sk)
'artist#Solar Fields': !btree
  'album#Leaving Home': { Genre: Electronic }
  'artist': { Variations: [ Solarfields ] }
'song#Solar Fields#Leaving Home': !btree
  'Air Song': { Duration: 741 }
  'Monogram': { Duration: 944 }

This spreads the load onto multiple partitions, which should fix throttling.

The downside is that list songs for artist is now a two-step operation: first one query for the albums, then one query per album for the songs. The upside is that the per-album queries can be done in parallel, which wasn't possible before.

A consequence of this design is that you need a GSI to list items of a specific type (otherwise, you have to do a full table scan). Of note, exceeding the GSI partition throughput limit will cause write throttling on the base table; in the absence of a natural high-cardinality GSI partition key, sharding or some other composite key can help.

A final benefit of using a single table is better utilization with provisioned mode: usage gets averaged across entities and tends to be smoother, and spikes can share the same spare capacity.

See also

• NoSQL design
• Data modeling foundations # Single table design
• Relational modeling # JOIN operations
• (blog) Single-table vs. multi-table design in Amazon DynamoDB
• (unofficial) The What, Why, and When of Single-Table Design with DynamoDB

GSI overloading #

GSI overloading is just single table design for indexes – you put different values in the GSI key attributes, depending on item type. This way you can index more attributes than the 20 GSIs per table quota, and it can be cheaper too, since, like with tables, fewer indexes make better use of spare provisioned capacity.

Example

For a table that contains both artist and album items, a single GSI can be used for entirely different purposes:

• artist: partition key artist#{Country} – list artists by country
• album: partition key album#{Genre} – list albums by genre

# table Music (partition key: Artist, sort key: sk)
2 Bit Pie: !btree
  'album#2 Pie Island': { gsi1pk: 'album#Electronic' }
  'artist': { gsi1pk: 'artist#United Kingdom' }
Ishome: !btree
  'album#Confession': { gsi1pk: 'album#Electronic' }
  'artist': { gsi1pk: 'artist#Russia' }

# GSI GSI1 (partition key: gsi1pk, sort key: Artist)
'artist#United Kingdom': !btree
  2 Bit Pie: { sk: 'artist' }
'artist#Russia': !btree
  Ishome: { sk: 'artist' }
'album#Electronic': !btree
  2 Bit Pie: { sk: 'album#2 Pie Island' }
  Ishome: { sk: 'album#Confession' }

See also

• GSI overloading

Partition key sharding #

Sometimes, a partition key composed of multiple natural attributes is not enough to spread the load evenly across partitions; you can deal with this by putting items with the same natural attributes on multiple partitions.

So, what partition key should you use? One option is to use a random suffix from a known range; this allows you to list items for a natural attribute value by doing multiple queries, one for each suffix.

Example

For a table of songs, using Album as the partition key won't work, since not all songs are released on an album; Artist always has a value, but some artists have hundreds or even thousands of songs, which can lead to throttling.

Instead, we can use {Artist}#{randrange(10)} as partition key, which allows ten times as many items before we reach throughput limits. To list an artist's songs:

for shard in range(10):
    for item in dynamodb.query(f"{artist}#{shard}"):
        yield item

A downside of random suffixes is that you can't get a specific item, because you don't know what its suffix is. A better option is to calculate the suffix from an attribute that you do know, for example using its hash modulo N.

Example

With primary key {Artist}#{hash(Song) % 10)}, we can get a song like this:

def hash(s):
    return int.from_bytes(sha256(s.encode()).digest())

shard = hash(song_title) % 10
dynamodb.get_item(f"{artist}#{shard}", song_title)

A lot of times you need to list items by a low-cardinality attribute, so sharding may be even more important for GSIs.

Example

Assuming dedicated album items, you can list all the albums by putting them in a single GSI partition key called albums, but this will definitely cause throttling.

To avoid it, you can use GSI partition key album#{hash(Album} % 100} if you don't care about the order, or something like album#{Album[:2].lower()} if you do (but likely more sophistication is needed – th will be a very common album title prefix, and some album titles don't contain letters at all).

Even if throttling is not an issue (e.g. single infrequent reader), sharding allows you to query multiple partitions in parallel, which can speed up getting the entire result set.

So, how many shards should you have? That depends on the number, size, and how often you access the items, and is also a trade-off – too many shards means additional queries and latency, too few shards means you still overload the partitions sometimes.

Importantly, increasing the number of shards is non-trivial. For tables, you usually need to rebalance the items in place. For indexes, it's cleaner to move to a new index, or if you just need to list items by type, you can put all new items on new shards.

Regardless, you have to support it in code, do a backfill, and orchestrate the migration, which all become more complex if downtime and inconsistencies are not acceptable (e.g. if you expose a pagination token based on LastEvaluatedKey, you may want to support both versions during the switch).

See also

• Write sharding
• GSI sharding

Sparse indexes #

An item with missing index partition/sort key attributes won't appear in the index, and you won't pay for it. This can be used deliberately to query a subset of the items in the table, like those of a specific type or in a specific state.

Example

Assuming dedicated album items, an alternative way to list all the albums is to have a GSI with {Album} as partition key, and just scan the entire index (the primary key has to be a dedicated attribute that only albums have, so that only album items appear in the index).

Or, you can use a dedicated GSI with CoverOf as primary key to list cover songs.

See also

• Sparse indexes

Base table indexes #

In some cases, GSIs won't cut it – maybe you need a strongly consistent index, or need to model a many-to-one relationship (indexes map one item in the base table to one item in the index).

Instead, you can maintain an index in the base table by having additional index items associated with the main item; to guarantee atomic updates, use transactions. You then go from the main item to the index items via a main item attribute, and from the index items to the main item via their partition key.

Example

Songs have different identifiers in external systems, such as ISRC, ISWC, or MBID. To query songs by multiple external ids, you'd structure your database like this:

• song
  • partition key song#{Artist}#{Album}
  • sort key {Song}
  • external_{type}: id, ...
• external ids
  • partition key external#{type}#{id}
  • sort key song#{Artist}#{Album}#{Song}

(Alternatively, you could have one sparse index per external id type, but then you lose strong consistency, and risk running out of GSIs).

Note that modeling one-to-many relationships isn't this involved, since it fits neatly into the related-items-same-partition variant of single table design.

See also

• Working with item collections (modeling one-to-many relationships)
• Many-to-many relationships

Optimistic locking #

Optimistic locking is a concurrency control method useful when conflicts are rare, so instead of acquiring a lock to do changes, you check if someone else changed the data right before commiting, as part of an atomic operation.

In DynamoDB, that operation is a conditional write; items get an integer version attribute, and every time you want to update an item, you:

1. read the item, including the version
2. increment the version and modify the item
3. update the item, using a condition expression to ensure the version matches
   1. if successful, you're done
   2. else, start over from the beginning

You can also do this in transactions to update groups of related items, like in the base table index pattern above, with only the main item needing a version.

The upside of optimistic locking is that it is faster on average, since updates usually succeed on the first try; for fewer conflicts, use strongly consistent reads.

The downside is that it requires explicit support – it must be possible to start over from the beginning, which complicates logic, especially if you need to interact with other systems besides updating the item (e.g. to send a notification).

See also

• Implementing version control via optimistic locking (Python example)
• Optimistic locking with version number (Java example)

Anyway, that's it for now.

See also

For mode details and examples, check out the official documentation:

• Data modeling
• Data modeling schemas (worked examples)

Learned something new today? Share it with others, it really helps! PyCoder's Weekly HN Bluesky linkedin Twitter

Want to know when new articles come out? Subscribe here to get new stuff straight to your inbox!

June 01, 2026 01:43 PM UTC

Real Python

Quiz: Regular Expressions: Regexes in Python (Part 1)

In this quiz, you’ll test your understanding of Regular Expressions: Regexes in Python (Part 1).

By working through this quiz, you’ll revisit how to use the re module to search for patterns, build character classes and anchors, group and capture substrings, and apply flags like re.IGNORECASE to control matching behavior.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

June 01, 2026 12:00 PM UTC

Speed Matters

Scandir Rs

layout: post title: scandir-rs tagline: Blazing-fast directory traversal for Python — up to 70× faster than os.walk. date: 2026-06-01 08:40:00 +0100 categories: posts —————–

scandir-rs: High-Performance Directory Traversal for Python

File system traversal is often a hidden bottleneck.

Whether you’re indexing files, collecting statistics, searching large directory trees, or building developer tools, performance matters. That’s why I created scandir-rs: a Rust-powered Python library designed to be a drop-in replacement for os.walk() and os.scandir(), while delivering dramatically better performance and additional functionality.

A new version (2.9.9) is available with following changes compared to the version I’ve introduced here the last time (2.7.1):

• Add support for Python 3.13 and 3.14
• Add support for MacOS on ARM64
• Add support for Linux on PPC64 and S390
• Add SSE builds (for older CPUs which have no AVX) for Linux and Windows
• Add optional argument follow_links.
• Changed skip_hidden to false by default

Why scandir-rs?

Because speed matters…

🚀 Significant Performance Improvements

Compared to Python’s built-in implementations:

• Walk is up to 13× faster on Linux
• Walk is up to 70× faster on Windows
• Scandir is up to 6.5× faster on both Linux and Windows

When processing millions of files, these speedups can turn minutes into seconds.

Benchmarks results for running scandir in linux-5.9 folder

🔍 Richer Metadata

Beyond the standard os.walk() and os.scandir() APIs, scandir-rs can return:

• Extended file metadata
• Hardlink detection
• Additional file type classification
• Error collection without interrupting traversal

⚡ Background Processing

Long-running scans can run asynchronously in the background, allowing your application to process results while scanning is still in progress.

Installation

pip install scandir-rs

Usage Examples

Directory Statistics

Get fast statistics for an entire directory tree:

import scandir_rs as scandir

print(scandir.Count("/usr").collect())

Extended Statistics

Include additional metadata and hardlink detection:

import scandir_rs as scandir

print(
    scandir.Count(
        "/usr",
        return_type=scandir.ReturnType.Ext
    ).collect()
)

Background Scanning

Process results while scanning continues in the background:

import scandir_rs as scandir

counter = scandir.Count("/usr")

with counter:
    while counter.busy:
        results = counter.results()
        # Process intermediate results

# Final results as JSON
results = counter.to_json()

Faster os.walk()

A familiar interface with significantly better performance:

import scandir_rs as scandir

for root, dirs, files in scandir.Walk("/usr"):
    # Process files

Extended Walk Information

Retrieve additional file categories and error information:

import scandir_rs as scandir

for root, dirs, files, symlinks, other, errors in scandir.Walk(
    "/usr",
    return_type=scandir.ReturnType.Ext
):
    # Process files

On Unix systems, other includes special file types such as pipes and devices.

Faster os.scandir()

Collect all entries at once:

import scandir_rs as scandir

entries, errors = scandir.Scandir("/usr").collect()

Or iterate lazily:

import scandir_rs as scandir

for entry in scandir.Scandir("/usr"):
    # Process entry

Extended Metadata

Request detailed information for each directory entry:

import scandir_rs as scandir

for entry in scandir.Scandir(
    "/usr",
    return_type=scandir.ReturnType.Ext
):
    # Process entry

Entries are returned as DirEntryExt objects. Errors are reported as tuples containing:

(relative_path, error_message)

allowing scans to continue even when individual files cannot be accessed.

Benchmark Results

Walk Performance

Scandir Performance

For detailed benchmark data and methodology, see the benchmark documentation:

https://github.com/brmmm3/scandir-rs/blob/master/pyscandir/doc/benchmarks.md

Get Started

If your application spends time traversing large directory trees, scandir-rs can provide substantial performance improvements with minimal code changes.

The API is intentionally familiar, making migration from os.walk() and os.scandir() straightforward while unlocking additional capabilities and significantly faster execution.

Source code, documentation, and issue tracker:

https://github.com/brmmm3/scandir-rs

Licensed under the MIT License.

June 01, 2026 12:00 AM UTC

Stéphane Wirtel

PyCon Ireland 2026: The Call for Proposals is Open

![[pycon-ireland-2026-cfp-banner.png]]

TL;DR

PyCon Ireland 2026 takes place on 17 October at Trinity College Dublin. The Call for Proposals is open until 30 August. Two tracks get special focus this year: Python security and AI with Python. First-time speakers are welcome. Financial aid up to €350 is available. Submit at 2026.pycon.ie/cfp.

I’m part of the team organising PyCon Ireland 2026, and the Call for Proposals opened on 25 May. If you’ve been carrying a Python idea around (something you built, broke, learned, or want to share), now is the time to write it up.

June 01, 2026 12:00 AM UTC

Bob Belderbos

AI Human-in-the-loop: News Digest Triage Telegram Bot

In my trend digest article I shared a quick tool to keep on top of tech trends, but it's a one-way street: the model gives information, but I still have to decide what to do with it. Let's build the second half: a Telegram bot that shows me each story, guesses a tag, and lets me confirm or overrule it with one tap.

Human-in-the-loop (HITL): the model proposes, you decide

AI makes suggestions but it can hallucinate, so it's important to have a human in the loop to catch mistakes. The model does the work of categorizing, the human makes the final decision. This is a good example of the control layer above the model and it's where you can make AI more reliable.

This is what we teach in week 4 of our Agentic AI cohort where things come together: expense parsing, AI category suggestion, and the human in the loop to confirm it. This requires the bot to keep state, route responses, and a way to be wrong gracefully. Below is a smaller version so you can get a taste for how this works.

We'll build it in seven steps. Grab the full script up front, or follow along piece by piece.

Step 1: create the bot and get a token

Telegram bots are created by another bot. Open Telegram and search for @BotFather (it has a blue checkmark):

1. Send /newbot.
2. Give it a display name (anything).
3. Give it a username ending in bot that is globally unique, e.g. alice_trend_bot.

BotFather replies with a token like 123456789:ABCdef.... Treat it like a password. Put it in a .env file next to your script (or export in your shell), together with your OpenAI key:

TELEGRAM_BOT_TOKEN=123456789:ABCdef...
OPENAI_API_KEY=sk-...

If the token ever leaks, send /revoke to BotFather for a fresh one.

Step 2: the dependencies

The whole thing is one file. I use a PEP 723 header so uv run resolves everything into its own environment, no virtualenv to manage. Put this at the top of trend_triage_bot.py:

# /// script
# requires-python = ">=3.12"
# dependencies = [
#   "python-telegram-bot>=21",
#   "openai>=1.40",
#   "httpx",
#   "python-decouple",
#   "pydantic",
# ]
# ///

If you would rather build this inside an existing project, the equivalent is:

uv init && uv add python-telegram-bot openai httpx python-decouple pydantic

Then the imports and a few constants:

import json
import logging
from pathlib import Path
from typing import Literal, Protocol

import httpx
from decouple import config
from openai import AsyncOpenAI
from pydantic import BaseModel
from telegram import InlineKeyboardButton, InlineKeyboardMarkup, Message, Update
from telegram.ext import (
    Application,
    CallbackQueryHandler,
    CommandHandler,
    ContextTypes,
)

logger = logging.getLogger(__name__)

TAGS = ["read", "lib", "tool", "skip"]
DEFAULT_TOPIC = "rust"
READING_LIST = Path("reading_list.jsonl")
LOBSTERS_FEED = "https://lobste.rs/t/{tag}.json"

Step 3: fetch the stories

Lobsters has a per-tag JSON feed, no auth required: https://lobste.rs/t/rust.json returns the latest Rust-tagged stories, .../t/python.json the Python ones, and so on. It's a tighter, more engineering-focused signal than a broad keyword search, and parameterizing the tag is what lets /digest rust and /digest python hit the same code. A Story is just a title and a URL.

Let's set up the model and fetch the latest five stories for a given tag:

from pydantic import BaseModel, HttpUrl

class Story(BaseModel):
    title: str
    url: HttpUrl


async def fetch_stories(tag: str, *, limit: int = 5) -> list[Story]:
    async with httpx.AsyncClient(
        timeout=10, headers={"User-Agent": "trend-triage-bot"}
    ) as http:
        response = await http.get(LOBSTERS_FEED.format(tag=tag))
        response.raise_for_status()
        return [
            Story(
                title=story["title"],
                url=story["url"] or f"https://lobste.rs/s/{story['short_id']}",
            )
            for story in response.json()[:limit]
            if story.get("title")
        ]

Two small details: Lobsters expects a User-Agent header, and a text/discussion post has an empty url, so we fall back to its comments page (/s/{short_id}), the same pattern you'd use for an HN self-post.

The * in the function signature makes the limit keyword-only, so you have to call fetch_stories("rust", limit=10), which is a nice safeguard against accidentally changing the default.

Step 4: let the LLM propose a tag

The model picks one of TAGS. As the digest topic is variable (/digest rust, /digest python), the tags have to be topic-agnostic, so they describe what a story is (read / lib / tool), not anything Rust- or Python-specific.

Content-type beats intent here: "is this a tool or a library" is answerable from a headline, whereas "will I read this or build with it" depends on me, not the title. And a tag the model can't infer is a tag you end up correcting every time.

And using structured outputs I get typed values back, not strings I have to parse and second-guess; consistent data types are the foundation of reliable AI.

SYSTEM = (
    "Tag this software/tech headline with one of: "
    "read (an article, post, or tutorial), "
    "lib (a library, framework, or package you import), "
    "tool (a CLI, app, or utility you run). "
    "Use 'skip' only if it is off-topic or clickbait."
)


class TagChoice(BaseModel):
    tag: Literal["read", "lib", "tool", "skip"]


class Classifier(Protocol):
    async def tag(self, story: Story) -> str: ...


class OpenAIClassifier:
    def __init__(self, api_key: str, model: str = "gpt-4o-mini") -> None:
        self._client = AsyncOpenAI(api_key=api_key)
        self._model = model

    async def tag(self, story: Story) -> str:
        completion = await self._client.beta.chat.completions.parse(
            model=self._model,
            messages=[
                {"role": "system", "content": SYSTEM},
                {"role": "user", "content": story.title},
            ],
            response_format=TagChoice,
        )
        choice = completion.choices[0].message.parsed
        return choice.tag if choice else "skip"


def _build_classifier() -> Classifier:
    return OpenAIClassifier(config("OPENAI_API_KEY"))

_build_classifier() constructs the client at runtime, not at import; we call it once in main() and stash the result (more on that in step 7).

This decoupling allows a test to inject a fake tagger without touching OpenAI. It's the same lazy-wiring trick I used demonstrating the repository pattern. The Protocol means any class with an async tag() method drops in. Protocols are more flexible here, because they don't require inheritance like ABCs do, so the test double doesn't have to know about the real classifier at all.

Filing a tagged story is a one-liner to a JSONL file. JSONL (or JSON Lines) is a way to store structured data; each line contains a single, valid JSON object.

def save_to_reading_list(story: Story, tag: str) -> None:
    with READING_LIST.open("a") as f:
        f.write(json.dumps({"tag": tag, **story.model_dump(mode="json")}) + "\n")

Note that model_dump() hands back a pydantic Url object (HttpUrl) that json.dumps can't serialize; mode="json" coerces it to a string first.

Step 5: the keyboard that highlights the guess

The AI's pick is prefixed with >>, but every other tag is one tap away. The callback_data stays plain (tag:read) so the handler never has to strip the decoration:

def triage_keyboard(suggested: str) -> InlineKeyboardMarkup:
    buttons = [
        InlineKeyboardButton(
            f">> {tag}" if tag == suggested else tag,
            callback_data=f"tag:{tag}",
        )
        for tag in TAGS
    ]
    rows = [buttons[i : i + 3] for i in range(0, len(buttons), 3)]
    return InlineKeyboardMarkup(rows)

The marker goes in front, and that detail matters: Telegram clips long button labels from the end, so my first attempt, wrapping the tag in >> tool <<, showed up as >> tool… with the closing marker eaten. The kind of bug you only catch by testing it on a real phone.

> marker" />

Step 6: two steps, one stashed queue

A simple bot is stateless: message in, reply out. This one is not. Step one (the /digest command) fetches and shows the first story; step two fires later, when I tap a button, and needs the queue from step one. context.user_data is a per-user dict the library keeps between handler calls, so I park the queue there:

async def start_digest(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
    if update.message is None:
        return
    topic = context.args[0].lower() if context.args else DEFAULT_TOPIC
    await update.message.reply_text(f"Fetching today's {topic} stories...")
    try:
        context.user_data["queue"] = await fetch_stories(topic)
    except httpx.HTTPStatusError:
        await update.message.reply_text(
            f"No feed for '{topic}'. Try a Lobsters tag like rust, python, or go."
        )
        return
    except httpx.RequestError:
        await update.message.reply_text(
            "Couldn't reach Lobsters right now — try again in a bit."
        )
        return
    await show_next(update.message, context)


async def show_next(message: Message, context: ContextTypes.DEFAULT_TYPE) -> None:
    queue: list[Story] = context.user_data.get("queue", [])
    if not queue:
        await message.reply_text("Inbox zero. That's all the trends today.")
        return
    story = queue[0]
    suggested = await context.bot_data["classifier"].tag(story)
    await message.reply_text(
        f"{story.title}\n{story.url}",
        reply_markup=triage_keyboard(suggested),
    )

context.args is whatever followed the command: /digest python gives ["python"], a bare /digest gives [] and falls back to DEFAULT_TOPIC.

A typo'd topic is a 404 from Lobsters, so I catch HTTPStatusError and reply with a helpful message, otherwise the user would just stare at a digest that never arrives. Validate at the boundary where untrusted input enters.

The second except covers the other failure mode: the request never gets an HTTP response at all. HTTPStatusError only fires once Lobsters answers with a 4xx/5xx — a connect timeout, read timeout, or DNS failure is an httpx.RequestError, which is a sibling of HTTPStatusError, not a subclass. Miss it and a flaky network crashes the handler with a traceback instead of a friendly reply. Catching RequestError covers every transport-level failure (ConnectTimeout, ReadTimeout, ConnectError) in one branch.

Read user_data with .get(...), never [...]. It lives in memory, so if the bot restarts mid-flow the dict is empty and you want a graceful reply, not a KeyError.

context.bot_data is its per-bot sibling: one dict shared across all users. That makes it the right home for the classifier, which holds no per-user state. We build it once in step 7 and read it back here, so every story reuses the same OpenAI client instead of constructing a fresh one each time.

Step 7: the callback, then wire it up

When I tap a button Telegram sends a callback query, not a message. Three rules keep it sane, numbered in the code:

async def on_tag(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
    query = update.callback_query
    if query is None or query.data is None:
        return
    await query.answer()  # 1. stop the spinner, first thing
    _, tag = query.data.split(":", 1)  # 2. "tag:read" -> "read"
    queue = context.user_data.get("queue", [])
    if not queue:  # stale button after a restart
        await query.edit_message_text("Session expired, send /digest again.")
        return
    story = queue.pop(0)
    if tag != "skip":
        save_to_reading_list(story, tag)  # the human's final say
    await query.edit_message_text(  # 3. edit, don't reply
        f"Filed under {tag}: {story.title}"
        if tag != "skip"
        else f"Skipped: {story.title}"
    )
    await show_next(query.message, context)

Call await query.answer() first or the loading spinner on the button never stops, even when everything else works. Edit the original message instead of replying, or the dead keyboard sits there inviting a second tap on a story you already filed. The same .get(...)-not-[...] rule applies here: an old keyboard from before a restart can still send a tap, and you want a "send /digest again" nudge, not a KeyError.

Routing is by prefix. The pattern="^tag:" is why a future second keyboard (say setcurrency:EUR) would not trip this handler:

async def on_error(update: object, context: ContextTypes.DEFAULT_TYPE) -> None:
    logger.exception("Handler failed", exc_info=context.error)


def main() -> None:
    logging.basicConfig(
        format="%(asctime)s %(name)s %(levelname)s %(message)s",
        level=logging.INFO,
    )
    logger.info("Starting trend triage bot, polling for updates")
    app = Application.builder().token(config("TELEGRAM_BOT_TOKEN")).build()
    app.bot_data["classifier"] = _build_classifier()
    app.add_handler(CommandHandler("digest", start_digest))
    app.add_handler(CallbackQueryHandler(on_tag, pattern="^tag:"))
    app.add_error_handler(on_error)
    app.run_polling()


if __name__ == "__main__":
    main()

Three things to notice here in main():

• logging.basicConfig(...) turns on output: run_polling() blocks silently otherwise, so without it a freshly started bot looks dead in the terminal even though it's happily polling.

• app.bot_data["classifier"] builds the OpenAI client once instead of per story.

• add_error_handler means a network blip or a rate limit gets logged through on_error rather than vanishing into the framework.

Run it

$ export OPENAI_API_KEY=sk-proj-...
$ export TELEGRAM_BOT_TOKEN=...
$ uv run trend_triage_bot.py
2026-06-01 13:19:42,866 __main__ INFO Starting trend triage bot, polling for updates
2026-06-01 13:19:43,190 httpx INFO HTTP Request: POST https://api.telegram.org/bot.../getMe "HTTP/1.1 200 OK"
2026-06-01 13:19:43,242 httpx INFO HTTP Request: POST https://api.telegram.org/bot.../deleteWebhook "HTTP/1.1 200 OK"
2026-06-01 13:19:43,244 telegram.ext.Application INFO Application started

Open your bot in Telegram and send /digest for the default topic, or use a tag like /digest python, /digest rust or any Lobsters tag.

The bot walks you through today's stories one at a time. Tap the highlighted tag to accept the model's guess, or any other tag to overrule it, until you hit inbox zero:

Same bot, any topic: finish the Rust queue, then /digest python and triage that, no code change:

Filed stories land in reading_list.jsonl:

{"tag": "read", "title": "One year of Roto, the compiled scripting language for Rust", "url": "https://blog.nlnetlabs.nl/one-year-of-roto-the-compiled-scripting-language-for-rust/"}
{"tag": "lib", "title": "Announcing Rust 1.96.0", "url": "https://blog.rust-lang.org/2026/05/28/Rust-1.96.0/"}
{"tag": "read", "title": "What kache actually caches", "url": "https://kunobi.ninja/blog/what-kache-actually-caches"}
{"tag": "read", "title": "Creusot helps you prove your Rust code is correct", "url": "https://github.com/creusot-rs/creusot/tree/master"}
{"tag": "tool", "title": "uv must be installed to build a standalone Python distribution", "url": "https://github.com/astral-sh/python-build-standalone/commit/c9c40c56eb53136587f0a32382cad9e5cd8d184a"}
{"tag": "tool", "title": "SPy: an interpreter and a compiler for a statically typed variant of Python", "url": "https://github.com/spylang/spy"}
{"tag": "read", "title": "Opaque Types in Python", "url": "https://blog.glyph.im/2026/05/opaque-types-in-python.html"}
{"tag": "read", "title": "uv is fantastic, but its package management UX is a mess", "url": "https://www.loopwerk.io/articles/2026/uv-ux-mess/"}

That file is the actual output; one JSON object per line, ready to feed into whatever reads it next. The whole script is in this gist.

The interesting question is not whether the model can tag a headline. It's pretty accurate, but it can get it wrong, and that's where you want to have a human in the loop. This has been a simple example to show the flow, but real workflows might involve more interesting things like approving trades, triaging support tickets, or moderating content. The model can do the heavy lifting of making a guess, but the human gets the final say, and that's where the value is.

Keep reading

• Build a daily AI digest in 200 lines of Python
• The control layer is the product, not the model
• How an AI expense agent is actually structured

June 01, 2026 12:00 AM UTC

Brian Okken

Markdown to PDF with pandoc and typst

I use a lot of markdown

I like writing in markdown.
So much so that I use it for everything.

• Work documents
• Personal documents
• Books
• Blog posts

Converting to pdf has been difficult

However, lots of those things are more easily shared with PDF. Converting has been a pain in the past.

Now it’s easy

Not anymore. And not for a while, but I just learned about this process.

Tools needed

June 01, 2026 12:00 AM UTC

May 30, 2026

Kay Hayen

Nuitka Release 4.1

This is to inform you about the new stable release of Nuitka. It is the extremely compatible Python compiler, “download now”.

This release adds many new features and corrections with a focus on async code compatibility, missing generics features, and Python 3.14 compatibility and Python compilation scalability yet again.

Bug Fixes

• Python 3.14: Fix, decorators were breaking when disabling deferred annotations. (Fixed in 4.0.1 already.)

• Fix, nested loops could have wrong traces lead to mis-optimization. (Fixed in 4.0.1 already.)

• Plugins: Fix, run-time check of package configuration was incorrect. (Fixed in 4.0.1 already.)

• Compatibility: Fix, __builtins__ lacked necessary compatibility in compiled functions. (Fixed in 4.0.1 already.)

• Distutils: Fix, incorrect UTF-8 decoding was used for TOML input file parsing. (Fixed in 4.0.1 already.)

• Fix, multiple hard value assignments could cause compile time crashes. (Fixed in 4.0.1 already.)

• Fix, string concatenation was not properly annotating exception exits. (Fixed in 4.0.2 already.)

• Windows: Fix, --verbose-output and --show-modules-output did not work with forward slashes. (Fixed in 4.0.2 already.)

• Python 3.14: Fix, there were various compatibility issues including dictionary watchers and inline values. (Fixed in 4.0.2 already.)

• Python 3.14: Fix, stack pointer initialization to localsplus was incorrect to avoid garbage collection issues. (Fixed in 4.0.2 already.)

• Python 3.12+: Fix, generic type variable scoping in classes was incorrect. (Fixed in 4.0.2 already.)

• Python 3.12+: Fix, there were various issues with function generics. (Fixed in 4.0.2 already.)

• Python 3.8+: Fix, names in named expressions were not mangled. (Fixed in 4.0.2 already.)

• Plugins: Fix, module checksums were not robust against quoting style of module-name entry in YAML configurations. (Fixed in 4.0.2 already.)

• Plugins: Fix, doing imports in queried expressions caused corruption. (Fixed in 4.0.2 already.)

• UI: Fix, support for uv_build in the --project option was broken. (Fixed in 4.0.2 already.)

• Compatibility: Fix, names assigned in assignment expressions were not mangled. (Fixed in 4.0.2 already.)

• Python 3.12+: Fix, there were still various issues with function generics. (Fixed in 4.0.3 already.)

• Clang: Fix, debug mode was disabled for clang generally, but only ClangCL and macOS Clang didn’t want it. (Fixed in 4.0.3 already.)

• Zig: Fix, --windows-console-mode=attach|disable was not working when using Zig. (Fixed in 4.0.3 already.)

• macOS: Fix, yet another way self dependencies can look like, needed to have support added. (Fixed in 4.0.3 already.)

• Python 3.12+: Fix, generic types in classes had bugs with multiple type variables. (Fixed in 4.0.3 already.)

• Scons: Fix, repeated builds were not producing binary identical results. (Fixed in 4.0.3 already.)

• Scons: Fix, compiling with newer Python versions did not fall back to Zig when the developer prompt MSVC was unusable, and error reporting could crash. (Fixed in 4.0.4 already.)

• Zig: Fix, the workaround for Windows console mode attach or disable was incorrectly applied on non-Windows platforms. (Fixed in 4.0.4 already.)

• Standalone: Fix, linking with Python Build Standalone failed because libHacl_Hash_SHA2 was not filtered out unconditionally. (Fixed in 4.0.4 already.)

• Python 3.6+: Fix, exceptions like CancelledError thrown into an async generator awaiting an inner awaitable could be swallowed, causing crashes. (Fixed in 4.0.4 already.)

• Fix, not all ordered set modules accepted generators for update. (Fixed in 4.0.5 already.)

• Plugins: Disabled warning about rebuilding the pytokens extension module. (Fixed in 4.0.5 already.)

• Standalone: Filtered libHacl_Hash_SHA2 from link libs unconditionally. (Fixed in 4.0.5 already.)

• Debugging: Disabled unusable unicode consistency checks for Python versions 3.4 to 3.6. (Fixed in 4.0.5 already.)

• Python3.12+ Avoided cloning call nodes on class level which caused issues with generic functions in combination with decorators. (Added in 4.0.5 already.)

• Python 3.12+: Added support for generic type variables in async def functions. (Added in 4.0.5 already.)

• UI: Fix, flushing outputs for prompts was not working in all cases when progress bars were enabled. (Fixed in 4.0.6 already.)

• UI: Fix, unused variable warnings were missing at C compile time when using zig as a C compiler. (Fixed in 4.0.6 already.)

• Scons: Fix, forced stdout and stderr paths as a feature was broken. (Fixed in 4.0.6 already.)

• Fix, replacing a branch did not accurately track shared active variables causing optimization crashes. (Fixed in 4.0.7 already.)

• macOS: Fix, failed to remove extended attributes because files need to be made writable first. (Fixed in 4.0.7 already.)

• Fix, dict pop and setdefault using with := rewrites lacked exception-exit annotations for un-hashable keys. (Fixed in 4.0.8 already.)

• Python 3.13: Fix, the __parameters__ attribute of generic classes was not working. (Fixed in 4.0.8 already.)

• Python 3.11+: Fix, starred arguments were not working as type variables. (Fixed in 4.0.8 already.)

• Python2: Fix, FileNotFoundError compatibility fallback handling was not working properly. (Fixed in 4.0.8 already.)

• Compatibility: Fix, loop ownership check in value traces was missing, causing issues with nested loops.

• Windows: Improved --windows-console-mode=attach to properly handle console handles, enabling cases like os.system to work nicely.

• Python2: Fix, there was a compatibility issue where providing default values to the mkdtemp function was failing.

• Windows: Fix, there were spurious issues with C23 embedding in 32-bit MinGW64 by switching to coff_obj resource mode for it as well.

• Plugins: Fix, the post-import-code execution could fail because the triggering sub-package was not yet available in sys.modules.

• UI: Fix, listing package DLLs with --list-package-dlls was broken due to recent plugin lifecycle changes.

• UI: Fix, --list-package-exe was not working properly on non-Windows platforms failing to detect executable files correctly.

• UI: Handled paths starting with {PROGRAM_DIR} the same as a relative path when parsing the --onefile-tempdir-spec option.

• Plugins: Followed multiprocessing forkserver changes for newer Python versions.

• Python 3.12+: Fix, generic class type parameters handling was incorrect.

• Python 3.12: Fix, deferred evaluation of type aliases was failing.

• Python 3.12+: Aligned sum built-in float summation with CPython’s compensated sum for better accuracy.

• Python 3.10+: Fix, uncompiled coroutine throw() return handling was incorrect, restoring completed coroutine results via StopIteration.value rather than exposing them as ordinary return values to the outer await chain.

• Python 3.13+: Fix, uncompiled coroutine cancel()/await suspension handling was incorrect, improved to ensure integration compatibility.

• macOS: Made finding create-dmg more robustly by also checking the Homebrew path for Intel and from PATH properly.

• Compatibility: Fix, class frames were not exposing frame locals.

• UI: Detected static-libpython problems, which affected some forms of Anaconda.

• Distutils: Rejected --project mixed with --main arguments as it is not useful.

• macOS: Fix, zig from PATH or from ziglang was not being used.

• Distutils: Fix, the wrong module-root config value was being checked for uv build backend.

• macOS: Fix, was attempting to change removed (rejected) DLLs, which of course failed and errored out.

• Python 3.14: Fix, tuple reuse was not fully compatible, potentially causing crashes due to outdated hash caches.

• Fix, fake modules were still being attempted to located when imported by other code, which could conflict with existing modules.

• Python 3.5+: Fix, failed to send uncompiled coroutines the sent in value in yield from.

• Fix, older gcc compilers lacking newer intrinsic methods had compilation issues that needed to be addressed.

• Standalone: Fix, multiphase module extension modules with post-load code were not working properly.

• Fix, Avoid using the non-inline copy of pkg_resources with the inline copy of Jinja2. These could mismatch and cause errors.

• Fix, loops could make releasing of previous values very unclear, causing optimization errors.

• Fix, incbin resource mode was not working with old gcc C++ fallback.

• Python 3.4 to 3.6: Fix, bytecode demotion was not working properly for these versions, also bytecode only files not working.

• Plugins: Added a check for the broken patchelf versions 0.10 and 0.11 to prevent breaking Qt plugins.

• Android: Allowed patchelf version 0.18 on Android.

• Windows: Fix, the header path for self uninstalled Python was not detected correctly.

• Release: Fix, inclusion of the pkg_resources inline copy for Python 2 to source distributions was missing.

• UI: Detected the OBS versions of SUSE Linux better.

• Suse: Allowed using patchelf 0.18.0 there too.

• Python 3.11: Fix, package and module dicts were not aligned close enough to avoid a CPython bug.

• Fix, unbound compiled methods could crash when called without an object passed.

• Standalone: Fix, multiphase module extension modules with postload. (Fixed in 4.0.8 already.)

• Onefile: Fix, while waiting for the child, it may already be terminated.

• macOS: Removed existing absolute rpaths for Homebrew and MacPorts.

• Python 3.14: Avoided warning in CPython headers.

• Python 3.14: Followed allocator changes more closely.

• Compatibility: Avoided using pkg_resources for Jinja2 template location for loading.

• No-GIL: Applied some bug fixes to get basic things to work.

Package Support

• Standalone: Add support for newer paddle version. (Added in 4.0.1 already.)

• Standalone: Add workaround for refcount checks of pandas. (Fixed in 4.0.1 already.)

• Standalone: Add support for newer h5py version. (Added in 4.0.2 already.)

• Standalone: Add support for newer scipy package. (Added in 4.0.2 already.)

• Plugins: Revert accidental os.getenv over os.environ.get changes in anti-bloat configurations that stopped them from working. Affected packages are networkx, persistent, and tensorflow. (Fixed in 4.0.5 already.)

• Standalone: Added missing DLLs for openvino. (Added in 4.0.7 already.)

• Enhanced the package configuration YAML schema by adding the relative_to parameter for from_filenames DLL specification, avoiding error-prone purely relative paths.

• Standalone: Fix, flet_desktop app assets were missing, now preserving the packaged runtime and sidecar DLLs.

• Standalone: Added support for the tyro package.

• Standalone: Added data files for the perfetto package.

• Standalone: Added support for anyio process forking.

• Standalone: Added support for the plotly.graph package.

• Anaconda: Fix, dependencies for the numpy conda package on Windows were incorrect.

• Plugins: Enhanced the auto-icon hack in PySide6 to use compatible class names.

• Standalone: Fix, Qt libraries were duplicated with PySide6 WebEngine framework support on macOS.

• Plugins: Fix, automatic detection of mypyc runtime dependencies was including all top level modules of the containing package by accident. (Fixed in 4.0.5 already.)

• Anaconda: Fix, delvewheel plugin was not working with Python 3.8+. This enhances compatibility with installed PyPI packages that use it for their DLLs. (Fixed in 4.0.6 already.)

• Plugins: Fix, our protection workaround could confuse methods used with PySide6.

New Features

• UI: Added the --recommended-python-version option to display recommended Python versions for supported, working, or commercial usage.

• UI: Add message to inform users about Nuitka[onefile] if compression is not installed. (Added in 4.0.1 already.)

• UI: Add support for uv_build in the --project option. (Added in 4.0.1 already.)

• Onefile: Allow extra includes as well. (Added in 4.0.2 already.)

• UI: Add nuitka-project-set feature to define project variables, checking for collisions with reserved runtime variables. (Added in 4.0.2 already.)

• Scons: Added new option to select --reproducible builds or not. (Added in 4.0.6 already.)

• Python 3.10+: Added support for importlib.metadata.package_distributions(). (Added in 4.0.8 already.)

• Plugins: Added support for the multiprocessing forkserver context. (Added in 4.0.8 already, for 4.1 Python 3.6 and earlier, as well as 3.14 support were added too.)

• Reports: Added structured resource usage (rusage) performance information to compilation reports.

• Reports: Included individual module-level C compiler caching (ccache/clcache) statistics in compilation reports.

• Added support for detecting and correctly resolving the Python prefix for the PyEnv on Homebrew Python flavor.

• macOS: Added support for rusage information for Scons.

• UI: Added the __compiled__.extension_filename attribute to give the real filename of the containing extension module.

• Windows: Added support for --clang or ARM. (Added in 4.0.8 already.)

• Windows: Added support for resources names as not just integers, important when we copy them from template files.

• MacPorts: Added basic support for this Python flavor. More work will be needed to get it to work fully though.

Optimization

• Avoid including importlib._bootstrap and importlib._bootstrap_external. (Added in 4.0.1 already.)

• Linux: Cached the syscall used for time keeping during compilation to avoid loading libc for each trace. (Added in 4.0.8 already.)

• UI: Output a warning for modules that remain unfinished after the third optimization pass.

• Added an extra micro pass trigger when new variables are introduced or variable usage changes severely, ensuring optimizations are fully propagated, avoiding unnecessary extra full passes.

• Provided scripts to compile Python statically with PGO tailored for Nuitka on Linux, Windows, and macOS.

• Added support for running the Data Composer tool from a compiled Nuitka binary without spawning an uncompiled Python process.

• Enhanced the usage of vectorcall for PyCFunction objects by directly checking for its presence instead of relying purely on flags, allowing more frequent use of this faster execution path.

• Cached frequently used declarations for top-level variables to speed up C code generation.

• Sped up trace collection merging by avoiding unnecessary set creation and using a set instead of a list for escaped traces.

• Optimized plugin hook execution by tracking overloaded methods and added an option to show plugin usage statistics.

• Improved performance of module location by avoiding unnecessary module name reconstruction and redundant filesystem checks for pre-loaded packages.

• Improved the caching of distribution name lookups to effectively avoid repeated IO operations across all package types.

• Plugins: Cached callback plugin dispatch for onFunctionBodyParsing and onClassBodyParsing to skip argument computation when no plugin overrides them.

• Python 3.13: Handled sub-packages of pathlib as hard modules.

• Handled hard attributes through merge traces as well.

• Made constant blobs more compact by avoiding repeated identifiers and unnecessary fields.

• Enhanced Python compilation scripts further. (Fixed in 4.0.8 already.)

• Recognized late incomplete variables better. (Fixed in 4.0.8 already.)

• Made constant blobs more compact. (Fixed in 4.0.8 already.)

• Optimized calls with only constant keywords and variable posargs too.

Anti-Bloat

• Fix, memory bloat occurred when C compiling sqlalchemy. (Fixed in 4.0.2 already.)

• Avoid using pydoc in PySimpleGUI. (Added in 4.0.2 already.)

• Avoided using doctest from zodbpickle. (Added in 4.0.5 already.)

• Avoided inclusion of cython when using pyav. (Added in 4.0.7 already.)

• Avoided including typing_extensions when using numpy. (Added in 4.0.7 already.)

Organizational

• UI: Relocated the warning about the available source code of extension modules to be evaluated at a more appropriate time.

• Debian: Remove recommendation for libfuse2 package as it is no longer useful.

• Debian: Used platformdirs instead of appdirs.

• Debugging: Removed Python 3.11+ restriction for clang-format as it is available everywhere, even Python 2.7, and we still want nicely formatted code when we read things. (Added in 4.0.6 already.)

• Removed no longer useful inline copy of wax_off. We have our own stubs generator project.

• Release: Added missing package to the CI container for building Nuitka Debian packages.

• Developer: Updated AI instructions for creating Minimal Reproducible Examples (MRE) to skip unneeded C compilation.

• Debugging: Added an internal function for checking if a string is a valid Python identifier.

• AI: Added a task in Visual Studio Code to export the currently selected Python interpreter path to a file, making it available as “python” and “pip” matching the selected interpreter. This makes it easier to use a specific version with no instructions needed.

• AI: Updated the rules to instruct AI to only generate useful comments that add context not present in the code.

• Containers: Added template rendering support for Jinja2 (.j2) container files in our internal Podman tools.

• Projects: Clarified the current status and rationale of Python 2.6 support in the developer manual.

• Debugging: Added experimental flag --experimental=ignore-extra-micro-pass to allow ignoring extra micro pass detection.

• Visual Code: Added integration scripts for bash and zsh autocompletion of Nuitka CLI options. These are now also integrated into Visual Studio Code terminal profiles and the Debian package.

• RPM: Included the Python compile script for Linux.

• RPM: Removed the requirement for distutils in the spec.

Tests

• Install only necessary build tools for test cases.

• Avoided spurious failures in reference counting tests due to Python internal caching differences. (Fixed in 4.0.3 already.)

• Fix, the parsing of the compilation report for reflected tests was incorrect.

• Python 3.14: Ignored a syntax error message change.

• Python 3.14: Added test execution support options to the main test runner to use this version as well.

• Fix, the runner binary path was mishandled for the third pass of reflected compilations.

• Removed the usage of obsolete plugins in reflected compilation tests.

• Debugging: Prevented boolean testing of namedtuples to avoid unexpected bugs.

• Added the Test suffix to syntax test files and disabled “python” mode and spell checking for them to resolve issues reported in IDEs.

• Fix, newline handling in diff outputs from the output comparison tool was incorrect.

• Covered post-import-code functionality with a new subpackage test case.

• Prevented the program test suite from running an unnecessary variant to save execution time.

• macOS: Ignored differences from GUI framework error traces in headless runs in output comparisons.

• Reflected test for Nuitka, where it compiles itself and compares its operation has been restored to functional state.

• Used the new method to clear internal caches if available for reference counts.

• Disabled running nested loops test with Python 2.6.

• Containers: Detected Python 2 defaulting containers in Podman tooling.

Cleanups

• UI: Fix, there was a double space in the Windows Runtime DLLs inclusion message. (Fixed in 4.0.1 already.)

• Onefile: Separated files and defines for extra includes for onefile boot and Python build.

• Scons: Provided nicer errors in case of “unset” variables being used, so we can tell it.

• Refactored the process execution results to correctly utilize our namedtuples variant, that makes it easier to understand what code does with the results.

• Quality: Enabled automatic conversion of em-dashes and en-dashes in code comments to the autoformat tool. AI won’t stop producing them and they can cause SyntaxError for older Python versions, nor is unnecessarily using UTF-8 welcome.

• Ensured that cloned outline nodes are assigned their correct names immediately upon creation, that avoids inconsistencies during their creation.

• Quality: Updated to the latest versions of black and adopted a faster isort execution by caching results.

• Quality: Modified the PyLint wrapper to exit gracefully instead of raising an error when no matching files require checking.

• Quality: Avoided checking YAML package configuration files twice, since autoformat already handles them.

• Quality: Ensured that YAML package configuration checks output the original filename instead of the temporary one when a failure occurs.

• Quality: Prevented pushing of tags from triggering git pre-push quality checks.

• Quality: Silenced the output of optipng and jpegoptim during image optimization auto-formatting.

• Visual Code: Added the generated Python alias path file to the ignore list.

• Quality: Enabled auto-formatting for the Nuitka devcontainer configuration file.

• Watch: Avoided absolute paths in compilation to make reports more comparable across machines.

• Quality: Changed mdformat checks to run only once and silently.

• Scons: Disabled format security errors in debug mode and moved Python-related warning disables into common build setup code.

• Quality: Updated to the latest deepdiff version.

• Scons: Avoided MSVC telemetry since it can produce outputs that break CI.

• Debugging: Enhanced non-deployment handler for importing excluded modules.

• Split import module finding functionality into more pieces for enhanced readability.

• Debugging: Added more assertions for constants loading and checking.

• macOS: Dropped the universal target arch.

• Debugging: Added more traces for deep hash verification.

Summary

This release builds on the scalability improvements established in 4.0, with enhanced Python 3.14 support, expanded package compatibility, and significant optimization work.

The --project option seems usable now.

Python 3.14 support remains experimental, but only barely made the cut, and probably will get there in hotfixes. Some of the corrections came in so late before the release, that it was just not possible to feel good about declaring it fully supported just yet.

May 30, 2026 10:00 PM UTC

Talk Python to Me

#550: AI Contributions and Maintainer Load in Open Source

You wake up, brew the coffee, open GitHub, and there it is. Another pull request on your open source project. Thirteen thousand lines added. No issue filed first. No discussion. Just "here, please review this for me." <br/> <br/> Over the past year, GitHub activity has spiked roughly twelve times in a few short months, and a huge chunk of that signal is landing on the same small group of maintainers who were already stretched thin. The curl bug bounty got buried under AI-generated noise. Jazzband, the home of Django classics like pip-tools and the Django debug toolbar, hit what its maintainer called an "apocalypse" and started sunsetting. Even CPython just shipped fresh guidelines on AI-assisted contributions this week. <br/> <br/> So what does all of this actually look like from the receiving end of the pull request? <br/> <br/> On this episode, Paolo Melchiorre joins us to tell that story from inside the maintainer's chair. Paolo is a director of the Django Software Foundation, an organizer of PyCon Italy, a Django Girls coach, and he has spent the past year carefully collecting examples of how AI is reshaping open source contributions. The good, the bad, and the extra fingers. <br/> <br/> We dig into his PyCon US talk on AI-assisted contributions and maintainer load, why AI is best understood as an amplifier rather than a new kind of contributor, the wildly different policies across 86 open source foundations, whether projects banning AI today are reacting to last year's models.<br/> <br/> <strong>Episode sponsors</strong><br/> <br/> <a href='https://talkpython.fm/agentfield-page'>AgentField AI</a><br> <a href='https://talkpython.fm/training'>Talk Python Courses</a><br/> <br/> <h2 class="links-heading mb-4">Links from the show</h2> <div><strong>Guest</strong><br/> <strong>Paolo Melchiorre</strong>: <a href="https://github.com/pauloxnet?featured_on=talkpython" target="_blank" >github.com</a><br/> <br/> <strong>DSF</strong>: <a href="https://www.djangoproject.com/foundation/?featured_on=talkpython" target="_blank" >www.djangoproject.com</a><br/> <strong>djangonaut-space</strong>: <a href="https://djangonaut.space/?featured_on=talkpython" target="_blank" >djangonaut.space</a><br/> <strong>PyCon Italia</strong>: <a href="https://2026.pycon.it/en?featured_on=talkpython" target="_blank" >2026.pycon.it</a><br/> <strong>uDjango</strong>: <a href="https://github.com/pauloxnet/uDjango?featured_on=talkpython" target="_blank" >github.com</a><br/> <strong>My PyCon US 2026 post</strong>: <a href="https://www.paulox.net/2026/05/21/my-pycon-us-2026/?featured_on=talkpython" target="_blank" >www.paulox.net</a><br/> <strong>AI-Assisted Contributions and Maintainer Load</strong>: <a href="https://www.paulox.net/2026/05/15/pycon-us-2026/?featured_on=talkpython" target="_blank" >www.paulox.net</a><br/> <strong>Senior Engineer Tries Vibe Coding</strong>: <a href="https://www.youtube.com/watch?v=_2C2CNmK7dQ" target="_blank" >www.youtube.com</a><br/> <strong>Code Rabbit AI PR Reviews</strong>: <a href="https://www.coderabbit.ai?featured_on=talkpython" target="_blank" >www.coderabbit.ai</a><br/> <strong>GitHub Usage Graphs</strong>: <a href="https://github.blog/news-insights/company-news/an-update-on-github-availability/?featured_on=talkpython" target="_blank" >github.blog</a><br/> <strong>Update on CPython's AI Policies</strong>: <a href="https://fosstodon.org/@mariatta/116610508567734365" target="_blank" >fosstodon.org</a><br/> <strong>High-Quality Chaos from Curl</strong>: <a href="https://daniel.haxx.se/blog/2026/04/22/high-quality-chaos/?featured_on=talkpython" target="_blank" >daniel.haxx.se</a><br/> <strong>The Generative AI Policy Landscape in Open Source</strong>: <a href="https://redmonk.com/kholterhoff/2026/02/26/generative-ai-policy-landscape-in-open-source/?featured_on=pythonbytes" target="_blank" >redmonk.com</a><br/> <br/> <strong>Watch this episode on YouTube</strong>: <a href="https://www.youtube.com/watch?v=1RJ1kkpTdow" target="_blank" >youtube.com</a><br/> <strong>Episode #550 deep-dive</strong>: <a href="https://talkpython.fm/episodes/show/550/ai-contributions-and-maintainer-load-in-open-source#takeaways-anchor" target="_blank" >talkpython.fm/550</a><br/> <strong>Episode transcripts</strong>: <a href="https://talkpython.fm/episodes/transcript/550/ai-contributions-and-maintainer-load-in-open-source" target="_blank" >talkpython.fm</a><br/> <br/> <strong>Theme Song: Developer Rap</strong><br/> <strong>🥁 Served in a Flask 🎸</strong>: <a href="https://talkpython.fm/flasksong" target="_blank" >talkpython.fm/flasksong</a><br/> <br/> <strong>---== Don't be a stranger ==---</strong><br/> <strong>YouTube</strong>: <a href="https://talkpython.fm/youtube" target="_blank" ><i class="fa-brands fa-youtube"></i> youtube.com/@talkpython</a><br/> <br/> <strong>Bluesky</strong>: <a href="https://bsky.app/profile/talkpython.fm" target="_blank" >@talkpython.fm</a><br/> <strong>Mastodon</strong>: <a href="https://fosstodon.org/web/@talkpython" target="_blank" ><i class="fa-brands fa-mastodon"></i> @talkpython@fosstodon.org</a><br/> <strong>X.com</strong>: <a href="https://x.com/talkpython" target="_blank" ><i class="fa-brands fa-twitter"></i> @talkpython</a><br/> <br/> <strong>Michael on Bluesky</strong>: <a href="https://bsky.app/profile/mkennedy.codes?featured_on=talkpython" target="_blank" >@mkennedy.codes</a><br/> <strong>Michael on Mastodon</strong>: <a href="https://fosstodon.org/web/@mkennedy" target="_blank" ><i class="fa-brands fa-mastodon"></i> @mkennedy@fosstodon.org</a><br/> <strong>Michael on X.com</strong>: <a href="https://x.com/mkennedy?featured_on=talkpython" target="_blank" ><i class="fa-brands fa-twitter"></i> @mkennedy</a><br/></div>

May 30, 2026 03:43 PM UTC

Bob Belderbos

The control layer is the product, not the model

Gary Bernhardt posted something this week that names a phenomenon we're teaching in our agentic AI cohort:

Everyone seems fixated on the models, but I think there's so much low-hanging fruit in the control layer above the model. "Agent" and "harness" sell that layer short. There's so much more that we can do beyond "read input, send to model, run commands it returns."

He's right. The model is a brain in a jar. Useful, fast, occasionally wrong, stateless. Everything that turns it into a product lives in the code that wraps it: the routing, the validation, the state, the audit trail. Gary calls that the control layer. I'm stealing the term.

One of the replies under the tweet nailed the design goal in a single question: do you actually know what the agent is going to do?

That's what a control layer buys you. Not magic, not autonomy, predictability. A workflow where, by the time the model is called, the next move is already constrained to something safe.

Why "agent" and "harness" sell it short

When a developer says "I'm building an agent", they usually mean a while True loop that pings an LLM, parses a tool call, runs it, feeds the result back, and repeats. That pattern works for demos. It rarely survives contact with a real workflow.

The word "harness" makes the wrapping code sound passive, a strap that holds the model in place. It's actually the control layer where the engineering happens. The model is a function call inside it. Once you flip that mental model, you stop asking "which LLM should I use" and start asking "what guarantees does my control layer make?" and "how can I make the inherently unpredictable model fit into a predictable workflow?"

These are the questions production teams have to answer.

Pattern 1: deterministic state machines, not unconstrained agents

An agent without constraints decides what to do next from inside the model. A state machine decides outside the model and gives the model one bounded job at each step. The pipeline runs categorize → validate → confirm → persist, and the LLM only ever gets called inside one of those buckets.

This shifts control flow back to your code, where you can test it, log it, and reason about it. The expense agent we build in our cohort, which I broke down in How an AI expense agent is actually structured, follows exactly this pattern: Protocol-defined LLM boundary, Pydantic-validated outputs, service layer holds the state, human-in-the-loop (HITL) confirms before anything writes. Four layers, no free-roaming agent, constraints at every step.

Pattern 2: the model behind a typed boundary

The model should be one swappable function call inside your control layer, not a dependency threaded through every layer. In our cohort the LLM lives behind a Python Protocol: a small interface the service layer depends on, so nothing downstream knows or cares whether the call goes to OpenAI or Anthropic.

Once the boundary is a Protocol, the decisions people reach for "routing" to solve become wiring instead of rewrites. Picking a cheap fast model for a 12-way classification and saving the expensive one for hard reasoning is a one-line change. Falling back to a second provider when the first is rate-limited is a small factory, not a refactor. Swapping OpenAI for Anthropic, two SDKs that disagree on almost every detail, touches one file because the boundary absorbs the difference.

And it makes the whole pipeline testable. Tests pass a mock that satisfies the Protocol, so you exercise every path without an API call incurring latency or cost.

Pattern 3: evaluators and guardrails

The model's output is not the user's output. Between the two sits validation: schema checks, business rules, PII filters, sometimes a second model grading the first one's work.

This is the generator-evaluator split and it's an important pattern (apart from HITL) I've found for AI code that has to be right. The generator proposes. The evaluator approves or rejects. When the evaluator rejects, control loops back with feedback, not a stack trace.

It's also the layer that catches the worst failure mode of multi-step agents. What production AI agents actually require goes deeper on the four questions the control layer answers before any action runs: state, idempotency, audit, rollback.

Pattern 4: structured generation

A raw string from the model is the start of your problems. You can't store it, validate it, or test it well. The fix is to constrain output at the boundary: the model is allowed to speak, but only in shapes your code understands.

Where the typed boundary in Pattern 2 decides where the model sits in your code, structured generation decides what shape it's allowed to emit.

Pydantic plus your model's structured outputs gives you typed data instead of strings, which means the next layer of your control flow becomes ordinary Python.

I covered this in Build the data layer before you touch the LLM, explaining why we teach students to build the schema before they make a single API call.

The frontier models make the headlines. The control layer ships the product. Gary's tweet names a gap that has been there the whole time, between the people optimizing benchmarks and the people building products. The control layer is the product, not the model. If you want to build AI products, that's where you need to spend your time.

If you want a working walkthrough of the patterns above, the 10 small agentic AI exercises Juanjo and I shipped, run in the browser and cover the arc from a 3-line model call to a complete loop with HITL. They're the conceptual map.

The cohort is the same map, end to end. Six weeks, no frameworks, the control layer built explicitly, with code review at every step. By the end you can answer that one question: you know what your agent is going to do.

May 30, 2026 12:00 AM UTC

May 29, 2026

Real Python

The Real Python Podcast – Episode #297: Improving Python Through PEPs and Protocols

Have you ever been confused by the naming of modules you're importing from a package? Is there a standard way to organize and name your Python virtual environments? This week on the show, Brett Cannon returns to discuss the Python Enhancement Proposals (PEPs) he's been working on recently.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

May 29, 2026 12:00 PM UTC

Quiz: Python's assert: Debug and Test Your Code Like a Pro

In this quiz, you’ll test your understanding of Python’s assert: Debug and Test Your Code Like a Pro.

By working through this quiz, you’ll revisit how assertions help you debug, test, and document your code, when to disable them in production, and which common pitfalls to avoid.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

May 29, 2026 12:00 PM UTC

Ned Batchelder

Snake way for ducklings

This is the mascot for Boston Python. It’s called Snake Way for Ducklings:

My son Ben drew it, which makes me very happy. He also drew Sleepy Snake. Wearing this image on a shirt around PyCon, I had to explain it a number of times. People in Boston understand it almost immediately, but others need more background.

In 1941, Robert McCloskey wrote a children’s book called Make Way for Ducklings. It’s a classic, selling millions of copies and never going out of print. We read it to our own children growing up many times.

The book is the story of Mrs. Mallard making her way through Boston guiding her eight ducklings (Jack, Kack, Lack, Mack, Nack, Oack, Pack, and Quack) to a pond in Boston’s Public Garden. It has charming pencil illustrations:

The book led to a sculpture in the Public Garden near the actual pond:

The sculpture is sized and placed for kids to play on, and is widely known and beloved in Boston. The ducks are dressed in costumes for all kinds of occasions: holidays, sports events, even Star Wars day. On Mother’s Day, there’s a duckling parade: families bring their children dressed as ducklings. In Boston, the ducklings are a big deal.

And it’s not just fiction.

So it seemed natural to Ben to riff on the ducklings for Boston Python. One observer thought a snake eating the ducklings seemed kind of dark, but you can see the ducklings are still quacking, so they are fine!

BTW, Boston also has Duck Boat tours, but that’s completely different.

May 29, 2026 11:29 AM UTC

PyCon Ireland

Call for Proposals Now Open

We’re excited to announce that the Call for Proposals (CFP) for PyCon Ireland 2026 is now open!

We Want to Hear From You

Whether you’re a first-time speaker or an experienced presenter, we’d love to hear your Python story. We welcome proposals on a wide range of topics, including:

• Web Development — Django, Flask, FastAPI, and beyond
• Data Science & ML — pandas, scikit-learn, PyTorch, and more
• DevOps & Infrastructure — CI/CD, containers, cloud-native Python
• Security — Best practices for secure Python applications
• Community & Education — Teaching Python, diversity initiatives, community building
• Core Python — Language features, CPython internals, typing

Talk Formats

• Full talks (30 minutes) — Deep dives into a topic
• Lightning talks (5 minutes) — Quick, focused presentations

How to Submit

Visit our proposal submission page to submit your talk. The deadline is 30 August 2026.

Don’t hesitate to reach out at contact@python.ie if you have any questions about your proposal.

May 29, 2026 12:00 AM UTC

Seth Michael Larson

How much “Super Mario” per year?

It's impossible to objectively quantify art, but we try anyway. For example: Is “Super Mario” a good video-game franchise?

Looking at review scores, Super Mario includes some of the most universally-acclaimed games ever published: Galaxy, Galaxy 2, and Odyssey are respectively the #4, #5, and #13 highest ranking video-games of all time on Metacritic, all with 97 overall. Chances seem good?

What if we tried quantifying art in a different and slightly more reductive way? This blog post introduces and calculates a new unit: “Super Mario per year”. If you enjoy this franchise like I do then this unit is of particular importance to you.

Calculating “Super Mario per year”

There have been ~19 titles (and two add-ons) published to what I consider the "main-line" Super Mario games, both 2D and 3D. Below is a table with every title, the year it was published, and the approximate duration to play. This last column is the most subjective, because there’s speed-runners, casual players, completionists. If you think any value is way off, send me an email.

Using the table above we can calculate approximately how much new Super Mario gameplay is published on average per year.

This table will help you calculate approximately how much Super Mario is coming in the next decade. The current 10-year window pace shows 5 hours of Super Mario per year.

Looking at the trends, it looks like we may have already passed peak 2D and 3D Mario individually. This table also shows how overdue we are for a new big 3D Super Mario title, the last entry being Super Mario Odyssey almost a decade ago in 2017.

If I were to somewhat morbidly apply these numbers I can estimate how much more new “Super Mario” gameplay I’m likely to experience. Let’s be optimistic and apply the “All-Time Average” instead of the “10-Year Average”: the resulting number is 256 hours. Around 10 games of similar size to “Super Mario Odyssey”... seems good to me!

Super Mario Blogroll

If you want to read more Super Mario writing here are a few personal selections from my blogroll:

• “Mario 101: For Super Players” by Drew Mackie
• “Super Mario Bros was designed on graph paper” by Nicolás Valencia
• “The most Mario colors” by Louie Mantia

Happy gaming!

Thanks for keeping RSS alive! ♥ What to do next? Share your thoughts with me on Mastodon, Bluesky, or email. I try to reply to everyone!Browse the blog archive. Check out my blogroll. Or maybe go outside (best option)?

May 29, 2026 12:00 AM UTC

May 28, 2026

Real Python

Quiz: BNF Notation: Dive Deeper Into Python's Grammar

In this quiz, you’ll test your understanding of BNF Notation: Dive Deeper Into Python’s Grammar.

By working through this quiz, you’ll revisit how to read Python’s grammar rules, recognize terminals and nonterminals, and interpret the BNF fragments that appear throughout the official documentation.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

May 28, 2026 12:00 PM UTC

Bob Belderbos

Two Python Scoping Bugs: A Lesson in Object Lifetimes

Your app works fine with one user. You open a second browser tab and the data is wrong. Your tests pass individually but fail when run together. The culprit: a global object created at module scope.

How it starts

I see this a lot in Python web projects:

# database.py
from sqlmodel import create_engine, Session

engine = create_engine("sqlite:///database.db")

def get_session():
    with Session(engine) as session:
        yield session

This 'innocent' engine is created the moment database.py is first imported. Every module that imports from database shares the same engine, the same connection pool, the same database file. For a simple script, this is fine. For a multi-module app, it creates hidden coupling and shared state.

The test isolation problem

I hit this recently in a FastAPI app:

# test_app.py
from myapp.database import engine, create_db_and_tables, clear_db_and_tables

@pytest.fixture(autouse=True)
def setup_database():
    clear_db_and_tables()
    create_db_and_tables()

def test_create_race(race_events):
    championship = create_races(2026, race_events)
    assert championship.id == 1  # Passes alone, fails in suite

That assert championship.id == 1 works when the test runs first. Run it after another test that inserts data, and the auto-increment ID comes back as 2. The fixture does its job, but state still leaks between tests in subtle ways: connection pool state, cached metadata, and SQLite's own bookkeeping on the shared file can carry over even with drop/recreate cycles.

The root cause is upstream: every test reaches for the same module-level engine pointed at the same on-disk database. If you want true isolation, the engine itself has to be per-test, not the cleanup ritual around it.

The fix is creating an engine per test session:

@pytest.fixture
def engine():
    engine = create_engine("sqlite://", echo=False)
    SQLModel.metadata.create_all(engine)
    yield engine
    engine.dispose()

@pytest.fixture
def session(engine):
    with Session(engine) as session:
        yield session

Now each test gets a fresh database (no scope defined on the fixture decorator means function scope = per test). No cleanup needed. No shared state.

Alternatively, keep the module-level engine and wrap each test in a transaction you roll back at teardown (sqlmodel's Session supports this).

If you omit engine.dispose() in the code above, you may see a ResourceWarning: unclosed database, but only when running pytest --cov. Coverage's sys.settrace() hook keeps frame locals alive longer, delaying GC of the engine.

The shared simulator problem

The database engine bug is about too much sharing. Here is the inverse: not enough sharing, which breaks in a different way.

Consider a race simulation dashboard. FakeDataSource wraps a RaceSimulator that holds the full mutable race state, driver positions, lap counter, cumulative changes, and advances it on each call:

class FakeDataSource(RaceDataSource):
    def __init__(self, data_file: Path, delay_ms: int = 100):
        drivers = self._load_drivers(data_file)
        self.simulator = RaceSimulator(drivers=drivers)  # mutable state lives here

    async def get_positions(self, fixture_id: str) -> list[Position]:
        self.simulator.tick()  # randomly swaps adjacent positions, advances lap
        return self.simulator.get_current_positions()

The FastAPI dependency looks like this:

def get_data_source() -> RaceDataSource:
    source_type = config("DATA_SOURCE", default="fake")
    if source_type == "fake":
        return FakeDataSource(data_file=..., delay_ms=...)  # new instance every call
    ...

def get_race_data_source() -> RaceDataSource:
    return get_data_source()

FastAPI calls get_race_data_source() once per request. Each browser tab that opens the SSE stream gets a brand new FakeDataSource with a brand new RaceSimulator starting at lap 1 with drivers in their initial order.

The random swaps then diverge independently: Tab A shows Verstappen in P1 at lap 12, Tab B shows Hamilton in P1 at lap 3. Neither reflects a shared reality, because there is no shared state at all.

Two fixes, from quick to idiomatic

1. cache: one line, works immediately

from functools import lru_cache

@lru_cache(maxsize=1)
def get_data_source() -> RaceDataSource:
    source_type = config("DATA_SOURCE", default="fake")
    if source_type == "fake":
        return FakeDataSource(...)
    return SportmonksDataSource()

One instance for the lifetime of the process. Simple, but hard to override in tests.

2. FastAPI app.state: idiomatic and testable

@asynccontextmanager
async def lifespan(app: FastAPI):
    app.state.data_source = get_data_source()
    yield

app = FastAPI(lifespan=lifespan)

def get_race_data_source(request: Request) -> RaceDataSource:
    return request.app.state.data_source

The data source is created once at startup, shared across all requests, and easy to replace in tests via app.dependency_overrides[get_race_data_source] = lambda: test_source.

Key takeaways

• Module-level objects are created at import time and shared everywhere: global mutable state, and a common source of subtle bugs.
• Tests that share a database engine aren't isolated, even with setup/teardown fixtures.
• Web apps that create per-request instances lose shared state; apps that share module-level instances lose testability.
• Use app.state or cache for shared runtime state; override the FastAPI dependency in tests for isolation.

The rule of thumb: if an object holds mutable state, pick its scope deliberately. Too broad (module scope) and tests leak into each other. Too narrow (per-request) and there's no shared reality. Match the scope to the object's intended lifetime.

Or put more sharply: any module-level object that holds mutable state or owns a resource (DB engines, HTTP clients, caches, queues, connection pools) should be encapsulated. Move it into a fixture, a Depends(), or app.state. Constants and pure values at module scope are fine; resources are not.

The cost of "just import it" is paid later, in test isolation, debugging, and concurrency. Under real concurrency the GIL hides this class of bug until it doesn't, see a race condition Rust wouldn't have let me write, where the same module-global pattern leaked one user's data into another user's response.

May 28, 2026 12:00 AM UTC

May 27, 2026

PyCharm

Build a Live Object Detection App for the Reachy Mini With TensorFlow and PyCharm

This is a guest post from Iulia Feroli, founder of the Back To Engineering YouTube community.

In this tutorial, we build a live object detection app using TensorFlow and PyCharm, then deploy it onto the Reachy Mini open-source robot for real-time object tracking.

Reachy Mini is a compact open-source robot built in collaboration by Pollen Robotics, Hugging Face, and Seeed Studio. It has been going viral lately, getting mentioned in NVIDIA videos and even in the keynotes at some of their conferences. What makes it particularly interesting is that not only is all the code open-source, the body is too, which means you can print your own parts and develop your own apps to run on it.

There is an app store of community-built projects you can explore and try, and easily contribute to. Anything conversational or camera-based is especially fun to build because of the hardware it ships with: a speaker, a microphone, and a camera, plus expressive antennas for emotions.

This really highlights the unique new type of robot that the Reachy Mini embodies: It almost feels like it is a physical representation of an LLM or an AI agent, rather than a robot that has AI added to it. It does not have a body that moves around or hands to grab things, so its main selling point is really its brain. That design choice shapes what is most interesting to build with it.

Let’s learn how to build a TensorFlow object detection app and deploy it on the Reachy Mini, which will then allow us to do live object tracking. You can head over to the PyCharm channel for the full code breakdown and try it at home. All the code is in the Reachy-mini-object-detection GitHub repository.

For an introduction to the robot, you can first watch Iulia’s video here:

What you’ll learn

• How to build a real-time TensorFlow object detection pipeline. 
• How to use SSD MobileNet V2 from TensorFlow Hub. 
• How to create a TensorFlow object detection example with OpenCV. 
• How to run live webcam inference in PyCharm notebooks. 
• How to deploy object detection on the Reachy Mini robot. 
• How to track detected objects using head movement logic. 
• How to stream annotated detections to a live dashboard.

What we are building

The project is split into two stages.

Stage 1 is a standalone notebook that runs entirely on your laptop using your webcam. No robot needed. This is where we make sure the detection pipeline works correctly before touching any hardware.

Stage 2 is a Reachy Mini app that integrates the same model with the robot: Her head moves to follow detected objects, her antennas wiggle when she spots something new, and a live web dashboard at http://0.0.0.0:8042 shows the annotated camera feed and detections.

You can follow along with the step-by-step video tutorial:

How TensorFlow object detection works: Step by step

1. Capture an image frame from the webcam. 

2. Convert the frame into a TensorFlow tensor.

3. Run inference through the pretrained model.

4. Receive bounding boxes, labels, and confidence scores.

5. Filter low-confidence detections.

6. Draw annotated results onto the frame.

7. Display the processed image in real time.

Prerequisites

• Python 3.12+.
• PyCharm with its Jupyter Notebook integration.
• A Reachy Mini for Stage 2 (Stage 1 runs entirely on your laptop).
• Some familiarity with TensorFlow basics – if you are brand new to it, the previous post in this series is a good starting point.

Stage 1: Building a Tensorflow object detection pipeline in PyCharm

Before connecting the robot, we want to make sure the TensorFlow part works independently. We are going to make a notebook that only executes through our object detection model and makes it run smoothly. PyCharm’s native notebook integration is a great fit here: You can inspect each step of the pipeline and visualize results inline.

The object detection model

We are using SSD MobileNet V2 from TensorFlow Hub, trained on Open Images V4. This popular model from Google provides SSD-based object detection and has been trained on a lot of open images. With a little bit of fine-tuning you can deploy it with your own use case, though for this tutorial, the general model works well without any fine-tuning at all.

It runs at around 10 FPS on CPU, which is fast enough for responsive real-time behavior on the robot.

Install dependencies

!pip install tensorflow tensorflow-hub opencv-python numpy Pillow

Load the model

import tensorflow as tf
import tensorflow_hub as hub
import numpy as np
import cv2
import time
from IPython.display import display, clear_output
from PIL import Image

MODEL_HANDLE = "https://tfhub.dev/google/openimages_v4/ssd/mobilenet_v2/1"

print(f"TensorFlow version: {tf.__version__}")
print("Loading model (first time downloads ~30MB)...")

detector = hub.load(MODEL_HANDLE)
print("Model loaded!")

The model is about 30 megabytes and gets cached locally after the first download. Because it is very generalized, it can work across a lot of different scenarios without needing additional training data, which makes it a lot easier to get started.

Detection and drawing helpers

We need two helper functions: one to run inference and return a list of detections, and one to draw the bounding boxes on the frame. These are the same functions we use later in the Reachy app.

def detect_objects(frame_bgr, min_score=0.5, max_detections=10):
    rgb = frame_bgr[:, :, ::-1]
    img_tensor = tf.image.convert_image_dtype(rgb, tf.float32)[tf.newaxis, ...]

    results = detector.signatures['default'](img_tensor)

    boxes = np.array(results["detection_boxes"])
    scores = np.array(results["detection_scores"])
    class_labels = np.array(results["detection_class_entities"])

    if boxes.ndim > 2:
        boxes = boxes[0]
    if scores.ndim > 1:
        scores = scores[0]
    if class_labels.ndim > 1:
        class_labels = class_labels[0]

    scores = np.atleast_1d(scores)
    indices = [i for i, score in enumerate(scores) if score >= min_score][:max_detections]

    detections = []
    for idx in indices:
        ymin, xmin, ymax, xmax = boxes[idx]
        label = class_labels[idx].decode('utf-8') if isinstance(class_labels[idx], bytes) else str(class_labels[idx])
        detections.append({
            "box": [ymin, xmin, ymax, xmax],
            "score": float(scores[idx]),
            "label": label
        })

    return detections


def draw_detections(frame_bgr, detections):
    h, w = frame_bgr.shape[:2]
    annotated = frame_bgr.copy()

    for det in detections:
        ymin, xmin, ymax, xmax = det["box"]
        x1, y1 = int(xmin * w), int(ymin * h)
        x2, y2 = int(xmax * w), int(ymax * h)

        color = (0, 255, 0)
        cv2.rectangle(annotated, (x1, y1), (x2, y2), color, 2)

        label = f"{det['label']} {det['score']:.0%}"
        font_scale, thickness = 0.6, 2
        (tw, th), _ = cv2.getTextSize(label, cv2.FONT_HERSHEY_SIMPLEX, font_scale, thickness)
        cv2.rectangle(annotated, (x1, y1 - th - 8), (x1 + tw + 4, y1), color, -1)
        cv2.putText(annotated, label, (x1 + 2, y1 - 4),
                    cv2.FONT_HERSHEY_SIMPLEX, font_scale, (0, 0, 0), thickness)

    return annotated

The detect_objects function runs inference using the model’s detect_objects entry point and handles flattening the batch dimension from the output tensors. Labels come back as bytes from the model, so we decode them to strings before returning.

Test on a single frame

cap = cv2.VideoCapture(0)
ret, frame = cap.read()
cap.release()

if not ret:
    print("ERROR: Could not access webcam. Make sure no other app is using it.")
else:
    print(f"Frame captured: {frame.shape}")

    t0 = time.time()
    detections = detect_objects(frame)
    elapsed = time.time() - t0

    print(f"Inference time: {elapsed:.2f}s ({1/elapsed:.1f} FPS)")
    print(f"Found {len(detections)} objects:")
    for d in detections:
        print(f"  - {d['label']}: {d['score']:.0%}")

    annotated = draw_detections(frame, detections)
    display(Image.fromarray(annotated[:, :, ::-1]))

This is the stage where you check that the model is detecting correctly and the bounding boxes are drawn in the right places. The inline image display in PyCharm’s notebook view makes it easy to see the result right there in the cell.

Running real-time TensorFlow object detection with OpenCV

Once the single-frame test looks good, you can run it continuously:

cap = cv2.VideoCapture(0)

if not cap.isOpened():
    print("ERROR: Could not open webcam.")
else:
    print("Running live detection... (interrupt kernel to stop)")
    try:
        while True:
            ret, frame = cap.read()
            if not ret:
                break

            t0 = time.time()
            detections = detect_objects(frame)
            fps = 1.0 / max(time.time() - t0, 0.001)

            annotated = draw_detections(frame, detections)
            cv2.putText(annotated, f"{fps:.1f} FPS", (10, 30),
                        cv2.FONT_HERSHEY_SIMPLEX, 1.0, (0, 0, 255), 2)

            clear_output(wait=True)
            display(Image.fromarray(annotated[:, :, ::-1]))

            labels = ", ".join(f"{d['label']} ({d['score']:.0%})" for d in detections)
            print(f"{fps:.1f} FPS | {len(detections)} objects: {labels or 'none'}")

    except KeyboardInterrupt:
        print("Stopped.")
    finally:
        cap.release()
        print("Camera released.")

At this point we have built a notebook that works with just having object detection and we can use this with a simple camera of whatever type you have around. Now, we can wrap it up and make it into an app that we can deploy on the Reachy.

Stage 2: Deploying the TensorFlow object detection app on the Reachy Mini

The Reachy Mini app lives in the reachy_mini_object_detector/ folder and extends the detection logic with head tracking, antenna reactions, and a web dashboard. We’ve followed the guidelines for building Reachy Apps laid out in this blog post. Particularly, we can leverage a helper LLM system like Claude by giving it the predefined Agent Helper documentation.

Project structure

reachy_mini_object_detector/
├── pyproject.toml
└── reachy_mini_object_detector/
    ├── detector.py       # TF Hub model wrapper
    ├── main.py           # App: head tracking + web dashboard
    └── static/           # Web UI assets (served at :8042)

The detector.py file wraps the model and the detect_objects logic. main.py imports from it and adds everything specific to the robot.

Installing the app

From the Reachy Mini dashboard, under Apps, or by manually adding:

pip install git+https://huggingface.co/spaces/backtoengineering/reachy_mini_object_detector

How head tracking works

The app runs two loops in parallel: an inference thread that grabs frames from the robot’s camera and runs detection, and a main control loop at around 50Hz that handles head movement and antenna control.

The head tracking feature maps the detected object’s position in the frame to a yaw and pitch offset for the head. The camera has a horizontal field of view of 60 degrees and a vertical field of view of 45 degrees. When an object is at the center of the frame its center_x is 0.5, so subtracting 0.5 and multiplying by the field of view gives the angle offset to track it:

target_yaw = -(largest.center_x - 0.5) * CAMERA_FOV_H_DEG
target_pitch = (largest.center_y - 0.5) * CAMERA_FOV_V_DEG

Rather than snapping the head instantly to that target, the app uses a smoothing factor (TRACKING_ALPHA = 0.15) so the movement looks natural:

self._current_yaw += TRACKING_ALPHA * (target_yaw - self._current_yaw)
self._current_pitch += TRACKING_ALPHA * (target_pitch - self._current_pitch)

When nothing is detected, the head slowly drifts back toward center rather than freezing in place.

Antenna wiggle

The antennas wiggle when a new object class is first detected, not on every frame. The app keeps track of which classes have already been seen in _seen_classes, and when something new appears it sets a wiggle timer for 1.5 seconds. During that window, the control loop drives a sinusoidal antenna movement as follows:

phase = (t - t0) * 8.0  # fast wiggle
antenna_val = np.deg2rad(20.0 * np.sin(phase))
antennas = np.array([antenna_val, -antenna_val]

This makes the interaction feel intentional: Reachy reacts when she sees something new, rather than wiggling constantly while tracking.

The web dashboard

The app serves a live dashboard (available at http://0.0.0.0:8042) with the annotated camera feed (as an MJPEG stream), the current detection list, an FPS counter, and a toggle to enable or disable head tracking. This is useful during development because you can see exactly what the model is detecting from the robot’s perspective in real time.

Where to go next

This is a great starting point and there are a lot of directions you can take it:

• Run the app with a specific use case in mind. The model is general, but if you want Reachy to recognize specific objects you can fine-tune on your own dataset using TensorFlow’s Object Detection API.
• Add more apps. There are many apps that users have already created in the Reachy Mini store, and building one that uses both the camera and the conversational capabilities together opens up a lot of possibilities.
• Connect to physical arms. Something I would really like to explore next is connecting Reachy to the SO-101 arms, so she can actually reach out and do things in the physical world as well as see them.

You can find all the code in the Reachy-mini-object-detection repository. Everything is open-source, so feel free to build on it, adapt it, or deploy your own version.

FAQs

What is TensorFlow object detection?

TensorFlow object detection is a computer vision technique that uses machine learning models to identify and locate objects within images or video streams.

What is the best TensorFlow object detection model for real-time applications?

SSD MobileNet V2 is commonly used for real-time TensorFlow object detection because it balances inference speed and accuracy efficiently.

Can TensorFlow object detection run on CPU?

Yes. Models like SSD MobileNet V2 can run entirely on CPU, making them suitable for laptops, edge devices, and robotics projects.

What is the difference between the TensorFlow Object Detection API and TensorFlow Hub?

TensorFlow Hub provides pretrained reusable models, while the TensorFlow Object Detection API offers a larger framework for training, evaluation, and deployment workflows.

Can I train TensorFlow object detection on custom data?

Yes. You can fine-tune pretrained models using your own labelled datasets to detect custom objects.

About the author

Iulia Feroli

Iulia Feroli is the founder of the Back To Engineering community on YouTube, where she builds robots, explores physical AI, and makes complex engineering topics accessible and fun. She has a background in data science, AI, cloud architecture, and open source.

May 27, 2026 02:06 PM UTC

Real Python

Sending Emails With Python

You probably found this tutorial because you want to send emails with Python to automate confirmation messages, password resets, or scheduled notifications. Python’s standard library covers the whole pipeline, from making a server connection to building the message and sending it to one or many recipients. This tutorial walks through every step in working code.

By the end of this tutorial, you’ll understand that:

• A safe testing setup uses a throwaway Gmail account with an app password, a local aiosmtpd debug server, or a privacy-focused provider like Posteo or Proton Mail.
• A secure SMTP session uses .SMTP_SSL() with ssl.create_default_context(), which validates the server certificate and encrypts your credentials and message content.
• The EmailMessage class from the email package assembles plain text, HTML alternatives, file attachments, and personalized fields through .set_content(), .add_alternative(), and .add_attachment().
• Setting msg["reply-to"] or any other RFC 5322 header on an EmailMessage routes replies to a different mailbox than the sender address.
• For high-volume sending, transactional email services like SendGrid, Mailgun, and Brevo provide deliverability, statistics, and API libraries that go beyond what smtplib alone offers.

Before you jump into the code, you’ll set up a throwaway email account or a local debug server so you can experiment freely without spamming real inboxes.

Get Your Code: Click here to download the free sample code you’ll use to learn how to send plain-text and HTML emails, attach files, and automate email delivery with Python.

Take the Quiz: Test your knowledge with our interactive “Sending Emails With Python” quiz. You’ll receive a score upon completion to help you track your learning progress:

---

Interactive Quiz

Sending Emails With Python

Use Python's standard library to send email through secure SMTP connections, attach files, include HTML content, and route replies.

Setting Up an Email Service

Email is sent from a client to an email server, and from one email server to another, using the Simple Mail Transfer Protocol, or SMTP, defined under RFC 821. Python comes with the built-in smtplib module, which implements this protocol, allowing you to programmatically send email through any accessible email server.

While you can certainly use your own email account for this tutorial, it’s recommended that you set up a throwaway email account instead. There are several free and paid email services you can use. In this tutorial, you’ll explore the following options:

• Setting up a Gmail account for development: You’ll learn how to create a dedicated testing account and use app passwords to satisfy modern security requirements.
• Setting up a local SMTP server: You’ll use the aiosmtpd library to run a server on your own machine, allowing you to inspect email content without sending any live messages.
• Setting up other email accounts for development: You’ll see how to connect to alternative services like Posteo or Proton Mail to ensure your code works across different providers.

Understanding the distinction between secure (encrypted) and insecure (unencrypted) connections is vital. Most modern providers require encryption via SSL or TLS to protect your data, while the local debugging server uses no encryption. By the end of this section, you’ll know how to choose the right connection type for your specific service choice.

Setting Up a Gmail Account for Development

To set up a Gmail account for testing your code, follow these steps:

1. Create a new Google account. You need to provide a name, a birthday, and a unique username for the account.
2. Set up two-factor authentication for the new account.
3. Add a new app password to allow password sign-ins to the account.

An app password is a temporary password generated by Google. Instead of using your main account password to authenticate with your username, you use the app password. You can delete and recreate app passwords whenever you like.

App passwords allow access to Gmail when modern security measures like OAuth2 aren’t available. When creating one, make sure you copy it to a secure location, as you won’t be able to review it after leaving the page.

If you don’t want to use an app password, check out Google’s documentation on how to obtain access credentials for your Python script using the OAuth2 authorization framework.

A nice feature of Gmail is that you can use the + sign to add modifiers to your email address right before the @ sign. For example, emails sent to my+person1@gmail.com and my+person2@gmail.com will both arrive at my@gmail.com. When testing email functionality, you can use this to simulate multiple addresses that all point to the same inbox.

Setting Up a Local SMTP Server

You can test email functionality by running a local Simple Mail Transfer Protocol (SMTP) debugging server with the aiosmtpd module. Rather than sending emails to a specific address, the local debug server discards the message after printing its content to the console. Running a local debugging server makes it unnecessary to deal with encryption of messages or use credentials to log in to an email server.

Note: aiosmtpd is a third-party library that replaces the former built-in smtpd module, which was initially deprecated in Python 3.4.7. Deprecation notices were repeated in 3.5.4 and 3.6.1, and the module was eventually removed in Python 3.12, as outlined in PEP 594.

Install the aiosmtpd module with the following command:

Language: Shell

$ python -m pip install aiosmtpd

Then, start a local SMTP debugging server with this command:

Language: Shell

$ python -m aiosmtpd -n

Read the full article at https://realpython.com/python-send-email/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

May 27, 2026 02:00 PM UTC

Quiz: Sending Emails With Python

In this quiz, you’ll test your understanding of Sending Emails With Python.

By working through this quiz, you’ll revisit how to build messages with the EmailMessage class, secure your SMTP connection, attach files, send HTML alternatives, route replies to a different mailbox, and address multiple recipients at once.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

May 27, 2026 12:00 PM UTC

Yair Lenga

jsonfold: Making Pretty-Printed JSON Compact and Readable Again in Python

Most JSON serializers force a choice between compact machine output and fully expanded pretty-printing. This article explores a middle ground: a hybrid JSON formatter for Python that keeps nested payloads readable while still remaining compact.

Example output:

{
  "meta": { "version": 1, "ok": true },
  "ids": [ 1, 2, 3, 4 ],
  "matrix": [ [1, 2], [3, 4] ]
}

The article also discusses streaming formatting, bounded buffering, width-aware layout decisions, and incremental processing without reparsing the full JSON tree.

Source code: github.com/yairlenga/jsonfold

May 27, 2026 12:00 PM UTC

PyPy

PyPy v7.3.23 release

PyPy v7.3.23: release of python 2.7, 3.11

The PyPy team is proud to release version 7.3.23 of PyPy after the previous release on April 26, 2026. This is a bug-fix release that fixes an overeager warning about unused coroutines, and some problems around multiple inheritance in c-extensions.

This version includes a change to the bytecode interpreter to use exception tables instead of dedicated opcodes. Now the PyPy disassembly will be closer to CPython format. So far it does not impact performance.

The release includes two different interpreters:

• PyPy2.7, which is an interpreter supporting the syntax and the features of Python 2.7 including the stdlib for CPython 2.7.18+ (the + is for backported security updates)

• PyPy3.11, which is an interpreter supporting the syntax and the features of Python 3.11, including the stdlib for CPython 3.11.15.

The interpreters are based on much the same codebase, thus the double release. This is a micro release, all APIs are compatible with the other 7.3 releases.

We recommend updating. You can find links to download the releases here:

https://pypy.org/download.html

We would like to thank our donors for the continued support of the PyPy project. If PyPy is not quite good enough for your needs, we are available for direct consulting work. If PyPy is helping you out, we would love to hear about it and encourage submissions to our blog via a pull request to https://github.com/pypy/pypy.org

We would also like to thank our contributors and encourage new people to join the project. PyPy has many layers and we need help with all of them: bug fixes, PyPy and RPython documentation improvements, or general help with making RPython's JIT even better.

If you are a python library maintainer and use C-extensions, please consider making a HPy / CFFI / cppyy version of your library that would be performant on PyPy. In any case, cibuildwheel supports building wheels for PyPy.

What is PyPy?

PyPy is a Python interpreter, a drop-in replacement for CPython. It's fast (PyPy and CPython performance comparison) due to its integrated tracing JIT compiler.

We also welcome developers of other dynamic languages to see what RPython can do for them.

We provide binary builds for:

• x86 machines on most common operating systems (Linux 32/64 bits, Mac OS 64 bits, Windows 64 bits)

• 64-bit ARM machines running Linux (aarch64) and macos (macos_arm64).

PyPy supports Windows 32-bit, Linux PPC64 big- and little-endian, Linux ARM 32 bit, RISC-V RV64IMAFD Linux, and s390x Linux but does not release binaries. Please reach out to us if you wish to sponsor binary releases for those platforms. Downstream packagers provide binary builds for debian, Fedora, conda, OpenBSD, FreeBSD, Gentoo, and more.

What else is new?

For more information about the 7.3.23 release, see the full changelog.

Please update, and continue to help us make pypy better.

Cheers, The PyPy Team

May 27, 2026 07:40 AM UTC

Python GUIs

Fixing Missing Icons in PyInstaller-Packaged PyQt6 Applications on Windows — Why your app icon disappears after packaging and how to fix it

I've packaged my PyQt application with PyInstaller, but the icon isn't showing up — both the executable icon and the running application icon are just the default Python/Windows icon. What's going on?

This is a common issue when packaging PyQt6 apps with PyInstaller on Windows. The good news is that it usually comes down to one of two straightforward causes: Windows icon caching, and missing resource files in your packaged output.

Setting the executable icon with PyInstaller

When you run PyInstaller, you can set the icon for the .exe file itself using the --icon flag:

pyinstaller --windowed --icon=myicon.ico myapp.py

This embeds the icon into the executable, so it shows up in File Explorer and on the desktop. The icon file needs to be in .ico format — .png or .svg won't work here.

After building, check the dist/ folder. Your .exe should display the custom icon. But sometimes... it doesn't.

Windows icon caching

Windows caches icons aggressively. If you've previously built your app without a custom icon, Windows may continue to show the old default icon even after you've rebuilt the app with the correct one.

This still catches me out, even though I know this. You'll reflexively start checking the config assuming something is wrong, and think you're going mad.

There are a few ways to deal with this:

• Rename the executable. Changing the filename forces Windows to look up the icon fresh. This is the quickest way to confirm that your icon is actually embedded correctly.
• Clear the Windows icon cache. You can do this by restarting Windows Explorer or by deleting the icon cache files manually. To manually clear the Windows icon cache open a Command Prompt and run:

ie4uinit.exe -show

After clearing the cache, the correct icon should appear.

You can also try turning your computer off and on again, or rather restarting Windows. That will also trigger the icon cache to rebuild.

Missing icon file at runtime

Setting the executable icon with --icon only affects what shows up in File Explorer. If your application also sets a window icon in code (using setWindowIcon), that icon file needs to be available at runtime too.

For example, if your code does this:

from PyQt6.QtWidgets import QApplication, QMainWindow
from PyQt6.QtGui import QIcon
import sys


class MainWindow(QMainWindow):
    def __init__(self):
        super().__init__()
        self.setWindowTitle("My Application")


app = QApplication(sys.argv)
app.setWindowIcon(QIcon("myicon.ico"))

window = MainWindow()
window.show()

app.exec()

Then myicon.ico needs to exist in the working directory when the packaged app runs. By default, PyInstaller doesn't include data files like .ico images unless you tell it to.

You can add the icon file to your build using the --add-data flag:

pyinstaller --windowed --icon=myicon.ico --add-data "myicon.ico;." myapp.py

On Linux or macOS, use : instead of ; as the separator:

pyinstaller --windowed --icon=myicon.ico --add-data "myicon.ico:." myapp.py

This copies myicon.ico into the output directory alongside your executable (or into the temporary directory if you're using --onefile).

An alternative approach (not available on PyQt6) is to use the Qt Resource System to embed your icon directly into your application, which avoids the need to bundle separate icon files entirely.

Handling --onefile builds

When you use --onefile, PyInstaller extracts everything to a temporary folder at runtime. Your code needs to know how to find files relative to that temporary folder. You can handle this by detecting the base path:

import sys
import os

if getattr(sys, 'frozen', False):
    # Running as a PyInstaller bundle
    basedir = sys._MEIPASS
else:
    # Running as a normal script
    basedir = os.path.dirname(__file__)

Then use basedir when constructing file paths:

app.setWindowIcon(QIcon(os.path.join(basedir, "myicon.ico")))

Taskbar grouping with an Application User Model ID

On Windows, the taskbar groups windows by their application identity. Without an explicit identity, Windows guesses — and sometimes guesses wrong. This can cause your app to show the Python icon in the taskbar, or to group instances inconsistently depending on where they were launched from.

You can fix this by setting an Application User Model ID before creating your QApplication. This tells Windows exactly which application this is:

import ctypes

myappid = "com.mycompany.myapp.1.0"
ctypes.windll.shell32.SetCurrentProcessExplicitAppUserModelID(myappid)

The string can be anything, but it's conventional to use a reverse-domain format. The value just needs to be unique to your application.

With an explicit app ID set, all instances of your app will group together in the taskbar regardless of where they were launched from — whether that's your IDE, the dist/ folder, or a --onefile build.

Complete working example

Here's a complete example that handles all of the above — the runtime base path, the window icon, and the application user model ID. If you're new to building PyQt6 applications, you may want to start with creating your first window before tackling packaging.

import sys
import os
import ctypes

from PyQt6.QtWidgets import QApplication, QMainWindow, QLabel
from PyQt6.QtGui import QIcon
from PyQt6.QtCore import Qt


# Set the app user model ID before creating QApplication (Windows only)
if sys.platform == "win32":
    myappid = "com.mycompany.myapp.1.0"
    ctypes.windll.shell32.SetCurrentProcessExplicitAppUserModelID(myappid)

# Determine the base directory for resource files
if getattr(sys, "frozen", False):
    basedir = sys._MEIPASS
else:
    basedir = os.path.dirname(__file__)


class MainWindow(QMainWindow):
    def __init__(self):
        super().__init__()
        self.setWindowTitle("My Application")
        label = QLabel("Hello, world!")
        label.setAlignment(Qt.AlignmentFlag.AlignCenter)
        self.setCentralWidget(label)


app = QApplication(sys.argv)
app.setWindowIcon(QIcon(os.path.join(basedir, "myicon.ico")))

window = MainWindow()
window.show()

app.exec()

To package this with PyInstaller:

pyinstaller --windowed --icon=myicon.ico --add-data "myicon.ico;." myapp.py

For an in-depth guide to building Python GUIs with PyQt6 see my book, Create GUI Applications with Python & Qt6.

May 27, 2026 06:00 AM UTC

Python Morsels

Selecting random values in Python

Python's random module provides utilities for generating pseudorandom numbers. For cryptographically-secure randomness, use the secrets module instead.

Table of contents

1. Generating random integers
2. Generating random floating point numbers
3. Selecting random items from a sequence
4. The random utilities are only pseudorandom
5. Cryptographically-secure randomness with the secrets module
6. Random and SystemRandom classes
7. Use random for pseudo-random numbers and secrets for true randomness

>>> from random import randint
>>> randint(1, 6)
4

>>> from random import randrange
>>> randrange(10)
7

>>> randrange(5)
2

>>> randrange(5, 10)
8

>>> randrange(0, 100, 10)
70

Read the full article: https://www.pythonmorsels.com/random-numbers/

May 27, 2026 04:00 AM UTC

May 26, 2026

PyCoder’s Weekly

Issue #736: Polars Sort-Merge Joins, Zen, Resolving Lazy Imports, and More (2026-05-26)

#736 – MAY 26, 2026
View in Browser »

FREE Python Error Tracking From Honeybadger – all Signal, no Noise

Production bugs don’t arrive one at a time. Honeybadger groups similar errors into a single issue and lets you pause or ignore alerts in a single click. More signal. Less noise. ⚡ Sign Up for Your FREE Developer Account →
HONEYBADGER sponsor

Articles & Tutorials

Projects & Code

Events

Happy Pythoning!
This was PyCoder’s Weekly Issue #736.
View in Browser »

[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]

May 26, 2026 07:30 PM UTC

• • RSS feed
  • Titles Only
  • Powered by Planet!
• Other Python Planets
  • Python Summer of Code
  • Planet Python Francophone
  • Planet Python Argentina
  • Planet Python Brasil
  • Planet Python Poland
• Python Libraries
  • OLPC
  • PySoy
  • SciPy
  • SymPy
  • Twisted
• Python/Web Planets
  • CherryPy
  • Django Community
  • Plone
  • Turbogears
• Other Languages
  • Haskell
  • Lisp
  • Parrot
  • Perl
  • Ruby
• Databases
  • MySQL
  • PostgreSQL
• Subscriptions
  • [OPML feed]
  • "Menno's Musings"
  • "Michael J.T. O'Kelly"
  • "Michael Kennedy's Thoughts on Technology"
  • "Michiel's Blog"
  • "Morphex's Blogologue"
  • "Speno's Pythonic Avocado"
  • "William's Journal"
  • 2degrees
  • A. Jesse Jiryu Davis
  • ABlog for Sphinx
  • Aahz
  • Abhijeet Pal
  • Abu Ashraf Masnun
  • Adam Pletcher
  • Adrarsh Divakaran
  • Agendaless Consulting
  • Ahmed Bouchefra
  • Al-Ahmadgaid Asaad
  • Alec Munro
  • Alex Grönholm
  • Alex Morozov
  • Alexander Limi
  • Alexandre Conrad
  • Alexandre Vassalotti
  • Alexey Evseev
  • Allison Kaptur
  • Amjith Ramanujam
  • AmvTek
  • Anarcat
  • Anatoly Techtonik
  • Andre Roberge
  • Andrea Grandi
  • Andrew Dalke
  • Andriy Kornatskyy
  • Andy Dustman
  • Andy R. Terrel
  • Anna Martelli Ravenscroft
  • Anthony Baxter
  • Anton Belyaev
  • Anton Bobrov
  • Antonio Cuni
  • Anwesha Das
  • Ari Lamstein
  • Armin Ronacher
  • Artem Golubin
  • Artem Rys
  • Ashish Vidyarthi
  • Astro Code School
  • Audrey M. Roy Greenfeld
  • Automating OSINT
  • Awesome Python Applications
  • Baiju Muthukadan
  • Bajusz Tamás
  • BeDjango
  • Ben Bass
  • Ben Cook
  • Ben Rousch
  • Benjamin Peterson
  • Benji York
  • Bertrand Mathieu
  • Bhavin Gandhi
  • Bhishan Bhandari
  • BioPython News
  • Bit of Cheese
  • Bob Belderbos
  • Bojan Mihelac
  • Brandon Rhodes
  • BreadcrumbsCollector
  • Brendan Scott
  • Brett Cannon
  • Brian Okken
  • Bruno Oliveira
  • Bruno Ponne / Coding The Past
  • Bruno Rocha
  • Caktus Consulting Group
  • Calvin Cheng
  • Calvin Spealman
  • Carl Chenet
  • Carl Düvel
  • Carl Trachte
  • Carlos Eduardo de Paula
  • Casey Duncan
  • Catherine Devlin
  • Ching-Hwa Yu
  • Chris Hager
  • Chris Miles
  • Chris Mitchell
  • Chris Moffitt
  • Chris Rose
  • Chris Warrick
  • Christian Heimes
  • Christian Ledermann
  • Christian Scholz
  • Christoph Schiessl
  • Christoph Zwerschke
  • CodeGrades
  • CodeSnipers
  • CodersLegacy
  • Coding Diet
  • Core Dispatch
  • Corey Gallon
  • Corey Goldberg
  • Cormoran Project
  • Cross-Platform Command Line Tools
  • CubicWeb
  • Curtis Miller
  • DSPIllustrations.com
  • DaPythonista
  • Daily Tech Video (Python)
  • Dallas Fort Worth Pythoneers
  • Dan Crosta
  • Dan Stromberg
  • Dan Yeaw
  • Daniel Bader
  • Daniel Nouri
  • Daniel Roy Greenfeld
  • Data Community DC
  • Data School
  • DataWars.io
  • Dave Beazley
  • David Amos
  • David Caron
  • David Goodger
  • David Lindelof
  • David MacIver
  • David Malcolm
  • David Szotten
  • Davide Moro
  • Davy Mitchell
  • Davy Wybiral
  • Declassed Art
  • Denis Kurov
  • Django Weblog
  • Djangostars
  • Doing Math with Python
  • Doug Hellmann
  • Dougal Matthews
  • Duncan McGreggor
  • Ed Crewe
  • Edward K. Ream
  • Eli Bendersky
  • Eniram Ltd.
  • Eray Özkural (examachine)
  • Erik Marsja
  • Etienne Desautels
  • EuroPython
  • EuroPython Society
  • Evennia
  • Everyday Superpowers
  • Fabio Zadrozny
  • Filip Wasilewski
  • Filipe Saraiva
  • First Institute of Reliable Software
  • Flavio Coelho
  • Flavio Percoco
  • Floris Bruynooghe
  • Frank Wierzbicki
  • François Dion
  • François Marier
  • Frederik Rietdijk
  • From Python Import Podcast
  • Full Stack Python
  • Gaël Varoquaux
  • Georges Dubus
  • Ghaandee on IT
  • Giampaolo Rodola
  • Giulio Fidente
  • Glenn Franxman
  • Glyph Lefkowitz
  • Go Deh
  • Gocept Weblog
  • Godson Gera
  • Graeme Cross
  • Graham Dumpleton
  • Graham Wheeler
  • Grant Baillie
  • Grant Rettke
  • Greg Taylor
  • Grig Gheorghiu
  • Grzegorz Śliwiński
  • Guido van Rossum
  • Gustavo Narea
  • Gustavo Niemeyer
  • Gökhan Sever
  • Hernan Grecco
  • Hilary Mason
  • Holger Peters
  • HoloViz
  • Hugo van Kemenade
  • Humberto Rocha
  • Hynek Schlawack
  • Ian Ozsvald
  • Ilian Iliev
  • Import Python
  • Inspired Python
  • Ionel Cristian Maries
  • IronPython-URLs
  • IslandT
  • Israel Fruchter
  • Itamar Turner Trauring
  • Ivan Velichko
  • J. Pablo Fernández
  • Jack Diederich
  • Jacob Perkins
  • Jahongir Rahmonov
  • Jaime Buelta
  • Jamal Moir
  • James Bennett
  • Janusworx
  • Jarrod Millman
  • Jean-Paul Calderone
  • Jeff Bisbee
  • Jeff Bradberry
  • Jeff Hinrichs
  • Jeff Shell
  • Jeremy Epstein
  • Jeremy Hylton
  • Jim Fulton
  • Joe Abbate
  • Joe Pitz
  • Johan Dahlin
  • John Burns
  • John Cook
  • John Jacobsen
  • John Ludhi/nbshare.io
  • Jon Parise
  • Jonathan Ellis
  • Jonathan Harrington
  • Jonathan Hartley
  • Jonathan Street
  • Jorgen Schäfer
  • Juan Manuel Contreras
  • Julien Danjou
  • Julien Tayon
  • Juri Pakaste
  • Just a little Python
  • Justin Mayer
  • Kai Lautaportti
  • Karim Elghamrawy
  • Kay Hayen
  • Kay Schluehr
  • Kelly Yancey
  • Kevin Renskers
  • Kodnito
  • Kogan Dev
  • Konrad Delong
  • Koodaamo
  • Kristján Valur Jónsson
  • Kriti Godey
  • Kulbir Saini
  • Kumar McMillan
  • Kumar Vipin Yadav
  • Kushal Das
  • Károly Nagy
  • LAAC Technology
  • Laurent Szyster
  • Leigh Honeywell
  • Lennart Regebro
  • Leonhard Vogt
  • Lintel Technologies
  • Linux Stans
  • ListenData
  • Logilab
  • Low Kian Seong
  • Luca Botti
  • Lucas Cimon
  • Ludovic Gasc
  • Ludvig Ericson
  • Luke Plant
  • Maciej Fijalkowsk
  • Made With Mu
  • Mahmoud Hashemi
  • Malthe Borch
  • Marc Richter
  • Marc-André Lemburg
  • Marcos Dione
  • Mariatta
  • Marius Gedminas
  • Mark Dufour
  • Mark McLoughlin
  • Mark McMahon
  • Martijn Pieters
  • Martin Fitzpatrick
  • Mats Kindahl
  • Matt Layman
  • Matthew Rocklin
  • Matthew Rollings
  • Matthew Wilson
  • Matthew Wright
  • Mattias Brändström
  • Mauveweb
  • Michael Becker
  • Michael Droettboom
  • Michael Foord
  • Michael Nelson
  • Michal Kwiatkowski
  • Michał Bultrowicz
  • Michele Simionato
  • Michy Alice
  • Mike C. Fletcher
  • Mike Driscoll
  • Mike Müller
  • Mikhail Korobov
  • Mikko Ohtamaa
  • Mirek Długosz
  • Mitchell Garnaat
  • Mitya Sirenef
  • Montreal Python User Group
  • Moshe Zadka
  • Moya Project
  • Mozilla Web Development
  • Muharem Hrnjadovic
  • Mycli
  • Nadav Samet
  • Naomi Ceder
  • Natan Zabkar
  • Ned Batchelder
  • Neil Schemenauer
  • Nick Coghlan
  • Nick Craig-Wood
  • Nick Efford
  • Nick Janetakis
  • Nicola Iarocci
  • Nicolas Dumazet
  • Nicolas Paris
  • Nigel Babu
  • Nikola
  • Nikolaos Diamantis
  • Not Invented Here
  • Nsukami Patrick
  • Obey the Testing Goat
  • Ofosos
  • Omaha Python Users Group
  • Ondřej Čertík
  • Paolo Melchiorre
  • Pathwright
  • Patrice Neff
  • Patrick Altman
  • Patrick Kennedy
  • Patrick Müller
  • Paul Everitt
  • Paul Harrison
  • Paul Redman
  • Paweł Fertyk
  • Pedro Lima
  • Peter Bengtsson
  • Peter Eisentraut
  • Peter Fankhänel
  • Peter Harkins
  • Peter Hoffmann
  • Phil Hassey
  • Philip Jenvey
  • Philipp von Weitershausen
  • Philippe Normand
  • Phillip J. Eby
  • Podcast.__init__
  • Polyglot.Ninja()
  • Possbility and Probability
  • Pradeep Gowda
  • Pranav Pandey
  • Praveen Gollakota
  • Programiz
  • Programming Ideas With Jake
  • Przemysław Kołodziejczyk
  • PyATL Bytecode
  • PyBites
  • PyCharm
  • PyCoder’s Weekly
  • PyCon
  • PyCon Ireland
  • PyPodcats
  • PyPy
  • PyTennessee
  • Python 4 Kids
  • Python 411 Podcast
  • Python Advocacy
  • Python Anywhere
  • Python Bytes
  • Python Celery - Weekly Celery Tutorials and How-tos
  • Python Circle
  • Python Data
  • Python Diary
  • Python Docs Editorial Board
  • Python Does What?!
  • Python Engineering at Microsoft
  • Python GUIs
  • Python Insider
  • Python Koans
  • Python Morsels
  • Python People
  • Python Piedmont Triad User Group
  • Python Pool
  • Python Software Foundation
  • Python Sweetness
  • Python User Groups
  • Python for Beginners
  • Python on Karan
  • Python with Myo
  • Python(x,y) News
  • PythonClub - A Brazilian collaborative blog about Python
  • PythonDebugging.com
  • Pythonicity
  • Pythonology
  • Python⇒Speed
  • Péter Szabó
  • Péter Zsoldos
  • Quansight Labs Blog
  • R David Murray
  • RMOTR
  • Ralph Bean
  • Ram Rachum
  • Randell Benavidez
  • Randle Taylor
  • Randy Zwitch
  • Raymond Hettinger
  • Read the Docs
  • Real Python
  • Red Hat Developers
  • Rene Dudfield
  • Reuven Lerner
  • Richard Tew
  • Richard Wall
  • Rickard Lindberg
  • Rishi Maker
  • Rob Galanakis
  • Robert Collins
  • Robert Zaremba
  • Robin Parmar
  • Robin Wilson
  • Rodrigo Araúj
  • Rodrigo Girão Serrão
  • RoseHosting Blog
  • Ruslan Spivak
  • Ryan Cox
  • S. Lott
  • S. R. Krishnan
  • SDJournal
  • SPE Weblog
  • STX Next
  • Salim Fadhley
  • Sandipan Dey
  • Sandro Tosi
  • Sean McGrath
  • Sebastian Pölsterl
  • Sebastian Witowski
  • Selena Deckelmann
  • Senthil Kumaran
  • Seth Michael Larson
  • Shannon -jj Behrens
  • ShiningPanda
  • Simeon Franklin
  • Simeon Visser
  • Simon
  • Simon Brunning
  • Simon Wittber
  • Simple is Better Than Complex
  • SoftFormance
  • Speed Matters
  • Spike ekipS
  • Spyder IDE
  • Stack Abuse
  • Stanislas Morbieu
  • Starzel.de
  • Stefan Behnel
  • Stefan Scherfke
  • Stefanie Molin
  • Stein Magnus Jodal
  • Stephen Ferg
  • Steve Holden
  • Steven Klass
  • Stuart Gordon Reid
  • Stéphane Wirtel
  • Sumana Harihareswara - Cogito, Ergo Sumana
  • Suresh Dasari/Tutlane.com
  • Swisscom ICT
  • Talk Python Blog
  • Talk Python to Me
  • Taylor Edmiston
  • TechBeamers Python
  • Techiediaries - Django
  • Terri Oda
  • Terry Jones
  • Test and Code
  • TestDriven.io
  • The Data Scientist
  • The Digital Cat
  • The Lunar Cowboy
  • The No Title® Tech Blog
  • The Occasional Occurrence
  • The Open Sourcerer
  • The Parcon Blog
  • The Python Coding Blog
  • The Python Coding Stack
  • The Python Papers
  • The Python Show
  • The Three of Wands
  • Thibauld Nion
  • ThisHosting.Rocks
  • Thomas Guest
  • Thomas Vander Stichele
  • Tibo Beijen
  • Tim Arnold / reachtim
  • Tim Gilbert
  • Tim Knapp
  • Tim Lesher
  • Tobias Ivarsson
  • Tom Christie
  • Tomasz Ducin
  • Tomaž Muraus
  • Tomer Filiba
  • Tony Breyal
  • Toshio Kuratomi
  • Travis Oliphant
  • Trey Hunner
  • Tryton News
  • Turnkey Linux
  • Twisted Matrix Labs
  • TypeThePipe
  • V.S. Babu
  • Varun Nischal
  • Vasudev Ram
  • Vinay Sajip
  • Vinay Sajip (Logging)
  • Vinayak Mehta
  • Virgil Dupras
  • Vladimir Perić
  • Wayne Witzel
  • Weekly Python Chat
  • Weekly Python StackOverflow Report
  • Wes Mason
  • Wesley Chun
  • Will Kahn-Greene
  • Will McGugan
  • Will Pierce
  • William Minchin
  • William Reade
  • Wing Tips
  • Wingware
  • Wyatt Baldwin
  • Yair Lenga
  • Yaniv Aknin
  • Yann Larrivée
  • Yuval Greenfield
  • Zaki Akhmad
  • Zato Blog
  • Zero to Mastery
  • Zero-with-Dot (Oleg Żero)
  • ZeroDB
  • bottlepy-dev
  • codeboje
  • death and gravity
  • eGenix.com
  • hypothesis.works articles
  • kdev-python
  • meejah.ca
  • nl-project
  • pgcli
  • py.CheckIO
  • pygame
  • pythonwise
  • qutebrowser development blog
  • saaj/recollection
  • scikit-learn
  • testmon
  • tryexceptpass
  • Éric Araujo
  • Łukasz Langa
  • To request addition or removal, open a PR or issue