Planet Python

Last update: September 01, 2026 04:48 PM UTC

September 01, 2026

Topics covered in this episode: <ul> <li><a href="https://github.com/openai/openai-python/blob/main/httpx2.md?featured_on=pythonbytes">OpenAI's Python SDK has migrated to HTTPX2</a></li> <li><a href="https://tmog.org?featured_on=pythonbytes">TMOG - Native Task Manager for macOS, Windows, and Linux</a></li> <li><a href="http://grahamdumpleton.me/posts/2026/08/introducing-wrapture/?featured_on=pythonbytes">wrapture - one wrapper for mocking, tracing, and observability</a></li> <li><a href="https://github.com/juanmanueldaza/linkedin2md?featured_on=pythonbytes">linkedin2md</a>: turn your LinkedIn export into 40+ Markdown files</li> <li>Extras</li> <li>Joke</li> </ul><a href='https://www.youtube.com/watch?v=4AUE4ewgN38' style='font-weight: bold;'data-umami-event="Livestream-Past" data-umami-event-episode="494">Watch on YouTube</a> About the show Sponsored by us! Support our work through: <ul> <li>Our <a href="https://training.talkpython.fm/?featured_on=pythonbytes">courses at Talk Python</a></li> <li>Consulting from <a href="https://sixfeetup.com/?featured_on=pythonbytes">Six Feet Up</a> </li> </ul> Connect with the hosts <ul> <li>Michael: <a href="https://fosstodon.org/@mkennedy">Mastodon</a> / <a href="https://bsky.app/profile/mkennedy.codes?featured_on=pythonbytes">BlueSky</a> / <a href="https://x.com/mkennedy?featured_on=pythonbytes">X</a> / <a href="https://www.linkedin.com/in/mkennedy/?featured_on=pythonbytes">LinkedIn</a></li> <li>Calvin: <a href="https://sixfeetup.social/@calvin?featured_on=pythonbytes">Mastodon</a> / <a href="https://bsky.app/profile/calvinhp.com?featured_on=pythonbytes">BlueSky</a> / <a href="https://x.com/calvinhp?featured_on=pythonbytes">X</a> / <a href="https://www.linkedin.com/in/calvinhp/?featured_on=pythonbytes">LinkedIn</a></li> <li>Show: <a href="https://fosstodon.org/@pythonbytes">Mastodon</a> / <a href="https://bsky.app/profile/pythonbytes.fm">BlueSky</a> / <a href="https://x.com/PythonBytes?featured_on=pythonbytes">X</a> </li> </ul> Join us on YouTube at <a href="https://pythonbytes.fm/stream/live">pythonbytes.fm/live</a> to be part of the audience. Usually Tuesday at 7am PT. Older video versions available there too. Finally, if you want an artisanal digest of every week of the show notes in email form? Add your name and email to <a href="https://pythonbytes.fm/friends-of-the-show">our friends of the show list</a>, we'll never share it. Calvin #1: <a href="https://github.com/openai/openai-python/blob/main/httpx2.md?featured_on=pythonbytes">OpenAI's Python SDK has migrated to HTTPX2</a> <ul> <li>The OpenAI Python SDK has migrated to HTTPX2, the Pydantic-stewarded fork of httpx. Pydantic picked it up citing "limited activity recently" in the original project, promising "a reliably maintained path forward."</li> <li>If you just use the default client, nothing to do. No code changes.</li> <li>The catch is TLS. Quoting the guide: HTTPX "previously verified certificates against the CA bundle provided by certifi. HTTPX2 instead uses the operating-system trust store, and the SDK no longer installs certifi."</li> <li>That "can break certificate verification in minimal container images without system CA certificates, environments using corporate TLS-inspecting proxies, and deployments that relied on a custom or modified certifi bundle."</li> <li>The fix is SSL_CERT_FILE or SSL_CERT_DIR, or pass your own ssl.SSLContext via verify.</li> <li>Deeper integrations need real edits: custom clients, auth handlers, hooks, and request mocking all take HTTPX2 objects now, and plain httpx is no longer pulled in transitively. So import httpx in your own code means declaring it yourself or moving over.</li> <li>Temporary escape hatch: a legacy HTTPX client</li> </ul> Michael #2: <a href="https://tmog.org?featured_on=pythonbytes">TMOG - Native Task Manager for macOS, Windows, and Linux</a> A native, deeply instrumented system monitor for macOS, Windows, and Linux, now in public beta - from Plummers' Software, i.e. Dave Plummer, who wrote the original Windows Task Manager and donated it to Microsoft in 1995. <a href="https://en.wikipedia.org/wiki/Dave_Plummer?featured_on=pythonbytes">Wikipedia</a> <ul> <li>Three real native apps: Swift/AppKit on macOS, Win32 on Windows, C++/Qt 6 on Linux, with a shared C++ core keeping metric semantics aligned - no browser shell anywhere.</li> <li>One dense summary: CPU, clocks, thermals, GPU, memory, storage, network, energy, and the processes responsible for the load, all click-through.</li> <li>Per-core honesty: logical processor and NUMA views, P and E cores color-coded, optional kernel time, 60 FPS live meters.</li> <li>Memory with context: pressure, wired, compressed, cached, committed, available, and swap, plus configurable scrolling history.</li> <li>Processes that act like processes: tree view, filtering, sorting, follow mode, and native verbs including service and launchd control.</li> <li>Phosphor themes: light, dark, green, amber, blue, or mono, with color and saturation you tune yourself.</li> </ul> Calvin #3: <a href="http://grahamdumpleton.me/posts/2026/08/introducing-wrapture/?featured_on=pythonbytes">wrapture - one wrapper for mocking, tracing, and observability</a> <ul> <li>Graham Dumpleton, author of wrapt and the original New Relic Python agent, has released wrapture. The name is wrapt plus capture. The core idea: wrap real code instead of replacing it, so the real code still runs while you watch every call.</li> <li>Name a method with wrapture.binding(Class, "method"), open a timeline(), and you get a tape of what actually happened. Real return values, real nesting, arguments normalised against real signatures. tape.tree() prints the call graph as it ran.</li> <li>One mechanism, three jobs: monkey patching with a real lifecycle (apply, remove, suspend, plus returns, raises, transforms_args), unit testing that asserts on real call flow instead of a flat MagicMock call list, and ad-hoc tracing of a running app.</li> <li>The testing pitch is error paths. Inject TimeoutError at the payment gateway, then assert the ledger was never written. Stubs and mocks are strict and spec-required, and there is deliberately no bare Mock().</li> <li>Tracing needs no code at all. A wrapture.toml naming targets and a sink, run with python -m wrapture <a href="http://main.py/?featured_on=pythonbytes">main.py</a>, and you get a live call tree with timings. It captures ordinary logging calls as nested events, and with the otel extra it exports spans, metrics and correlated logs with W3C trace ids that join across services.</li> <li>Every line of code and docs was AI-written under their direction, and they say so up front. Two weeks from first commit, eleventh alpha, over 1000 tests, 150+ pages of docs. Alpha on PyPI, needs Python 3.12+ and wrapt 2.4.0+.</li> </ul> Michael #4: <a href="https://github.com/juanmanueldaza/linkedin2md?featured_on=pythonbytes">linkedin2md</a>: turn your LinkedIn export into 40+ Markdown files Via Juan Manuel Daza - a Python CLI that unpacks LinkedIn's data-export ZIP into clean, per-category Markdown you can drop straight into an LLM. <ul> <li>One command: <code>linkedin2md Complete_LinkedInDataExport.zip</code>, plus <code>o</code> for output dir, <code>-lang en|es</code>, and <code>-pdf</code>.</li> <li>40+ output files: profile, experience, education, skills, connections, posts, comments, reactions, recommendations, endorsements, job applications, even ad targeting and LinkedIn's inferences about you.</li> <li>Built for LLM analysis: the README pitches NotebookLM, Claude Projects, Obsidian, and Ollama, with example prompts like "what patterns do you see in my career transitions?"</li> <li>PDF resume mode: <code>-pdf</code> renders an A4 CV via weasyprint, and degrades gracefully to Markdown-only if it isn't installed.</li> <li>Dependency note: "pure Python / zero-dep" holds for the Markdown path only - the PDF path needs weasyprint and markdown installed.</li> <li>Install: <code>pipx install linkedin2md</code> recommended, <code>pip</code> in a venv otherwise - 86% Python, 10 releases, v0.3.1 in May.</li> <li>Agentic dev angle: repo ships opencode config and an N3RV subagent pipeline, including a "judgment day" dual-model adversarial PR review.</li> </ul> Extras Calvin: <ul> <li><a href="https://www.eveonline.com/news/view/the-move-to-python-3-begins?featured_on=pythonbytes">EVE Online Migrates to Python 3</a></li> </ul> Michael: <ul> <li><a href="https://dinkus.textualize.io?featured_on=pythonbytes">Dinkus</a> by Will McGugan</li> </ul> Joke: <a href="https://www.talisman.org/tao/?featured_on=pythonbytes">Tao of Programming: Book 5 Maintenance</a>

September 01, 2026 03:39 PM UTC

Hugo van Kemenade

Help test Python 3.15!

Calling all Python library maintainers! 🐍

The second and final Python 3.15 release candidate is out! 🎉

It’s your last chance to join the cool testers’ club and say that YOU helped test 3.15 before the big day on 1st October! 😎

PEP 790 defines the release schedule for Python 3.15.0:

The first release candidate came out on 4th August 2026
The second release candidate came out on 1st September 2026
And the full release is set for 1st October 2026

In his announcement, Hugo van Kemenade (👋 hi, that’s me!), release manager for Python 3.14 and 3.15, gave a call to action:

We strongly encourage maintainers of third-party Python projects to prepare their projects for 3.15 during this phase, and publish Python 3.15 wheels on PyPI to be ready for the final release of 3.15.0, and to help other projects do their own testing. Any binary wheels built against Python 3.15.0 release candidates will work with future versions of Python 3.15. As always, report any issues to the Python bug tracker.

Test with 3.15 #

It’s now time for us library maintainers to start testing our projects with 3.15. There’s two big benefits:

There have been removals and changes in Python 3.15. Testing now helps us make our code compatible and avoid any big surprises (for us and our users) at the big launch in October.
We might find bugs in Python itself! Reporting those will help get them fixed and help everyone.

How #

To test the latest alpha, beta or release candidate on GitHub Actions with actions/setup-python, add 3.15 and allow-prereleases: true to your workflow matrix.

For example:

jobs:
 test:
 runs-on: ubuntu-latest
 strategy:
 fail-fast: false
 matrix:
 python-version: ["3.11", "3.12", "3.13", "3.14", "3.15"]

 steps:
 - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

 - name: Set up Python ${{ matrix.python-version }}
 uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
 with:
 python-version: ${{ matrix.python-version }}
 allow-prereleases: true

(We can instead use 3.15-dev and omit allow-prereleases: true, but I find the above a bit neater, and when 3.15.0 final is released in October, it will continue testing with full release versions.)

When to support 3.15? #

Now is also a good time to declare support and add the Programming Language :: Python :: 3.15 Trove classifier. Many projects already have!

Especially if you have extension modules and other projects depend on yours, a release with binary wheels will help them test and prepare.

ABI breaks? #

From the announcement:

There will be no ABI changes from this point forward in the 3.15 series, and the goal is that there will be as few code changes as possible.

Let’s start testing 3.15 now! 🚀

The code under test

An order service which takes a payment through a gateway, records it in a ledger, and sends a notification. If the ledger write fails, the payment is refunded and the error propagates. The collaborators can be injected through the constructor, so there is a seam for a mock to sit behind, and the payment step goes through a private method on the service itself.

class Gateway:
    def charge(self, amount, currency="USD"):
        return {"id": f"ch_{amount}", "amount": amount}

    def refund(self, charge_id):
        return {"id": f"re_{charge_id}"}


class Ledger:
    def record(self, entry):
        return f"led_{entry['id']}"


class Notifier:
    def send(self, message):
        return True


class OrderService:
    def __init__(self, gateway=None, ledger=None, notifier=None):
        self.gateway = Gateway() if gateway is None else gateway
        self.ledger = Ledger() if ledger is None else ledger
        self.notifier = Notifier() if notifier is None else notifier

    def place(self, amount):
        charge = self._take_payment(amount)
        try:
            self.ledger.record(charge)
        except Exception:
            self.gateway.refund(charge["id"])
            raise
        self.notifier.send(f"order {charge['id']} placed")
        return charge

    def _take_payment(self, amount):
        return self.gateway.charge(amount)

Where the two look the same

Stubbing a return value is the bread and butter of both tools, and on the surface there is nothing to choose between them:

from unittest.mock import patch

def test_stub_with_mock():
    with patch.object(Gateway, "charge", return_value={"id": "stub", "amount": 0}):
        assert OrderService().place(500)["id"] == "stub"

import wrapture

def test_stub_with_wrapture():
    with wrapture.binding(Gateway, "charge").on_call.returns({"id": "stub", "amount": 0}):
        assert OrderService().place(500)["id"] == "stub"

Even here there is a difference under the surface. A mock only checks a stubbed call against the real signature if you asked for autospec=True, so without it a call that has drifted from the method's signature returns the stub happily:

def test_drifted_call_with_mock():
    with patch.object(Gateway, "charge", return_value={"id": "stub"}):
        assert Gateway().charge(500, bogus=True) == {"id": "stub"}

That test passes. A wrapture binding is strict by default, so a call the real method would have rejected is rejected by the stub too:

def test_drifted_call_with_wrapture():
    with wrapture.binding(Gateway, "charge").on_call.returns({"id": "stub"}):
        with pytest.raises(TypeError):
            Gateway().charge(500, bogus=True)

The error names the site and the problem:

TypeError: orders:Gateway.charge (stubbed): got an unexpected keyword argument 'bogus'

There is a strict=False option for the rare patch which genuinely means to accept a different shape, but the default is the direction I care about. A test which passes because the stub was more forgiving than the real code is a test that will be wrong in production.

Calls an object makes to itself

The mock approach to testing OrderService is to inject a MagicMock as the gateway and assert on what it recorded:

from unittest.mock import MagicMock

def test_self_call_with_mock():
    gateway = MagicMock()
    service = OrderService(gateway=gateway)
    service.place(500)
    gateway.charge.assert_called_once_with(500)

Look at what the mock actually saw, by printing gateway.mock_calls after the call:

[call.charge(500),
 call.charge().__getitem__('id'),
 call.charge().__getitem__().__str__(),
 call.charge().__getitem__('id'),
 call.charge().__getitem__().__str__()]

The charge() call is there, followed by a trail of fabricated chains as the service reached into a return value that was never a real dictionary. What is not there, and cannot be, is _take_payment(). The call from place() to _take_payment() never crosses the seam the mock sits behind, so as far as the test can tell the private method does not exist. If you instead reach for patch.object(OrderService, "_take_payment") to get at it, you have replaced it, and now the real payment logic does not run and the gateway is never charged. Either the method is invisible or it is gone.

With wrapture the binding is on the class, so a call the object makes to itself passes through the wrapper like any other:

def test_self_call_with_wrapture():
    take_payment = wrapture.binding(OrderService, "_take_payment")
    charge = wrapture.binding(Gateway, "charge")

    with wrapture.timeline(take_payment, charge) as tape:
        OrderService().place(500)

        take_payment.events.with_args(amount=500).assert_once()
        assert tape.parent_of(charge.events.first) is take_payment.events.first

The second assertion says the charge happened inside the payment step, and tape.tree() shows the same thing:

orders:OrderService._take_payment(amount=500)  -> {'id': 'ch_500', 'amount': 500}
  orders:Gateway.charge(amount=500, currency='USD')  -> {'id': 'ch_500', 'amount': 500}

Real arguments, normalised against the real signature so the currency default appears even though the caller never passed it, and real return values, nested the way the calls actually nested.

Running the real code while changing one thing

This is where substitution runs out of road entirely. Mock(wraps=real) will forward calls to the real method, but it cannot change the arguments the real method receives, and it cannot touch the result on the way back. The standard library has no way to say "run the real method, but change one thing about it".

For wrapture that is the ordinary case. Here the real charge() runs and only the id in its result is rewritten, which is useful when a real id would be unstable across runs but everything else about the result matters:

def test_pinned_result_with_wrapture():
    charge = wrapture.binding(Gateway, "charge")
    charge.on_call.transforms_result(lambda r: {**r, "id": "ch_TEST"})

    with charge:
        assert OrderService().place(500) == {"id": "ch_TEST", "amount": 500}

transforms_args() does the same on the way in, and validates_args() and validates_result() check without changing. These are stages, and they compose, so a binding can rewrite one argument and check the result at the same time while the real code does the actual work in between.

Asserting on what did not happen

The tests that matter most are usually on the error paths, and the interesting fact on an error path is often an absence. When the ledger write fails, the refund must be issued and the notification must not be sent. Here it is with mock, and it takes three doubles to write:

def test_error_path_with_mock():
    gateway = MagicMock()
    ledger = MagicMock()
    ledger.record.side_effect = OSError("disk full")
    notifier = MagicMock()

    service = OrderService(gateway, ledger, notifier)

    with pytest.raises(OSError):
        service.place(500)

    gateway.refund.assert_called_once_with(gateway.charge.return_value["id"])
    notifier.send.assert_not_called()

The refund assertion is the awkward part. Because the gateway is a mock, the charge id is a fabricated MagicMock rather than "ch_500", so the only way to assert on it is to ask the mock what it invented. The test cannot say "the refund was for the charge that was taken", only "the refund was passed whatever charge() returned", which is the same thing only if you trust the code you are testing. (With a plain Mock rather than MagicMock the test does not even get that far, since charge["id"] fails with a TypeError about subscripting.)

With wrapture the failure is injected at the ledger and nothing else is touched:

def test_error_path_with_wrapture():
    charge = wrapture.binding(Gateway, "charge")
    refund = wrapture.binding(Gateway, "refund")
    record = wrapture.binding(Ledger, "record")
    send = wrapture.binding(Notifier, "send")

    record.on_call.raises(OSError("disk full"))

    with wrapture.timeline(charge, refund, record, send) as tape:
        with pytest.raises(OSError):
            OrderService().place(500)

        refund.events.with_args(charge_id="ch_500").assert_once()
        send.events.assert_never()
        tape.assert_order(charge, record, refund)

The real gateway was charged, so the refund is asserted against the real charge id. The notifier is real and was never called. And assert_order() says the refund came after the failed ledger write, across three different bindings. The tape shows exactly that:

orders:Gateway.charge(amount=500, currency='USD')  -> {'id': 'ch_500', 'amount': 500}
orders:Ledger.record(entry={'id': 'ch_500', 'amount': 500})  !! OSError (injected)
orders:Gateway.refund(charge_id='ch_500')  -> {'id': 're_ch_500'}

When an assertion fails, the message shows what was recorded rather than just the count that was wrong. Asserting on a refund for the wrong id gives:

AssertionError: expected exactly 1 event(s), got 0
<EventLog orders:Gateway.refund[charge_id='ch_999']: 0 event(s)>
    (no events)
  filtered from:
    <EventLog orders:Gateway.refund: 1 event(s)>
        orders:Gateway.refund(charge_id='ch_500')

The "filtered from" section is there because an over-narrowed filter producing an empty log is the easiest way to get a wrong assertion, and showing what the filter discarded is the fastest way to see it.

How the tests are shaped

Beyond what the assertions can say, the shape of the test code changes in a few ways that are worth pointing out.

A binding declares a target without touching it. binding() never patches, so bindings can be created at module scope, given behaviour, and shared, with each test applying and removing them. Nothing happens until apply(), a with block, or a timeline. That separation of declaration from effect is what lets the four bindings in the error path test be declared up front and read like a cast list.

The with block is the primitive, and for a test that binds one or two targets around its whole body there is a decorator form that says the same thing without the nesting. The bindings arrive as keyword arguments, and expectations can be declared on the decorator and are verified when the test finishes:

@wrapture.taped()
@wrapture.bound(Ledger, "record").on_call.raises(OSError("disk full"))
@wrapture.bound(Gateway, "refund").expect_once()
@wrapture.bound(Notifier, "send").expect_never()
def test_error_path_with_decorators(tape, record, refund, send):
    with pytest.raises(OSError):
        OrderService().place(500)

That is the same test as before with the assertions moved to the top as a contract, and a body that only performs the action. An expectation with nothing recording is an error rather than a silent pass.

Fixtures work the way you would expect, and because a fixture yields the binding a test can reconfigure it mid-flight, which is how a test walks a collaborator through failing and then recovering:

@pytest.fixture
def stub_charge():
    with wrapture.binding(Gateway, "charge").on_call.returns({"id": "stub", "amount": 0}) as charge:
        yield charge


def test_gateway_recovers(stub_charge):
    stub_charge.on_call.raises(TimeoutError("down"))
    with pytest.raises(TimeoutError):
        OrderService().place(500)

    stub_charge.on_call.returns({"id": "retry", "amount": 0})
    assert OrderService().place(500)["id"] == "retry"

Finally there is an opt-in pytest plugin, enabled with one line in conftest.py, which fails any test that leaves a binding applied and attaches the tape's tree to the failure report of any test that recorded one. The leak sweep is the thing I would turn on first. A patch that leaks changes the behaviour of every test that runs after it, and I suspect plenty of people have lost hours to that without ever knowing which test was the culprit.

Where mock still fits

Everything above follows one rule, which is to wrap the real code and record what actually flowed, with one deliberate opt-out. When the test itself must supply the thing being called, because the code under test receives a callback or a collaborator rather than importing one, stub() supplies a single callable and mock(Spec) a whole object. Both are strict, in that signatures are checked and a name the spec does not have raises, and both record onto the same tape as everything else.

What wrapture does not provide is a spec-less MagicMock() whose attributes exist on first touch and whose call chains all answer. That is the thing which would let a misspelled gateway.chargee.assert_not_called() pass silently in the tests above, and it is left out on purpose. If a test wants an object invented as it is touched, unittest.mock is the tool for that, and it has the other advantage of being in the standard library, on every team's common ground. The two coexist in one suite without difficulty, and the comparison page is there for translating between them.

What's next

The error path test leaned on the timeline and the tape without much explanation of what they are or what an event holds. That is the subject of the next post, where the example is a resource leak, which is a bug nothing in a return value will ever tell you about.

September 01, 2026 07:38 AM UTC

Python Insider

Python 3.15.0 candidate 2 is here!

Last chance to test before the big day!

September 01, 2026 12:00 AM UTC

Graham Dumpleton

Introducing wrapture

For the best part of two decades, through wrapt, I have been dealing with the mechanics of monkey patching in Python. Anyone who has followed wrapt will know I am quite pedantic about correctness, to the point of caring whether a wrapper preserves every last introspectable detail of the thing it wraps. For much of that time I have wanted the same standard from the tools I use when testing code, and unittest.mock, which does what it does well enough, was never designed to give it. A fabricated Mock answers every method call and verifies nothing. A patched call records a flat list of calls, with no return values and no sense of what was called from what. The calls an object makes to itself are invisible, because the substitute never runs the real code at all. What I want a test to be sure of, that the right calls happened, in the right order, with the real code actually running, sits just outside what substitution can express.

Seeing the real calls as they happen, with the real code still doing the work, was also something I had already spent years on for a quite different purpose. I was the original author of the New Relic Python agent, written while I worked there, and that left me with a lasting interest in instrumenting Python programs. Attaching observation to code you do not control, recording what flows through it, and doing so without disturbing the program being watched, is a problem I have never really stopped thinking about.

Testing and tracing look like different problems, but from where I sat they wanted the same thing, and I had long believed that wrapt's approach of wrapping real code rather than replacing it could serve both. wrapture (the name being wrapt plus capture) is me finally getting around to finding out whether that belief held up.

Wrap anything, capture everything

The one idea everything in wrapture sits on is to wrap rather than replace. A binding names a location in code, a method of a class or a function in a module, and when applied installs a wrapt wrapper around the real callable. Unless you tell it otherwise the wrapper is transparent. The real code runs, with wrapture in a position to watch the call, change it, or answer it instead.

Take a small call graph where an order service charges a payment gateway and then records the result in a ledger:

import wrapture

class Gateway:
    def charge(self, amount, currency="USD"):
        return {"id": f"ch_{amount}", "amount": amount}

class Ledger:
    def record(self, entry):
        return f"led_{entry['id']}"

class OrderService:
    def __init__(self):
        self.gateway = Gateway()
        self.ledger = Ledger()

    def place(self, amount):
        result = self.gateway.charge(amount)
        self.ledger.record(result)
        return result

None of these classes import wrapture or know they are about to be observed. Bindings are created by naming the methods, and a timeline opens a recording scope in which every call through them lands on a tape:

place = wrapture.binding(OrderService, "place")
charge = wrapture.binding(Gateway, "charge")
record = wrapture.binding(Ledger, "record")

with wrapture.timeline(place, charge, record) as tape:
    OrderService().place(500)

print(tape.tree())

Running this the output is:

__main__:OrderService.place(amount=500)  -> {'id': 'ch_500', 'amount': 500}
  __main__:Gateway.charge(amount=500, currency='USD')  -> {'id': 'ch_500', 'amount': 500}
  __main__:Ledger.record(entry={'id': 'ch_500', 'amount': 500})  -> 'led_ch_500'

That is the call graph as it actually ran. The arguments are normalised against the real signatures (so charge(500) and charge(amount=500) look the same), the return values are the real ones, and the nesting comes from what really called what. The tape.tree() call is a convenience for debugging and for demonstrations like this one; in a test you would query the tape instead, and outside of a test the events would be going to a sink, which I will come to.

The same bindings intervene as well as observe. The real method can be stubbed out, made to fail, or left running while one thing about the call is changed on the way in or out:

gateway = Gateway()

with wrapture.binding(Gateway, "charge").on_call.raises(TimeoutError("down")):
    gateway.charge(500)

Inside the block the call raises TimeoutError, and after the block exits the original method is back exactly as it was.

Three uses of one mechanism

That is the whole mechanism. What makes it interesting is that it serves three purposes which are usually handled by three different tools.

The first is plain monkey patching. wrapt's wrap_object() has always been able to patch a target, but it leaves the bookkeeping to you. wrapture adds a lifecycle and a vocabulary over the top of it. A binding declares a target without touching it, apply() installs the patch, remove() restores the original, suspend() makes it inert in place, and a group of bindings applies and removes as one unit. Behaviour is configured on the binding, with returns(), raises(), transforms_args(), transforms_result() and a few others, and can be scripted to change over time, so "succeed twice, then time out" is three lines rather than a hand-written counter. This layer is useful on its own with nothing else switched on.

The second is unit testing, which is where the recording comes in. Because the real code runs, a test can assert on how calls actually flowed through it, and the interesting cases are the error paths. Inject a failure at the gateway, then check that the ledger was never written:

with wrapture.timeline(place, charge, record) as tape:
    charge.on_call.raises(TimeoutError("down"))

    try:
        OrderService().place(500)
    except TimeoutError:
        pass

    record.events.assert_never()

    print(tape.tree())

With the assertion passing, the tree shows where the failure was injected and how it propagated:

__main__:OrderService.place(amount=500)  !! TimeoutError
  __main__:Gateway.charge(amount=500, currency='USD')  !! TimeoutError (injected)

When a test must supply a stand-in, because the code under test receives a collaborator rather than importing it, wrapture provides stub() for a callable and mock(Spec) for a whole object, and these record onto the same tape as everything else. Both are strict: signatures are checked and nothing is invented on first touch. There is deliberately no spec-less Mock() equivalent, and the comparison with unittest.mock in the documentation explains why, alongside a mapping of each mock idiom to its wrapture counterpart. An opt-in pytest plugin sweeps each test for patches left applied and attaches recordings to failure reports.

The third use is ad-hoc tracing of a running application, including one you cannot modify or redeploy. Take the bindings, drop the test around them, and the only remaining question is where the events go. A sink answers that. In practice this is done with a wrapture.toml file naming the targets and the sink, and no code at all. Here is one for a slightly bigger version of the shop above, where the gateway declines some cards and the order service logs a warning when it does:

[[observe]]
target = "shop:OrderService"
name = "place"

[[observe]]
target = "shop:PaymentGateway"
match = "*"
exclude = "_*"

[[observe]]
target = "shop:Ledger"
name = "record"

[[log]]
name = "shop.*"

[[sink]]
type = "printer"

Running the program as python -m wrapture main.py applies the config before the program starts, so the patches are in place before the application imports anything, and the printer sink writes the trace to stderr as it happens:

shop:OrderService.place(order_id='order-1', amount=30, card='5100-0010')
  shop:PaymentGateway.charge(amount=30, card='5100-0010')
  shop:PaymentGateway.charge -> 'ch_30' [7us]
  shop:Ledger.record(order_id='order-1', amount=30)
  shop:Ledger.record -> 'ledger:order-1:30' [5us]
shop:OrderService.place -> 'ch_30' [234us]
shop:OrderService.place(order_id='order-2', amount=240, card='5100-0020')
  shop:PaymentGateway.charge(amount=240, card='5100-0020')
  shop:PaymentGateway.charge -> 'ch_240' [4us]
  shop:Ledger.record(order_id='order-2', amount=240)
  shop:Ledger.record -> 'ledger:order-2:240' [3us]
shop:OrderService.place -> 'ch_240' [105us]
shop:OrderService.place(order_id='order-3', amount=75, card='4000-0030')
  shop:PaymentGateway.charge(amount=75, card='4000-0030')
  shop:PaymentGateway.charge !! PaymentDeclinedError [4us]
  log shop.orders WARNING 'order order-3 declined'
shop:OrderService.place !! PaymentDeclinedError [260us]

Unlike the tidy reconstruction from tape.tree(), this is the live view, with an opening line as each call begins and a closing line with the outcome and how long it took. The [[log]] entry captures the application's ordinary logging calls as events too, so the warning appears nested inside the call that logged it rather than somewhere in a separate log file. With autowrapt installed, even the launcher is unnecessary: AUTOWRAPT_BOOTSTRAP=wrapture in the environment applies the same config at interpreter startup, so the program runs with plain python. That covers the case where something else owns the command line, like a container entry point or a WSGI server.

The printer is the simplest sink. Others stream events to disk as JSON lines, count without retaining, and compose with fan-out, sampling and filtering. Sitting on top of the tracing layer is OpenTelemetry export: with the wrapture[otel] extra installed, one [otel] table in the config sends the same events to any OTLP backend as spans, metrics and correlated logs. Every tree of events carries a W3C trace id, and the id arrives and leaves in traceparent headers, so two services both observed by wrapture join up as one distributed trace without either of them calling an OpenTelemetry API.

The point I want to land is that these are layers of one mechanism and not separate products. The binding vocabulary that stubs a method in a test is the same one that traces it in production, and the config that names methods for a printed call tree is the config that exports spans. What starts as a monkey patch or a test assertion can grow into observability without the code being rewritten along the way.

Where it sits beside what already exists

Part of why I built this is that nothing I could find did all of it. unittest.mock records a flat call list with no nesting and no return values, and a patched call returns a fabricated MagicMock rather than running the real code. Span assertion tools such as OpenTelemetry's InMemorySpanExporter require the code to already be instrumented. Tools built on sys.settrace give you a firehose with no assertion API. APM agents are all-or-nothing products rather than toolkits, and their auto-instrumentation only covers the frameworks they already know about. wrapture needs none of that. You point at your own methods by name and a trace appears, and the same pointing is how it lands in a test, a terminal, or a backend.

Just as important is what it is not. It is not a fabrication tool, and unittest.mock remains the right thing for invented objects. It is not a production APM, although it is a toolkit that APM-like things could be built on. And it is not an OpenTelemetry competitor; it emits to OpenTelemetry rather than trying to replace it.

Pre-built instrumentation

Pointing at your own methods is the core of wrapture, but for common third-party packages the pointing has already been done. The companion wrapture-instrumentation package provides ready-made instrumentation, with Flask and Jinja2 covered so far. Each records a request or a template render as one structured tree, and enabling one is an [[instrument]] entry in wrapture.toml naming the target. Installing the package brings in wrapture and nothing else; the instrumentation for a package you do not have is inert. It is being built target by target, and the instrumentation packages guide describes how to write one for a package not yet covered.

Built with AI, on purpose

Every line of code and documentation in wrapture was written by an AI assistant working under my direction. I want to be upfront about that, and equally upfront about what it was not. This was not vibe coding, where a one-shot prompt produces a pile of generated code and the person driving hopes for the best because they lack the knowledge to judge what came back. Vibe coding has earned its bad reputation. I engineered wrapture carefully from the start. I have spent a long time in this particular corner of Python and knew exactly what the result needed to be, and the AI was the means of producing it rather than the source of the design.

The experiment had two halves. The first was whether an idea I had carried around for years actually held up once built. The second, and just as much the point, was whether a library of this kind could be produced this way, with an AI doing the writing and me doing the directing, to a standard I would be happy to put my name to.

The process is what makes the result worth trusting or not, so it deserves describing. The work started well before any code, with days spent on design documents setting out the goals, the scope, the shape of the API and the layers it would be built in, which the AI and I argued over before implementation began. From there it proceeded in layers, each one specified, discussed, implemented, tested and documented before the next began. Documentation grew with the code rather than after it, and every example in the docs runs as a doctest, so the docs are continually proven against the implementation. Writing them repeatedly exposed designs that read worse than they demoed. The test suite runs against every supported Python version, including the free-threaded builds, on every change. The overhead of Python instrumentation is the usual objection to it, so the recording path was also put through a performance pass, with the cost of a call observed and exported through wrapture measured against the same call instrumented directly with the OpenTelemetry SDK and in the style of its instrumentation packages. The result was comparable per call, with the figures in the OpenTelemetry export guide.

The step I would most recommend to anyone attempting something similar came late. I took the unit test suites of well-known Python packages that lean heavily on unittest.mock and had the AI replicate their tests using wrapture instead, side by side with the originals. Every point of friction became a decision: sometimes a documented position on why wrapture deliberately differs, and sometimes a missing feature that got specified, built and documented like everything else. Several pieces of wrapture exist only because a real test suite could not be expressed cleanly without them.

Throughout, the division of labour was consistent. The AI wrote the code, the tests and the prose. I set the direction, made the design calls, reviewed what came back, and sent plenty of it back. My experience with wrapt and with Python's darker corners is all through the result, in what was asked for as much as in what was refused.

The first commit was in the middle of August and the current release is the eleventh alpha, so this all happened in a bit over two weeks. In that time it accumulated over 1000 tests and over 150 pages of documentation. The documentation is admittedly quite dense in places and needs some work still, but it is complete in the sense that every part of the package is covered. One thing a brand new library has over an old one is coherence. Mature packages accrete features one release at a time, and there is never a moment when the whole API can be redesigned to match what was learned along the way. Because wrapture arrived in a compressed period with the whole design still in view, when validation showed a design could be better it was redesigned rather than worked around.

I know some people are firmly opposed to using AI-written software. If that is you, I understand, and I am not going to argue with your position. It is a reasonable one to hold, and this post exists so you can make the call with the facts in hand rather than discover them later. If AI involvement rules wrapture out for you then wrapture is not for you, and that does not worry me one bit. This has been about finding out whether the process works, and I now have my answer to that. The longer version of all this is on the how wrapture was built page in the documentation.

What's next

wrapture is in alpha, with pre-releases on PyPI. Until 1.0.0 is final a plain pip install wrapture picks up the latest pre-release, so there is no need to pin a version. It requires Python 3.12 or later and wrapt 2.4.0 or later. The API is complete for the three uses described above and I am not expecting it to break, so code written against it today should carry forward to 1.0.0.

What it needs now is use. unittest.mock and OpenTelemetry's own instrumentation are the established tools for the two halves of what wrapture does, and the open question is whether an alternative that does both from one mechanism is something people want. Reports of it working, or not, on real code, and of what confused or was missing, are what will decide whether anything changes before a beta. They go to the issue tracker.

To be clear, wrapture was never premised on anyone else using it. I built it because I wanted to see it exist, not because I had identified a gap in the market. If people find it useful and pick it up, that is great, and I will aim to support it. If there is no interest, I will keep treating it as an experiment and work on it for my own purposes. Either way the questions got answered, and getting them answered was the point.

There is a lot more in wrapture than fits in an introduction, and I expect to write about specific parts of it in follow-up posts, starting with how it can be used for unit testing, and then tracing a Flask application through to an OpenTelemetry backend without touching the application code. For now the getting started page is the place to begin.

September 01, 2026 12:00 AM UTC

Seth Michael Larson

Buying GameCube games from other regions

This blog post was inspired by discovering that Pokémon Box: Ruby & Sapphire, a legendarily rare and expensive GameCube game in the United States (>$2,500 USD), could be purchased complete-in-box (CIB) in Japan for $50 USD. That's a savings of $2450, or ~98% of the US price. This is a somewhat extreme example: in the US market Pokémon Box was only sold online and at a single store: the Pokémon Center in New York City.

This big price difference between regions got me thinking: what other games are cheaper in other regions and by how much? Please note that the below data was taken from Pricecharting on August 30th, 2026. Shipping costs aren't included in the price, shipping from Japan to the US is typically $15-$20 USD on eBay, but many games are already imported to the US.

Game		Diff.	Price (US)	Price (PAL)	Price (JP)
Pokémon Box	CIB	-$2348	$2400	$204	$52
Pokémon Box	Loose	-$1553	$1574	$137	$21
Cubivore	CIB	-$612	$640	—	$28
Cubivore	Loose	-$318	$341	—	$23
Gotcha Force	CIB	-$405	$474	$137	$69
Gotcha Force	Loose	-$204	$249	$121	$45
Resident Evil: Code Veronica X	CIB	-$239	$299	$80	$60
Resident Evil: Code Veronica X	Loose	-$151	$213	$62	—
Chibi Robo	CIB	-$189	$210	$223	$34
Chibi Robo	Loose	-$136	$144	$161	$25
Pokémon XD: Gale of Darkness	CIB	-$170	$209	$232	$62
Pokémon XD: Gale of Darkness	Loose	-$129	$150	$183	$54
Fire Emblem: Path of Radiance	CIB	-$166	$214	$187	$48
Fire Emblem: Path of Radiance	Loose	-$101	$145	$145	$44
Resident Evil 2	CIB	-$90	$161	$78	$71
Resident Evil 2	Loose	-$77	$106	$29	$44
Pokémon Colosseum	CIB	-$98	$142	$64	$44
Pokémon Colosseum	Loose	-$83	$109	$49	$26
Legend of Zelda: Twilight Princess	CIB	-$108	$196	$115	$88
Legend of Zelda: Twilight Princess	Loose	-$66	$146	$107	$80
Resident Evil 3: Nemesis	CIB	-$97	$156	$59	$74
Resident Evil 3: Nemesis	Loose	-$52	$99	$47	$69
Legend of Zelda: Four Swords Adventures	CIB	-$92	$77	$130	$38
Legend of Zelda: Four Swords Adventures	Loose	-$58	$56	$79	$21
Eternal Darkness	CIB	-$82	$114	$32	$34
Eternal Darkness	Loose	-$60	$85	$25	$33
Billy Hatcher and the Giant Egg	CIB	-$66	$89	$23	$34
Billy Hatcher and the Giant Egg	Loose	-$55	$65	$10	$34
Wario World	CIB	-$55	$80	$60	$25
Wario World	Loose	-$44	$62	$45	$18

Even more excitingly, there is a mod patch for Pokémon Box enabling the Japanese version to communicate and trade Pokémon between US, JP, and PAL Game Boy Advance games. This means if you wanted to use Pokémon Box functionality you could buy a Japanese copy for $50, patch the software, and use the software as if it were from any region.

This fact carries beyond GameCube games, as you'd imagine. Legend of Zelda: Minish Cap is $90 for a loose US copy and only $40 for a Japanese copy. Surprisingly it's not always the case that the US release is cheaper than the Japanese release. For example, the Sonic Gems Collection for GameCube costs $55 CIB for a US release compared to $180 CIB for JP.

I suspect this is because the soundtrack for Sonic CD is different for each region, JP Sonic Gems Collection carrying the far-superior and sought after JP Sonic CD soundtrack. The US Sonic CD soundtrack was included in the US and PAL Sonic Gems Collection releases. The Japanese Sonic CD soundtrack was released for the first time outside Japan in Sonic Origins.

Why shouldn't you buy other regions' games?

The primary reason is if you don't speak the language. Many PAL GameCube titles include English as one of the many supported languages, but this can't be said for NTSC-J. Most Japanese titles are exclusively in Japanese, and I don't speak the language. For some games this is fine, not knowing Japanese won't hamper games that aren't text-heavy, but RPGs or story-based adventures will impact the fun-factor. There are some modifications you can apply to translate the Japanese text back into English.

Many consoles are region-locked, and thus require either purchasing a matching region console, modifying a console to support all regions, or dumping ROMs and using emulators (usually with a dedicated device or console modification). So you can't use any region's software without at least some extra work or hardware. The Nintendo GameCube has three main regions: NTSC-J (Japan), NTSC-M (Americas), and PAL (and maybe NTSC-K (Korea)? I cannot tell if this is actually its own region).

I've gone the way of modifying my GameCube hardware with a solderless mod-chip called the FlippyDrive, but other open source options for a similar device are now available. This modification allows running any regions games on any region hardware.

If you are using unmodified hardware you may run into issues with frame rate if you're mixing software and hardware from an NTSC region (US, JP, KR) and the PAL region. I am not super knowledgeable in this area, look around for solutions to this issue before buying.

Thanks for reading ♥ I would love to hear your thoughts! Contact me via Mastodon, Bluesky, or email. Browse the blog archive. Check out my blogroll.

September 01, 2026 12:00 AM UTC

August 31, 2026

PyCharm

Fine-Tuning SOTA Object Detection Models on Real-World Datasets

In our previous blog post in this series, we discussed state-of-the-art models for object detection: the architectures, the theory, and what makes YOLO12, YOLO26, and RF-DETR tick. If you want the theoretical background on these models, start there.

This post is the practical follow-up: how to actually use these models, how to fine-tune them on diverse, specialized datasets that look nothing like their training data, and how to evaluate the results – all within PyCharm.

Why fine-tune at all?

Every pretrained detector you download was trained on some distribution of images, almost always COCO, which is ~118k training images of everyday scenes containing 80 common object categories (people, cars, dogs, chairs, etc.).

Real-world deployment data rarely looks like COCO. Things that object detection might actually be applied to, such as damaged industrial cables, bone fractures on X-rays, or densely stacked soda bottles on a shelf, are:

Out of vocabulary: “Bone fracture” is not one of COCO’s 80 classes, so the model literally has no output category for it.
Out of visual distribution: X-ray imagery, industrial close-ups, and heavily occluded shelf scenes differ drastically from consumer photos in texture, viewpoint, and object density.

Deploying a detector on off-distribution data therefore requires fine-tuning. But before we break the models, let’s establish that we get similar results on our hardware to the ones reported by developers.

The models

For the purposes of this experiment, we’ll focus on three current SOTA object detection families and examine two sizes of each model:

Family	Variants	Implementation
YOLO12	`yolov12n, yolov12m`	Original authors’ repo
YOLO26	`yolo26n, yolo26m`	Ultralytics PyPI package
RF-DETR	`RFDETRNano, RFDETRBase`	Roboflow PyPI package

Sanity check: Reproducing COCO `val2017` baselines

We’re going to be working with six pretrained checkpoints: Two different sizes of each of the three models. To check that these models are behaving as expected, we evaluated all of them on the full 5,000-image COCO validation dataset (val2017) to verify the numbers reported in the previous post:

Model	Params (M)	mAP50	mAP50-95	Latency (ms)
YOLOv12-N	2.55	0.5548	0.4021	23.9
YOLO26-N	2.57	0.5498	0.3952	12.3
YOLOv12-M	19.67	0.6953	0.5259	72.4
YOLO26-M	21.90	0.6906	0.5181	13.9
RF-DETR Nano	30.47	0.6750	0.4835	12.4
RF-DETR Base	32.17	0.7210	0.5325	12.9

Three things stand out even before we leave COCO behind:

Larger models (mostly) have better performance. RF-DETR Base leads (0.5325 mAP50-95), but the medium YOLOs get remarkably close (0.5259 / 0.5181) with ~10M fewer parameters.
YOLO26’s NMS-free design pays off in throughput. YOLO26-N is the fastest model in the lineup (12.3 ms latency) at essentially the same mAP50-95 as YOLOv12-N, which, despite being the smallest model here, is considerably slower (23.9 ms latency). Attention is expensive. (If you want more detail on the model architectures and how they affect performance, see the previous blog post in this series.)
RF-DETR Nano is not “nano” by parameter count (~30M – more than YOLO26-M), but it is well-optimized: With 12.4 ms latency, it is the second-fastest overall.

Published papers report optimized inference latency: That is, they measure the model’s forward pass in isolation, stripped of the surrounding stages of the object detection pipeline. We deliberately skipped that aggressive optimization so our numbers reflect what you’d actually see when deploying these models.

As a result, our latency figures don’t line up with the benchmarks in the models’ white papers. There are two main reasons for this:

Hardware: We used different hardware from the NVIDIA T4 GPU that serves as the de facto standard in object detection benchmarking.
Unoptimized computation graph: We ran the models in their native framework rather than converting them to TensorRT. TensorRT compiles a network into a hardware-specific engine, fusing layers, selecting the fastest kernels for the target GPU, and optionally running in reduced precision. That can cut latency substantially, but the resulting engine is tied to one GPU and requires an extra build step, so it doesn’t represent how these models perform out of the box.

Accuracy is a different story: While our latencies diverge from the published ones, our mAP50-95 results fall within reasonable noise bounds of the reported figures.

Now that we’ve seen what our pretrained models can do on COCO, the dataset they were trained on, let’s see what happens when they’re tested off distribution.

The datasets

For evaluation, we used RF100-VL, a large-scale collection of 100 multimodal datasets covering concepts deliberately chosen to be rare in object detection models’ pretraining data. These datasets contain exactly the off-distribution targets we care about. These targets also mirror common real-life applications for object detection, giving us a realistic test of these models’ capabilities out in the wild.

We picked three datasets that stress test the models in different ways:

Dataset	Domain	Why it’s hard	Classes
`cable-damage`	Technical/industrial	Fine-grained damage types on visually similar backgrounds	`break, thunderbolt`
`bone-fracture`	Medical (X-ray)	Entirely different imaging modality; subtle features	`angle, fracture, line, messed_up_angle`
`soda-bottles`	Retail	Heavy occlusion, many near-identical instances per image	`coca-cola, fanta, sprite`

Tutorial: Fine-tuning all three models in PyCharm 🙂

Step 1: Setting up the project

One of the first challenges we had to overcome in this project was that the three implementations do not share a compatible set of dependencies. In particular, the two different generations of YOLO require different versions of the ultralytics package. PyCharm offers a clean solution for this: one PyCharm project with three isolated uv environments – one per model family.

We’ll run our computations on a remote GPU. Configuring a remote interpreter in PyCharm follows the same workflow as a local one: the same dialog and the same dropdown as in the local case. Note that remote interpreters require PyCharm Professional; Community Edition supports local environments only.

Firstly, we need to instantiate our three uv virtual environments via:

cd yolov12 && uv venv .venv --python 3.11 

cd yolov26 && uv venv .venv --python 3.11 

cd rf-detr && uv venv .venv --python 3.11

Once your uv virtual environments exist, register each one as an existing interpreter. Go to Settings | Python | Interpreter, click Add Interpreter → Add Local Interpreter, choose Environment as Select existing, and point the interpreter field at that environment’s bin/python. PyCharm doesn’t create anything here, it just picks up the environment uv already built.

Repeat for each environment. From then on, switching is a matter of picking one from the Settings | Python | Interpreter dropdown, or from the interpreter widget in the bottom-right-hand status bar.

You can find the full list of dependencies required for each model in their respective project repositories. You can either install all the projects’ dependencies in PyCharm’s built-in Terminal tool window or install individual packages using the Python Packages tool window (including selecting specific versions of packages). You can access both of these tool windows by clicking the relevant icons in the lower left-hand corner of the PyCharm toolbar.

For a step-by-step guide on setting up the environments for all three models, see our GitHub implementation of this tutorial.

Step 2: Getting the datasets

To obtain the out-of-COCO-distribution datasets, we can install our datasets via the rf-detr virtual environment, since it has roboflow as one of its core dependencies. We then set the Roboflow API key as an environment variable so that it is available to the API when downloading the datasets.

pip install roboflow

export ROBOFLOW_API_KEY="your_key_here" # you can get API key here: https://docs.roboflow.com/reference/authentication/authentication/find-your-roboflow-api-key

After setting everything up, now you can run the Python script below to get the three datasets we’re going to use in our tutorial:

import os
from roboflow import Roboflow

api_key = os.environ.get("ROBOFLOW_API_KEY")

if not api_key:
    raise RuntimeError("ROBOFLOW_API_KEY is not set")

DATASETS = [
    "bone-fracture-7fylg",
    "cable-damage",
    "soda-bottles",
]

VERSION = 2          # RF100 projects are generally published at version 2
FORMAT = "yolov8"    # or "coco", "voc", "yolov5"
rf = Roboflow(api_key=api_key)
workspace = rf.workspace("rf100")

for slug in DATASETS:
    print(f"Downloading {slug} ...")

    try:
        project = workspace.project(slug)
        dataset = project.version(VERSION).download(FORMAT)
        print(f"  -> {dataset.location}")

    except Exception as e:
        print(f"  !! failed: {e}")

This script connects to the Roboflow cloud service via its Python API client and downloads three specified RF100 datasets in YOLOv8 format. It loops through each dataset, reports where successful downloads are saved, and prints an error if any download fails.

Step 3: Getting a zero-shot baseline by using pretrained models on custom data

Before fine-tuning, we’re going to evaluate the COCO-pretrained checkpoints directly on our three datasets, to see whether the fine-tuning is actually necessary. The result was unambiguous: The models predicted essentially nothing.

Zero-shot mAP50-95 on the test splits of our three datasets:

Model	`cable-damage`	`soda-bottles`
RF-DETR Nano	0.0004	0.0027
RF-DETR Base	0.0005	0.0004
YOLOv12-N	0.0007	0.0266
YOLO26-N	0.0000	0.0033
YOLOv12-M	0.0000	0.0160
YOLO26-M	0.0000	0.0012

This is to be expected; it’s not a bug! As the models are closed-vocabulary detectors, that is, they have a finite number of predefined target classes, they physically cannot output a class like fracture that isn’t in their 80-class COCO head.

This is the punchline of this whole post: A model scoring 0.72 mAP50 on COCO scores 0.00 on bone fractures. Pretrained ≠ deployable, even when the model is state of the art. Basic machine learning principles still apply, even in the age of AI!

Step 4: Fine-tuning

All models were fine-tuned on a single A100 GPU for 10 epochs. We used standard Ultralytics/RF-DETR fine-tuning pipelines in order to fine-tune the models on our three datasets. We fine-tuned a model for each dataset. The full fine-tuning pipeline can be found in finetune_rf100.py scripts in the project repo, under the folders for each model.

You can see the core of the training setup below. Both YOLO and RF-DETR are built on PyTorch under the hood, but the training loops are abstracted behind higher-level library APIs: Ultralytics’ YOLO.train() for the YOLO models, and RF-DETR’s own train() functionality.

YOLO12 and YOLO26

train_model = YOLO(args.model)

train_res = train_model.train(
                data=str(yaml_path),
                epochs=args.epochs,
                imgsz=args.imgsz,
                batch=args.batch,
                device=args.device,
                project=args.project,
                name=run_name,
                exist_ok=True,
                verbose=False,
            )

RF-DETR

ModelClass().train(
                dataset_dir=str(coco_dir),
                output_dir=str(output_dir),
                epochs=args.epochs,
                batch_size=args.batch_size,
                grad_accum_steps=args.grad_accum,
                lr=args.lr,
                resolution=resolution,
                early_stopping=True,
                checkpoint_interval=1,
            )

Step 5: Results

Fine-tuning transforms the picture. You can see the results on the test set after training:

On the left, we have the pretrained models’ results for the COCO validation dataset. As we showed earlier, accuracy (mAP50-95) fell between 0.39 and 0.53, and all models except for YOLOv12-M showed low latency. The fine-tuned models on the right showed a similar range of accuracy for the cable-damage and soda-bottle detection tasks, only falling lower for the bone-fracture task. Moreover, the fine-tuned models were comparable in latency to the pretrained models for their intended tasks, and for YOLOv12-M, they were even faster. This suggests that, after fine-tuning to the target domain, the models achieve performance that’s broadly comparable to the pretrained performance on their original training domain.

Let’s now have a closer look at the fine-tuned models’ performance, breaking it down by mAP50 and mAP50-95 for the three separate RF-100 datasets:

Model	`cable-damage`	`bone-fracture`	`soda-bottles`
RF-DETR Nano	0.9195 (0.4391)	0.2317 (0.1136)	0.9617 (0.6223)
RF-DETR Base	0.9281 (0.4456)	0.4474 (0.1915)	0.9688 (0.6332)
YOLOv12-N	0.9236 (0.4378)	0.0911 (0.0532)	0.9677 (0.6343)
YOLO26-N	0.8165 (0.3681)	0.0193 (0.0064)	0.9148 (0.5896)
YOLOv12-M	0.8266 (0.3649)	0.1500 (0.0635)	0.9706 (0.6422)
YOLO26-M	0.8707 (0.3896)	0.2194 (0.1038)	0.9596 (0.6304)

What the numbers say:

The soda-bottles target is the easy win. Every model lands in the 0.91–0.97 mAP50 band. This is likely due to the fact that the domain (consumer products in photos) is visually close to existing classes in COCO, so only the vocabulary was new. Interestingly, the attention model family does great here, with YOLOv12-M taking the top spot (0.6422 mAP50-95).
cable-damage: Detection is easy, but localization is hard. mAP50 reaches 0.93, but mAP50-95 tops out at 0.446. It appears that the models find the damage reliably, yet they struggle to box thin, elongated defects precisely. If your application needs tight boxes at high IoU, this gap would be a significant issue.
bone-fracture remains genuinely hard. The best model (RF-DETR Base, 0.447 mAP50) is far from production-ready, and the performance spread across models is huge. The modality shift from photos to X-rays means the pretrained backbone features transfer poorly. The different image modality and small, sometimes almost indistinguishable bone fractures make the detection task way harder than the one employed on common objects identification. This is the dataset that would most benefit from domain-specific pretraining, more data, or longer fine-tuning.
RF-DETR Base is the most consistent performer, winning on two out of three datasets and challenging seriously for the third. The DETR-style architecture seems to transfer more robustly to unfamiliar domains.

Qualitative results

To visually assess how these models perform, we can overlay the predicted bounding boxes on the images. Let’s look at the objects our models detected in six random images per class:

We can see this confirms the accuracy values we saw above: The noisy images of soda bottles in fridges are labeled accurately, with tight bounding boxes for each object. The cable damage is identified less consistently, with some models failing to find the damage altogether, and others creating unnecessarily large bounding boxes. Finally, the images of broken bones contrast sharply with the other two, with less than half of the images having any break identified, and different models identifying different potential breakage points.

Conclusions

Pretrained object detectors are powerful, based on advancements in model architecture over the past five years, but as we’ve seen here, pretrained does not necessarily mean deployable. All six models performed well on COCO, yet when we applied those same checkpoints directly to our specialized datasets, their performance fell close to zero. However, fine-tuning completely changed that picture.

After only 10 epochs of fine-tuning, all three model families were able to adapt well to both the cable-damage and soda-bottle datasets. As we noted, the soda-bottle task was particularly transferable, likely because it contained objects similar to those contained in COCO. cable-damage was also detected relatively reliably, although the larger gap between mAP50 and mAP50-95 showed that precisely locating these tiny defects was still challenging for all of the models. However, bone-fracture was a completely different story, likely because moving from the sort of natural images contained in COCO to X-rays is a much larger domain shift. While RF-DETR handled this jump best, even its performance shows the limits of fine-tuning, and there are times when you might need to consider more data, longer training, or even domain-specific pretraining.

The broader takeaway is that there is no single “best” detector: It is dependent on the task. Model size, latency requirements, licensing restrictions, and most importantly, the similarity between the model’s pretraining data and your target domain all affect the outcome. It is important to refrain from unquestioningly trusting the numbers reported by model providers and explore the fit of a specific model for your own particular task.

Get started with PyCharm today

In this post, we’ve gone from validating pretrained YOLO12, YOLO26, and RF-DETR checkpoints on COCO to testing them zero-shot on specialized data, to fine-tuning them on three very different object detection tasks, and then finally, comparing the resulting accuracy and latency. Along the way, we’ve seen how PyCharm can help manage the practical side of a project like this, where multiple model families require different dependency sets and training environments.

PyCharm helps you keep these workflows together in a single project while using isolated Python environments for each model family. Its interpreter management, built-in terminal, Python Packages tool window, and support for remote development make it easier to move between environments and run training on remote GPU hardware without having to manage each part of this workflow separately.

If you’d like to try these experiments yourself, maybe look into fine-tuning these models for your own specific object detection use case! PyCharm is available to download and try. You can use the accompanying project code to reproduce our COCO baselines, download the RF100 datasets, fine-tune the models, and evaluate them using the held-out test splits.

You can find the full code for this project on GitHub. And if you’d like to learn more about object detection, including the architectures behind the models we used in this post, check out the previous post in this series.

August 31, 2026 01:50 PM UTC

Python Software Foundation

Kojo Idrissa: 2026 PSF Board Election Candidate Interview

Who are you?

I'm a software engineer at REVSYS. Before that I was an accountant. I was also a university instructor in the US and China. I taught English, Western Culture, Problem-Solving, Accounting and MIS.

What would you bring to the PSF Board of Directors?

I bring business acumen from my accounting degree, MBA, and prior career as an accountant. I've also been on a non-profit board, when I was a board member for DEFNA, the non-profit that puts on DjangoCon US. I bring a global perspective of the Python community. I'm American, but I've spoken at conferences in Canada, Mexico, the Dominican Republic, Europe, Africa, and Australia. I used those opportunities to build ties with those local communities and increase my awareness of how their needs differ from the communities in the US, where the PSF is based.

What motivated you to run for the PSF Board of Directors?

I hadn't thought about running for the board until someone suggested I consider it, based on my prior community activity. As I thought about it, I realized I had some expertise and experience that I could use to benefit the broader Python community. In addition to my business background, I've made lots of connections with people from different parts of the global community.

What problem or challenge do you want to address if you are on the board?

The decline in the community's confidence in the board.
What I see (and hear, from other community members) is more a set of related challenges, instead of a single problem. The word "consistency" keeps coming up when discussing the PSF, both in action and communication. It comes up when discussing the Fiscal Sponsorees program and the Grants program. When do these programs open/close? Who are they available to and why/how? What type of support do they offer?

There's also an issue of "focus". The PSF's core activities are, "Infrastructure", "Community", and "Investing in Python & Tools". But at what rates are PSF resources allocated to those different activities. And do those allocation rates need to change?

Where do you see the PSF in 5 years from now?

If we handle things properly, I see the PSF thriving. I see a PSF more focused on the the parts of the core mission that only the PSF can best perform ("Infrastructure" and "Investing in Python & Tools"). I also see a PSF with better internal infrastructure in place to provide Community support in a clear, consistent, sustainable fashion.

However, if the board doesn't improve its internal infrastructure to provide Community support in a reliable fashion, I can see the global community "moving on" from the PSF and building its own mechanisms for support.

What areas of the Python community are you involved with?

I've been a DjangoCon US organizer since 2016 and the DEFNA North American Ambassador since 2017. I'm also a former DEFNA board member. I'm the Orientation Chair for DjangoCon US and I've helped lead the Newcomer's Orientation for PyCon US since 2022. I'm also the co-founder of Black Python Devs. I've spoken at multiple Python and Django conferences, been a Django Girls coach multiple times.

------

Note from the election administrators:

Want to learn more about this candidate or ask them a question?

Check out their nomination statement.

Check out their AMA thread on discuss.python.org.

August 31, 2026 10:45 AM UTC

Ed Crewe

From Routing Checks to Trajectory Testing: Evaluating an Agentic Chatbot

Which Agentic Chatbot?

I have been working on a Python based AI test framework for a chatbot interface for my company's product, Postgres AI Hybrid Manager. The manager allows the setup of Postgres clusters across cloud or on-prem and attaching various AI tools such as Langflow. So a combination of more traditional Postgres backup, migration, telemetry and analytics features along with LLM workflows leveraging the data it holds.

The product already has a control plane UI for managing Postgres estates. It also has full help for the product, all Postgres versions, analytics, AI and add ons. The chatbot brings all these things together: ask a question, get the relevant help, or ask it to do something such as migrate a cluster, or evaluate telemetry that would otherwise require clicking through the UI.

That makes it a pretty handy interface, especially for the less technical. However it is not simple to test and ensure good quality responses.

A normal deterministic API test is simple. Send a request, check the status code, check the JSON body, perhaps check the database state. An LLM-backed agent does not pass or fail so clearly. It can route to the wrong capability and still return fluent text. It can pick a plausible but wrong tool. It can miss half the task and still sound confident. It can complete the first turn of a conversation and lose the plot on the second. It could get malformed or missing data from tooling that leads it to deliver a misleading conclusion. It might only provide help to something that should be from tool data or was a request for an action such as create a cluster.

So the testing problem was not “does the chatbot return a reasonable response?” It was “how do we test the whole chat path is doing the right thing?”

This is the story of how our agent-eval test framework evolved as we worked to see that our chatbot was not only getting the right answer, 42 , but whether it was asking all the right questions of the right tools to get that answer. Known as trajectory testing ...

You're Golden

Before we can tell our story we need to define some terms.

A Golden is an example of a perfect desired output from a test input. They often refer to more complex outputs that may need saving as separate files, but a simple assertable output such as 42, is a golden too!
Whilst complex goldens may be used and marked for semantic similarity against the test output. It is more common for complex outputs to be described by a rubric. A rubric is a checklist of qualitative properties a good answer must exhibit, written in plain English as opposed to a golden example of an answer.

For AI testing the tests are termed evals, ie they evaluate the tool, but not by strict assertions, because one thing you can be sure of with an LLM is that given the same input, you usually get subtly different output, ie they are non-deterministic. Which means for LLM outputs the only way to test them is to use an LLM-as-judge, ie give that LLM the test output and a rubric or golden and let it mark it against that. Then you set a pass threshold for that mark, to translate your complex output into a pass or fail.

You can also total up all the passes to give you a Task Completion Rate, TCR. So with complex AI agentic LLM interactions a 100% pass of all evals is often not realistic. Hence you set a TCR below 100% for the whole test suite of evals to pass. Start with the smallest useful test. The core principle of evals is not complicated, you want the input to give you the expected output.

But for an Agentic application this may require a sequence of LLM calls and tools: Making the final output dependent on the route that should be chosen, the tool(s) that should be called, the actions to be taken, further LLM calls that may be necessary and finally the core data that the response to the user should contain.

Our first version did not try to solve every part of that. It started with routing, simple and deterministic.

Routing is the starting point

The chatbot originally had an agent per tool. The tool being the code and API calls that performed actions or returned data or help.

Different specialist agents owned different parts of the product surface: Control-plane actions, Postgres database operations, schema design, roles and permissions, cluster reporting, migration, and so on.

Before any specialist can help, something has to choose the right specialist.

So the first eval suite asked a narrow question:

Given this user prompt, did the chatbot route to the expected tool?

That gave us a fast health check. We could keep a corpus of prompts, map each one to an expected destination, run them through either a direct model path or against the real deployment and its tools, and score whether the selected destination tool matched the golden.

A golden here is just the name of the tool:

- id: "core-iam-001"
    prompt: "List all my projects"
    expected_tool: "control-plane"
    tags: ["core", "control-plane", "project"]

And the check on the other end is deliberately dumb — an equality test, not a semantic one:

self.success = tool_match(predicted_tool, expected_tool)

Agents became skills, but routing remained

The design moved away from “one agent per tool family” toward a more consolidated orchestrating agent with skills that could better compose different tool use for common tasks.

That is a better fit for how modern agent systems are evolving. A skill = instructions, constraints, and a subset of tools that are relevant for a task. It is a form of progressive disclosure. Give the model the minium it needs at each step to save tokens.

But this did not make routing irrelevant.

Instead of asking “did we transfer to the right sub-agent?”, the eval asks “was the right skill made visible and selected for this task?” The labels changed but a skill could still use the wrong tool.

Routing evals stayed valuable because they were fast, explainable, and easy to run in CI. But they are limited, routing should always be correct but it doesn't mean that the final agent response is too.

TCR jumps to the endpoint, the response

Task Completion Rate, or TCR, was the next step.

The user asked for a cluster comparison, or a schema recommendation, or help diagnosing a database issue. We need to know whether the full response actually completed these tasks.

Responses are complex goldens so they need the LLM-as-a-judge pattern: run the chatbot, take the actual response, and ask a judge model to score it against expected sections.

The eval has a rubric here for judging the output:

- id: "tcr-core-014"
  prompt: "Compare CPU usage between these two clusters"
  expected_sections:
    - "identifies which cluster has higher CPU usage"
    - "cites at least one supporting metric"
    - "suggests a plausible next step"

The judge gets one simple instruction: score each expected_sections between 0.0–1.0 A metric class then just thresholds it for pass / fail:

self.success = score >= 0.7

The judge must be calibrated and a consistent model used for comparing runs over time. Consistent judging enables skill and/or prompt tuning from metric trends. The rubric must be specific enough to avoid marking waffle as success. But it turns a non-deterministic complex output into a simple pass and fail. It also separated two different levels of QA:

Can the underlying model answer the task if given the right context?
Does the deployed chatbot complete the task through the real product path?

That led to two execution modes.

Direct mode calls the model with simulated context. It is faster and useful for prompt and rubric development.

Proxy mode calls the real chatbot. It is slower, but it exercises the production path: routing, skill selection, tool calls, guardrails, streaming responses, conversation state, and the actual service wiring.

Both matter. Direct mode tells you whether the model is capable of the answer. Proxy mode tells you whether your product is capable of really delivering it via agents running your deployment's tools.

This is the major difference from standard AI LLM testing, the model is only a small pluggable engine for the full agentic skill set that requires the actual deployment domain of data, actions and tools. Direct mode testing of only the model, is occasionally useful but E2E testing of the Chatbot deployment is required for agentic AI Chatbot QA, tuning and validation.

Multi-step conversations changed the unit of testing

Single-turn TCR is still too small for many real chatbot tasks.

Users do not always provide all required information in one message. They ask to create a cluster, then pick a project, then choose a size, then confirm. They ask for a schema review, then refine the problem, then ask for a migration path. They troubleshoot by adding information over time.

So the framework has to exercise test cases that are conversations, not just single prompts.

That sounds like a minor data-model change. It was not. Once a test has steps, the eval runner has to preserve conversation state. In proxy mode, that means carrying the real conversation_id returned by the chatbot and sending each follow-up as part of the same server-side conversation. In direct mode, it means building a synthetic conversation history so the model sees the prior turns.

In code that split is about as literal as it sounds. Proxy mode threads a real id through each call:

response = client.send_message(prompt=msg, conversation_id=conversation_id)
conversation_id = response.conversation_id  # captured on turn 1, reused after

Direct mode has no server-side conversation to lean on, so it fakes one by re-rendering the transcript into the prompt itself, every turn:

full_prompt = f"## Conversation History\n{render(history)}\n\n{next_prompt}"

Same test case, same expected outcome, but a different code path depending on which half of the system is actually holding the conversation state. That's impacts multi-turn evals because conversation memory is part of the harness code for the actual deployment not just a model issue.

The scoring also becomes more interesting. You want per-step checks, because the assistant should ask the right clarifying question at the right time. You also want an overall score, because a conversation can have reasonable individual turns and still fail to complete the user's goal.

Coding it yourself: deepeval underneath

Everything above sits on top of deepeval, the open-source LLM eval library. We add a Synthesize → Execute → Evaluate pipeline, a plugin system, YAML goldens, CI wiring, and Langfuse push on top of it But the core library underneath is plain deepeval, and you do not need any of the surrounding machinery we used. Here are routing, TCR and multi-step just built directly on deepeval (simplified deepeval 3.6.9)

A test case is just an input/output pair. LLMTestCase is the base unit everything else scores:

from deepeval.test_case import LLMTestCase

test_case = LLMTestCase(
    input="List all my projects",
    actual_output=chatbot_response_text,       # what the system under test said
    expected_output="control-plane",                  # the golden - a skill label here, not prose
    additional_metadata={"predicted_skill": predicted_skill},
)

Routing is a custom metric, not a built-in one. deepeval ships plenty of semantic metrics, but “did it route to the right skill” is an exact-match business rule, so you write your own BaseMetric. This is a simplified version of the same shape our real AgentMatch metric takes:

from deepeval.metrics import BaseMetric
from deepeval.test_case import LLMTestCase

class AgentMatch(BaseMetric):
    def __init__(self, threshold: float = 1.0):
        self.threshold = threshold
        self.async_mode = False  # routing checks are cheap; no need for async here

    def measure(self, test_case: LLMTestCase) -> float:
        predicted = test_case.additional_metadata["predicted_skill"]
        expected = test_case.expected_output
        self.score = 1.0 if tool_match(predicted, expected) else 0.0
        self.success = self.score >= self.threshold
        return self.score

    async def a_measure(self, test_case: LLMTestCase) -> float:
        return self.measure(test_case)

    def is_successful(self) -> bool:
        return bool(self.success)

    @property
    def __name__(self):
        return "Agent Match"

tool_match is the check from earlier. Run it with deepeval's own runner rather than hand-rolled assertions, and you get retries, pretty output, and a result object for free:

from deepeval import evaluate

evaluate(test_cases=[test_case], metrics=[AgentMatch()])

TCR is where deepeval's built-in GEval earns its keep. GEval is deepeval's off-the-shelf LLM-as-judge metric, you give it criteria (or explicit evaluation steps) and it handles the judge prompt, the JSON parsing, and the scoring for you. Our rubric-per-line expected_sections maps onto evaluation_steps almost directly:

from deepeval.metrics import GEval
from deepeval.test_case import LLMTestCase, LLMTestCaseParams

task_completion = GEval(
    name="TaskCompletion",
    evaluation_steps=[
        "Check whether the response identifies which cluster has higher CPU usage",
        "Check whether the response cites at least one supporting metric",
        "Check whether the response suggests a plausible next step",
    ],
    evaluation_params=[LLMTestCaseParams.INPUT, LLMTestCaseParams.ACTUAL_OUTPUT],
    threshold=0.7,
)

test_case = LLMTestCase(
    input="Compare CPU usage between these two clusters",
    actual_output=chatbot_response_text,
)

evaluate(test_cases=[test_case], metrics=[task_completion])

Multi-step conversations get their own test case type. ConversationalTestCase takes a list of Turns instead of a single input/output pair, and pairs with a BaseConversationalMetric instead of BaseMetric:

from deepeval.test_case import ConversationalTestCase, Turn

convo = ConversationalTestCase(
    turns=[
        Turn(role="user", content="Create a new cluster"),
        Turn(role="assistant", content="Sure - which project should it go in?"),
        Turn(role="user", content="acme-prod"),
        Turn(role="assistant", content=final_response_text),
    ],
    expected_outcome="A cluster is created in acme-prod after resolving the missing project name",
)

deepeval has a conversational counterpart to GEval too (ConversationalGEval), scored against the whole turn sequence rather than a single response which is the natural fit for “did the assistant ask the right clarifying question at the right time”, the per-step-plus-overall shape TCR needed once prompts became conversations.

Put together, that is the whole starting kit: LLMTestCase plus a hand-written BaseMetric for hard business rules like routing, GEval for rubric-style task completion, ConversationalTestCase plus ConversationalGEval once a prompt becomes a conversation, and evaluate() to run the lot and get a result object back.

Everything else we built, the YAML goldens, the plugin architecture, the CI wiring, the Langfuse push exists to run more of these at scale and make the failures easy to find. But none of it is required to get started. If you are testing your own agentic chatbot, this is how to begin.

This is where instrumentation started to matter much more.

For a single-turn answer, a markdown report with pass/fail rows is often enough to start debugging. For multi-step conversations, that is thin. You need to know which turn failed, whether the route changed, whether the wrong tool was called, whether the tool call used correct arguments, whether the model forgot earlier context, or whether the final answer simply missed a required section.

That is why we added span-level telemetry and pushed eval traces into Langfuse.

Langfuse made the failures inspectable

The useful thing about Langfuse is not just having another pretty dashboard. Although that is important for spotting quality regressions over time via regular CI/CD automated runs.

The vital thing was being able to treat an eval run as a set of traces. A run becomes a session. Each test case becomes a trace. The trace carries the prompt, response, scores, tags, model, mode, scenario, and the spans emitted by the proxy.

For a chatbot path, those spans are where the debugging starts. You can see routing, tool execution, LLM calls, latency, and token usage where it is available. You can filter by scenario and model. You can compare runs. You can look at a failing conversation and see whether the problem began at route selection, tool selection, tool arguments, or final synthesis.

That changes the tuning loop.

Without traces, an eval failure says “this case failed”. With traces, it can say why it failed.

That distinction matters because the fix lands in different places...
Is it a routing rule?
Is it a skill description?
Is it a tool schema?
Is it the judge rubric?
Is it that the eval has has an expectation that the product has never actually promised?

Trajectory testing -> knitted the pieces together

Routing and TCR started as separate signals.

Routing asked whether the right capability was selected. TCR asked whether the final task was completed. Multi-step testing asked whether that held across a conversation. Instrumentation showed what happened between those points.

Trajectory testing is the next natural step: score the path itself.

For an agentic product, the fully correct path is essential to response quality.
So trajectory tests add expectations about intermediate actions:

which tool or flow should be used
whether the arguments are valid
whether the conversation reached the right state
whether the final answer completed the task

The label-based routing tests are still useful as fast canaries. They tell us whether the classifier shape has drifted and distinguish tiers - see the next section.
But full trajectory tests judge the route by consequence: did the system actually follow the tool path that would satisfy the user?

So retain the fast determisitc routing tests, but move more user-visible behavioural coverage into trajectory and TCR.

Sovereign AI makes the eval problem tiered S/M/L/XL

There is one more constraint that makes this more than a generic chatbot-testing story.

Our chatbot has to work for sovereign and air-gapped deployments. In those environments, prompts, tool results, schema details, and operational data cannot be sent to a hosted frontier model outside the customer's trust boundary. The inference model may run inside the customer's environment.

That usually means a smaller model.

Smaller models are not just cheaper versions of larger ones. They have different context limits, weaker tool-selection behaviour, and less tolerance for an over-wide capability surface. If you show a smaller model every possible tool and skill, you have increased the chance that it chooses a bad one.

So the architecture becomes tiered. Models are effectively T-shirt sized. A small self-hosted model sees a curated subset of reliable skills. A larger model can be allowed to see more. Some experimental or complex skills only make sense for the highest tiers.

That changes the meaning of a routing eval again.

The correct visible skill set is no longer universal. It depends on the model tier. A prompt that should route to an advanced skill for an XL model may need to be dropped, refused, or handled differently for a smaller model that should not see that skill at all.

This is why trajectory testing and routing need to be tier-aware. We are not only asking whether the chatbot can complete a task. We are asking whether it can complete the task through the capability surface that a deployment's LLM size allows.

What I would keep from the journey

The final shape was not obvious at the start.

We began with routing because it was the first integration failure point and the cheapest one to isolate. We added TCR because correct routing did not prove task completion. We added multi-step cases because real users have conversations, not isolated prompts. We added telemetry because multi-step failures are otherwise too hard to debug. We moved toward trajectory testing because the route, tools, arguments, and answer need to be judged as one path.

If I were starting another agentic product eval framework, I would keep that order.

Do not start by trying to build a grand universal benchmark. Start with the smallest failure point that would embarrass the product if it regressed. Then move the signal closer to the user's actual goal.

For a chatbot wired into a real control plane, that means testing more than the output text. It means testing the route, the skill, the tool call, the arguments, the conversation state, the final answer, and the model tier that made those options visible in the first place.

That is the difference between checking that an AI system said something vaguely relevant and checking that it actually did all the things the user asked of it.

August 31, 2026 09:28 AM UTC

August 30, 2026

Paolo Melchiorre

My DjangoCon US 2026

My DjangoCon US 2026 experience in Chicago, captured through Mastodon posts about talks, my UUID talk, Django community activities, people, and moments along the way.

August 30, 2026 10:00 PM UTC

August 28, 2026

Django Weblog

Django Developers Survey 2026 results

The results from the 2026 Django Developers Survey are now available. This is the fifth annual report conducted from May to July 2026 by the Django Software Foundation in collaboration with JetBrains PyCharm.

The full report includes infographics, quotes, and dedicated sections so you can easily navigate the results. There is also a The State of Django 2026: Boring is so back blog post highlighting key Django trends in 2026 and actionable ideas for your own Django development.

The Django Chat podcast also covers the survey in a special summer episode, from Django 6.1 and HTMX to async, AI, deployment, testing, and Python tooling.

August 28, 2026 02:55 PM UTC

PyCharm

The State of Django 2026: Boring is so back

Welcome to the highlights from the fifth annual Django Developers Survey, a collaboration between the Django Software Foundation and PyCharm. This year’s report draws on responses from nearly 3,500 Django developers across more than 40 countries — from students in their first year to veterans with decades of experience.

In software, “boring” is a compliment. It means a technology works so reliably you can stop thinking about it. By that standard, Django in 2026 is thriving: PostgreSQL has been the database of choice for 76–79% of respondents for five consecutive years, Django’s template engine has held steady at around 80%, and nearly half of developers upgrade with every stable release — 43% are already on Django 6.0, months after it shipped. The core is boring in the best possible way. Mature, but not static.

Everything around that core, though, is moving fast. AI is now an everyday tool for most developers, and agents are beginning to move beyond answering questions to editing files, running commands, and completing larger tasks. Newer tools are consolidating workflows that once required several separate utilities, typing is becoming standard practice, and the boundaries between editors, terminals, automation, and AI are getting blurrier.

That may be Django’s particular advantage in 2026: it is mature enough to be dependable, active enough to keep moving, and stable enough to give developers room to change almost everything else.

As an open-source framework, Django depends on its community and needs funding to remain healthy and secure. The PyCharm team runs an annual fundraiser to support Django.

Until September 10, get PyCharm at 30% off, and JetBrains will donate 100% of your purchase amount to the Django Software Foundation.

Get PyCharm at 30% off, and support Django

1. AI is mainstream, but no workflow has won

AI has become part of the normal Django development workflow. Only 10% of respondents said they regularly use no AI tools for coding, while 58% of AI users use them every day and another 27% several times a week.

What remains unsettled is how developers use them. The interfaces are split almost evenly across the browser, the IDE, and the command line, and no tool dominates: Claude Code leads at 35%, with ChatGPT just behind at 33% and GitHub Copilot at 23%. And for all the attention on agents, a majority of AI users — 56% — still use it purely for chat and advice.

2. Dominating AI workflow is still supervised

Nearly half of AI users already work with AI through an IDE integration. But despite the growth of coding agents, the dominant workflow is still supervised: 59% have AI generate code and then apply the changes themselves, 44% let it edit files or run commands when instructed, and only 27% use it to autonomously complete multi-step tasks.

The emerging pattern is therefore less “replace the IDE with an agent” and more “bring the agent into the development environment.” Developers are adopting AI quickly, but the editor remains their home base for understanding the codebase, reviewing changes, and deciding what makes it into the project.

Nowhere is the shift clearer than in how developers learn. Django’s official documentation remains the top resource at 67%, but AI tools are now second at 51% — ahead of YouTube, reading source code, and Stack Overflow.

AI is already routine for writing code, debugging, research, and learning. But developers are still experimenting with where it belongs: in the browser, inside the IDE, at the terminal, or increasingly, acting directly on the codebase.

AI is changing the IDE faster than it is replacing it.

3. Python tooling is consolidating fast

Two tools that barely existed a few years ago are already near the top of the survey.

uv, released in February 2024, is already used by 43% of respondents for managing Python environments—second only to venv at 63% and ahead of Docker at 31%.

Ruff tells a similar story. At 43%, it is now the most widely used code-quality and formatting tool in the survey, ahead of IDE inspections at 27%, Black at 25%, pre-commit at 20%, and Flake8 at 17%.

The shift is not that Python’s older tools have disappeared. It is that newer tools increasingly cover jobs that once required several separate utilities. The result is a Python toolchain beginning to consolidate around fewer, faster, more capable tools.

4. Type hints are winning. The type checker race is wide open.

Type hints are becoming the norm in Django development: 57% already use them, and another 26% plan to. What is much less settled is how developers check those types.

Among developers who use type hints, the most popular option isn’t a standalone type checker at all: 40% rely on the checker built into their IDE. Mypy follows at 32%, Ruff at 29%, and Pyright/Pylance at 22%. Newer entrants are already appearing too, with Astral’s ty reaching 12% and Meta’s Pyrefly at 4%.

The practice, in other words, is converging faster than the tooling. Django developers increasingly agree that types are useful, but there is still no consensus on which tool should enforce them—or whether a separate tool is necessary at all.

That makes type checking an interesting space to watch in 2027: will one of the newer standalone tools break away from the pack, or will type checking increasingly become something developers simply expect their IDE to provide?

5. As AI writes more code, verification matters more

AI is moving beyond suggestions and into the codebase. That makes automated verification more important, not less.

The survey can’t tell us whether AI is driving greater adoption of tests or CI. What it does show is that most Django developers already have the infrastructure agents need: pytest is used by 45% of respondents and unittest by 43%, with pytest-django at 34%.

That testing culture sits alongside widespread CI/CD adoption — GitHub Actions is now used by a majority of respondents at 51%, with GitLab CI/CD at another 26%. Together, these create a natural feedback loop for agentic development: an agent can make a change, run the test suite, respond to failures, and hand the developer a result that has already passed the project’s checks.

Not everyone has that loop in place: 19% of respondents write no automated tests at all. As more code is delegated to agents, that fifth of developers is working without the safety net that makes delegation trustworthy.

The emerging agentic workflow may depend as much on verification as generation. The more code we delegate, the more valuable it becomes to have tests and pipelines that can quickly tell both developers and agents whether a change actually works.

Agents can generate code. Tests and pipelines tell them whether it works.

6. One framework, two ways to build

Two distinct ways of building with Django are now firmly mainstream: letting Django render the interface, or using it as the backend for a separate frontend. 72% of respondents use server-rendered templates, while 53% use Django for API-only applications and 46% use it as the backend for a single-page application or dedicated JavaScript frontend.

The balance is even clearer when developers are asked for their primary approach. Half primarily build server-rendered applications, while 44% primarily use Django for either APIs or dedicated JavaScript frontends.

The JavaScript numbers tell the sharper story. React has barely moved in five years — 37% in 2021, 38% today. What’s changed is everything around it: jQuery has fallen from 37% to 23%, Vue from 28% to 17%, while htmx has climbed from just 5% to 34%. htmx isn’t taking share from React — it’s modernizing the server-rendered side of the divide that jQuery used to own.

That flexibility is one of Django’s strengths. The same framework can sit at the center of a hypermedia application or behind an API consumed by React, mobile apps, or other clients.

Django remains unusually comfortable on both sides of the frontend divide.

Conclusion

Across the survey, the pattern is consistent: developers are changing their tools and workflows far faster than they are changing the framework underneath them. The parts of Django they value most remain familiar — models, the admin, authentication — and even deployment stays defiantly unfashionable, with 54% shipping monoliths and 44% self-hosting.

Even where Django itself is evolving, it does so deliberately: 33% of respondents use its async features and another 40% plan to — change offered as an opt-in, not a rewrite.

That stability is increasingly valuable. Developers can experiment with a new agent, replace several tools with Ruff or uv, add htmx to a template, or adopt a new type checker without having to rethink the framework underneath their application.

Django’s advantage in 2026 is not that it is the newest thing. It is that it gives developers a solid, dependable — yes, boring — place from which to try the newest things. Boring is so back.

PyCharm for Django Fundraiser

Get a new PyCharm Pro license or renew your existing one at 30% off, with 100% of your purchase amount going to the Django Software Foundation.

Get PyCharm at 30% off, and support Django

Explore the complete 2026 Django Developers Survey Results.

August 28, 2026 02:17 PM UTC

Security Incident Affecting JetBrains Cadence

We are investigating a security incident affecting JetBrains Cadence. Cadence is a JetBrains-hosted service that integrates with PyCharm through an optional plugin, and lets you run your projects on cloud compute resources. Our investigation has confirmed unauthorized access to the service and the exposure of customer data associated with its use.

We have contacted affected users directly and have taken steps to contain the incident.

This post provides the latest information about the incident, its potential impact, and the actions we recommend Cadence users take. We will update it as our investigation progresses and additional information becomes available.

Last updated: September 1, 2026, 12:05 CEST

September 1, 2026, 12:05 CEST

Our investigation is nearing completion, and most mitigation and response actions are now finalized. We have not identified any additional compromised resources or data since our last update, and we are wrapping up a small number of remaining verification activities. The recommended actions for affected users remain unchanged.

August 31, 2026, 12:56 CEST

Our investigation remains ongoing. At this time, we have not identified any additional compromised resources or data.

We have confirmed that the threat actor accessed data contained in the Cadence server backup from 2024. At this time, there is no evidence to suggest that the threat actor extracted data, including secrets, from the current Cadence environment.

The recommended actions described below remain unchanged.

August 28, 2026, 11:50 CEST

Cadence is a JetBrains-hosted service integrated with PyCharm through an optional plugin, that lets you run your projects on cloud compute resources. Cadence uses JetBrains TeamCity to orchestrate this work. We recently disclosed CVE-2026-63077, a critical vulnerability in TeamCity that can allow an unauthenticated attacker to execute arbitrary commands on a vulnerable server.

We have since confirmed the Cadence environment was vulnerable to CVE-2026-63077 and was exploited through this vulnerability.

Cadence users should immediately revoke or rotate all credentials and secrets that may have been used to run their Cadence executions. They should also treat all executions, including their inputs and outputs in your Cadence project, as potentially untrusted.

Actions required immediately

We strongly recommend that Cadence users:

Revoke and rotate all credentials and secrets that may have been used to run Cadence executions.
Review connected systems for suspicious activity, particularly AWS accounts, S3 buckets, deployment environments, package/container registries, and other systems accessible using credentials mentioned above.
Review source code repositories for unauthorized changes made during the affected period.
Review any source code or project files synchronized to Cadence from PyCharm and rotate any credentials, tokens, or other sensitive information contained within them.
Treat all executions, including their inputs and outputs in your Cadence project, as potentially untrusted.

Cadence users can contact us to request an inventory of the credentials and secrets associated with their Cadence usage. This may help users identify which credentials need to be revoked or rotated, but the inventory should not be considered exhaustive.

We have collated a list of Indicators of Compromise (IoCs) below. These indicators are not exhaustive, and the absence of these indicators does not confirm that an account or system was unaffected:

Activity occurring from August 8, 2026, onwards, particularly authentication or activity using credentials previously stored in or accessible through Cadence.
IP addresses associated with observed exploitation activity:
- 150.109.230.104
- 43.153.227.206
- 62.210.127.48
- 210.247.242.190
- 15.235.225.205
- 152.233.30.18
Authentication or other activity from unexpected IP addresses or locations.
Unexpected repository clones or downloads, and unexpected commits to repositories.
Changes to repository secrets, webhooks, collaborators, or permissions.
New or modified personal access tokens, API tokens, or SSH keys in external services.
New service accounts created in external services.
Unexpected changes to cloud IAM roles, policies, or permissions.
Unexpected access to cloud storage, including S3 buckets and objects, in services such as AWS and Google Cloud.
Unexpected publication or modification of packages or releases.

Affected server

We have confirmed that the following Cadence server was successfully exploited: api.cadence.jetbrains.com.

Affected period

August 8, 2026, to August 24, 2026.

What happened

The Cadence server used TeamCity to orchestrate workloads and was vulnerable to CVE-2026-63077. Threat actors exploited the vulnerability and gained unauthorized access to the affected Cadence environments, with activity identified from August 8, 2026. We discovered the exploitation on August 23, 2026, and took the affected server offline on August 24, 2026, while we continued our investigation.

What we know

Our investigation is ongoing, but we have confirmed that the threat actors:

Accessed personal data and extracted it from the affected environment. Confirmed affected personal data includes usernames, real names, email addresses, last-login timestamps, and last accessed IP addresses.
Compromised a full backup of the Cadence server dating from 2024. This means credentials, configuration, artifacts, logs, or other data present in that backup must also be treated as potentially exposed.
Compromised multiple AWS IAM users and associated credentials/secrets used with Cadence, including IAM users belonging to JetBrains employees who used the service. These credentials were present in the compromised 2024 backup.
Accessed files stored in S3 buckets within JetBrains AWS accounts used by Cadence. We are still determining the full scope of the data accessed. We do not currently know whether the threat actors accessed storage buckets in customer accounts. However, some users may have configured Cadence to access their own storage buckets, and the credentials used for those connections may have also been exposed.
May have accessed source code synchronized from PyCharm projects to the affected server. If you used PyCharm to upload or synchronize project files for execution in Cadence, you should treat that code, and any credentials or configuration contained within it, as potentially compromised.

The likely consequences of the personal data exposure include an increased risk of targeted phishing, social engineering, impersonation, and other unsolicited or malicious communications using the affected names and email addresses.

As the threat actors gained access to the Cadence server, any credentials or secrets stored in Cadence, contained in the compromised backup, or made available to executions on the affected server should be considered compromised and must be revoked or rotated.

This includes but is not limited to:

Cloud credentials, including AWS, Azure, and Google Cloud.
Source control credentials and tokens, including GitHub, GitLab, and Bitbucket.
Package repository credentials, including npm, Maven, NuGet, PyPI, and similar services.
Container registry credentials, including Docker Hub, ECR, GCR, ACR, and other registries.
Slack tokens, webhooks, API tokens, SSH/deployment keys, service account credentials, signing keys/certificates, and credentials for any other external systems used by your Cadence executions.

Actions JetBrains has taken

We took the Cadence server offline on August 24, 2026, while we continue to investigate the incident. At present, we have confirmed that the incident is limited to data associated with the Cadence host mentioned above.

The server should have been patched as part of our response to the vulnerability, but it was not. We sincerely apologize for this failure and the impact it may have on you.

We have invalidated all access tokens used by the JetBrains Cadence plugin in PyCharm to connect to Cadence, and took the server offline on August 24, 2026.

We are also notifying the relevant authorities and taking the necessary steps to protect the data of Cadence users.

Further updates

We will publish further findings and guidance here as our investigation progresses. We recommend checking this page frequently for the latest information. We will also contact affected users directly if we identify any important new information that may require action on their part.

For more information about the underlying vulnerability, please see our original security advisory to TeamCity customers and users.

If you previously used Cadence and need assistance identifying which credentials may have been exposed or have any questions regarding this incident, contact the JetBrains Security team at security@jetbrains.com.

We recognize the seriousness of this incident and apologize again for the impact.

August 28, 2026 09:50 AM UTC

Python Software Foundation

Georgi Ker: 2026 PSF Board Election Candidate Interview

Who are you?

Hi, I’m Georgi, an independent entrepreneur, open source community organiser, and leader. I’m also a PSF Fellow and a recipient of the PSF Community Service Award.

Although I currently live in Amsterdam, I am proud to represent communities across Asia. I have served on the PSF Board since 2023. I also helped design PyCon US branding and websites since 2022 and for other open source community events and projects as well.

My contributions to the Python community span over many years, beginning mainly in Asia and becoming more international over time. Much of my work sits where people, governance, and community meet.

I care especially about the people doing the quiet work that keeps open source communities alive. They may not always be on the main stage, but the community would not exist without them.

What would you bring to the PSF Board of Directors?

Three years on the Board have taught me where the PSF is strong, where it is fragile, and where good intentions are not becoming results.

As a former Treasurer, I raised concerns about the lack of a clear financial runway and pushed for stronger budgeting and regular financial planning. Through the Executive Committee, I have also participated in staff discussions, document reviews, difficult organisational decisions, and helped finalize the PSF’s long-delayed strategic planning process with the board.

My perspective is also strongly shaped by communities around the world. The work began from Asia and now includes PyLadiesCon, EuroPython, and regular discussions with global community leaders through the PSF Diversity and Inclusion Workgroup. Global representation at the PSF has improved, but it still remains incomplete.

I would bring institutional knowledge, experience in financial oversight and a global community perspective. And yes, I am willing to do the unglamorous work.

I know more now than when I joined. I know what I am signing up for and how much work remains.

What motivated you to run for the PSF Board of Directors?

I am running because the past three years have shown me much more clearly what the PSF needs next. I’ve seen where the Foundation works well. I’ve also seen outdated financial assumptions, limited organisational capacity, unclear responsibilities, and decisions that could not be implemented because the necessary systems or that expertise were missing.

These are not exciting campaign topics but unfortunately, they matter.

Python is now part of the world’s infrastructure. Companies, governments, researchers, schools, and millions of developers depend on it. The organisation responsible for protecting Python must be able to plan beyond the next conference or sponsorship cycle.

I would like to continue helping the PSF build a more realistic base, improve its financial planning, strengthen its internal systems, and communicate more honestly with the community.

The next few years matter. That is why I am running again.

What problem or challenge do you want to address if you are on the Board?

The problem I most want to address is the PSF’s financial sustainability.

For years, PyCon US was a reliable source of income for the Foundation. After COVID, the economics and risks of large conferences changed. PyCon US should remain an important community event, but one conference should not be expected to carry the financial future of the organisation protecting Python.

The PSF needs realistic budgets and financial projections. The Board needs to understand the Foundation’s runway, commitments, risks, and future operating costs. Hope is useful in open source but less useful in accounting.

We also need to build sponsorship around Python itself. Companies benefit from Python every day, not only when their logo appears at a conference. Their support should help fund Python’s infrastructure, security, legal protection, trademarks, and global community.

This requires a serious review of the PSF’s business model. It also requires the right staff capacity and expertise to turn Board decisions into results.

I would like the PSF to professionalise the support around the community without professionalising the community out of Python. That distinction really matters. Volunteers and local organisers are not a cheap workforce. They are the reason Python has a community in the first place.

Where do you see the PSF 5 years from now?

In five years, I would like to see the PSF be an organisation that plans with evidence instead of habit.

The PSF should be better equipped to protect Python’s trademarks, copyright, infrastructure, and identity. As Python becomes more important, this will require greater legal, financial, security, and organisational expertise. Goodwill alone cannot do all that work.

It should have a clear financial runway, realistic budgets, and dependable sources of income. PyCon US should continue bringing people together, but its financial performance should not decide whether the Foundation can support Python properly.

I would also like a more transparent PSF. The community should understand what the Board is working on, what the Foundation can afford, why major decisions were made, and what progress followed.

Finally, the PSF should remain global and human. It should develop new leaders, support communities outside North America and Europe, and recognise capable people who may not be famous conference speakers. In fact, the board should consider reserving a few board seats for appointed directors with expertise that the Foundation needs.

The PSF should become more professional in how it operates without becoming corporate in how it treats people. Python’s community is not a side project. It is one of its greatest strengths and part of its unique model.

What areas of the Python community are you involved with?

I have served on the PSF Board since 2023, including as Treasurer and currently as Vice Chair.

I currently chair the PSF Diversity and Inclusion Workgroup. I am also one of the organisers of PyLadiesCon and a member of the EuroPython Code of Conduct team. My earlier involvement included PyCon Thailand, PyCon APAC, founding PyLadies Bangkok, and other regional communities. I am also involved with the podcast PyPodcats featuring underrepresented Pythonistas.

Apart from that, I am developing Open Community Leadership through the fellowship with the Sovereign Tech Agency. The project focuses on mentorship, succession planning, sustainability, and helping communities prepare their next generation of leaders.

------

Note from the election administrators:

Want to learn more about this candidate or ask them a question?

Check out their nomination statement.

Check out their AMA thread on discuss.python.org.

August 28, 2026 08:44 AM UTC

August 27, 2026

Python Morsels

When to use NotImplemented

When should you return NotImplemented from a dunder method? Why not return False or raise an exception instead?

Table of contents

Dunder methods return `NotImplemented`

Integers and floating point numbers can be compared with equality:

>>> a = 3
>>> b = 3.0
>>> a == b
True

This is powered by the __eq__ method.

But strangely, if we call the __eq__ method on an integer, giving it a floating point number, we'll get back NotImplemented:

>>> a = 3
>>> b = 3.0
>>> a.__eq__(b)
NotImplemented

But if we do the same thing on a floating point number, giving it an integer, we get the answer we're expecting:

>>> a = 3
>>> b = 3.0
>>> b.__eq__(a)
True

The same thing happens with many other operations in Python.

For example, the + operator is powered by the __add__ method:

>>> a = 3
>>> b = 3.0
>>> b + a
6.0

But just as with equality, floating point numbers can be added to integers, but integers cannot be added to floating point numbers:

>>> a = 3
>>> b = 3.0
>>> b.__add__(a)
6.0
>>> a.__add__(b)
NotImplemented

What is the NotImplemented value and why do we get it?

`NotImplemented` means "I don't know"

Dunder methods that perform an …

Read the full article: https://www.pythonmorsels.com/when-to-use-notimplemented/

August 27, 2026 09:45 PM UTC

Robin Wilson

How to fix a weird pandas and pyarrow issue with BirdNetPi

Summary: If you get Python crashing immediately in BirdNetPi, try uninstalling the pyarrow Python package.

I’ve got a BirdNetPi set up at home. This is a bit of software that runs on a Raspberry Pi and listens on a microphone (I’ve mounted mine on the outside of an upstairs window, using a 3D printed mount/cover to protect it from the rain) for bird noises. It then classifies them using a machine learning model and tells you what birds it’s heard. It has a nice web interface which shows graphs like this:

file

as well as allowing you to listen to the recordings, correct errors, do more detailed statistics etc.

When I first installed it, I had some problems. One of the big ones, in fact, was that the graph above wasn’t present on the homepage (I didn’t actually know there should be a graph there, so it took me a while to realise it was missing) and when loading the detailed Species Stats page I just got a message from Streamlit saying "Connecting…".

Investigating this in more depth, I found that a lot (but not all) of the Python scripts involved in creating the visualisations crashed as soon as you started them, giving an ‘Illegal instruction’ error. This is a remarkably low-level error to get from Python, and suggested something was going wrong with a package that had some sort of native (C/C++) extension.

My first guess was that it was something to do with pandas, so I tried uninstalling and reinstalling pandas – but that didn’t fix it. I installed pandas in a new virtual environment and I could import it fine – so it wasn’t pandas itself that was causing the problem.

Then I realised I could use part of Python’s profiling tools to help narrow down what import was the problem. I ran birdnet/bin/python -X importtime -c 'import pandas', which uses the BirdNet Python interpreter to run import pandas while printing out timing information about how long each import took. This should allow me to see what package was imported just before the crash – and it did. The output looked like this:

import time:      1623 |      10550 |     pandas.compat.numpy
import time:       309 |        309 |         gc
import time:      1121 |       1121 |         pyarrow._generated_version

The most recently imported package at the time of the crash was pyarrow – a Python interface for the arrow file format. I believe it was originally installed as an optional dependency of pandas. I confirmed this was the problem by running import pyarrow in the BirdNet Python interpreter, and it immediately crashed.

This narrowed down the problem, but I wasn’t sure how I’d get the correct version of pyarrow installed. However, before doing that, I thought I’d try and see whether BirdNetPi actually used pyarrow anywhere. Searching through the git repo for arrow found no uses of it, so I thought it was worth trying uninstalling the package entirely.

This solved the problem straight away – and I haven’t found anything in BirdNetPi that has broken as a result of it.

August 27, 2026 05:57 PM UTC

The Python Show

57 - Python Developer Tooling Handbook with Tim Hopper

In this episode of the Python Show Podcast, I am pleased to have Tim Hopper as my guest. Tim is the author of the Python Developer Tooling Handbook, a free online resource for everyone.

We chat about Python tooling, writing, Tim’s background, AI, and much more!

You can check out Tim’s free handbook and some of his other work here:

Python Developer Tooling Handbook
pythonplot.com — a guide to Python plotting libraries
Into the Hopper — podcast on ML and software engineering

August 27, 2026 05:03 PM UTC

Python Software Foundation

Jeremy Tanner: 2026 PSF Board Election Candidate Interview

Who are you?

I'm Jeremy Tanner, an organizer, speaker, sponsor, Python developer, and community member. I'm based in North America, though many of my favorite people, places, and events are not. I've spent much of my career in the open Python ecosystem; packaging infrastructure, developer tooling, and the supply chains that get Python into the hands of pythonistas building everything imaginable. I believe the Python community should look like the world, not just the English-speaking, North American corner of it, and that sustainable finances are what make that vision achievable. I'm running for the PSF Board to do both: raise sustainable funds and resources, and make sure they reach everywhere.

What would you bring to the PSF Board of Directors?

Sponsorship for Python Foundations, Events, Organizations: I have direct experience securing corporate partnerships and sponsorships for the Python ecosystem, including navigating the internal processes of large companies to provide support to open source projects, infrastructure, and community gatherings. I’ve been on the organizing side as well, responsible for fundraising, programming, and attendee experience.

What motivated you to run for the PSF Board of Directors?

I wouldn't be here if it weren't for Python.

The PSF is the organizational backbone of the most important programming language in the world, and it is perpetually underfunded relative to its mission. Companies and the community benefit enormously from the PSF's work sustaining PyPI, funding sprints, and keeping the ecosystem healthy. Many are interested and capable of supporting further, but unsure how. I want to change that, and I want the resources generated to flow toward a Python community that genuinely reflects the mission's diverse and global users.

What problem or challenge do you want to address if you are on the board?

Build a sustainable corporate partnership program

The PSF has sponsorship tiers, but lacks a structured program for engaging companies at the scale their Python dependency warrants. I’m interested in working with PSF staff to design and execute a partnership program that makes it easier, and compelling, for companies, and others to contribute meaningfully. I've done this work both from the corporate side, and the position of organizer as well. I know what makes it succeed.

Invest in regional and international events

PyCon US has been the largest gathering, but is no longer running profitably. Python is spoken in Lagos, São Paulo, Jakarta, Warsaw, and the events, meetups, and communities in those places are where most of the world's Python developers live. The PSF grants program is the primary lever for supporting these communities, and it is chronically under-resourced. I’ll advocate for dedicated, predictable funding for regional and international events, not one-off grants that organizers have to re-apply for every year, but structural support that lets community leaders plan, grow, and mentor the next generation of organizers in their own languages and time zones. A Python community that looks like the world requires the PSF to invest across the globe.

Connect packaging infrastructure investment to the PSF's mission

PyPI is critical infrastructure, and its funding has historically been precarious. My background in packaging and distribution gives me the context to make the case, both to the board and to corporate partners, for why sustained investment in Python's packaging ecosystem is a strategic priority, not an afterthought. I want the PSF to be a more confident and articulate advocate for the infrastructure that millions of developers depend on daily.

Where do you see the PSF 5 years from now?

Celebrating Python's 40th anniversary. Strong, sustainable, with a growing staff & membership. Not a member yet? Please consider joining the PSF as a Supporting Member

What areas of the Python community are you involved with?

I've spoken at, sponsored, or participated in PyCon US, as well as PyLadies, PyCarribean, PyTexas, PyGotham, SciPy, PyData (Texas, London, New York, Seattle), North Bay Python, and meetups, have kept me grounded in what pythonistas across the community are building, struggling with, and asking for.

Packaging: Working with package maintainers and partners in order to see that all pythonistas are able to get the software they need.

------

Note from the election administrators:

Want to learn more about this candidate?

Check out their nomination statement.

Check out their AMA thread on discuss.python.org.

August 27, 2026 02:57 PM UTC

PyCharm

OpenTelemetry Comes to IntelliJ IDEA, GoLand, PyCharm, and WebStorm.

The OpenTelemetry plugin has broken out of the confines of JetBrains Rider. No sandbox exploit was involved – this escape was planned by our developers. With the 2026.2 release, the OpenTelemetry plugin is now available in IntelliJ IDEA, GoLand, PyCharm and WebStorm. Rider users needn’t worry – it still works there too.

The plugin brings logs, metrics, traces, and the service map from local applications into the IDE. You can inspect them without setting up a separate local observability backend.

Get the OTel plugin from the Marketplace

What is the OpenTelemetry plugin?

While developing with your IDE, you can use the plugin to:

Find what happened immediately before an error.
See which services and dependencies a request reached.
Pinpoint where a request spent its time.
Check whether the application emitted the expected logs, metrics, and traces.
Ask your coding agent to do all of the above via MCP.

The plugin’s functionality complements what the IDE offers out of the box. Use a debugger to step through code, a profiler to analyze performance, and a production monitoring platform to watch deployed systems. Use the OpenTelemetry plugin to inspect runtime behavior while you build or test the application.

Imagine this

Say you are testing a feature that calls several services and a database before publishing a message to a queue. The request fails, and the console shows the exception, but not the path that led to it.

In the OpenTelemetry tool window, you can search the logs for the error. Open the relevant trace to see how the request moved through the system and where it failed. The Service Map shows the services and infrastructure involved.

Explore runtime data in the IDE

Search and inspect logs

Console output is manageable until several services start writing at once. The Logs view puts OpenTelemetry log records in a searchable table. You can filter by severity or content, then open a record to inspect its attributes.

Detailed log view showing timestamps, levels, and messages for different log types

Check metrics

Select a metric from the metric tree to plot its values while you use the application. This view is not a replacement for your production dashboards, but it lets you inspect what the application will export before you send the data to a production observability platform. If a new library adds noisy or unnecessary metrics, you can catch them locally and adjust the instrumentation before your DevOps and SRE colleagues have to handle them.

The metrics viewer displays CPU utilization data with real-time charts.

Follow a request through its trace

Open a trace to see its spans across services. Each span includes timing and attributes, so you can follow the request from start to finish and zero in on the operation that failed or slowed things down. Development is also a good time to look at the trace itself. Does it contain the spans and details you’d need during a real incident? It’s much easier to fix those gaps now than to discover them in production.

The trace table view with filtering capabilities and detailed information for the selected trace.

The trace viewer showing details of the POST request involving multiple internal DB and HTTP calls.

See observed relationships in the Service Map

Architecture diagrams go stale. The Service Map builds its relationships from collected traces, so it reflects the traffic the plugin has seen between services, endpoints, databases, and message queues.

Expected one HTTP call or database query, but the diagram shows several? Finding that during development gives you time to fix it before release.

Automatically generated service map showing the relationship between API endpoints, internal services, and PostgreSQL based on actual runtime traces.

Connect an instrumented application

The plugin does not add OpenTelemetry libraries or agents to your application. It configures where an already instrumented application sends its data.

Start an instrumented Java, Python, Go, or .NET application from a supported IDE run configuration, and the plugin passes OpenTelemetry Protocol (OTLP) environment variables that point to its built-in receiver. It passes the same variables to new integrated terminal sessions.

You can also point the application’s OTLP exporter at the endpoint shown by the plugin. If you already use a local OpenTelemetry Collector, add the plugin as an OTLP destination and keep the rest of your pipeline.

Runtime context for coding agents

The plugin has experimental MCP support through the JetBrains MCP server. A compatible coding agent can query the telemetry collected in the IDE with these tools:

get_log_records
get_spans
get_services
get_service_map

These tools give the agent evidence from a particular run, so that it can inspect the logs and spans or query the observed service relationships.

Getting started

Install the OpenTelemetry plugin from JetBrains Marketplace for your favorite IDE.
Instrument your application with OpenTelemetry libraries or agents.
Start it from a supported IDE run configuration or a new integrated terminal session. The plugin will pass the OTLP environment variables. Alternatively, point the application’s exporter or your local collector at the endpoint shown by the plugin.
Open the OpenTelemetry tool window and exercise the part of the application you want to inspect.

The plugin shows the signals it receives. If your application exports only traces, the Logs and Metrics views stay empty.

Full setup instructions are available in the OpenTelemetry plugin documentation.

Install it and tell us what you find

Get the OTel plugin from the Marketplace

Try it on a real project, then tell us in the comments or through the issue tracker what worked and what still sent you to another tool.

Some final notes

This plugin is the product of a collaborative effort between the Rider Execution team and the Dynamic Program Analysis Research team at JetBrains Research. This launch is the first step in exploring how telemetry analysis can help us build development tools that better adapt to real-world developer workflows.

August 27, 2026 10:41 AM UTC

PyCon Ireland

PyCon Ireland 2026 Updates & Final Call for Proposals

PyCon Ireland 2026 is now three months away! Please join us on 21 November in Dublin’s city centre to hear some wonderful talks and workshops from the Python community.

You have just a few more days to submit your proposals for talks and workshops. Share your Python knowledge and experience with the community.

We’re looking for talks and workshops on the Python core and internals, open source, security, testing, libraries, scientific computing, data science, machine learning, and everything Pythonic. We have lots of AI-related submissions already but there’s room for more.

Submit your proposals no later than 30 August on Sessionize.

See you soon!

August 27, 2026 12:00 AM UTC

Core Dispatch

Core Dispatch #10

Welcome back to Core Dispatch! This edition covers August 5 through August 27, 2026. Python 3.12.14, 3.11.16, and 3.10.21 shipped on August 12. Next up is Python 3.15.0 release candidate 2, due September 1.

There are five new PEPs to catch up on. PEP 805 proposes a model for safe parallel Python. PEP 843 and PEP 844 offer two approaches to explicit module exports. PEP 841 proposes syntax for frozen types, while PEP 839 adds C APIs for building immutable collections. PEP 833 also reached Final, freezing the HTML form of the simple repository API. HTML support stays, but future standards work moves to JSON. Speaking of PEPs, PEP 11 was also updated to list RISC-V as a tier 3 platform.

Many core team members spoke about Python core development at EuroPython this year. All nine talks are in this edition's Core Team Musings, covering the CPython ABI, the JIT, free-threading, interpreter internals, security, and contributing to Python. Voters can also catch up on nominee AMAs for the PSF Board and inaugural Python Packaging Council elections before ballots open on September 1.

Python 3.15 is in the release candidate phase, so this is the moment for maintainers to test their projects and publish Python 3.15 wheels to PyPI. Wheels built against the 3.15 release candidates will work with future 3.15 releases, and publishing them now helps downstream projects test too. Please file any CPython issues you find.

Upcoming Releases

Python 3.15.0 release candidate 2 — Sep 01

Official News

RISC-V is now officially supported by CPython — By Stan Ulbrych
2026 Python Packaging Council election candidates announced — By Pradyun Gedam
Python 3.12.14, 3.11.16, and 3.10.21 are now available — By Thomas Wouters
2026 PSF Board candidates announced — By Marie Nordin

PEP Updates

Steering Council Updates

Merged PRs

One More Thing

"No CVE, no merg-e"
— Seth Larson

Credits

Savannah Ostrowski

August 27, 2026 12:00 AM UTC

August 26, 2026

Talk Python to Me

#560: Building a Research OS: From Django to 30,000 Samples

In 2020, a gastroenterologist in Glasgow did the math on his new research study and came up with 30,000 samples, arriving over two years from three cities and a dozen hospitals. He asked around about how researchers keep track of that. The answer was Microsoft Excel. Shaun Chuah had written some HTML by hand in Notepad back in high school and that was about the whole of his programming experience, so he opened the Django tutorial and started reading. Six years later that app is Foundry120, holding 10 terabytes of clinical and genomics data with an agentic AI running on top of it. Episode sponsors <a href='https://talkpython.fm/sentry'>Sentry Error Monitoring, Code talkpython26</a> <a href='https://talkpython.fm/course-certifications'>Talk Python Courses</a> <a href='https://talkpython.fm/training'>Talk Python Courses</a> <h2 class="links-heading mb-4">Links from the show</h2> <div>Guest Shaun Chuah: <a href="https://github.com/shaunchuah?featured_on=talkpython" target="_blank" >github.com</a> Up and Running with Rust Course: <a href="https://training.talkpython.fm/courses/up-and-running-with-rust" target="_blank" >talkpython.fm</a> Foundry120: <a href="https://www.foundry120.com/?featured_on=talkpython" target="_blank" >www.foundry120.com</a> Designing Data Intensive Applications: <a href="https://www.oreilly.com/library/view/designing-data-intensive-applications/9781491903063/?featured_on=talkpython" target="_blank" >www.oreilly.com</a> Microsoft Foundry: <a href="https://ai.azure.com/home?featured_on=talkpython" target="_blank" >ai.azure.com</a> ChatIBD: <a href="https://www.chatibd.com/?featured_on=talkpython" target="_blank" >www.chatibd.com</a> Blog: <a href="https://shaunchuah.github.io/?featured_on=talkpython" target="_blank" >shaunchuah.github.io</a> @drshaunchuah: <a href="https://x.com/drshaunchuah?featured_on=talkpython" target="_blank" >x.com</a> github.com/shaunchuah: <a href="http://github.com/shaunchuah?featured_on=talkpython" target="_blank" >github.com</a> Watch this episode on YouTube: <a href="https://www.youtube.com/watch?v=zwL1-VQMUo0" target="_blank" >youtube.com</a> Episode #560 deep-dive: <a href="https://talkpython.fm/episodes/show/560/building-a-research-os-from-django-to-30-000-samples#takeaways-anchor" target="_blank" >talkpython.fm/560</a> Episode transcripts: <a href="https://talkpython.fm/episodes/transcript/560/building-a-research-os-from-django-to-30-000-samples" target="_blank" >talkpython.fm</a> Theme Song: Developer Rap 🥁 Served in a Flask 🎸: <a href="https://talkpython.fm/flasksong" target="_blank" >talkpython.fm/flasksong</a> ---== Don't be a stranger ==--- YouTube: <a href="https://talkpython.fm/youtube" target="_blank" > youtube.com/@talkpython</a> Bluesky: <a href="https://bsky.app/profile/talkpython.fm" target="_blank" >@talkpython.fm</a> Mastodon: <a href="https://fosstodon.org/web/@talkpython" target="_blank" > @talkpython@fosstodon.org</a> X.com: <a href="https://x.com/talkpython" target="_blank" > @talkpython</a> Michael on Bluesky: <a href="https://bsky.app/profile/mkennedy.codes?featured_on=talkpython" target="_blank" >@mkennedy.codes</a> Michael on Mastodon: <a href="https://fosstodon.org/web/@mkennedy" target="_blank" > @mkennedy@fosstodon.org</a> Michael on X.com: <a href="https://x.com/mkennedy?featured_on=talkpython" target="_blank" > @mkennedy</a> </div>

August 26, 2026 08:04 PM UTC

Rodrigo Girão Serrão

Why OOP exists

Learn the fundamental principles behind OOP and how they connect to the syntax of Python.

Introduction

Welcome! This article will teach you the core ideas behind object-oriented programming, commonly known as OOP. This is the article I wish I read many years ago, when I was first learning about OOP in Python.

If you're new to OOP, this article will explain why OOP exists, how it works, and how to work with OOP in Python. If you think you already know OOP, this article will change the way you think about programming and Python.

Code is about real-world things

Let me tell you about a programming project I had to do in college where you had to write a library management service. We were expected to work in pairs and I was paired with my good friend Tito.

Tito and I sat down and started going through the problem statement, figuring out what we needed to implement. We got to a point where Tito turned to me and said:

“It's not obvious to me what's the best way to represent a book in our program. Maybe you can start implementing the search functionality and I'll think about this for a while.”

The search functionality was a set of functions that the problem statement required us to implement:

find_by_title(catalog, search_term): returns a sublist with the books whose title contains the given search term
find_by_genre(catalog, genre): returns a sublist with the books of the given genre
find_by_author(catalog, author): returns a sublist with the books written by the given author

I nodded, but then I thought about it for a second. If I don't know anything about how to work with books, there's no way I can implement these three functions. Tito agreed with me and told me he'd provide me with these functions:

book_title(book): returns the title of the given book
book_genre(book): returns the genre of the given book
book_author(book): returns the author of the given book

He told me to think of these functions as auxiliary functions that he would implement. I didn't have them yet, but I could write my search functions trusting he'd implement them correctly.

In OOP, you have entities with associated data (books with authors, titles, and genres) and a set of functions to operate on those entities (the functions find_by_xxx).

Now, think about it for a second. Can you implement the functions find_by_title, find_by_genre, and find_by_author, using the auxiliary functions that Tito will implement? How would you go about it?

I worked on it for a bit, and eventually used a list comprehension to define the function find_by_title:

def find_by_title(catalog, search_term):
    search_term = search_term.casefold()
    return [
        book
        for book in catalog
        if search_term in book_title(book).casefold()
    ]

The function find_by_title goes through the list of books called catalog with a loop and uses the auxiliary function book_title to retrieve the title. It then uses casefold to perform a case-insensitive search.

Something worth...

August 26, 2026 02:01 PM UTC

Planet Python

September 01, 2026

Test with 3.15 #

How #

When to support 3.15? #

ABI breaks? #

See also #

How to Vote

An Informed Vote is a Great Vote

Who can vote?

Find Your Ballot

Review the Nominees

Who Can Vote?

The code under test

Where the two look the same

Calls an object makes to itself

Running the real code while changing one thing

Asserting on what did not happen

How the tests are shaped

Where mock still fits

What's next

Wrap anything, capture everything

Three uses of one mechanism

Where it sits beside what already exists

Pre-built instrumentation

Built with AI, on purpose

What's next

Why shouldn't you buy other regions' games?

August 31, 2026

Why fine-tune at all?

The models

Sanity check: Reproducing COCO val2017 baselines

The datasets

Tutorial: Fine-tuning all three models in PyCharm 🙂

Step 1: Setting up the project

Step 2: Getting the datasets

Step 3: Getting a zero-shot baseline by using pretrained models on custom data

Step 4: Fine-tuning

YOLO12 and YOLO26

RF-DETR

Step 5: Results

Qualitative results

Conclusions

Get started with PyCharm today

Who are you?

What would you bring to the PSF Board of Directors?

What motivated you to run for the PSF Board of Directors?

What problem or challenge do you want to address if you are on the board?

Where do you see the PSF in 5 years from now?

What areas of the Python community are you involved with?

Note from the election administrators:

Which Agentic Chatbot?

You're Golden

Routing is the starting point

Agents became skills, but routing remained

TCR jumps to the endpoint, the response

Multi-step conversations changed the unit of testing

Coding it yourself: deepeval underneath

Langfuse made the failures inspectable

Trajectory testing -> knitted the pieces together

Sovereign AI makes the eval problem tiered S/M/L/XL

What I would keep from the journey

August 30, 2026

August 28, 2026

1. AI is mainstream, but no workflow has won

2. Dominating AI workflow is still supervised

3. Python tooling is consolidating fast

4. Type hints are winning. The type checker race is wide open.

5. As AI writes more code, verification matters more

6. One framework, two ways to build

Conclusion

PyCharm for Django Fundraiser

September 1, 2026, 12:05 CEST

August 31, 2026, 12:56 CEST

August 28, 2026, 11:50 CEST

Actions required immediately

Affected server

Affected period

What happened

What we know

Sanity check: Reproducing COCO `val2017` baselines

Dunder methods return `NotImplemented`

`NotImplemented` means "I don't know"