Planet Python
Last update: February 28, 2025 01:43 AM UTC
February 28, 2025
Zero to Mastery
Python Monthly Newsletter 💻🐍
63rd issue of Andrei Neagoie's must-read monthly Python Newsletter: which AI tool to use in 2025, Django vs. FastAPI, and much more. Read the full newsletter to get up-to-date with everything you need to know from last month.
February 28, 2025 01:43 AM UTC
February 27, 2025
Bojan Mihelac
Django-treebeard: Converting an Existing Model to MP_Node
This article explains how to convert an existing Django model into an MP_Node model in django-treebeard, handling migrations, data population, and field constraints.
February 27, 2025 07:37 PM UTC
Daniel Roy Greenfeld
TIL: Undecorating a functools.wraps decorated function
Another reason to use functools.wraps!
February 27, 2025 08:25 AM UTC
February 26, 2025
Sebastian Pölsterl
scikit-survival 0.24.0 released
It’s my pleasure to announce the release of scikit-survival 0.24.0.
A highlight of this release the addition of cumulative_incidence_competing_risks() which implements a non-parameteric estimator of the cumulative incidence function in the presence of competing risks. In addition, the release adds support for scikit-learn 1.6, including the support for missing values for ExtraSurvivalTrees.
Analysis of Competing Risks
In classical survival analysis, the focus is on the time until a specific event occurs. If no event is observed during the study period, the time of the event is considered censored. A common assumption is that censoring is non-informative, meaning that censored subjects have a similar prognosis to those who were not censored.
Competing risks arise when each subject can experience an event due to one of $K$ ($K \geq 2$) mutually exclusive causes, termed competing risks. Thus, the occurrence of one event prevents the occurrence of other events. For example, after a bone marrow transplant, a patient might relapse or die from transplant-related causes (transplant-related mortality). In this case, death from transplant-related mortality precludes relapse.
The bone marrow transplant data from Scrucca et al., Bone Marrow Transplantation (2007) includes data from 35 patients grouped into two cancer types: Acute Lymphoblastic Leukemia (ALL; coded as 0), and Acute Myeloid Leukemia (AML; coded as 1).
from sksurv.datasets import load_bmt
bmt_features, bmt_outcome = load_bmt()
diseases = bmt_features["dis"].cat.rename_categories(
{"0": "ALL", "1": "AML"}
)
diseases.value_counts().to_frame()
dis | count |
---|---|
AML | 18 |
ALL | 17 |
During the follow-up period, some patients might experience a relapse of the original leukemia or die while in remission (transplant related death). The outcome is defined similarly to standard time-to-event data, except that the event indicator specifies the type of event, where 0 always indicates censoring.
import pandas as pd
status_labels = {
0: "Censored",
1: "Transplant related mortality",
2: "Relapse",
}
risks = pd.DataFrame.from_records(bmt_outcome).assign(
label=lambda x: x["status"].replace(status_labels)
)
risks["label"].value_counts().to_frame()
label | count |
---|---|
Relapse | 15 |
Censored | 11 |
Transplant related mortality | 9 |
The table above shows the number of observations for each status.
Non-parametric Estimator of the Cumulative Incidence Function
If the goal is to estimate the probability of relapse, transplant-related death is a competing risk event. This means that the occurrence of relapse prevents the occurrence of transplant-related death, and vice versa. We aim to estimate curves that illustrate how the likelihood of these events changes over time.
Let’s begin by estimating the probability of relapse using the complement of the Kaplan-Meier estimator. With this approach, we treat deaths as censored observations. One minus the Kaplan-Meier estimator provides an estimate of the probability of relapse before time $t$.
import matplotlib.pyplot as plt
from sksurv.nonparametric import kaplan_meier_estimator
times, km_estimate = kaplan_meier_estimator(
bmt_outcome["status"] == 1, bmt_outcome["ftime"]
)
plt.step(times, 1 - km_estimate, where="post")
plt.xlabel("time $t$")
plt.ylabel("Probability of relapsing before time $t$")
plt.ylim(0, 1)
plt.grid()
However, this approach has a significant drawback: considering death as a censoring event violates the assumption that censoring is non-informative. This is because patients who died from transplant-related mortality have a different prognosis than patients who did not experience any event. Therefore, the estimated probability of relapse is often biased.
The cause-specific cumulative incidence function (CIF) addresses this problem by estimating the cause-specific hazard of each event separately. The cumulative incidence function estimates the probability that the event of interest occurs before time $t$, and that it occurs before any of the competing causes of an event. In the bone marrow transplant dataset, the cumulative incidence function of relapse indicates the probability of relapse before time $t$, given that the patient has not died from other causes before time $t$.
from sksurv.nonparametric import cumulative_incidence_competing_risks
times, cif_estimates = cumulative_incidence_competing_risks(
bmt_outcome["status"], bmt_outcome["ftime"]
)
plt.step(times, cif_estimates[0], where="post", label="Total risk")
for i, cif in enumerate(cif_estimates[1:], start=1):
plt.step(times, cif, where="post", label=status_labels[i])
plt.legend()
plt.xlabel("time $t$")
plt.ylabel("Probability of event before time $t$")
plt.ylim(0, 1)
plt.grid()
The plot shows the estimated probability of experiencing an event at time $t$ for both the individual risks and for the total risk.
Next, we want to to estimate the cumulative incidence curves for the two cancer types — acute lymphoblastic leukemia (ALL) and acute myeloid leukemia (AML) — to examine how the probability of relapse depends on the original disease diagnosis.
_, axs = plt.subplots(2, 2, figsize=(7, 6), sharex=True, sharey=True)
for j, disease in enumerate(diseases.unique()):
mask = diseases == disease
event = bmt_outcome["status"][mask]
time = bmt_outcome["ftime"][mask]
times, cif_estimates, conf_int = cumulative_incidence_competing_risks(
event,
time,
conf_type="log-log",
)
for i, (cif, ci, ax) in enumerate(
zip(cif_estimates[1:], conf_int[1:], axs[:, j]), start=1
):
ax.step(times, cif, where="post")
ax.fill_between(times, ci[0], ci[1], alpha=0.25, step="post")
ax.set_title(f"{disease}: {status_labels[i]}", size="small")
ax.grid()
for ax in axs[-1, :]:
ax.set_xlabel("time $t$")
for ax in axs[:, 0]:
ax.set_ylim(0, 1)
ax.set_ylabel("Probability of event before time $t$")
The left column shows the estimated cumulative incidence curves (solid lines) for patients diagnosed with ALL, while the right column shows the curves for patients diagnosed with AML, along with their 95% pointwise confidence intervals. The plot indicates that the estimated probability of relapse at $t=40$ days is more than three times higher for patients diagnosed with ALL compared to AML.
If you want to run the examples above yourself, you can execute them interactively in your browser using binder.
February 26, 2025 09:26 PM UTC
Real Python
How to Work With Polars LazyFrames
A Polars LazyFrame provides an efficient way to handle large datasets through lazy evaluation. Unlike traditional DataFrames, LazyFrames don’t contain data but instead store a set of instructions known as a query plan. Query plans perform operations like predicate and projection pushdown, ensuring only necessary rows and columns are processed. LazyFrames also support the parallel execution of query plans, further enhancing performance.
By the end of this tutorial, you’ll understand that:
- A Polars LazyFrame allows efficient data processing by storing query instructions instead of data.
- Lazy evaluation in LazyFrames optimizes query plans before data materialization.
- Predicate and projection pushdown minimize unnecessary data processing in LazyFrames.
- You create a LazyFrame using functions like
scan_parquet()
orscan_csv()
. - Switching between lazy and eager modes is sometimes necessary for certain operations.
Dive into this tutorial to discover how LazyFrames can transform your data processing tasks, providing both efficiency and flexibility for managing large datasets.
Before you start your learning journey, you should already be comfortable with the basics of working with DataFrames. This could be from any previous Polars experience you have, or from using any other DataFrame library, such as pandas.
In addition, you may consider using Jupyter Notebook as you work through many of the examples in this tutorial. Alternatively, JupyterLab will enhance your notebook experience, but any Python environment you’re comfortable with will be fine.
To get started, you’ll need some data. For the main part of this tutorial, you’ll use the rides.parquet
file included in the downloadable materials. You can download this by clicking the link below:
Get Your Code: Click here to download the free sample code that shows you how work with Polars LazyFrames.
The rides.parquet
file is a doctored version of the taxi ride data freely available on the New York City Taxi and Limousine Commission (TLC) website. The dataset contains edited information about New York taxi cab rides from July 2024. Before you go any further, you’ll need to download the file and place it in your project folder.
Note: The Parquet format is a format for storing large volumes of data efficiently. Parquet files use compression to minimize storage space. They also maintain metadata about each column allowing columns to be searched efficiently, often in parallel, and without the need to read the entire file. Because this metadata is useful to LazyFrames when they need to investigate a file’s content, Parquet is an excellent format for LazyFrames to use.
The table below shows details of the rides.parquet
file’s columns, along with their Polars data types. The text in parentheses beside each data type shows how these types are annotated in a DataFrame header when Polars displays its results:
Column Name | Polars Data Type | Description |
---|---|---|
pick_up |
String (str) |
Pick-up borough |
drop_off |
String (str) |
Drop-off borough |
passengers |
Int32 (i32) |
Number of passengers |
distance |
Int32 (i32) |
Trip distance (miles) |
fare |
Int32 (i32) |
Total fare ($) |
As a starting point, you’ll create a LazyFrame and take a first look at its data. To use Polars, you first need to install the Polars library into your Python environment. You might also like to install Matplotlib as well. You’ll use this to view the inner workings of a LazyFrame graphically later on. To install both from a command prompt you use:
In a Jupyter Notebook, the command is !python -m pip install polars matplotlib
.
You can then begin to use the Polars library and all of its cool features. Here’s what your data looks like, and here’s how to create your first LazyFrame:
>>> import polars as pl
>>> rides = pl.scan_parquet("rides.parquet")
>>> rides.collect()
shape: (3_076_903, 5)
┌───────────┬───────────┬────────────┬──────────┬──────┐
│ pick_up ┆ drop_off ┆ passengers ┆ distance ┆ fare │
│ --- ┆ --- ┆ --- ┆ --- ┆ --- │
│ str ┆ str ┆ i32 ┆ i32 ┆ i32 │
╞═══════════╪═══════════╪════════════╪══════════╪══════╡
│ Manhattan ┆ Manhattan ┆ 1 ┆ 3 ┆ 24 │
│ Queens ┆ Manhattan ┆ 1 ┆ 19 ┆ 75 │
│ Manhattan ┆ Queens ┆ 1 ┆ 1 ┆ 16 │
│ Queens ┆ Manhattan ┆ 0 ┆ 9 ┆ 60 │
│ Queens ┆ Manhattan ┆ 1 ┆ 17 ┆ 90 │
│ … ┆ … ┆ … ┆ … ┆ … │
│ Manhattan ┆ Manhattan ┆ null ┆ 5 ┆ 27 │
│ Manhattan ┆ Manhattan ┆ null ┆ 4 ┆ 26 │
│ Queens ┆ Brooklyn ┆ null ┆ 4 ┆ 26 │
│ Manhattan ┆ Manhattan ┆ null ┆ 3 ┆ 24 │
│ Manhattan ┆ Manhattan ┆ null ┆ 8 ┆ 35 │
└───────────┴───────────┴────────────┴──────────┴──────┘
First of all, you import the Polars library using the conventional pl
alias. You then use the scan_parquet()
function to read rides.parquet
. This makes your LazyFrame aware of the data file’s content. A LazyFrame doesn’t contain data but instead contains a set of instructions detailing what processing is to be carried out. To access the data, you need to materialize your LazyFrame by calling its .collect()
method. This creates a DataFrame and reads the data.
In this example, .collect()
shows there are 3,076,903 rows and five columns of data, as indicated by its shape
.
Using LazyFrames may seem like a strange way of working given that you have to materialize them into DataFrames to view the data. You might wonder why not just stick with DataFrames instead. As you’ll see later, despite their name, LazyFrames offer an extremely efficient way to work with data. With their lazy evaluation capabilities, LazyFrames should be your preferred way to work with data in Polars whenever possible.
Next, you’ll learn the main ways you can create LazyFrames.
Take the Quiz: Test your knowledge with our interactive “How to Work With Polars LazyFrames” quiz. You’ll receive a score upon completion to help you track your learning progress:
Interactive Quiz
How to Work With Polars LazyFramesThis quiz will challenge your knowledge of working with Polars LazyFrames. You won't find all the answers in the tutorial, so you'll need to do some extra investigating. By finding all the answers, you're sure to learn some interesting things along the way.
How to Create a Polars LazyFrame
You can create LazyFrames in three main ways:
Read the full article at https://realpython.com/polars-lazyframe/ »
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
February 26, 2025 02:00 PM UTC
PyPy
PyPy v7.3.19 release
PyPy v7.3.19: release of python 2.7, 3.10 and 3.11 beta
The PyPy team is proud to release version 7.3.19 of PyPy. This is primarily a bug-fix release fixing JIT-related problems and follows quickly on the heels of the previous release on Feb 6, 2025.
This release includes a python 3.11 interpreter. There were bugs in the first beta that could prevent its wider use, so we are continuing to call this release "beta". In the next release we will drop 3.10 and remove the "beta" label.
The release includes three different interpreters:
PyPy2.7, which is an interpreter supporting the syntax and the features of Python 2.7 including the stdlib for CPython 2.7.18+ (the
+
is for backported security updates)PyPy3.10, which is an interpreter supporting the syntax and the features of Python 3.10, including the stdlib for CPython 3.10.16.
PyPy3.11, which is an interpreter supporting the syntax and the features of Python 3.11, including the stdlib for CPython 3.11.11.
The interpreters are based on much the same codebase, thus the triple release. This is a micro release, all APIs are compatible with the other 7.3 releases. It follows after 7.3.17 release on August 28, 2024.
We recommend updating. You can find links to download the releases here:
We would like to thank our donors for the continued support of the PyPy project. If PyPy is not quite good enough for your needs, we are available for direct consulting work. If PyPy is helping you out, we would love to hear about it and encourage submissions to our blog via a pull request to https://github.com/pypy/pypy.org
We would also like to thank our contributors and encourage new people to join the project. PyPy has many layers and we need help with all of them: bug fixes, PyPy and RPython documentation improvements, or general help with making RPython's JIT even better.
If you are a python library maintainer and use C-extensions, please consider making a HPy / CFFI / cppyy version of your library that would be performant on PyPy. In any case, both cibuildwheel and the multibuild system support building wheels for PyPy.
What is PyPy?
PyPy is a Python interpreter, a drop-in replacement for CPython It's fast (PyPy and CPython performance comparison) due to its integrated tracing JIT compiler.
We also welcome developers of other dynamic languages to see what RPython can do for them.
We provide binary builds for:
x86 machines on most common operating systems (Linux 32/64 bits, Mac OS 64 bits, Windows 64 bits)
64-bit ARM machines running Linux (
aarch64
) and macos (macos_arm64
).
PyPy supports Windows 32-bit, Linux PPC64 big- and little-endian, Linux ARM 32 bit, RISC-V RV64IMAFD Linux, and s390x Linux but does not release binaries. Please reach out to us if you wish to sponsor binary releases for those platforms. Downstream packagers provide binary builds for debian, Fedora, conda, OpenBSD, FreeBSD, Gentoo, and more.
What else is new?
For more information about the 7.3.19 release, see the full changelog.
Please update, and continue to help us make pypy better.
Cheers, The PyPy Team
February 26, 2025 12:00 PM UTC
Real Python
Quiz: How to Work With Polars LazyFrames
In this quiz, you’ll test your understanding of the techniques covered in How to Work With Polars LazyFrames.
By working through the questions, you’ll review your understanding of why LazyFrames are an efficient and preferred way of working in Polars.
You’ll need to do some research outside of the tutorial to answer all the questions, so let this challenge take you on a learning journey.
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
February 26, 2025 12:00 PM UTC
Zato Blog
Automate Microsoft 365 Like a Pro: Skip the OAuth Headaches
Automate Microsoft 365 Like a Pro: Skip the OAuth Headaches
Are you tired of fighting with Microsoft 365 APIs and complex authentication flows? This practical tutorial cuts through the complexity, showing you how to build production-ready Python automations for Outlook, Calendar, and SharePoint in just 60 minutes.
No OAuth2 struggles, no Graph API complexity—just working code you can deploy today.
What you'll learn in the tutorial:
- Connect to Microsoft 365 APIs with minimal Python code
- Build real email, calendar, and SharePoint automations
- Schedule background tasks without programming
- Access Entra ID (Azure AD) user data effortlessly
- Implement enterprise-grade patterns that solve actual business problems
More resources
➤ Python API integration tutorials
➤ What is a Network Packet Broker? How to automate networks in Python?
➤ What is an integration platform?
➤ Python Integration platform as a Service (iPaaS)
➤ What is an Enterprise Service Bus (ESB)? What is SOA?
➤ Open-source iPaaS in Python
February 26, 2025 08:00 AM UTC
Python GUIs
Which Python GUI library should you use? — Comparing the Python GUI libraries available in 2025
Python is a popular programming used for everything from scripting routine tasks to building websites and performing complex data analysis. While you can accomplish a lot with command line tools, some tasks are better suited to graphical interfaces. You may also find yourself wanting to build a desktop front-end for an existing tool to improve usability for non-technical users. Or maybe you're building some hardware or a mobile app and want an intuitive touchscreen interface.
To create graphical user interfaces with Python, you need a GUI library. Unfortunately, at this point things get pretty confusing -- there are many different GUI libraries available for Python, all with different capabilities and licensing. Which Python GUI library should you use for your project?
In this article, we will look at a selection of the most popular Python GUI frameworks currently available and why you should consider using them for your own projects. You'll learn about the relative strengths of each library, understand the licensing limitations and see a simple Hello, World! application written in each. By the end of the article you should feel confident choosing the right library for your project.
tldr If you're looking to build professional quality software, start with PySide6 or PyQt6. The Qt framework is batteries-included — whatever your project, you'll be able to get it done. We have a complete PySide6 tutorial and PyQt6 tutorial as well as a Github respository full of Python GUI examples to get you started.
Tkinter
Best for simple tool GUIs, small portable applications
Tkinter is the defacto GUI framework for Python. It comes bundled with Python on both Windows and macOS. (On Linux, it may require downloading an additional package from your distribution's repo.) Tkinter is a wrapper written around the Tk GUI toolkit. Its name is an amalgamation of the words Tk and Interface.
Tkinter is a simple library with support for standard layouts and widgets, as well as more complex widgets such as tabbed views & progressbars. Tkinter is a pure GUI library, not a framework. There is no built-in support for GUIs driven from data sources, databases, or for displaying or manipulating multimedia or hardware. However, if you need to make something simple that doesn't require any additional dependencies, Tkinter may be what you are looking for. Tkinter is cross-platform however the widgets can look outdated, particularly on Windows.
Installation Already installed with Python on Windows and macOS. Ubuntu/Debian Linux sudo apt install python3-tk
A simple hello world application in Tkinter is shown below.
- Standard
- Class-based
import tkinter as tk
window = tk.Tk()
window.title("Hello World")
def handle_button_press(event):
window.destroy()
button = tk.Button(text="My simple app.")
button.bind("", handle_button_press)
button.pack()
# Start the event loop.
window.mainloop()
from tkinter import Tk, Button
class Window(Tk):
def __init__(self):
super().__init__()
self.title("Hello World")
self.button = Button(text="My simple app.")
self.button.bind("", self.handle_button_press)
self.button.pack()
def handle_button_press(self, event):
self.destroy()
# Start the event loop.
window = Window()
window.mainloop()
Hello world application built using Tkinter, running on Windows 11
Tkinter was originally developed by Steen Lumholt and Guido Van Rossum, who designed Python itself. Both the GUI framework and the language are licensed under the same Python Software Foundation (PSF) License. While the license is compatible with the GPL, it is a 'permissive' license (similar to the MIT License) that allows it to be used for proprietary applications and modifications.
PyQt or PySide
Best for commercial, multimedia, scientific or engineering desktop applications
PyQt and PySide are wrappers around the Qt framework. They allow you to easily create modern interfaces that look right at home on any platform, including Windows, macOS, Linux and even Android. They also have solid tooling with the most notable being Qt Creator, which includes a WYSIWYG editor for designing GUI interfaces quickly and easily. Being backed by a commercial project means that you will find plenty of support and online learning resources to help you develop your application.
Qt (and by extension PyQt & PySide) is not just a GUI library, but a complete application development framework. In addition to standard UI elements, such as widgets and layouts, Qt provides MVC-like data-driven views (spreadsheets, tables), database interfaces & models, graph plotting, vector graphics visualization, multimedia playback, sound effects & playlists and built-in interfaces for hardware such as printing. The Qt signals and slots models allows large applications to be built from re-usable and isolated components.
While other toolkits can work great when building small & simple tools, Qt really comes into its own for building real commercial-quality applications where you will benefit from the pre-built components. This comes at the expense of a slight learning curve. However, for smaller projects Qt is not really any more complex than other libraries. Qt Widgets-based applications use platform native widgets to ensure they look and feel at home on Windows, macOS and Qt-based Linux desktops.
Installation pip install pyqt6
or pip install pyside6
A simple hello world application in PyQt6, using the Qt Widgets API is shown below.
- PyQt6
- PySide6
from PyQt6.QtWidgets import QMainWindow, QApplication, QPushButton
import sys
class MainWindow(QMainWindow):
def __init__(self):
super().__init__()
self.setWindowTitle("Hello World")
button = QPushButton("My simple app.")
button.pressed.connect(self.close)
self.setCentralWidget(button)
self.show()
app = QApplication(sys.argv)
w = MainWindow()
app.exec()
from PySide6.QtWidgets import QMainWindow, QApplication, QPushButton
import sys
class MainWindow(QMainWindow):
def __init__(self):
super().__init__()
self.setWindowTitle("Hello World")
button = QPushButton("My simple app.")
button.pressed.connect(self.close)
self.setCentralWidget(button)
self.show()
app = QApplication(sys.argv)
w = MainWindow()
app.exec()
As you can see, the code is almost identical between PyQt & PySide, so it's not something to be concerned about when you start developing with either: you can always migrate easily if you need to.
Hello world application built using PyQt6, running on Windows 11
Before the Qt Company (under Nokia) released the officially supported PySide library in 2009, Riverbank Computing had released PyQt in 1998. The main difference between these two libraries is in licensing. The free-to-use version of PyQt is licensed under GNU General Public License (GPL) v3 but PySide is licensed under GNU Lesser General Public License (LGPL). This means that PyQt is limited GPL-licensed applications unless you purchase its commercial version, while PySide may be used in non-GPL applications without any additional fee. However, note that both these libraries are separate from Qt itself which also has a free-to-use, open source version and a paid, commercial version.
For a more information see our article on PyQt vs PySide licensing.
-
PySide6
-
PyQt6
-
PyQt5
PyQt/PySide with QML
Best for Raspberry Pi, microcontrollers, industrial and consumer electronics
When using PyQt and PySide you actually have two options for building your GUIs. We've already introduced the Qt Widgets API which is well-suited for building desktop applications. But Qt also provides a declarative API in the form of Qt Quick/QML.
Using Qt Quick/QML you have access to the entire Qt framework for building your applications. Your UI consists of two parts: the Python code which handles the business logic and the QML which defines the structure and behavior of the UI itself. You can control the UI from Python, or use embedded Javascript code to handle events and animations.
Qt Quick/QML is ideally suited for building modern touchscreen interfaces for microcontrollers or device interfaces -- for example, building interfaces for microcontrollers like the Raspberry Pi. However you can also use it on desktop to build completely customized application experiences, like those you find in media player applications like Spotify, or to desktop games.
Installation pip install pyqt6
or pip install pyside6
A simple Hello World app in PyQt6 with QML. Save the QML file in the same folder as the Python file, and run as normally.
- main.py
- main.qml
import sys
from PyQt6.QtGui import QGuiApplication
from PyQt6.QtQml import QQmlApplicationEngine
app = QGuiApplication(sys.argv)
engine = QQmlApplicationEngine()
engine.quit.connect(app.quit)
engine.load('main.qml')
sys.exit(app.exec())
import QtQuick 2.15
import QtQuick.Controls 2.15
ApplicationWindow {
visible: true
width: 600
height: 500
title: "HelloApp"
Text {
anchors.centerIn: parent
text: "Hello World"
font.pixelSize: 24
}
}
Licensing for Qt Quick/QML applications is the same as for other PyQt/PySide apps.
-
PyQt
-
PySide
Hello world application built using PyQt6 & QML, running on Windows 11
Kivy
Best for Python mobile app development
While most other GUI frameworks are bindings to toolkits written in other programming languages, Kivy is perhaps the only framework which is primarily written in pure Python. If you want to create touchscreen-oriented interfaces with a focus on mobile platforms such as Android and iOS, this is the way to go. This does run on desktop platforms (Windows, macOS, Linux) as well but note that your application may not look and behave like a native application. However, there is a pretty large community around this framework and you can easily find resources to help you learn it online.
The look and feel of Kivy is extremely customizable, allowing it to be used as an alternative to libraries like Pygame (for making games with Python). The developers have also released a number of separate libraries for Kivy. Some provide Kivy with better integration and access to certain platform-specific features, or help package your application for distribution on platforms like Android and iOS. Kivy has it's own design language called Kv, which is similar to QML for Qt. It allows you to easily separate the interface design from your application's logic.
There is a 3rd party add-on for Kivy named KivyMD that replaces Kivy's widgets with ones that are compliant with Google's Material Design.
A simple hello world application in Kivy is shown below.
Installation pip install kivy
A simple hello world application in Kivy is shown below.
from kivy.app import App
from kivy.uix.boxlayout import BoxLayout
from kivy.uix.button import Button
from kivy.core.window import Window
Window.size = (300, 200)
class MainWindow(BoxLayout):
def __init__(self):
super().__init__()
self.button = Button(text="Hello, World?")
self.button.bind(on_press=self.handle_button_clicked)
self.add_widget(button)
def handle_button_clicked(self, event):
self.button.text = "Hello, World!"
class MyApp(App):
def build(self):
self.title = "Hello, World!"
return MainWindow()
app = MyApp()
app.run()
Hello world application built using Kivy, running on Windows 11
An equivalent application built using the Kv declarative language is shown below.
- main.py
- controller.kv
import kivy
kivy.require('1.0.5')
from kivy.uix.floatlayout import FloatLayout
from kivy.app import App
from kivy.properties import ObjectProperty, StringProperty
class Controller(FloatLayout):
'''Create a controller that receives a custom widget from the kv lang file.
Add an action to be called from the kv lang file.
'''
def button_pressed(self):
self.button_wid.text = 'Hello, World!'
class ControllerApp(App):
def build(self):
return Controller()
if __name__ == '__main__':
ControllerApp().run()
#:kivy 1.0
:
button_wid: custom_button
BoxLayout:
orientation: 'vertical'
padding: 20
Button:
id: custom_button
text: 'Hello, World?'
on_press: root.button_pressed()
The name of the Kv file must match the name of the class from the main application -- here Controller
and controller.kv
.
Hello world application built using Kivy + Kv, running on Windows 11
In February 2011, Kivy was released as the spiritual successor to a similar framework called PyMT. While they shared similar goals and was also led by the current core developers of Kivy, where Kivy differs is in its underlying design and a professional organization which actively develops and maintains it. Kivy is licensed under the MIT license, which is a 'permissive' license that allows you to use it freely in both open source and proprietary applications. As such, you are even allowed to make proprietary modifications to the framework itself.
PySimpleGUI
Best for quickly building UIs for simple tools, very portable
PySimpleGUI 5 uses a paid subscription model for commercial software. Non-commercial distribution requires both developers and recipients to have an active PySimpleGUI subscription.
PySimpleGUI aims to simplify GUI application development for Python. It doesn't reinvent the wheel but provides a wrapper around other existing frameworks such as Tkinter, Qt (PySide 2), WxPython and Remi. By doing so, it lowers the barrier to creating a GUI but also allows you to easily migrate from one GUI framework to another by simply changing the import statement.
While there is a separate port of PySimpleGUI for each of these frameworks, the Tkinter version is considered the most feature complete. Wrapping other libraries comes at a cost however — your applications will not be able to exploit the full capabilities or performance of the underlying libraries. The pure-Python event loop can also hinder performance by bottlenecking events with the GIL. However, this is only really a concern when working with live data visualization, streaming or multimedia applications.
There is a fair amount of good resources to help you learn to use PySimpleGUI, including an official Cookbook and a Udemy course offered by the developers themselves. According to their project website, PySimpleGUI was initially made (and later released in 2018) because the lead developer wanted a 'simplified' GUI framework to use in his upcoming project and wasn't able to find any that met his needs.
Installation pip install pysimplegui
import PySimpleGUI as sg
layout = [
[sg.Button("My simple app.")]
]
window = sg.Window("Hello World", layout)
while True:
event, values = window.read()
print(event, values)
if event == sg.WIN_CLOSED or event == "My simple app.":
break
window.close()
Hello world application built using PySimpleGUI, running on Windows 11
PySimpleGUI is licensed under the same LGPL v3 license as PySide, which allows its use in proprietary applications but modifications to the framework itself must be released as open source.
WxPython
Best for simple portable desktop applications
WxPython is a wrapper for the popular, cross-platform GUI toolkit called WxWidgets. It is implemented as a set of Python extension modules that wrap the GUI components of the popular wxWidgets cross platform library, which is written in C++.
WxPython uses native widgets on most platforms, ensure that your application looks and feels at home. However, WxPython is known to have certain platform-specific quirks and it also doesn't provide the same level of abstraction between platforms as Qt for example. This may affect how easy it is to maintain cross-platform compatibility for your application.
WxPython is under active development and is also currently being reimplemented from scratch under the name 'WxPython Phoenix'. The team behind WxWidgets is also responsible for WxPython, which was initially released in 1998.
Installation pip install wxpython
import wx
class MainWindow(wx.Frame):
def __init__(self, parent, title):
wx.Frame.__init__(self, parent, title=title, size=(200, -1))
self.button = wx.Button(self, label="My simple app.")
self.Bind(
wx.EVT_BUTTON, self.handle_button_click, self.button
)
self.sizer = wx.BoxSizer(wx.VERTICAL)
self.sizer.Add(self.button)
self.SetSizer(self.sizer)
self.SetAutoLayout(True)
self.Show()
def handle_button_click(self, event):
self.Close()
app = wx.App(False)
w = MainWindow(None, "Hello World")
app.MainLoop()
Hello world application built using WxPython, running on Windows 11
Both WxWidgets and WxPython are licensed under a WxWindows Library License, which is a 'free software' license similar to LGPL (with a special exception). It allows both proprietary and open source applications to use and modify WxPython.
PyGObject (GTK+)
Best for developing applications for GNOME desktop
If you intend to create an application that integrates well with GNOME and other GTK-based desktop environments for Linux, PyGObject is the right choice. PyGObject itself is a language-binding to the GTK+ widget toolkit. It allows you to create modern, adaptive user interfaces that conform to GNOME's Human Interface Guidelines (HIG).
It also enables the development of 'convergent' applications that can run on both Linux desktop and mobile platforms. There are a few first-party and community-made, third-party tools available for it as well. This includes the likes of GNOME Builder and Glade, which is yet another WYSIWYG editor for building graphical interfaces quickly and easily.
Unfortunately, there aren't a whole lot of online resources to help you learn PyGObject application development, apart from this one rather well-documented tutorial. While cross-platform support does exist (e.g. Inkscape, GIMP), the resulting applications won't feel completely native on other desktops. Setting up a development environment for this, especially on Windows and macOS, also requires more steps than for most other frameworks in this article, which just need a working Python installation.
Installation Ubuntu/Debian sudo apt install python3-gi python3-gi-cairo gir1.2-gtk-4.0
, macOS Homebrew brew install pygobject4 gtk+4
import gi
gi.require_version("Gtk", "4.0")
from gi.repository import Gtk
def on_activate(app):
win = Gtk.ApplicationWindow(application=app)
btn = Gtk.Button(label="Hello, World!")
btn.connect('clicked', lambda x: win.close())
win.set_child(btn)
win.present()
app = Gtk.Application(application_id='org.gtk.Example')
app.connect('activate', on_activate)
app.run(None)
Hello world application built using PyGObject, running on Ubuntu Linux 21.10
PyGObject is developed and maintained under the GNOME Foundation, who is also responsible for the GNOME desktop environment. PyGObject replaces several separate Python modules, including PyGTK, GIO and python-gnome, which were previously required to create a full GNOME/GTK application. Its initial release was in 2006 and it is licensed under an older version of LGPL (v2.1). While there are some differences with the current version of LGPL (v3), the license still allows its use in proprietary applications but requires any modification to the library itself to be released as open source.
Remi
Best for web based UIs for Python applications
Remi, which stands for REMote Interface, is the ideal solution for applications that are intended to be run on servers and other headless setups. (For example, on a Raspberry Pi.) Unlike most other GUI frameworks/libraries, Remi is rendered completely in the browser using a built-in web server. Hence, it is completely platform-independent and runs equally well on all platforms.
That also makes the application's interface accessible to any computer or device with a web browser that is connected to the same network. Although access can be restricted with a username and password, it doesn't implement any security strategies by default. Note that Remi is meant to be used as a desktop GUI framework and not for serving up web pages. If more than one user connects to the application at the same time, they will see and interact with the exact same things as if a single user was using it.
Remi requires no prior knowledge of HTML or other similar web technologies. You only need to have a working understanding of Python to use it, which is then automatically translated to HTML. It also comes included with a drag n drop GUI editor that is akin to Qt Designer for PyQt and PySide.
import remi.gui as gui
from remi import start, App
class MyApp(App):
def main(self):
container = gui.VBox(width=120, height=100)
# Create a button, with the label "Hello, World!"
self.bt = gui.Button('Hello, World?')
self.bt.onclick.do(self.on_button_pressed)
# Add the button to the container, and return it.
container.append(self.bt)
return container
def on_button_pressed(self, widget):
self.bt.set_text('Hello, World!')
start(MyApp)
Remi is licensed under the Apache License v2.0, which is another 'permissive' license similar to the MIT License. The license allows using it in both open source and proprietary applications, while also allowing proprietary modifications to be made to the framework itself. Its main conditions revolve around the preservation of copyright and license notices.
Hello world application built using Remi, running on Chrome on Windows 11
Conclusion
If you're looking to build GUI applications with Python, there is probably a GUI framework/library listed here that fits the bill for your project. Try and weigh up the capabilities & licensing of the different libraries with the scale of your project, both now and in the future.
Don't be afraid to experiment a bit with different libraries, to see which feel the best fit. While the APIs of GUI libraries are very different, they share many underlying concepts in common and things you learn in one library will often apply in another.
You are only limited by your own imagination. So go out there and make something!
February 26, 2025 06:00 AM UTC
Ahmed Bouchefra
Python 3.13 in 2025 Breakthroughs: No-GIL, JIT, and iOS Support Explained
Python 3.13 dropped its first stable release on October 7, 2024, and with the latest maintenance update—Python 3.13.2—released on February 6, 2025, the community’s still buzzing about what this version brings to the table. From iOS support to experimental no-GIL dreams, let’s break down the highlights, the chatter on X, and what it all means for Pythonistas in 2025.
Python 3.13: TLDR
Python 3.13 (released October 2024, latest update 3.13.2 in February 2025) brings significant changes:
- Performance boosts: 5-15% faster than Python 3.12, with experimental JIT compiler (PEP 744) showing up to 30% speedups for computation-heavy tasks
- Experimental no-GIL mode: Optional free-threading via PEP 703 for better parallelism (use with
--free-threading
flag) - iOS support: PEP 730 adds iOS build targets, opening doors for Python mobile development
- Improved typing: Enhanced TypedDict with
Required
andNotRequired
operators for more precise type annotations (typing docs) - Better pattern matching: More intuitive syntax including matching on attributes and properties (pattern matching docs)
- Cleaner codebase: PEP 594 removed deprecated modules like smtpd and cgi
- Memory efficiency: 7% smaller memory footprint compared to 3.12
- Data science improvements: Better NumPy integration and optimized statistical functions
- Framework support: Django 5.1 and FastAPI already supporting 3.13 features
The community sentiment: Excited about iOS possibilities and the no-GIL experiment, though the latter still has compatibility issues with some C extensions. Python 3.13 represents a stepping stone toward a faster Python that maintains its developer-friendly nature. Source
The Big Wins in Python 3.13
First off, Python 3.13 isn’t just another incremental update—it’s packing some serious punches. The improved interactive interpreter, inspired by PyPy, is a game-changer for anyone who lives in the REPL. It’s snappier, more intuitive, and honestly, just feels good to use. Then there’s the experimental free-threaded mode (thanks, PEP 703), which ditches the Global Interpreter Lock (GIL) for those brave enough to opt in. Pair that with a preliminary JIT compiler (PEP 744), and you’ve got Python flirting with performance levels we’ve only dreamed of. Oh, and deprecated modules? Gone—PEP 594 cleaned house, so say goodbye to relics like smtpd and cgi.
The latest maintenance release, 3.13.2, doesn’t add shiny new toys but keeps things humming with around 250 bug fixes. Think stability: better IPv6 handling, an updated libexpat, and a smoother ride overall. It’s the kind of polish that keeps Python reliable while the experimental stuff matures.
Significant Performance Improvements
One of the most noticeable changes in Python 3.13 is the performance boost. Early benchmarks show a 5-15% improvement over Python 3.12 for common operations. The JIT compiler, while still experimental, shows promising results with some computation-heavy workloads seeing up to 30% speedups when properly configured.
The memory footprint has also been reduced by roughly 7% compared to 3.12, making Python more efficient on resource-constrained systems. This is particularly important as Python continues expanding into embedded systems and mobile devices.
Python 3.13 in Action: Where It Shines (and Where It Doesn’t)
So, where does Python 3.13 actually make a difference? Let’s break it down:
- Data Science & Machine Learning — Thanks to the JIT compiler, operations on large datasets in NumPy and Pandas are noticeably faster. Some benchmarks report a 15-20% reduction in execution time for common matrix computations.
- Web Development — Django 5.1 and FastAPI are already adapting to Python 3.13, leveraging better typing support and free-threading optimizations.
- Mobile Development — The biggest wildcard is Python’s new iOS support. While it’s still early days, developers are experimenting with Python-based iOS apps, and frameworks like Kivy might see a resurgence.
- Concurrency & Async — The free-threading experiment is exciting, but many popular C extensions aren’t yet fully compatible. Libraries like Twisted and Tornado need updates before they can truly benefit.
X Is Talking: iOS and No-GIL Steal the Show
Hop over to X, and you’ll see Python 3.13’s got people excited—and a little speculative. A wave of posts around mid-February (like from @DrGaryHoworth and @polyglotengine1) lit up with links to an article called “Python for iOS: A New Era with Python 3.13” by Pouya Hallaj. Turns out, PEP 730’s addition of an iOS target in the build system has devs dreaming of mobile Python apps. Imagine coding on your iPhone or shipping Python-powered tools to the App Store—that’s the vibe here, and it’s got traction.
Meanwhile, the free-threading experiment’s got its own fanbase. @_salvasfreewill (in Persian, no less) traced the no-GIL journey from Python 3.11’s PEP 734 to interface tweaks in 3.14’s early notes, calling 3.13 a pivotal step. @dystopiabreaker chimed in too, praising the GIL-free release but grumbling about async support still lagging. The sentiment? Optimism with a side of “let’s see where this goes.”
The New Pattern Matching Refinements
Python 3.13 builds on the pattern matching introduced in 3.10 with more intuitive syntax and better performance. One of the most requested features—the ability to match on attributes and properties—has finally landed:
match user:
case User(name="Admin", role=AdminRole()) as admin:
return admin.get_permissions()
case User(name=name, role="editor") if is_senior(name):
return editor_permissions()
case _:
return default_permissions()
This syntax is cleaner and more natural than previous workarounds, making pattern matching truly Pythonic.
Hidden Gems in Python 3.13
Beyond the headline features, Python 3.13 includes several smaller but impactful changes:
- Improved Debugging — The
traceback
module now provides richer error messages, making debugging a smoother experience. - JSON Enhancements — The
json
module is faster and more memory-efficient, reducing serialization/deserialization overhead. - Improved Unicode Handling — Python 3.13 refines Unicode support, particularly in file handling and string manipulations.
Why It Matters in 2025
Python 3.13 feels like a bridge. It’s stable enough for production with 3.13.2’s fixes, yet it’s teasing a future where Python could rival faster languages without losing its charm. The iOS push signals Python’s not just for servers or data science anymore—it’s eyeing your phone. And with each maintenance release (like the 400+ fixes in 3.13.1 last December), the core keeps getting stronger.
For enterprises, Python 3.13’s performance improvements mean potential cost savings on compute resources, especially for data-intensive applications. The typing enhancements also make large codebases more maintainable, addressing one of Python’s traditional weaknesses in enterprise settings.
Migration Considerations
If you’re planning to upgrade from Python 3.12, the transition should be relatively smooth for most applications. The deprecated modules removed in PEP 594 are the main breaking change, but they were already marked as deprecated for several versions, so well-maintained codebases should be prepared.
For those interested in experimenting with the no-GIL mode, the Python core team recommends starting with isolated components rather than converting entire applications. The command-line flag --free-threading
enables the mode, but be prepared for potential compatibility issues with C extensions.
Want More?
Curious about a feature? The Python docs or @ThePSF’s updates are goldmines. Me, I’m betting the iOS hype keeps growing, and that no-GIL dream? It’s closer than ever.
The Python Language Summit scheduled for April 2025 will likely focus heavily on the future of the no-GIL implementation and whether it might become the default in Python 3.15 or beyond. If you’re invested in Python’s performance journey, that’s an event worth watching.
What’s your take—trying 3.13 yet, or waiting for the dust to settle?
Official Resources
- Python 3.13.2 Release Page
- What’s New in Python 3.13
- Python Developer’s Guide
- Python Software Foundation
February 26, 2025 12:00 AM UTC
February 25, 2025
PyCoder’s Weekly
Issue #670: pyproject.toml, DuckDB, Flet, and More (Feb. 25, 2025)
#670 – FEBRUARY 25, 2025
View in Browser »
How to Manage Python Projects With pyproject.toml
Learn how to manage Python projects with the pyproject.toml configuration file. In this tutorial, you’ll explore key use cases of the pyproject.toml file, including configuring your build, installing your package locally, managing dependencies, and publishing your package to PyPI.
REAL PYTHON
Mastering DuckDB When You’re Used to pandas or Polars
Why use DuckDB / SQL at all if you’re used to dataframes? This article makes the case for some reasons why, and shows how to perform some operations which in dataframe are basic but in SQL aren’t necessarily obvious
MARCO GORELLI • Shared by Marco Gorelli
Lighter, Faster Object Segmentation with Intel AI, Powered by OpenVINO
Meet EfficientSAM—a high-performance version of SAM, designed for greater efficiency. Optimized to run locally and open source, it’s built so you can see a needle in a haystack. Get the code on GitHub.
INTEL CORPORATION sponsor
Update on Flet: Python + Flutter UIs
Talk Python interviews Feodor Fitsner and they talk about Flet, a Python UI framework that is distributed and executed on the Flutter framework.
KENNEDY & FITSNER podcast
Discussions
Articles & Tutorials
Telling Effective Stories With Your Python Visualizations
How do you make compelling visualizations that best convey the story of your data? What methods can you employ within popular Python tools to improve your plots and graphs? This week on the show, Matt Harrison returns to discuss his new book “Effective Visualization: Exploiting Matplotlib & Pandas.”
REAL PYTHON podcast
Concatenating Strings in Python Efficiently
In this video course, you’ll learn how to concatenate strings in Python. You’ll use different tools and techniques for string concatenation, including the concatenation operators and the .join() method. You’ll also explore other tools that can also be handy for string concatenation in Python.
REAL PYTHON course
Postman AI Agent Builder Is Here: The Quickest Way to Build AI Agents. Start Building
Postman AI Agent Builder is a suite of solutions that accelerates agent development. With centralized access to the latest LLMs and APIs from over 18,000 companies, plus no-code workflows, you can quickly connect critical tools and build multi-step agents — all without writing a single line of code →
POSTMAN sponsor
Building a Python Command-Line To-Do App With Typer
In this step-by-step video course, you’ll create a to-do application for your command line using Python and Typer. While you build this app, you’ll learn the basics of Typer, a modern and versatile library for building command-line interfaces (CLI).
REAL PYTHON course
Adding OpenTelemetry to Django
“OpenTelemetry is an open source, vendor-neutral way to add monitoring features to your application.” This post shows you how to set it up and track lots of things in your Django project.
JESSICA GARSON
Implicit Casting in Dataframe Concatenation
This quick TIL article covers how to do non-strict, vertical concatenation of DataFrames in Polars with the parameter how='vertical'
.
RODRIGO GIRÃO SERRÃO
Ugly Code and Dumb Things
An opinion piece by Armin Ronacher that posits: why ugly and dumb code sometimes blinds engineers from the ingenuity behind it.
ARMIN RONACHER
Smoke Testing vs. Sanity Testing
Learn the key differences between smoke testing vs sanity testing, their purposes, processes, and when to use them.
ANTONELLO ZANINI
FastAPI Deconstructed: Anatomy of an ASGI Framework
This article, based on a PyCon APAC talk, covers just what FastAPI does under the hood.
RAFIQUL HASAN
Benchmarking Utility for Python
Eli attempts to mimic Go’s built-in bench marking capabilities in Python.
ELI BENDERSKY
Projects & Code
when: Convert Timezones by City, IANA, or TZ Alias
GITHUB.COM/DAKRAUTH • Shared by David Krauth
Events
Weekly Real Python Office Hours Q&A (Virtual)
February 26, 2025
REALPYTHON.COM
Python Weekend Abuja
February 27, 2025
CODECAMPUS.COM.NG
SPb Python Drinkup
February 27, 2025
MEETUP.COM
PyCon APAC 2025
March 1 to March 3, 2025
PYTHON.PH
Python Weekend Abuja
March 1, 2025
CODECAMPUS.COM.NG
Happy Pythoning!
This was PyCoder’s Weekly Issue #670.
View in Browser »
[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]
February 25, 2025 07:30 PM UTC
Real Python
Single and Double Underscore Naming Conventions in Python
Python has a few important naming conventions that are based on using either a single or double underscore character (_
). These conventions allow you to differentiate between public and non-public names in APIs, write safe classes for subclassing purposes, avoid name clashes, and more.
Following and respecting these conventions allows you to write code that looks Pythonic and consistent in the eyes of other Python developers. This skill is especially useful when you’re writing code that’s intended for other developers to work with.
In this video course, you’ll:
- Learn about Python naming conventions that rely on using underscores (
_
) - Differentiate public and non-public names by using a single leading underscore
- Use double leading underscores to leverage name mangling in Python classes
- Explore other common uses of underscores in Python names
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
February 25, 2025 02:00 PM UTC
PyPy
Low Overhead Allocation Sampling with VMProf in PyPy's GC
Introduction
There are many time-based statistical profilers around (like VMProf or py-spy just to name a few). They allow the user to pick a trade-off between profiling precision and runtime overhead.
On the other hand there are memory profilers such as memray. They can be handy for finding leaks or for discovering functions that allocate a lot of memory. Memory profilers typlically save every single allocation a program does. This results in precise profiling, but larger overhead.
In this post we describe our experimental approach to low overhead statistical memory profiling. Instead of saving every single allocation a program does, it only saves every nth allocated byte. We have tightly integrated VMProf and the PyPy Garbage Collector to achieve this. The main technical insight is that the check whether an allocation should be sampled can be made free. This is done by folding it into the bump pointer allocator check that the PyPy’s GC uses to find out if it should start a minor collection. In this way the fast path with and without memory sampling are exactly the same.
Background
To get an insight how the profiler and GC interact, lets take a brief look at both of them first.
VMProf
VMProf is a statistical time-based profiler for PyPy. VMProf samples the stack of currently running Python functions a certain user-configured number of times per second. By adjusting this number, the overhead of profiling can be modified to pick the correct trade-off between overhead and precision of the profile. In the resulting profile, functions with huge runtime stand out the most, functions with shorter runtime less so. If you want to get a little more introduction to VMProf and how to use it with PyPy, you may look at this blog post
PyPy’s GC
PyPy uses a generational incremental copying collector. That means there are two spaces for allocated objects, the nursery and the old-space. Freshly allocated objects will be allocated into the nursery. When the nursery is full at some point, it will be collected and all objects that survive will be tenured i.e. moved into the old-space. The old-space is much larger than the nursery and is collected less frequently and incrementally (not completely collected in one go, but step-by-step). The old space collection is not relevant for the rest of the post though. We will now take a look at nursery allocations and how the nursery is collected.
Bump Pointer Allocation in the Nursery
The nursery (a small continuous memory area) utilizes two pointers to keep track from where on the nursery is free and where it ends. They are called nursery_free
and nursery_top
. When memory is allocated, the GC checks if there is enough space in the nursery left. If there is enough space, the nursery_free
pointer will be returned as the start address for the newly allocated memory, and nursery_free
will be moved forward by the amount of allocated memory.
def allocate(totalsize): # Save position, where the object will be allocated to as result result = gc.nursery_free # Move nursery_free pointer forward by totalsize gc.nursery_free = result + totalsize # Check if this allocation would exceed the nursery if gc.nursery_free > gc.nursery_top: # If it does => collect the nursery and allocate afterwards result = collect_and_reserve(totalsize) # result is a pointer into the nursery, obj will be allocated there return result def collect_and_reserve(size_of_allocation): # do a minor collection and return the start of the nursery afterwards minor_collection() return gc.nursery_free
Understanding this is crucial for our allocation sampling approach, so let us go through this step-by-step.
We already saw an example on how an allocation into a non-full nursery will look like. But what happens, if the nursery is (too) full?
As soon as an object doesn't fit into the nursery anymore, it will be collected. A nursery collection will move all surviving objects into the old-space, so that the nursery is free afterwards, and the requested allocation can be made.
(Note that this is still a bit of a simplification.)
Sampling Approach
The last section described how the nursery allocation works normally. Now we'll talk how we integrate the new allocation sampling approach into it.
To decide whether the GC should trigger a sample, the sampling logic is integrated into the bump pointer allocation logic. Usually, when there is not enough space in the nursery left to fulfill an allocation request, the nursery will be collected and the allocation will be done afterwards. We reuse that mechanism for sampling, by introducing a new pointer called sample_point
that is calculated by sample_point = nursery_free + sample_n_bytes
where sample_n_bytes
is the number of bytes allocated before a sample is made (i.e. our sampling rate).
Imagine we'd have a nursery of 2MB and want to sample every 512KB allocated, then you could imagine our nursery looking like that:
We use the sample point as nursery_top
, so that allocating a chunk of 512KB would exceed the nursery top and start a nursery collection. But of course we don't want to do a minor collection just then, so before starting a collection, we need to check if the nursery is actually full or if that is just an exceeded sample point. The latter will then trigger a VMprof stack sample. Afterwards we don't actually do a minor collection, but change nursery_top
and immediately return to the caller.
The last picture is a conceptual simplification. Only one sampling point exists at any given time. After we created the sampling point, it will be used as nursery top, if exceeded at some point, we will just add sample_n_bytes
to that sampling point, i.e. move it forward.
Here's how the updated collect_and_reserve
function looks like:
def collect_and_reserve(size_of_allocation): # Check if we exceeded a sample point or if we need to do a minor collection if gc.nursery_top == gc.sample_point: # One allocation could exceed multiple sample points # Sample, move sample_point forward vmprof.sample_now() gc.sample_point += sample_n_bytes # Set sample point as new nursery_top if it fits into the nursery if sample_point <= gc.real_nursery_top: gc.nursery_top = sample_point # Or use the real nursery top if it does not fit else: gc.nursery_top = gc.real_nursery_top # Is there enough memory left inside the nursery if gc.nursery_free + size_of_allocation <= gc.nursery_top: # Yes => move nursery_free forward gc.nursery_free += size_of_allocation return gc.nursery_free # We did not exceed a sampling point and must do a minor collection, or # we exceeded a sample point but we needed to do a minor collection anyway minor_collection() return gc.nursery_free
Why is the Overhead ‘low’
The most important property of our approach is that the bump-pointer fast path is not changed at all. If sampling is turned off, the slow path in collect_and_reserve
has three extra instructions for the if at the beginning, but are only a very small amount of overhead, compared to doing a minor collection.
When sampling is on, the extra logic in collect_and_reserve
gets executed. Every time an allocation exceeds the sample_point
, collect_and_reserve
will sample the Python functions currently executing. The resulting overhead is directly controlled by sample_n_bytes
. After sampling, the sample_point
and nursery_top
must be set accordingly. This will be done once after sampling in collect_and_reserve
. At some point a nursery collection will free the nursery and set the new sample_point
afterwards.
That means that the overhead mostly depends on the sampling rate and the rate at which the user program allocates memory, as the combination of those two factors determines the amount of samples.
Since the sampling rate can be adjusted from as low as 64 Byte to a theoretical maximum of ~4 GB (at the moment), the tradeoff between number of samples (i.e. profiling precision) and overhead can be completely adjusted.
We also suspect linkage between user program stack depth and overhead (a deeper stack takes longer to walk, leading to higher overhead), especially when walking the C call stack to.
Sampling rates bigger than the nursery size
The nursery usually has a size of a few megabytes, but profiling long-runningor larger applications with tons of allocations could result in very high number of samples per second (and thus overhead). To combat that it is possible to use sampling rates higher than the nursery size.
The sampling point is not limited by the nursery size, but if it is 'outside' the nursery (e.g. because sample_n_bytes
is set to twice the nursery size) it won't be used as nursery_top
until it 'fits' into the nursery.
After every nursery collection, we'd usually set the sample_point
to nursery_free + sample_n_bytes
, but if it is larger than the nursery, then the amount of collected memory during the last nursery collection is subtracted from sample_point
.
At some point the sample_point
will be smaller than the nursery size, then it will be used as nursery_top
again to trigger a sample when exceeded.
Differences to Time-Based Sampling
As mentioned in the introduction, time-based sampling ‘hits’ functions with high runtime, and allocation-sampling ‘hits’ functions allocating much memory. But are those always different functions? The answer is: sometimes. There can be functions allocating lots of memory, that do not have a (relative) high runtime.
Another difference to time-based sampling is that the profiling overhead does not solely depend on the sampling rate (if we exclude a potential stack-depth - overhead correlation for now) but also on the amount of memory the user code allocates.
Let us look at an example:
If we’d sample every 1024 Byte and some program A allocates 3 MB and runs for 5 seconds, and program B allocates 6 MB but also runs for 5 seconds, there will be ~3000 samples when profiling A, but ~6000 samples when profiling B. That means we cannot give a ‘standard’ sampling rate like time-based profilers use to do (e.g. vmprof uses ~1000 samples/s for time sampling), as the number of resulting samples, and thus overhead, depends on sampling rate and amount of memory allocated by the program.
For testing and benchmarking, we usually started with a sampling rate of 128Kb and then halved or doubled that (multiple times) depending on sample counts, our need for precision (and size of the profile).
Evaluation
Overhead
Now let us take a look at the allocation sampling overhead, by profiling some benchmarks.
The x-axis shows the sampling rate, while the y-axis shows the overhead, which is computed as runtime_with_sampling / runtime_without_sampling
.
All benchmarks were executed five times on a PyPy with JIT and native profiling enabled, so that every dot in the plot is one run of a benchmark.
As you probably expected, the Overhead drops with higher allocation sampling rates. Reaching from as high as ~390% for 32kb allocation sampling to as low as < 10% for 32mb.
Let me give one concrete example: One run of the microbenchmark at 32kb sampling took 15.596 seconds and triggered 822050 samples.
That makes a ridiculous amount of 822050 / 15.596 = ~52709
samples per second.
There is probably no need for that amount of samples per second, so that for 'real' application profiling a much higher sampling rate would be sufficient.
Let us compare that to time sampling.
This time we ran those benchmarks with 100, 1000 and 2000 samples per second.
The overhead varies with the sampling rate. Both with allocation and time sampling, you can reach any amount of overhead and any level of profiling precision you want. The best approach probably is to just try out a sampling rate and choose what gives you the right tradeoff between precision and overhead (and disk usage).
The benchmarks used are:
microbenchmark
- https://github.com/Cskorpion/microbenchmark
pypy microbench.py 65536
gcbench
- https://github.com/pypy/pypy/blob/main/rpython/translator/goal/gcbench.py
- print statements removed
pypy gcbench.py 1
pypy translate step
- first step of the pypy translation (annotation step)
pypy path/to/rpython --opt=0 --cc=gcc --dont-write-c-files --gc=incminimark --annotate path/to/pypy/goal/targetpypystandalone.py
interpreter pystone
- pystone benchmark on top of an interpreted pypy on top of a translated pypy
pypy path/to/pypy/bin/pyinteractive.py -c "import test.pystone; test.pystone.main(1)"
All benchmarks executed on:
- Kubuntu 24.04
- AMD Ryzen 7 5700U
- 24gb DDR4 3200MHz (dual channel)
-
SSD benchmarking at read: 1965 MB/s, write: 227 MB/s
- Sequential 1MB 1 Thread 8 Queues
-
Self built PyPy with allocation sampling features
-
Modified VMProf with allocation sampling support
Example
We have also modified vmprof-firefox-converter to show the allocation samples in the Firefor Profiler UI. With the techniques from this post, the output looks like this:
While this view is interesting, it would be even better if we could also see what types of objects are being allocated in these functions. We will take about how to do this in a future blog post.
Conclusion
In this blog post we introduced allocation sampling for PyPy by going through the technical aspects and the corresponding overhead. In a future blog post, we are going to dive into the actual usage of allocation sampling with VMProf, and show an example case study. That will be accompanied by some new improvements and additional features, like extracting the type of an object that triggered a sample.
So far all this work is still experimental and happening on PyPy branches but we hope to get the technique stable enough to merge it to main and ship it with PyPy eventually.
-- Christoph Jung and CF Bolz-Tereick
February 25, 2025 10:16 AM UTC
Python Insider
Python 3.14.0 alpha 4 is out
Hello, three dot fourteen dot zero alpha four!
https://www.python.org/downloads/release/python-3140a4/
This is an early developer preview of Python 3.14
Major new features of the 3.14 series, compared to 3.13
Python 3.14 is still in development. This release, 3.14.0a4, is the fourth of seven planned alpha releases.
Alpha releases are intended to make it easier to test the current state of new features and bug fixes and to test the release process.
During the alpha phase, features may be added up until the start of the beta phase (2025-05-06) and, if necessary, may be modified or deleted up until the release candidate phase (2025-07-22). Please keep in mind that this is a preview release and its use is not recommended for production environments.
Many new features for Python 3.14 are still being planned and written. Among the new major new features and changes so far:
- PEP 649: deferred evaluation of annotations
- PEP 741: Python configuration C API
- PEP 761: Python 3.14 and onwards no longer provides PGP signatures for release artifacts. Instead, Sigstore is recommended for verifiers.
- Improved error messages
- Many removals of deprecated classes, functions, methods and parameters in various standard library modules.
- New deprecations, many of which are scheduled for removal from Python 3.16
- C API removals and deprecations
- (Hey, fellow core developer, if a feature you find important is missing from this list, let Hugo know.)
The next pre-release of Python 3.14 will be 3.14.0a5, currently scheduled for 2025-02-11.
More resources
- Online documentation
- PEP 745, 3.14 Release Schedule
- Report bugs at https://github.com/python/cpython/issues
- Help fund Python and its community
And now for something completely different
In Python, you can use Greek letters as constants. For example:
from math import pi as π
def circumference(radius: float) -> float:
return 2 * π * radius
print(circumference(6378.137)) # 40075.016685578485
Enjoy the new release
Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organisation contributions to the Python Software Foundation.
Regards from a slushy, slippery Helsinki,
Your release team,
Hugo van Kemenade
Ned Deily
Steve Dower
Łukasz Langa
February 25, 2025 06:50 AM UTC
February 24, 2025
Łukasz Langa
A peek into a possible future of Python in the browser
My Python code was too slow, so I made it faster with Python. For some definition of “Python”.
February 24, 2025 07:03 PM UTC
Real Python
How to Use sorted() and .sort() in Python
Sorting in Python is a fundamental task that you can accomplish using sorted()
and .sort()
. The sorted()
function returns a new sorted list from the elements of any iterable, without modifying the original iterable. On the other hand, the .sort()
method modifies a list in place and doesn’t return a value. Both methods support customization through optional keyword arguments like key
and reverse
.
By the end of this tutorial, you’ll understand that:
- You can sort any iterable with the
sorted()
function. - The
sorted()
function returns a new sorted list. - The
.sort()
method sorts the list in place. - You sort items in descending order by setting the
reverse
argument toTrue
. - The
key
argument accepts a function to customize the sort order.
In this tutorial, you’ll learn how to sort various types of data in different data structures, customize the order, and work with two different ways of sorting in Python. You’ll need a basic understanding of lists and tuples as well as sets. These are the data structures you’ll be using to perform some basic operations.
Get Your Cheat Sheet: Click here to download a free cheat sheet that summarizes how to use sorted() and .sort() in Python.
Take the Quiz: Test your knowledge with our interactive “How to Use sorted() and .sort() in Python” quiz. You’ll receive a score upon completion to help you track your learning progress:
Interactive Quiz
How to Use sorted() and .sort() in PythonIn this quiz, you'll test your understanding of sorting in Python using sorted() and .sort(). You'll revisit how to sort various types of data in different data structures, customize the order, and work with two different ways of sorting in Python.
Ordering Values With sorted()
In Python, you can sort iterables with the sorted()
built-in function. To get started, you’ll work with iterables that contain only one data type.
Sorting Numbers
You can use sorted()
to sort a list in Python. In this example, a list of integers is defined, and then sorted()
is called with the numbers
variable as the argument:
>>> numbers = [6, 9, 3, 1]
>>> sorted(numbers)
[1, 3, 6, 9]
>>> numbers
[6, 9, 3, 1]
The output from this code is a new, sorted list. When the original variable is printed, the initial values are unchanged.
This example shows four important characteristics of sorted()
:
- You don’t have to define the
sorted()
function. It’s a built-in function that’s available in any standard installation of Python. - You’re ordering the values in
numbers
from smallest to largest when you callsorted(numbers)
. When you pass no additional arguments or parameters,sorted()
orders the values innumbers
in ascending order. - You don’t change the original
numbers
variable becausesorted()
provides sorted output and doesn’t update the original value in place. - You get an ordered list as a return value when you call
sorted()
.
These points mean that sorted()
can be used on a list, and the output can immediately be assigned to a variable:
>>> numbers = [6, 9, 3, 1]
>>> numbers_sorted = sorted(numbers)
>>> numbers_sorted
[1, 3, 6, 9]
>>> numbers
[6, 9, 3, 1]
In this example, a new variable called numbers_sorted
now stores the output of the sorted()
function.
You can confirm all of these observations by calling help()
on sorted()
:
>>> help(sorted)
Help on built-in function sorted in module builtins:
sorted(iterable, /, *, key=None, reverse=False)
Return a new list containing all items from the iterable in ascending order.
A custom key function can be supplied to customize the sort order, and the
reverse flag can be set to request the result in descending order.
You’ll cover the optional arguments key
and reverse
later in the tutorial.
The first parameter of sorted()
is an iterable. That means that you can use sorted()
on tuples and sets very similarly:
>>> numbers_tuple = (6, 9, 3, 1)
>>> sorted(numbers_tuple)
[1, 3, 6, 9]
>>> numbers_set = {5, 10, 1, 0}
>>> sorted(numbers_set)
[0, 1, 5, 10]
Notice how even though the input was a set and a tuple, the output is a list because sorted()
returns a new list by definition. The returned object can be cast to a new type if it needs to match the input type. Be careful if attempting to cast the resulting list back to a set, as a set by definition is unordered:
>>> numbers_tuple = (6, 9, 3, 1)
>>> numbers_set = {5, 10, 1, 0}
>>> numbers_tuple_sorted = sorted(numbers_tuple)
>>> numbers_set_sorted = sorted(numbers_set)
>>> numbers_tuple_sorted
[1, 3, 6, 9]
>>> numbers_set_sorted
[0, 1, 5, 10]
>>> tuple(numbers_tuple_sorted)
(1, 3, 6, 9)
>>> set(numbers_set_sorted)
{0, 1, 10, 5}
When you cast the numbers_set_sorted
value to a set
, it’s unordered, as expected. If you’re curious about how sets work in Python, then you can check out the tutorial Sets in Python.
Read the full article at https://realpython.com/python-sort/ »
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
February 24, 2025 02:00 PM UTC
Quiz: How to Use sorted() and .sort() in Python
In this quiz, you’ll test your understanding of sorting in Python.
By working through this quiz, you’ll revisit how to sort various types of data in different data structures, customize the order, and work with two different ways of sorting in Python.
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
February 24, 2025 12:00 PM UTC
DataWars.io
7 free Machine Learning projects to practice using Python | DataWars
February 24, 2025 09:25 AM UTC
Django Weblog
Call for Proposals for DjangoCon Africa 2025 is now open!
The call for proposals for DjangoCon Africa 2025 is officially open! 💃🏻 Come be a part of this headline event by submitting a talk.
Submit a proposal for DjangoCon Africa 2025
Why speak at DjangoCon Africa
Simply put, it’s an excellent opportunity to put your ideas out there, share knowledge with fellow Djangonauts, and give back to our community. You get to reach both a passitonate local audience, and the global Django community once your talk is published online.
If you’re interested in our Opportunity Grants, being an approved speaker or tutorial presenter also puts you first in line to receive that.
What to cover
We’re looking for proposals from first-time speakers as well as veterans. We want talks (20 - 45 min), workshops and tutorials, (60 - 90 min), and also lightning talks (5 min). As far as topics, here are suggested ones:
- Django internals and challenges in modern web development.
- Wild ideas, clever hacks, surprising or cool use cases.
- Improving Django and Python developers’ lives.
- Pushing Django to its limits.
- The Django and Python community, culture, history, past, present & future, the why, the who and the what of it all.
- Security
- Emerging technologies and industries – AI, Blockchain, Open Source etc.
- Diversity, Equity and Inclusion
- Whatever you deem appropriate - it’s your conference after all
Ubuntu
In addition to Django, this year's edition will feature a new Pan-African open source event running alongside DjangoCon Africa - UbuCon at DjangoCon Africa!
We invite proposals on any of these topics, and more: Desktop, Cloud and Infrastructure, Linux Containers and Container Orchestration, DevOps, Virtualisation, Automation, Networking Windows Subsystem for Linux(WSL), IoT, Embedded, Robotics, Appliances, Packaging, Documentation, QA and Bug triage, Security, Compliance and Kernel, Data and AI, Video, Audio and Image editing, Open source tools, Community, Diversity, Local Outreach and Social Context.
I’m in! What do I do?
Great! 🤘 Go submit your proposal. You have until the end of March to do that but no need to wait – submit now and you can always edit the proposal later.
And if you’d like to increase your changes, make sure to review our Speaking at DjangoCon Africa 2025 documentation, and the Speakers resources.
Submit a proposal for DjangoCon Africa 2025
Not convinved yet? Check out our Connections that count: Reflecting on DjangoCon Africa 2023 in Zanzibar to hear from our 2023 participants on what the conference meant for them.
February 24, 2025 09:05 AM UTC
Talk Python to Me
#495: OSMnx: Python and OpenStreetMap
On this episode, I’m joined by Dr. Jeff Boeing, an assistant professor at the University of Southern California whose research spans urban planning, spatial analysis, and data science. We explore why OpenStreetMap is such a powerful source of global map data—and how Jeff’s Python library, OSMnx, makes that data easier to download, model, and visualize. Along the way, we talk about what shapes city streets around the world, how urban design influences everything from daily commutes to disaster resilience, and why turning open data into accessible tools can open up completely new ways of understanding our cities. If you’ve ever wondered how to build or analyze your own digital maps in Python, or what it takes to manage a project that transforms raw geographic data into meaningful research, you won’t want to miss this conversation.<br/> <br/> <strong>Episode sponsors</strong><br/> <br/> <a href='https://talkpython.fm/ppm'>Posit</a><br> <a href='https://talkpython.fm/podcastlater'>Podcast Later</a><br> <a href='https://talkpython.fm/training'>Talk Python Courses</a><br/> <br/> <h2 class="links-heading">Links from the show</h2> <div><strong>City Street Orientations World</strong>: <a href="https://geoffboeing.com/2018/07/city-street-orientations-world/?featured_on=talkpython" target="_blank" >geoffboeing.com</a><br/> <strong>OSMnx Documentation</strong>: <a href="https://osmnx.readthedocs.io/en/stable/?featured_on=talkpython" target="_blank" >readthedocs.io</a><br/> <strong>OSMnx GitHub</strong>: <a href="https://github.com/gboeing/osmnx?featured_on=talkpython" target="_blank" >github.com</a><br/> <strong>OpenStreetMap</strong>: <a href="https://www.openstreetmap.org?featured_on=talkpython" target="_blank" >openstreetmap.org</a><br/> <strong>Open Database License</strong>: <a href="https://opendatacommons.org/licenses/odbl/1-0/?featured_on=talkpython" target="_blank" >opendatacommons.org</a><br/> <strong>ID Editor (Web Editor)</strong>: <a href="https://wiki.openstreetmap.org/wiki/ID?featured_on=talkpython" target="_blank" >wiki.openstreetmap.org</a><br/> <strong>Planet OSM</strong>: <a href="https://planet.openstreetmap.org/?featured_on=talkpython" target="_blank" >planet.openstreetmap.org</a><br/> <strong>Overpass API</strong>: <a href="https://wiki.openstreetmap.org/wiki/Overpass_API?featured_on=talkpython" target="_blank" >wiki.openstreetmap.org</a><br/> <strong>GeoPandas</strong>: <a href="https://geopandas.org/?featured_on=talkpython" target="_blank" >geopandas.org</a><br/> <strong>NetworkX</strong>: <a href="https://networkx.org/?featured_on=talkpython" target="_blank" >networkx.org</a><br/> <strong>Shapely</strong>: <a href="https://shapely.readthedocs.io/?featured_on=talkpython" target="_blank" >shapely.readthedocs.io</a><br/> <strong>Watch this episode on YouTube</strong>: <a href="https://www.youtube.com/watch?v=PGE2fwNc0zE" target="_blank" >youtube.com</a><br/> <strong>Episode transcripts</strong>: <a href="https://talkpython.fm/episodes/transcript/495/osmnx-python-and-openstreetmap" target="_blank" >talkpython.fm</a><br/> <br/> <strong>--- Stay in touch with us ---</strong><br/> <strong>Subscribe to Talk Python on YouTube</strong>: <a href="https://talkpython.fm/youtube" target="_blank" >youtube.com</a><br/> <strong>Talk Python on Bluesky</strong>: <a href="https://bsky.app/profile/talkpython.fm" target="_blank" >@talkpython.fm at bsky.app</a><br/> <strong>Talk Python on Mastodon</strong>: <a href="https://fosstodon.org/web/@talkpython" target="_blank" ><i class="fa-brands fa-mastodon"></i>talkpython</a><br/> <strong>Michael on Bluesky</strong>: <a href="https://bsky.app/profile/mkennedy.codes?featured_on=talkpython" target="_blank" >@mkennedy.codes at bsky.app</a><br/> <strong>Michael on Mastodon</strong>: <a href="https://fosstodon.org/web/@mkennedy" target="_blank" ><i class="fa-brands fa-mastodon"></i>mkennedy</a><br/></div>
February 24, 2025 08:00 AM UTC
Python Bytes
#421 22 years old
<strong>Topics covered in this episode:</strong><br> <ul> <li><strong><a href="https://github.com/cle-b/httpdbg?featured_on=pythonbytes">httpdbg</a></strong></li> <li><strong><a href="https://socket.dev/blog/pypi-now-supports-ios-and-android-wheels-for-mobile-python-development?featured_on=pythonbytes">PyPI Now Supports iOS and Android Wheels for Mobile Python Development</a></strong></li> <li><strong><a href="https://github.com/pythonarcade/arcade/blob/development/CHANGELOG.md?featured_on=pythonbytes">Arcade Game Platform</a> goes 3.0</strong></li> <li><strong><a href="https://peps.python.org/pep-0765/?featured_on=pythonbytes">PEP 765 – Disallow return/break/continue that exit a finally block</a></strong></li> <li><strong>Extras</strong></li> <li><strong>Joke</strong></li> </ul><a href='https://www.youtube.com/watch?v=P0VHDhUaWlo' style='font-weight: bold;'data-umami-event="Livestream-Past" data-umami-event-episode="421">Watch on YouTube</a><br> <p><strong>About the show</strong></p> <p>Sponsored by us! Support our work through:</p> <ul> <li>Our <a href="https://training.talkpython.fm/?featured_on=pythonbytes"><strong>courses at Talk Python Training</strong></a></li> <li><a href="https://courses.pythontest.com/p/the-complete-pytest-course?featured_on=pythonbytes"><strong>The Complete pytest Course</strong></a></li> <li><a href="https://www.patreon.com/pythonbytes"><strong>Patreon Supporters</strong></a></li> </ul> <p><strong>Connect with the hosts</strong></p> <ul> <li>Michael: <a href="https://fosstodon.org/@mkennedy"><strong>@mkennedy@fosstodon.org</strong></a> <strong>/</strong> <a href="https://bsky.app/profile/mkennedy.codes?featured_on=pythonbytes"><strong>@mkennedy.codes</strong></a> <strong>(bsky)</strong></li> <li>Brian: <a href="https://fosstodon.org/@brianokken"><strong>@brianokken@fosstodon.org</strong></a> <strong>/</strong> <a href="https://bsky.app/profile/brianokken.bsky.social?featured_on=pythonbytes"><strong>@brianokken.bsky.social</strong></a></li> <li>Show: <a href="https://fosstodon.org/@pythonbytes"><strong>@pythonbytes@fosstodon.org</strong></a> <strong>/</strong> <a href="https://bsky.app/profile/pythonbytes.fm"><strong>@pythonbytes.fm</strong></a> <strong>(bsky)</strong></li> </ul> <p>Join us on YouTube at <a href="https://pythonbytes.fm/stream/live"><strong>pythonbytes.fm/live</strong></a> to be part of the audience. Usually <strong>Monday</strong> at 10am PT. Older video versions available there too.</p> <p>Finally, if you want an artisanal, hand-crafted digest of every week of the show notes in email form? Add your name and email to <a href="https://pythonbytes.fm/friends-of-the-show">our friends of the show list</a>, we'll never share it.</p> <p><strong>Michael #1:</strong> <a href="https://github.com/cle-b/httpdbg?featured_on=pythonbytes">httpdbg</a></p> <ul> <li>A tool for Python developers to easily debug the HTTP(S) client requests in a Python program.</li> <li>To use it, execute your program using the pyhttpdbg command instead of python and that's it. Open a browser to http://localhost:4909 to view the requests</li> </ul> <p><strong>Brian #2:</strong> <a href="https://socket.dev/blog/pypi-now-supports-ios-and-android-wheels-for-mobile-python-development?featured_on=pythonbytes">PyPI Now Supports iOS and Android Wheels for Mobile Python Development</a></p> <ul> <li>Sara Gooding</li> <li>“the Python Packaging Index (PyPI) has officially begun accepting and distributing pre-compiled binary packages, known as "wheels," for both iOS and Android platforms. “</li> <li>Next up, “cibuildwheel Updates Are in Progress to Simplify iOS and Android Wheel Creation”</li> </ul> <p><strong>Michael #3:</strong> <a href="https://github.com/pythonarcade/arcade/blob/development/CHANGELOG.md?featured_on=pythonbytes">Arcade Game Platform</a> goes 3.0</p> <ul> <li>via Maic Siemering</li> <li>This is our first major release since 2022.</li> <li>It keeps the beginner-friendly API while adding power and efficiency.</li> <li>Arcade now supports both standard OpenGL and ShaderToy (<a href="https://www.shadertoy.com?featured_on=pythonbytes">www.shadertoy.com)</a> a-shaders through a compatibility layer.</li> <li>Since 3.0 is a major release, the full list of changes is over in</li> <li><a href="https://github.com/pythonarcade/arcade/blob/development/CHANGELOG.md?featured_on=pythonbytes">github.com/pythonarcade/arcade/blob/development/CHANGELOG.md</a></li> </ul> <p><strong>Brian #4:</strong> <a href="https://peps.python.org/pep-0765/?featured_on=pythonbytes">PEP 765 – Disallow return/break/continue that exit a finally block</a></p> <ul> <li>Accepted for Python 3.14</li> <li>I wouldn’t have thought to do this anyway, but it’s weird, so don’t.</li> <li>Will become a SyntaxWarning catchable by running with -We</li> </ul> <p><strong>Extras</strong> </p> <p>Brian:</p> <ul> <li>Correction: <a href="https://mastodon.online/@nikitonsky/113691789641950263?featured_on=pythonbytes">Niki Tonsky was originator of</a><a href="https://mastodon.online/@nikitonsky/113691789641950263?featured_on=pythonbytes"> </a><a href="https://mastodon.online/@nikitonsky/113691789641950263?featured_on=pythonbytes">“Pride</a><a href="https://mastodon.online/@nikitonsky/113691789641950263?featured_on=pythonbytes"> Versioning”</a>. <a href="https://fosstodon.org/@kytta/114034442981727301">Thanks Nikita</a></li> <li>Correction: <a href="https://filip.lajszczak.dev/my-land-of-scheme-2025.html?featured_on=pythonbytes">Scheme is actually awesome. Brian is just a curmudgeon</a></li> <li>Also: <a href="https://filip.lajszczak.dev/exposing-flaky-tests-with-pytest-rerunfailures.html?featured_on=pythonbytes">pytest-rerunfailures is good for exposing flaky tests</a></li> <li>And apparently me being wrong was a great to get at least one person to blog more. <ul> <li>Cheers Filip Łajszczak</li> </ul></li> </ul> <p>Michael:</p> <ul> <li>Tea pot follow up <ul> <li>While you're right that some software actually had this implemented, Python does not. It's not an officially accepted HTTP status code, it was proposed in a 'joke' RFC. I guess Python - even though its name comes from the funny TV series Monty Python - is not so funny. httpx, your (or at least -my-) favorite HTTP module for python, does have the I_AM_A_TEAPOT constant.</li> <li>By the way, there are some HTTP status codes that changed their names in RFC 9110, for instance, http.HTTPStatus.UNPROCESSABLE_CONTENT (422, previously UNPROCESSABLE_ENTITY)</li> </ul></li> <li>Pride follow up <a href="https://fosstodon.org/@kytta/114034442981727301">fosstodon.org/@kytta/114034442981727301</a></li> <li><a href="https://blobs.pythonbytes.fm/bench.png">Time to upgrade your mini</a>?</li> </ul> <p><strong>Joke:</strong> <a href="https://www.reddit.com/r/programminghumor/comments/1im5tj0/i_can_see_why_its_tough/?share_id=nzdSINSpAss2mrfKH9nCE&utm_content=2&utm_medium=ios_app&utm_name=iossmf&utm_source=share&utm_term=22&featured_on=pythonbytes">How old is she</a>?</p>
February 24, 2025 08:00 AM UTC
February 21, 2025
Real Python
The Real Python Podcast – Episode #240: Telling Effective Stories With Your Python Visualizations
How do you make compelling visualizations that best convey the story of your data? What methods can you employ within popular Python tools to improve your plots and graphs? This week on the show, Matt Harrison returns to discuss his new book "Effective Visualization: Exploiting Matplotlib & Pandas."
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
February 21, 2025 12:00 PM UTC
Talk Python to Me
#494: Update on Flet: Python + Flutter UIs
As Python developers, we're incredibly lucky to have over half a million packages that we can use to build our applications with over at PyPI. However, when it comes to choosing a UI framework, the options get narrowed down very quickly. Intersect those choices with the ones that work on mobile, and you have a very short list. Flutter is a UI framework for building desktop and mobile applications, and is in fact the one that we used to build the Talk Python courses app, you'd find at <a href="https://talkpython.fm/apps">talkpython.fm/apps</a>. That's why I'm so excited about Flet. Flet is a Python UI framework that is distributed and executed on the Flutter framework, making it possible to build mobile apps and desktop apps with Python. We have Feodor Fitsner back on the show after he launched his project a couple years ago to give us an update on how close they are to a full featured mobile app framework in Python.<br/> <br/> <strong>Episode sponsors</strong><br/> <br/> <a href='https://talkpython.fm/connect'>Posit</a><br> <a href='https://talkpython.fm/podcastlater'>Podcast Later</a><br> <a href='https://talkpython.fm/training'>Talk Python Courses</a><br/> <br/> <h2 class="links-heading">Links from the show</h2> <div><strong>Flet</strong>: <a href="https://flet.dev?featured_on=talkpython" target="_blank" >flet.dev</a><br/> <strong>Flet on Github</strong>: <a href="https://github.com/flet-dev/flet?featured_on=talkpython" target="_blank" >github.com</a><br/> <strong>Packaging apps with Flet</strong>: <a href="https://flet.dev/docs/publish?featured_on=talkpython" target="_blank" >flet.dev/docs/publish</a><br/> <br/> <strong>Flutter</strong>: <a href="https://flutter.dev/?featured_on=talkpython" target="_blank" >flutter.dev</a><br/> <strong>React vs. Flutter</strong>: <a href="https://trends.stackoverflow.co/?tags=flutter,react-native&featured_on=talkpython" target="_blank" >trends.stackoverflow.co</a><br/> <strong>Kivy</strong>: <a href="https://kivy.org?featured_on=talkpython" target="_blank" >kivy.org</a><br/> <strong>Beeware</strong>: <a href="https://beeware.org/?featured_on=talkpython" target="_blank" >beeware.org</a><br/> <strong>Mobile forge from Beeware</strong>: <a href="https://github.com/beeware/mobile-forge?featured_on=talkpython" target="_blank" >github.com</a><br/> <br/> <strong>The list of built-in binary wheels</strong>: <a href="https://flet.dev/docs/publish/android#binary-python-packages" target="_blank" >flet.dev/docs/publish/android#binary-python-packages</a><br/> <strong>Difference between dynamic and static Flet web apps</strong>: <a href="https://flet.dev/docs/publish/web?featured_on=talkpython" target="_blank" >flet.dev/docs/publish/web</a><br/> <strong>Integrating Flutter packages</strong>: <a href="https://flet.dev/docs/extend/integrating-existing-flutter-packages?featured_on=talkpython" target="_blank" >flet.dev/docs/extend/integrating-existing-flutter-packages</a><br/> <strong>serious_python</strong>: <a href="https://pub.dev/packages/serious_python?featured_on=talkpython" target="_blank" >pub.dev/packages/serious_python</a><br/> <strong>Watch this episode on YouTube</strong>: <a href="https://www.youtube.com/watch?v=zNyTE8W_5OM" target="_blank" >youtube.com</a><br/> <strong>Episode transcripts</strong>: <a href="https://talkpython.fm/episodes/transcript/494/update-on-flet-python-flutter-uis" target="_blank" >talkpython.fm</a><br/> <br/> <strong>--- Stay in touch with us ---</strong><br/> <strong>Subscribe to Talk Python on YouTube</strong>: <a href="https://talkpython.fm/youtube" target="_blank" >youtube.com</a><br/> <strong>Talk Python on Bluesky</strong>: <a href="https://bsky.app/profile/talkpython.fm" target="_blank" >@talkpython.fm at bsky.app</a><br/> <strong>Talk Python on Mastodon</strong>: <a href="https://fosstodon.org/web/@talkpython" target="_blank" ><i class="fa-brands fa-mastodon"></i>talkpython</a><br/> <strong>Michael on Bluesky</strong>: <a href="https://bsky.app/profile/mkennedy.codes?featured_on=talkpython" target="_blank" >@mkennedy.codes at bsky.app</a><br/> <strong>Michael on Mastodon</strong>: <a href="https://fosstodon.org/web/@mkennedy" target="_blank" ><i class="fa-brands fa-mastodon"></i>mkennedy</a><br/></div>
February 21, 2025 08:00 AM UTC
Python Morsels
Multiline strings
Need to represent multiple lines of text in Python? Use Python's multi-line string syntax!
A string with line breaks in it
Here we have a Python program called stopwatch.py
that acts like a timer:
from itertools import count
from time import sleep
import sys
arguments = sys.argv[1:]
usage = "Welcome to stopwatch!\nThis script counts slowly upward,\none second per tick.\n\nNo command-line arguments are accepted."
if arguments:
sys.exit(usage)
for n in count():
print(f"{n} sec")
sleep(1)
It counts upward one second at a time, until we manually exit it by hitting Ctrl + C
:
$ python3 stopwatch.py
0 sec
1 sec
2 sec
3 sec
4 sec
5 sec
^CTraceback (most recent call last):
File "/home/trey/stopwatch.py", line 13, in <module>
sleep(1)
KeyboardInterrupt
If we run this program with any command-line arguments at all, it prints out a usage statement instead of counting:
~ $ python3 stopwatch.py --help
Welcome to stopwatch!
This script counts slowly upward,
one second per tick.
No command-line arguments are accepted.
This usage statement is represented by multiple lines of text in a single string.
usage = "Welcome to stopwatch!\nThis script counts slowly upward,\none second per tick.\n\nNo command-line arguments are accepted."
This string has \n
characters in it, which are newline characters.
Using string concatenation to build up long strings
Currently our string is pretty …
Read the full article: https://www.pythonmorsels.com/multi-line-strings/
February 21, 2025 04:48 AM UTC
February 20, 2025
PyBites
NLP Made Easy: How We Prioritize Exercise Improvements with a Few Lines of Code
We highly appreciate user feedback for continuous improvement.
With hundreds of Bite exercises and thousands of reviews, it’s easy to get overwhelmed by the data.
How do you uncover insights from this sea of feedback? Use code!
Enter TextBlob, a Python library that abstracts away the complexities of Natural Language Processing (NLP).
This article walks you through a practical example: analyzing the sentiment of Bite reviews to identify which exercises need the most immediate attention and which ones delight people the most (perhaps to highlight these more on social media …)
Using TextBlob alongside SQLAlchemy’s automap feature, we’ll show you how to make sense of user sentiment with relatively little effort.
Problem Statement
Managing user feedback is challenging when dealing with:
- High volumes of unstructured text data.
- The need to quickly identify areas requiring improvement.
- Limited time / resources to implement a sophisticated NLP pipeline.
Leveraging TextBlob to perform sentiment analysis on Bite reviews, we quickly got actionable insights into which exercises have the most positive vs. negative feedback.
Setup
To get started:
- Clone the repository.
- Run
make setup
to create a virtual environment + install dependencies using uv. - Ensure your database connection URL is set in an
.env
file using theDATABASE_URL
variable, e.g.DATABASE_URL=postgresql://postgres:password@0.0.0.0:5432/reviews
- To create this database and add some fake data, run
make db
– this will execute the includeddata.sql
file.
$ git clone https://github.com/bbelderbos/nlp-bite-feedback
...
$ cd nlp-bite-feedback
$ make setup
uv sync
Using CPython 3.13.0
Creating virtual environment at: .venv
...
+ textblob==0.19.0
+ tqdm==4.67.1
+ typing-extensions==4.12.2
$ echo "DATABASE_URL=postgresql://postgres:password@0.0.0.0:5432/reviews" > .env
$ make db
createdb reviews && psql -U postgres -d reviews -f data.sql
CREATE TABLE
INSERT 0 6
New to Makefiles, check out our article or YouTube video:
The Script in Action
How It Works
The script:
- Queries the database for Bite reviews using SQLAlchemy’s automap.
- Uses TextBlob to calculate the sentiment polarity for each review.
- Aggregates the results to display:
- Average sentiment scores.
- The number of comments per Bite.
- Detailed reviews for a specific Bite upon request.
Example Output
Running the script without arguments displays the sentiment scores for all Bites – it’s clear that Bite # 276 needs attention:
$ uv run python script.py
bite id | # comments | avg sentiment score
276 | 2 | -0.2375
142 | 2 | 0.53125
229 | 2 | 0.83
To view the reviews for a specific Bite, pass its ID as an argument:
$ uv run python script.py 229
0.75 | Nice Bite! Learned (once again) to always proof-read my code.
0.91 | I have always struggled with loops, so this was very good practice.
$ uv run python script.py 276
-0.25 | It was only difficult because I forgot why we were defining.
-0.23 | Not sure if I am missing something on this one.
Key Concepts
Sentiment Analysis with TextBlob
TextBlob makes sentiment analysis effortless:
- Polarity: Measures how positive (1.0) or negative (-1.0) a comment is.
- Subjectivity: Determines how subjective (opinion-based) or objective a comment is.
For example:
from textblob import TextBlob
comment = "This Bite was incredibly helpful and fun!"
sentiment = TextBlob(comment).sentiment
print(sentiment.polarity) # 0.8 (positive)
print(sentiment.subjectivity) # 0.75 (fairly subjective)
SQLAlchemy Automap
SQLAlchemy’s automap dynamically maps database tables to Python objects:
from sqlalchemy.ext.automap import automap_base
Base = automap_base()
Base.prepare(engine, reflect=True)
REVIEW_TABLE = Base.classes.bites_biteconsumer
This eliminates the need to define models manually as you would typically do with ORMs. As per the SQLAlchemy Automap docs:
Define an extension to the
sqlalchemy.ext.declarative
system which automatically generates mapped classes and relationships from a database schema, typically though not necessarily one which is reflected. –
Here is another example where I used this:
Take-aways
- Fast Results: TextBlob abstracts NLP complexities, enabling rapid prototyping.
- Actionable Insights: Sentiment analysis highlights areas needing improvement.
- Scalable: Combine with dashboards or alerts for real-time insights into user sentiment.
By leveraging simple but powerful tools, you can uncover patterns in user feedback and continuously improve your platform.
Taking it a Step Further
While TextBlob provides a quick and effective way to analyze sentiment, there’s room for refinement.
One improvement could be fine-tuning sentiment thresholds—for example, adjusting how strongly negative reviews are flagged based on historical trends or specific keywords.
Additionally, for more nuanced sentiment analysis, you could incorporate an AI-based model (e.g., OpenAI’s API or a fine-tuned Hugging Face model).
These models can better handle sarcasm, context, and domain-specific language, making sentiment classification even more accurate.
Actually let’s put this to the test. Here is a quick script to use Marvin AI to do a sentiment analysis given a text input (if new to Inline Script Metadata, check this article).
And it works pretty well!
$ uv run sentiment.py --text "Oh great, another bug in production! This just made my day."
Polarity: -0.5
Subjectivity: 0.8
Summary: The sentiment is quite negative with a high level of subjectivity, expressing frustration and sarcasm.
$ uv run sentiment.py --text "The weather was cold, crisp, and refreshing. I loved it"
Polarity: 0.5
Subjectivity: 0.6
Summary: The sentiment is positive, reflecting enjoyment and appreciation of the weather.
$ uv run sentiment.py --text "The app crashed twice, but the debugging logs made it easy to fix."
Polarity: 0.0
Subjectivity: 0.5
Summary: The sentiment is neutral with balanced positive and negative statements about the app.
As demonstrated, AI appears more capable than TextBlob, which returned the following scores for the same texts:
$ uv run sentiment_textblob.py
Oh great, another bug in production! This just made my day.
1.0 # did not catch frustration / sarcasm!
0.75
The weather was cold, crisp, and refreshing. I loved it.
0.2125 # could have been more positive
0.8041666666666667
The app crashed twice, but the debugging logs made it easy to fix.
0.43333333333333335 # more on the positive side
0.8333333333333334
As seen in the examples, TextBlob struggles with sarcasm and contextual sentiment, whereas AI-powered models handle nuance significantly better.
So for the ease of TextBlob, if we’re willing to pay a bit of money, using openai’s API we can make this even more sophisticated with almost equally little code.
Combining this with alerts or dashboards could provide real-time insights into user sentiment, allowing for faster and more targeted exercise improvements.
Try running one of those scripts, or something similar, on your own data and share your findings in our community.