skip to navigation
skip to content

Planet Python

Last update: October 23, 2021 10:40 AM UTC

October 22, 2021


Ben Cook

TorchVision Datasets: Getting Started

The TorchVision datasets subpackage is a convenient utility for accessing well-known public image and video datasets. You can use these tools to start training new computer vision models very quickly.

TorchVision Datasets Example

To get started, all you have to do is import one of the Dataset classes. Then, instantiate it and access one of the samples with indexing:

from torchvision import datasets

dataset = datasets.MNIST(root="./", download=True)
img, label = dataset[10]
img.size

# Expected result
# (28, 28)

You’ll get a tuple with a Pillow image and an integer label back:

torchvision datasets mnist 3

The TorchVision datasets implement __len__() and __getitem__() methods, which means that in addition to getting specific elements by index, you can also get the number of samples with the len() function:

len(dataset)

# Expected result
# 60000

Additionally, DataLoader classes can use TorchVision Dataset objects to create automatic batches for training.

Since they mostly return Pillow images, you do need to pass in a transform to convert the image to a tensor:

import torch
from torchvision import transforms

dataset = datasets.MNIST(
    root="./",
    transform=transforms.ToTensor()
)
data_loader = torch.utils.data.DataLoader(dataset, batch_size=4)

x, y = next(iter(data_loader))

x.shape

# Expected result
# torch.Size([4, 1, 28, 28])

API

The interface for the TorchVision Dataset classes is somewhat inconsistent because every dataset has a slightly different set of constraints. For example, many of the datasets return (PIL.Image, int) tuples, but this obviously wouldn’t work for videos (TorchVision packs them into tensors).

But generally speaking, the constructors take the following arguments:

A word about ImageNet

ImageNet is no longer available for small companies or independent researchers. This is a real shame because pre-trained classifiers in model zoos are almost always trained on ImageNet.

However, it is possible to download most of the ImageNet dataset from Academic Torrents. I cannot endorse this strategy because I don’t know if it’s allowed.

If you did want to download the train and validation sets from ImageNet 2012, here are some steps you could follow:

  1. Launch an Amazon Linux EC2 instance with at least 200GB of storage. The whole process takes about 2 hours on a c5.xlarge instance.

2. Install the aria2c command-line tool (instructions here).

3. Download the tar files:

# Download the validation set
aria2c https://academictorrents.com/download/dfa9ab2528ce76b907047aa8cf8fc792852facb9.torrent

# Download the train set
aria2c https://academictorrents.com/download/a306397ccf9c2ead27155983c254227c0fd938e2.torrent

4. Make sure the files match the MD5 hashes (helpfully provided by the TorchVision team):

# Check the validation file
md5sum ILSVRC2012_img_val.tar

# Expected result
# 29b22e2961454d5413ddabcf34fc5622

# Check the train file
md5sum ILSVRC2012_img_train.tar

# Expected result
# 1d675b47d978889d74fa0da5fadfb00e

5. Upload the files to S3 — hosting the files costs a little over $3 per month.

6. Terminate the instance.

7. Send the Academic Torrents team some Bitcoin to say thank you.

Summary

And that’s all you need to know to get started with TorchVision Datasets. For production machine learning pipelines, you probably want to implement your own Dataset class, but the datasets that come out of the box with TorchVision are a great way to experiment quickly!

The post TorchVision Datasets: Getting Started appeared first on Sparrow Computing.

October 22, 2021 09:59 PM UTC


Python for Beginners

Compare two lists in Python

While programming in python, comparison has to be done very often for checking different conditions. We may need to compare two variables or two groups of variables for a single condition checking. In this article we will try different ways to compare two lists in python. While comparing, we will have to check if both the lists contain the same elements or not irrespective of the order in which the elements are present in the lists. Accordingly, we will have to print the result.

Compare two lists using sort() method

To check whether two lists contain the same elements or not, we can use  the sort() method to sort the elements of the lists first. Then, we can compare the two lists.

For comparison,first we will check if the length of the lists are equal or not. If the lengths are not equal, the lists will be automatically considered as different. 

If the length of the lists are the same, we will sort the two lists. Then we will compare the lists using the == operator to check whether the lists are equal or not. If the sorted lists are equal, it will establish that both the original lists contain the same elements. This can be implemented as follows.

# function to compare lists
def compare(l1, l2):
    # here l1 and l2 must be lists
    if len(l1) != len(l2):
        return False
    l1.sort()
    l2.sort()
    if l1 == l2:
        return True
    else:
        return False


list1 = [1, 2, 3, 4]
list2 = [1, 4, 3, 2]
list3 = [2, 3, 4, 5]
print("list1 is:",list1)
print("list2 is:",list2)
print("list3 is:",list3)

# comparing list1 and list 2
print("list1 and list2 contain same elements:",compare(list1, list2))
#comparing list2 and list3
print("list1 and list3 contain same elements:",compare(list1, list3))

Output:

list1 is: [1, 2, 3, 4]
list2 is: [1, 4, 3, 2]
list3 is: [2, 3, 4, 5]
list1 and list2 contain same elements: True
list1 and list3 contain same elements: False

Compare using sets in Python

To compare two lists in python, we can use sets. A set in python only allows unique values in it. We can use this property of sets to find if two lists have the same elements or not.

For comparison,first we will check if the length of the lists are equal or not. If the lengths are not equal, the lists will be automatically flagged as different.

After that, we will convert the lists into sets using set() constructor. We can compare the two sets using the == operator to check if both the sets are equal or not. If both sets are equal, it will be established that both the lists contain equal values. Following example illustrates this concept.

# function to compare lists
def compare(l1, l2):
    # here l1 and l2 must be lists
    if len(l1) != len(l2):
        return False
    set1 = set(l1)
    set2 = set(l2)
    if set1 == set2:
        return True
    else:
        return False


list1 = [1, 2, 3, 4]
list2 = [1, 4, 3, 2]
list3 = [2, 3, 4, 5]
print("list1 is:", list1)
print("list2 is:", list2)
print("list3 is:", list3)

# comparing list1 and list 2
print("list1 and list2 contain same elements:", compare(list1, list2))
# comparing list2 and list3
print("list1 and list3 contain same elements:", compare(list1, list3))

Output:

list1 is: [1, 2, 3, 4]
list2 is: [1, 4, 3, 2]
list3 is: [2, 3, 4, 5]
list1 and list2 contain same elements: True
list1 and list3 contain same elements: False

Compare two lists using frequency counter

We can also compare two lists without comparing their lengths. For this, first we will have to create a python dictionary for each list which will keep track of the frequency of the elements in the lists. After creating the dictionaries where elements of the lists are stored as keys and their frequency are stored as values, we can compare the frequency of each element in the two dictionaries. If the frequency of each elements becomes equal, it will be confirmed that both the lists contained equal elements.

For this task, we can use the counter() method. The counter() method, when invoked on a list, creates a python dictionary and stores the elements as keys and their frequency as values in it. After invoking the counter() method, we can compare the created dictionary using == operator to check whether the frequency of every element is equal or not. If the result is True, the lists contain equal elements. Otherwise not. This can be seen from the following example.

import collections
# function to compare lists
def compare(l1, l2):
    # here l1 and l2 must be lists
    if len(l1) != len(l2):
        return False
    counter1 = collections.Counter(l1)
    counter2=collections.Counter(l2)
    if counter1 == counter2:
        return True
    else:
        return False


list1 = [1, 2, 3, 4]
list2 = [1, 4, 3, 2]
list3 = [2, 3, 4, 5]
print("list1 is:", list1)
print("list2 is:", list2)
print("list3 is:", list3)

# comparing list1 and list 2
print("list1 and list2 contain same elements:", compare(list1, list2))
# comparing list2 and list3
print("list1 and list3 contain same elements:", compare(list1, list3))

Output:

list1 is: [1, 2, 3, 4]
list2 is: [1, 4, 3, 2]
list3 is: [2, 3, 4, 5]
list1 and list2 contain same elements: True
list1 and list3 contain same elements: False

Conclusion

In this article, we have seen three different ways to compare two lists in python and have checked if they contain the same elements without considering the position of the elements. To read more about lists, read this article on list comprehension.

In the compare() function used in the examples, it may be possible that user passes two other objects instead of lists. In such cases, the program may run into error. To avoid this, we can use exception handling using python try except to avoid runtime errors by applying type checking using type() method in try-except block to check if the objects passed as arguments are lists.

The post Compare two lists in Python appeared first on PythonForBeginners.com.

October 22, 2021 12:58 PM UTC


Real Python

The Real Python Podcast – Episode #83: Ready to Publish Your Python Packages?

Are you interested in sharing your Python project with the broader world? Would you like to make it easily installable using pip? How do you create Python packages that share your code in a scalable and maintainable way? This week on the show, Real Python author and former guest Dane Hillard returns to talk about his new book, "Publishing Python Packages."


[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

October 22, 2021 12:00 PM UTC


Stack Abuse

Integrating Matplotlib Charts in a PDF in Python With borb

Introduction

The Portable Document Format (PDF) is not a WYSIWYG (What You See is What You Get) format. It was developed to be platform-agnostic, independent of the underlying operating system and rendering engines.

To achieve this, PDF was constructed to be interacted with via something more like a programming language, and relies on a series of instructions and operations to achieve a result. In fact, PDF is based on a scripting language - PostScript, which was the first device-independent Page Description Language.

In this guide, we'll be using borb - a Python library dedicated to reading, manipulating and generating PDF documents. It offers both a low-level model (allowing you access to the exact coordinates and layout if you choose to use those) and a high-level model (where you can delegate the precise calculations of margins, positions, etc to a layout manager).

Matplotlib is a data visualization library that fueled an entire generation of engineers to start visualizing data, and the engine behind many other popular libraries such as Seaborn.

Given how common PDF documents are for creating reports (which oftentimes include graphs), we'll take a look at how to integrate Matplotlib charts in a PDF document using borb.

Installing borb (and Matplotlib)

borb can be downloaded from source on GitHub, or installed via pip:

$ pip install borb

Matplotlib can be installed via pip:

$ pip install matplotlib

Integrating Matplotlib Charts in PDF Documents with borb

Before we can create a chart, such as a Pie Chart, we're going to write a small utility function that generates N colors, evenly distributed among the color-spectrum.

This will help us whenever we need to create a plot and color each section:

from borb.pdf.canvas.color.color import HSVColor, HexColor
from decimal import Decimal
import typing

def create_n_colors(n: int) -> typing.List[str]:
  # The base color is borb-blue
  base_hsv_color: HSVColor = HSVColor.from_rgb(HexColor("56cbf9"))
  # This array comprehension creates n HSVColor objects, transforms then to RGB, and then returns their hex string
  return [HSVColor(base_hsv_color.hue + Decimal(x / 360), Decimal(1), Decimal(1)).to_rgb().to_hex_string() for x in range(0, 360, int(360/n))]

Note: HSL (hue, saturation, lightness) and HSV/HSB (hue, saturation, value/hue, saturation, brightness) are alternative representations of the RGB color model.

HSL and HSV/HSB were designed in the 1970s by computer graphics researchers to more closely align with the way human vision perceives color-making attributes. In these models, colors of each hue are arranged in a radial slice, around a central axis of neutral colors which ranges from black at the bottom to white at the top:

hsv cone spectrum
Credits: Wikimedia (CC BY-SA 3.0) license

The advantage of using this representation for Color is that we can easily divide the color-spectrum in equal parts.

Now we can define a create_pie_chart() function (or a function for other types of plots):

# New import(s)
import matplotlib.pyplot as plt
from borb.pdf.canvas.layout.image.chart import Chart
from borb.pdf.canvas.layout.layout_element import Alignment

def create_piechart(labels: typing.List[str], data: typing.List[float]):

  # Symetric figure to ensure equal aspect ratio
  fig1, ax1 = plt.subplots(figsize=(4, 4))
  ax1.pie(
    data,
    explode=[0 for _ in range(0, len(labels))],
    labels=labels,
    autopct="%1.1f%%",
    shadow=True,
    startangle=90,
    colors=create_n_colors(len(labels)),
  )

  ax1.axis("equal")  # Equal aspect ratio ensures that pie is drawn as a circle.

  return Chart(
    plt.gcf(),
    width=Decimal(200),
    height=Decimal(200),
    horizontal_alignment=Alignment.CENTERED,
  )

Here, we've used Matplotlib to create a pie chart, via the pie() function.

If you'd like to learn more about creating Pie Charts, read our Guide to Matplotlib Pie Charts!

The gcf() function of the PyPlot instance returns the current figure (get current figure). This figure can be embedded into a PDF document, by injecting it in a Chart constructor, alongside your customization arguments such as the width, height and horizontal_alignment.

That's it! You just supply a Matplotlib figure to the Chart contructor.

Adding a Matplotlib Chart to a PDF Document

Now it's time to create our basic PDF Document and add content to it.

# New import(s)
from borb.pdf.document import Document
from borb.pdf.page.page import Page
from borb.pdf.pdf import PDF
from borb.pdf.canvas.layout.page_layout.multi_column_layout import MultiColumnLayout
from borb.pdf.canvas.layout.page_layout.page_layout import PageLayout
from borb.pdf.canvas.layout.text.paragraph import Paragraph

# Create empty Document
pdf = Document()

# Create empty Page
page = Page()

# Add Page to Document
pdf.append_page(page)

# Create PageLayout
layout: PageLayout = MultiColumnLayout(page)

# Write title
layout.add(Paragraph("About Lorem Ipsum", 
                     font_size=Decimal(20), 
                     font="Helvetica-Bold"))

We'll be using hyphenation in this PDF to ensure the text can be laid out even more smoothly. Hyphenation in borb is pretty straightforward:

# New import(s)
from borb.pdf.canvas.layout.hyphenation.hyphenation import Hyphenation

# Create hyphenation algorithm
hyphenation_algorithm: Hyphenation = Hyphenation("en-gb")

# Write paragraph
layout.add(Paragraph(
    """
    Lorem Ipsum is simply dummy text of the printing and typesetting industry. 
    Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, 
    when an unknown printer took a galley of type and scrambled it to make a type specimen book. 
    It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged. 
    It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, 
    and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum.
    """, text_alignment=Alignment.JUSTIFIED, hyphenation=hyphenation_algorithm))

Now we can add a piechart using the function we declared earlier;

# Write graph
layout.add(create_piechart(["Loren", "Ipsum", "Dolor"], 
                           [0.6, 0.3, 0.1]))

Next we're going to write three more Paragraph objects.
One of them is going to be more of a quote (border on the side, different font, etc).

# Write paragraph
layout.add(Paragraph(
    """
    Contrary to popular belief, Lorem Ipsum is not simply random text. 
    It has roots in a piece of classical Latin literature from 45 BC, making it over 2000 years old. 
    Richard McClintock, a Latin professor at Hampden-Sydney College in Virginia, looked up one of the more obscure Latin words, 
    consectetur, from a Lorem Ipsum passage, and going through the cites of the word in classical literature, 
    discovered the undoubtable source.
    """, text_alignment=Alignment.JUSTIFIED, hyphenation=hyphenation_algorithm))

# Write paragraph
layout.add(Paragraph(
    """
    Lorem Ipsum is simply dummy text of the printing and typesetting industry. 
    """, 
    font="Courier-Bold",
    text_alignment=Alignment.JUSTIFIED, 
    hyphenation=hyphenation_algorithm,
    border_color=HexColor("56cbf9"),
    border_width=Decimal(3),
    border_left=True,
    padding_left=Decimal(5),
    padding_bottom=Decimal(5),
))

# Write paragraph
layout.add(Paragraph(
    """
    Lorem Ipsum comes from sections 1.10.32 and 1.10.33 of "de Finibus Bonorum et Malorum" 
    (The Extremes of Good and Evil) by Cicero, written in 45 BC. 
    This book is a treatise on the theory of ethics, very popular during the Renaissance.
    """, text_alignment=Alignment.JUSTIFIED, hyphenation=hyphenation_algorithm))

Let's add another plot

# Write graph
layout.add(create_piechart(["Loren", "Ipsum", "Dolor", "Sit", "Amet"], 
                           [600, 30, 89, 100, 203]))

And one more Paragraph of content

# Write paragraph
layout.add(Paragraph(
    """
    It is a long established fact that a reader will be distracted by the readable content of a page when looking at its layout. 
    The point of using Lorem Ipsum is that it has a more-or-less normal distribution of letters, as opposed to using 'Content here, content here', 
    making it look like readable English. Many desktop publishing packages and web page editors now use Lorem Ipsum as their default model text, 
    and a search for 'lorem ipsum' will uncover many web sites still in their infancy. 
    Various versions have evolved over the years, sometimes by accident, sometimes on purpose (injected humour and the like).
    """, text_alignment=Alignment.JUSTIFIED, hyphenation=hyphenation_algorithm))

Finally, we can store the Document:

# Write to disk
with open("output.pdf", "wb") as pdf_file_handle:
  PDF.dumps(pdf_file_handle, pdf)

Running this code results in a PDF document that looks like this:

integrating matplotlib charts in pdf with python and borb

Conclusion

In this guide you've learned how to integrate Matplotlib charts in a PDF using borb. From here, the sky is the limit! The more creative you get with data visualization, the nicer your PDFs will be.

October 22, 2021 08:30 AM UTC


Test and Code

167: React, TypeScript, and the Joy of Testing - Paul Everitt

Paul has a tutorial on testing and TDD with React and TypeScript.
We discuss workflow and the differences, similarities between testing with React/TypeScript and Python.
We also discuss what lessons that we can bring from front end testing to Python testing.

Special Guest: Paul Everitt.

Sponsored By:

Support Test & Code in Python

Links:

<p>Paul has a tutorial on testing and TDD with React and TypeScript. <br> We discuss workflow and the differences, similarities between testing with React/TypeScript and Python.<br> We also discuss what lessons that we can bring from front end testing to Python testing.</p><p>Special Guest: Paul Everitt.</p><p>Sponsored By:</p><ul><li><a href="https://www.patreon.com/testpodcast" rel="nofollow">Patreon Supporters</a>: <a href="https://www.patreon.com/testpodcast" rel="nofollow">Help support the show with as little as $1 per month and be the first to know when new episodes come out.</a></li></ul><p><a href="https://www.patreon.com/testpodcast" rel="payment">Support Test & Code in Python</a></p><p>Links:</p><ul><li><a href="https://www.jetbrains.com/webstorm/guide/tutorials/react_typescript_tdd/" title="React, TypeScript, and TDD" rel="nofollow">React, TypeScript, and TDD</a> &mdash; Paul Everitt's tutorial</li><li><a href="https://testing-library.com/docs/react-testing-library/intro/" title="React Testing Library" rel="nofollow">React Testing Library</a></li></ul>

October 22, 2021 04:15 AM UTC

October 21, 2021


Ben Cook

Development containers in VS Code: a quick start guide

If you’re building production ML systems, dev containers are the killer feature of VS Code. Dev containers give you full VS Code functionality inside a Docker container. This lets you unify your dev and production environments if production is a Docker container. But even if you’re not targeting a Docker deployment, running your code in a container prevents you from fighting with local Python environments across development machines: if you configure your dev container correctly, it will work exactly the same for every member of your team.

In this post, I’ll give you a quick look at how development containers work in VS Code so you can try them for yourself. There are other ways to go about this, but for a basic workflow, I recommend a few things:

This means, for a new project, the folder structure will look something like the following:

.
├── .devcontainer
│   └── devcontainer.json
├── Dockerfile
├── LICENSE
├── README.md
├── main.py
├── requirements-dev.txt
└── requirements.txt

The Dockerfile can be very flexible. Typically you’ll want to install dependencies unless your Docker image already has them installed. Here’s a very basic one you can use to start:

FROM python:3.9

COPY requirements.txt .
RUN pip install requirements.txt

The devcontainer.json file is also highly customizable, but a few things are particularly important:

Here’s what a basic devcontainer.json looks like:

{
  "name": "Python 3",
  "build": {
    "dockerfile": "../Dockerfile",
  },
  // Set *default* container specific settings.json values on container create.
  "settings": {
    "python.pythonPath": "/usr/local/bin/python",
    "python.languageServer": "Pylance",
    "python.linting.enabled": true,
    "python.formatting.blackPath": "/usr/local/bin/black",
    "python.formatting.provider": "black",
    "editor.formatOnSave": true,
  },
  // Add the IDs of extensions you want installed when the container is created.
  "extensions": [
    "ms-python.python",
    "ms-python.vscode-pylance"
  ],
  "postCreateCommand": "pip install -r requirements-dev.txt"
}

Once you have these files setup in your VS Code session, you can open the project in a container. Open the Command Palette with F1 and run Remote-Containers: Open Folder in Container... From there, you can open a terminal and run your code.

Notice if you modify print("hello world") to print('hello world') (with a single quotation) Black will automatically change it for you on save. Nifty!

One other note: the community seems to be switching to Poetry for dependency management in Python and I think this is a good trend. Version conflicts are a huge pain in the regular pip toolchain. I will save that for another post, but just know that you can swap out the pip requirements files for Poetry.

The post Development containers in VS Code: a quick start guide appeared first on Sparrow Computing.

October 21, 2021 09:42 PM UTC


Łukasz Langa

PEP 563 and PEP 649

As the author of PEP 563, I can summarize my position as follows: had PEP 563 never been implemented, it would be easy to accept PEP 649. However, in the current situation it’s not that clear because I find it very important to allow writing a single codebase that works on Python 3.7 - 3.11. If we can secure this behavior, I’m +1 to accepting PEP 649. If not, I have an alternative idea.

October 21, 2021 07:54 PM UTC


Reuven Lerner

Stop fearing Python decorators

If I mention “decorators,” even if you’ve been programming with Python for a while, what comes to mind?

To many, decorators are mysterious and powerful — and beyond the reach of mere mortals. Their @ signs pop up in all sorts of places, from “pytest” to Flask to properties… but it’s not really clear what they are, what they’re doing, how they work, or how they can be useful.

Here’s the thing: If you are comfortable writing functions and classes in Python, then you can write decorators. And once you start, you’ll wonder how you lived without them.

I’ve taught decorators to engineers at some of the world’s largest companies, and also at conferences like PyCon and Euro Python. And this Sunday, I’ll be teaching a live, online course all about decorators.

Once you’re done with the course, you’ll be able to write and understand a wide variety of decorators, from those that filter arguments to those that cache function results and monitor your code’s performance.

I’m running these class like I do my corporate training: With lots of time for Q&A, and a ton of hands-on exercises. Without slides, as I live-code into Jupyter. We’ll even have a private forum, so you can ask questions when the class is done. And of course, you’ll have access to the video recording.

If you want to finally understand decorators, then you want this class.  Sign up here: https://store.lerner.co.il/live-class-decorators

Questions? Comments? Eligible for a discount (for students, retirees/pensioners, or people living outside the world’s 30 richest countries)? E-mail me at reuven@lerner.co.il or get me on Twitter as @reuvenmlerner, and I’ll answer ASAP.

The post Stop fearing Python decorators appeared first on Reuven Lerner.

October 21, 2021 07:35 PM UTC


PyCharm

PyCharm 2021.3 EAP #3

The third PyCharm 2021.3 EAP build is out! Please give it a try, and don’t forget to share your feedback on Twitter (by mentioning @pycharm) or by reporting issues to our tracker!

Important:

DOWNLOAD PYCHARM 2021.3 EAP

Poetry support

One of the most popular issues in our tracker is getting closer to being marked as “fixed”, as this EAP build bundles Poetry support out of the box. We’ve also bundled the TOML plugin so you will also get code insight for your pyproject.toml file out of the box. Kudos to Koudai Aono, who developed the original plugin, which is now being fine-tuned and will become available for all PyCharm users by default.

poetry-support-screenshot

Endpoints tool window for FastAPI and Flask projects

If you develop web applications using FastAPI and Flask, you likely work with endpoints. Contrary to the name, endpoints are where your work as a Python developer often begins. Although working with them is generally simple in minor projects, it can become cumbersome in larger ones.

Our team is always looking for ways to make you more productive and remove roadblocks from your workflow. This is why we’ve made the Endpoints tool window default for FastAPI and Flask projects. When you start working on a new or existing project, PyCharm will scan its routes and list them as endpoints in a dedicated tool window. You’ll also get code completion and refactoring capabilities for your URLs.

fastapi-and-flask-endpoints-tool-window

The tool window will present relevant documentation, and you’ll have full navigation capabilities, including the ability to go to an endpoint’s declaration and to find usages of it across your project. You’ll also have quick access to the HTTP Client, an ad hoc tool that allows you to create requests and receive their results quickly for testing purposes.

endpoints-code-completion

Bug fixes and improvements

Ready to join the EAP?

Some ground rules

How to download

You can download this EAP build from our website. Alternatively, you can use the JetBrains Toolbox App to stay up to date throughout the entire EAP. If you’re on Ubuntu 16.04 or later, you can use snaps to make sure you always have the latest PyCharm EAP build.

The PyCharm team

October 21, 2021 06:08 PM UTC


qutebrowser development blog

CVE-2021-41146: Arbitrary command execution in qutebrowser on Windows via URL handler

I'm happy to announce that I just released qutebrowser v2.4.0!

This release fixes a high-severity arbitrary command execution on Windows via URL handlers, see the security advisory and commit message for details.

Windows users are urged to update as soon as possible. For everyone else, this is a …

October 21, 2021 05:28 PM UTC


Ben Cook

NumPy Any: Understanding np.any()

The np.any() function tests whether any element in a NumPy array evaluates to true:

np.any(np.array([[1, 0], [0, 0]]))

# Expected result
# True

The input can have any shape and the data type does not have to be boolean (as long as it’s truthy). If none of the elements evaluate to true, the function returns false:

np.any(np.array([[0, 0], [0, 0]]))

# Expected result
# False

Passing in a value for the axis argument makes np.any() a reducing operation. Say we want to know which rows in a matrix have any truthy elements. We can do that by passing in axis=-1:

np.any(np.zeros((2, 3)), axis=-1)

# Expected result
# array([False, False])

There are two rows and for each of them, none of the elements evaluate to true. The -1 value here is shorthand for “the last axis”.

Easy enough! NumPy also has a function called np.all() which has the same API as np.any() but returns true when all of the elements evaluate to true.

The post NumPy Any: Understanding np.any() appeared first on Sparrow Computing.

October 21, 2021 02:42 PM UTC

TorchVision Transforms: Image Preprocessing in PyTorch

TorchVision, a PyTorch computer vision package, has a simple API for image pre-processing in its torchvision.transforms module. The module contains a set of common, composable image transforms and gives you an easy way to write new custom transforms. As you would expect, these custom transforms can be included in your pre-processing pipeline like any other transform from the module.

Let’s start with a common use case, preparing PIL images for one of the pre-trained TorchVision image classifiers:

import io

import requests
import torchvision.transforms as T

from PIL import Image

resp = requests.get('https://sparrow.dev/assets/img/cat.jpg')
img = Image.open(io.BytesIO(resp.content))

preprocess = T.Compose([
   T.Resize(256),
   T.CenterCrop(224),
   T.ToTensor(),
   T.Normalize(
       mean=[0.485, 0.456, 0.406],
       std=[0.229, 0.224, 0.225]
   )
])

x = preprocess(img)
x.shape

# Expected result
# torch.Size([3, 224, 224])

Here, we apply the following in order:

  1. Resize a PIL image to (<height>, 256), where <height> is the value that maintains the aspect ratio of the input image.
  2. Crop the (224, 224) center pixels.
  3. Convert the PIL image to a PyTorch tensor (which also moves the channel dimension to the beginning).
  4. Normalize the image by subtracting a known ImageNet mean and standard deviation.

Let’s go a notch deeper to understand exactly how these transforms work.

Transforms

TorchVision transforms are extremely flexible – there are just a few rules. In order to be composable, transforms need to be callables. That means you can actually just use lambdas if you want:

times_2_plus_1 = T.Compose([
    lambda x: x * 2,
    lambda x: x + 1,
])

x.mean(), times_2_plus_1(x).mean()

# Expected result
# (tensor(1.2491), tensor(3.4982))

But often, you’ll want to use callable classes because they give you a nice way to parameterize the transform at initialization. For example, if you know you want to resize images to have height of 256 you can instantiate the T.Resize transform with a 256 as input to the constructor:

resize_callable = T.Resize(256)

Any PIL image passed to resize_callable() will now get resized to (<height>, 256):

resize_callable(img).size

# Expected result
# (385, 256)

This behavior is important because you will typically want TorchVision or PyTorch to be responsible for calling the transform on an input. We actually saw this in the first example: the component transforms (ResizeCenterCropToTensor, and Normalize) were chained and called inside the Compose transform. And the calling code would not have knowledge of things like the size of the output image you want or the mean and standard deviation for normalization.

Interestingly, there is no Transform base class. Some transforms have no parent class at all and some inherit from torch.nn.Module. This means that if you’re writing a transform class, the constructor can do whatever you want. The only requirement is that there must be a __call__() method to ensure the instantiated object is callable. Note: when transforms override the torch.nn.Module class, they will typically define the forward() method and then the base class takes care of __call__().

Additionally, there are no real constraints on the callable’s inputs or outputs. A few examples:

NumPy arrays may also be a good choice sometimes.

Ok. Now that we know a little about what transforms are, let’s look at an example that TorchVision gives us out of the box.

Example Transform: Compose

The T.Compose transform takes a list of other transforms in the constructor and applies them sequentially to the input. We can take a look at the __init__() and __call__() methods from a recent commit hash to see how this works:

class Compose:
    def __init__(self, transforms):
        self.transforms = transforms

    def __call__(self, img):
        for t in self.transforms:
            img = t(img)
        return img

Very simple! You can pass the T.Compose constructor a list (or any other in-memory sequence) of callables and it will dutifully apply them to any input one at a time. And notice that the input img can be any type you want. In the first example, the input was PIL and the output was a PyTorch tensor. In the second example, the input and output were both tensors. T.Compose doesn’t care!

Let’s instantiate a new T.Compose transform that will let us visualize PyTorch tensors. Remember, we took a PIL image and generated a PyTorch tensor that’s ready for inference in a TorchVision classifier. Let’s take a PyTorch tensor from that transformation and convert it into an RGB NumPy array that we can plot with Matplotlib:

torchvision transforms on a cat image
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np

reverse_preprocess = T.Compose([
    T.ToPILImage(),
    np.array,
])

plt.imshow(reverse_preprocess(x));

The T.ToPILImage transform converts the PyTorch tensor to a PIL image with the channel dimension at the end and scales the pixel values up to int8. Then, since we can pass any callable into T.Compose, we pass in the np.array() constructor to convert the PIL image to NumPy. Not too bad!

Functional Transforms

As we’ve now seen, not all TorchVision transforms are callable classes. In fact, TorchVision comes with a bunch of nice functional transforms that you’re free to use. If you look at the torchvision.transforms code, you’ll see that almost all of the real work is being passed off to functional transforms.

For example, here’s the functional version of the resize logic we’ve already seen:

import torchvision.transforms.functional as F

F.resize(img, 256).size

# Expected result
# (385, 256)

It does the same work, but you have to pass additional arguments in when you call it. My advice: use functional transforms for writing custom transform classes, but in your pre-processing logic, use callable classes or single-argument functions that you can compose.

At this point, we know enough about TorchVision transforms to write one of our own.

Custom Transforms

Let’s write a custom transform that erases the top left corner of an image with the color of a randomly selected pixel. We’ll use the F.erase() function and we’ll allow the caller to specify what how many pixels they want to erase in both directions:

import torch

class TopLeftCornerErase:
    def __init__(self, n_pixels: int):
        self.n_pixels = n_pixels
    
    def __call__(self, img: torch.Tensor) -> torch.Tensor:
        all_pixels = img.reshape(3, -1).transpose(1, 0)
        idx = torch.randint(len(all_pixels), (1,))[0]
        random_pixel = all_pixels[idx][:, None, None]
        return F.erase(img, 0, 0, self.n_pixels, self.n_pixels, random_pixel)

In the constructor, all we do is take the number of pixels as a parameter from the caller. The magic happens in the __call__() method:

  1. Create a reshaped view of the image tensor as a (n_pixels, 3) tensor
  2. Randomly select a pixel index using torch.randint()
  3. Add two dummy dimensions to the tensor. This is because F.erase() and to the image, which has these two dimensions.
  4. Call and return F.erase(), which takes five arguments: the tensor, the i coordinate to start at, the j coordinate to start at, the height of the box to erase, the width of the box to erase and the random pixel.

We can apply this custom transform just like any other transform. Let’s use T.Compose to both apply this erase transform and then convert it to NumPy for plotting:

torchvision transforms on a cat with corner erased
torch.manual_seed(1)

erase = T.Compose([
    TopLeftCornerErase(100),
    reverse_preprocess,
])

plt.imshow(erase(x));

We’ve seen this type of transform composition multiple times now. One thing that is important to point out is that you need to call torch.manual_seed() if you want a deterministic (and therefore reproducible) result for any TorchVision transform that has random behavior in it. This is new as of version 0.8.0.

And that’s about all there is to know about TorchVision transforms! They’re lightweight and flexible, but using them will make your image preprocessing code much easier to reason about.

The post TorchVision Transforms: Image Preprocessing in PyTorch appeared first on Sparrow Computing.

October 21, 2021 01:51 PM UTC

October 20, 2021


Ben Cook

Poetry for Package Management in Machine Learning Projects

When you’re building a production machine learning system, reproducibility is a proxy for the effectiveness of your development process. But without locking all your Python dependencies, your builds are not actually repeatable. If you work in a Python project without locking long enough, you will eventually get a broken build because of a transitive dependency (that is, a dependency of a dependency).

But a broken build isn’t the most dangerous problem. A transitive dependency can change in a way that affects an algorithm’s results without you ever knowing it. Imagine being unable to reproduce an algorithm result because an ancestor dependency that never got pinned changes in an unexpected way. Not good.

Enter Poetry. Poetry is an environment management tool for Python projects. Admittedly, there are a lot of environment management tools for Python and there’s no guarantee that Poetry will win. But the Python development community does seem to be slowly standardizing around Poetry.

What is Poetry?

Virtual environment tools like virtualenv and conda have been an important part of modern Python development for a while. But there are a few problems:

Ideally, you want all your builds to be precisely repeatable and you want a straightforward solution that you can use across all your projects. This is where Poetry shines.

Think of Poetry like npm for Python. It’s centered around pyproject.toml, a single config file that defines your entire Python environment and it comes with a lock file, poetry.lock, which pins all dependencies (including transitive dependencies).

Poetry configuration

Poetry extends the pyproject.toml file (introduced in PEP 517) to specify build requirements, project dependencies and development dependencies. It also includes project-level configuration (things like the name, description and license), scripts, extra dependencies and more.

Importantly, the Poetry spec for pyproject.toml works for both Python packages and Python applications, which prevents you from using a different approach across Python projects. This is important for machine learning development because reusable utilities like config and basic calculations like bounding box math often fit better in Python packages than in your actual machine learning application.

Lock file

The poetry.lock file is similar in concept to running pip freeze > requirements.txt every time you add or remove a package. It’s just that Poetry manages this for you when you add or remove a dependency.

And Poetry comes with a good dependency resolver. Since some packages on PyPI don’t specify all dependencies in their metadata, it can make adding a new package a little slower in Poetry. But it does ensure that you won’t get broken builds down the road as dependencies evolve, which is important.

Quick Start

To install the latest Poetry into your Python environment, run:

curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python

After installation, the script will print instructions to your screen to add poetry to your path.

To initialize a new project, run:

poetry init

This will guide you interactively through the process of setting up a new Python project with Poetry. You will add information about the project, the version of Python to use and (optionally) dependencies and development dependencies.

Say you want to add a dependency, like NumPy, after your project has been initialized. You can do that with the poetry add command:

poetry add numpy@~1.20

This will install NumPy with version >=1.20.0,<1.21.0. Alternatively, you could run poetry add numpy@^1.20 to install version >=1.20.0,<2.0.0. When you run this command Poetry does a few things:

  1. Update pyproject.toml to specify the new dependency.
  2. Use the dependency resolver to find the set of package versions that best fit the configuration. In this case, just numpy 1.20.3, but this will also include other packages if they’re specified.
  3. Install all the packages found by the resolver. By default, these will be installed into a virtual environment managed by Poetry. But you can also tell Poetry to install dependencies directly to the system Python.
  4. Freeze all dependencies and save them to poetry.lock so the exact build can be repeated in the future.

If you add the --dev flag to the poetry add command, Poetry treats the dependency as a development dependency. For example:

poetry add black --dev

This distinction is useful because for production builds, you can install all non-development dependencies by adding the --no-dev flag:

poetry install --no-dev

This prevents unnecessary bloat in your production environment where you don’t need tools like formatters and linters.

Finally, you can enter a shell with the virtual environment managed by Poetry activated:

poetry shell
python

Recommended Setup

As mentioned above, Poetry manages virtual environments for you and gives you a dedicated shell. But for production machine learning projects, I think it’s usually better to develop inside a Docker container.

In this case, I recommend turning off Poetry’s virtual environment management. You can do that with a config command:

poetry config virtualenvs.create false

Or you can use an environment variable, for example:

POETRY_VIRTUALENVS_CREATE=false poetry install

And now, you should know everything you need to get started using Poetry to manage your Python environment machine learning projects! Of course, there’s a ton more to learn about Poetry. Fortunately, their documentation is pretty good.

Happy ML engineering!

The post Poetry for Package Management in Machine Learning Projects appeared first on Sparrow Computing.

October 20, 2021 09:33 PM UTC


Real Python

Using the len() Function in Python

In many situations, you’ll need to find the number of items stored in a data structure. Python’s built-in function len() is the tool that will help you with this task.

There are some cases in which the use of len() is straightforward. However, there are other times when you’ll need to understand how this function works in more detail and how to apply it to different data types.

In this tutorial, you’ll learn how to:

  • Find the length of built-in data types using len()
  • Use len() with third-party data types
  • Provide support for len() with user-defined classes

By the end of this article, you’ll know when to use the len() Python function and how to use it effectively. You’ll know which built-in data types are valid arguments for len() and which ones you can’t use. You’ll also understand how to use len() with third-party types, such as ndarray in NumPy and DataFrame in pandas, and with your own classes.

Free Bonus: Click here to get a Python Cheat Sheet and learn the basics of Python 3, like working with data types, dictionaries, lists, and Python functions.

Getting Started With Python’s len()

The function len() is one of Python’s built-in functions. It returns the length of an object. For example, it can return the number of items in a list. You can use the function with many different data types. However, not all data types are valid arguments for len().

You can start by looking at the help for this function:

>>>
>>> help(len)
Help on built-in function len in module builtins:
len(obj, /)
    Return the number of items in a container.

The function takes an object as an argument and returns the length of that object. The documentation for len() goes a bit further:

Return the length (the number of items) of an object. The argument may be a sequence (such as a string, bytes, tuple, list, or range) or a collection (such as a dictionary, set, or frozen set). (Source)

When you use built-in data types and many third-party types with len(), the function doesn’t need to iterate through the data structure. The length of a container object is stored as an attribute of the object. The value of this attribute is modified each time items are added to or removed from the data structure, and len() returns the value of the length attribute. This ensures that len() works efficiently.

In the following sections, you’ll learn about how to use len() with sequences and collections. You’ll also learn about some data types that you cannot use as arguments for the len() Python function.

Using len() With Built-in Sequences

A sequence is a container with ordered items. Lists, tuples, and strings are three of the basic built-in sequences in Python. You can find the length of a sequence by calling len():

>>>
>>> greeting = "Good Day!"
>>> len(greeting)
9

>>> office_days = ["Tuesday", "Thursday", "Friday"]
>>> len(office_days)
3

>>> london_coordinates = (51.50722, -0.1275)
>>> len(london_coordinates)
2

When finding the length of the string greeting, the list office_days, and the tuple london_coordinates, you use len() in the same manner. All three data types are valid arguments for len().

The function len() always returns an integer as it’s counting the number of items in the object that you pass to it. The function returns 0 if the argument is an empty sequence:

>>>
>>> len("")
0
>>> len([])
0
>>> len(())
0

In the examples above, you find the length of an empty string, an empty list, and an empty tuple. The function returns 0 in each case.

A range object is also a sequence that you can create using range(). A range object doesn’t store all the values but generates them when they’re needed. However, you can still find the length of a range object using len():

>>>
>>> len(range(1, 20, 2))
10

This range of numbers includes the integers from 1 to 19 with increments of 2. The length of a range object can be determined from the start, stop, and step values.

In this section, you’ve used the len() Python function with strings, lists, tuples, and range objects. However, you can also use the function with any other built-in sequence.

Read the full article at https://realpython.com/len-python-function/ »


[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

October 20, 2021 02:00 PM UTC


Python for Beginners

Iterator in Python

You must have used different data structures like python dictionary, list, tuple and set while programming. We often need to access the elements of these data structures in a sequential manner. To iterate these data structures sequentially, we generally use for loops and index of elements. In this article, we will try to understand how we can access elements of a list or a tuple without using a for loop or  the indices of the elements. So, let’s dive into the concept of iterator and iterables in Python.

What is an iterable in Python? 

A container object like a list or set can contain a lot of elements. If we can access the member elements of the container objects one at a time then the container object is called an iterable. In our programs we use different iterables like list, tuple, set, or dictionary. 

Elements of any iterable can be accessed one at a time using a for loop as follows.

myList = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
print("Elements of the list are:")
for element in myList:
    print(element)

Output:

Elements of the list are:
1
2
3
4
5
6
7
8
9
10

In a for loop, we access all the elements from start till end. But, Consider a situation where we have to access the next element of the iterable only when some specific event occurs. 

For example, Let us take a scenario in which we have a list of 100 elements. We continually ask the user to input a number and whenever they enter an even number, we print the next element of the list. 

Now, this cannot be done using a for loop as we cannot predict when the user will enter an even number.  Having no order or sequence in the inputs stops us from using the for loop to iterate the list in our program. Iterators can be a handy tool to access the elements of iterables in such situations. So, Let’s learn what an iterator is and how we can create an iterator in python to access the elements of an iterable.

What is an iterator in Python?

An iterator is an object that can be iterated upon. In other words, we can  access all the elements in an iterable object using an iterator. 

In python, we can create an iterator for any container object using the iter() method. The iter() method takes an iterable object as input and returns an iterator for the same object.

myList = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
myIter = iter(myList)
print("list is:", myList)
print("Iterator for the list is:", myIter)

Output:

list is: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
Iterator for the list is: <list_iterator object at 0x7f4c21734070>

In the output, you can see that a list_iterator object has been created by passing a list to the iter() function.

How to traverse an iterator in Python?

The simplest way to traverse an iterator is by using a for loop. We can access each element in the iterator using a for loop as follows.

myList = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
myIter = iter(myList)
print("list is:", myList)
print("Elements in the iterator are:")
for element in myIter:
    print(element)

Output:

list is: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
Elements in the iterator are:
1
2
3
4
5
6
7
8
9
10

As discussed earlier, for loop won’t work if we have no order or sequence for traversing the iterator. For this, we can use two ways.

The first way to access an element of the iterator is by using the __next__() method. The __next__() method, when invoked on the iterator, returns an element next to the previously traversed element. It always retains the information about the element which was returned last time and whenever it is invoked, it returns only the next element which has not been traversed yet. We can understand this using the following example.

myList = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
myIter = iter(myList)
print("list is:", myList)
print("Elements in the iterator are:")
try:
    print(myIter.__next__())
    print(myIter.__next__())
    print(myIter.__next__())
    print(myIter.__next__())
    print(myIter.__next__())
    print(myIter.__next__())
    print(myIter.__next__())
    print(myIter.__next__())
    print(myIter.__next__())
    print(myIter.__next__())
    print(myIter.__next__())
    print(myIter.__next__())
except StopIteration as e:
    print("All elements in the iterator already traversed. Raised exception", e)

Output:

list is: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
Elements in the iterator are:
1
2
3
4
5
6
7
8
9
10
All elements in the iterator already traversed. Raised exception 

Another way to access the elements of the iterator is by using the next() function. The next() function takes the iterator as input and returns the next element that has not been traversed yet, just like the __next__() method. 

myList = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
myIter = iter(myList)
print("list is:", myList)
print("Elements in the iterator are:")
try:
    print(next(myIter))
    print(next(myIter))
    print(next(myIter))
    print(next(myIter))
    print(next(myIter))
    print(next(myIter))
    print(next(myIter))
    print(next(myIter))
    print(next(myIter))
    print(next(myIter))
    print(next(myIter))
except StopIteration as e:
    print("All elements in the iterator already traversed. Raised exception", e)

Output:

list is: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
Elements in the iterator are:
1
2
3
4
5
6
7
8
9
10
All elements in the iterator already traversed. Raised exception 

You can observe from the above two examples that the functioning of the __next__() method and next() function are almost similar. Also, both the next() function and the __next__() method raise an StopIteration error when all the elements of the iterator are already traversed. So, it is advised to use exception handling using python try except blocks.

Example use case of an iterator

Now let us take a closer look at the situation discussed above where we have to access the elements from a list only when the user gives an even number as input. 

Now, we have the next() function that can access the element from an iterator whenever we call the function. We can use it to access the elements from the list. For this, we will create an iterator for the list and then we will access the elements from the list using the next() function whenever the user gives an even number as input. We can implement a python function for this as follows.

myList = range(0, 101)
myIter = iter(myList)
while True:
    try:
        user_input = int(input("Input an even number to get an output, 0 to exit:"))
        if user_input == 0:
            print("Good Bye")
            break
        elif user_input % 2 == 0:
            print("Very good. Here is an output for you.")
            print(next(myIter))
        else:
            print("Input an even number.")
            continue
    except StopIteration as e:
        print("All the output has been exhausted.")
        

Output:

Input an even number to get an output, 0 to exit:1
Input an even number.
Input an even number to get an output, 0 to exit:2
Very good. Here is an output for you.
0
Input an even number to get an output, 0 to exit:4
Very good. Here is an output for you.
1
Input an even number to get an output, 0 to exit:3
Input an even number.
Input an even number to get an output, 0 to exit:6
Very good. Here is an output for you.
2
Input an even number to get an output, 0 to exit:0
Good Bye

Conclusion

In this article, we have discussed how we can access elements from different iterable objects using an iterator in Python. To learn more about python programming, you can read this article on list comprehension. You may also like this article on the linked list in Python.

The post Iterator in Python appeared first on PythonForBeginners.com.

October 20, 2021 01:41 PM UTC


PyCharm

PyCharm 2021.2.3 Is Out!

The third minor release of PyCharm 2021.2 contains multiple bug fixes:

Download PyCharm 2021.2.3

For the full list of issues addressed in PyCharm 2021.2.3, please see the release notes.
Found a bug? Please report it using our bug tracker.

October 20, 2021 01:36 PM UTC


Python Software Foundation

Announcing Python Software Foundation Fellow Members for Q3 2021! 🎉

The PSF is pleased to announced its third batch of PSF Fellows for 2021! Let us welcome the new PSF Fellows for Q3! The following people continue to do amazing things for the Python community:

Anthony Sottile

Twitch, YouTube, GitHub Sponsors, Twitter

Bernát Gábor

Twitter, Website, GitHub

Cristián Danilo Maureira-Fredes

Website, Twitter, GitHub

Michael Iyanda

LinkedIn, GitHub

Nicolás Demarchi

Twitter, GitHub, LinkedIn


Thank you for your continued contributions. We have added you to our Fellow roster online.

The above members help support the Python ecosystem by being phenomenal leaders, sustaining the growth of the Python scientific community, maintaining virtual Python communities, maintaining Python libraries, creating educational material, organizing Python events and conferences, starting Python communities in local regions, and overall being great mentors in our community. Each of them continues to help make Python more accessible around the world. To learn more about the new Fellow members, check out their links above.

Let's continue recognizing Pythonistas all over the world for their impact on our community. The criteria for Fellow members is available online: https://www.python.org/psf/fellows/. If you would like to nominate someone to be a PSF Fellow, please send a description of their Python accomplishments and their email address to psf-fellow at python.org. We are accepting nominations for quarter 4 through November 20, 2021.

Are you a PSF Fellow and want to help the Work Group review nominations? Contact us at psf-fellow at python.org.

October 20, 2021 11:26 AM UTC


Python GUIs

[updated for PyQt6] Creating your first app with PyQt — A simple Hello World! application with Python and Qt5

PyQt is a Python library for creating GUI applications using the Qt toolkit. Created by Riverbank Computing, PyQt is free software (GPL licensed) and has been in development since 1999. The latest version PyQt6 -- based on Qt 6 -- was released in 2021 and the library continues to be updated.

There are two major versions currently in use: PyQt5 based on Qt5 and PyQt6 based on Qt6. Both versions are almost completely compatible aside from imports, and lack of support for some advanced modules in Qt6. PyQt6 also makes some changes to how namespaces and flags work, but these are easily manageable.

In this tutorial we'll learn how to use PyQt to create desktop applications with Python.

First we'll create a series of simple windows on your desktop to ensure that PyQt is working and introduce some of the basic concepts. Then we'll take a brief look at the event loop and how it relates to GUI programming in Python. Finally we'll look at Qt's QMainWindow which offers some useful common interface elements such as toolbars and menus. These will be explored in more detail in the subsequent tutorials.

This PyQt tutorial features both PyQt5 and PyQt6 code examples. You can use whichever you prefer, but currently there are more examples online for PyQt5.

Creating an application

Let's create our first application! To start create a new Python file — you can call it whatever you like (e.g. app.py) and save it somewhere accessible. We'll write our simple app in this file.

We'll be editing within this file as we go along, and you may want to come back to earlier versions of your code, so remember to keep regular backups.

The source code for the application is shown below. Type it in verbatim, and be careful not to make mistakes. If you do mess up, Python will let you know what's wrong.

python
from PyQt5.QtWidgets import QApplication, QWidget

# Only needed for access to command line arguments
import sys

# You need one (and only one) QApplication instance per application.
# Pass in sys.argv to allow command line arguments for your app.
# If you know you won't use command line arguments QApplication([]) works too.
app = QApplication(sys.argv)

# Create a Qt widget, which will be our window.
window = QWidget()
window.show()  # IMPORTANT!!!!! Windows are hidden by default.

# Start the event loop.
app.exec()


# Your application won't reach here until you exit and the event
# loop has stopped.

python
from PyQt6.QtWidgets import QApplication, QWidget

# Only needed for access to command line arguments
import sys

# You need one (and only one) QApplication instance per application.
# Pass in sys.argv to allow command line arguments for your app.
# If you know you won't use command line arguments QApplication([]) works too.
app = QApplication(sys.argv)

# Create a Qt widget, which will be our window.
window = QWidget()
window.show()  # IMPORTANT!!!!! Windows are hidden by default.

# Start the event loop.
app.exec()


# Your application won't reach here until you exit and the event
# loop has stopped.

First, launch your application. You can run it from the command line like any other Python script, for example --

bash
python3 app.py

Run it! You will now see your window. Qt automatically creates a window with the normal window decorations and you can drag it around and resize it like any window.

What you'll see will depend on what platform you're running this example on. The image below shows the window as displayed on Windows, macOS and Linux (Ubuntu).

Our window, as seen on Windows, macOS and Linux. Our window, as seen on Windows, macOS and Linux.

Stepping through the code

Let's step through the code line by line, so we understand exactly what is happening.

First, we import the PyQt classes that we need for the application. Here we're importing QApplication, the application handler and QWidget, a basic empty GUI widget, both from the QtWidgets module.

python
from PyQt5.QtWidgets import QApplication, QWidget
python
from PyQt6.QtWidgets import QApplication, QWidget

The main modules for Qt are QtWidgets, QtGui and QtCore.

You could do from <module> import * but this kind of global import is generally frowned upon in Python, so we'll avoid it here.

Next we create an instance of QApplication, passing in sys.arg, which is Python list containing the command line arguments passed to the application.

python
app = QApplication(sys.argv)

If you know you won't be using command line arguments to control Qt you can pass in an empty list instead, e.g.

python
app = QApplication([])

Next we create an instance of a QWidget using the variable name window.

python
window = QWidget()
window.show()

In Qt all top level widgets are windows -- that is, they don't have a parent and are not nested within another widget or layout. This means you can technically create a window using any widget you like.

Widgets without a parent are invisible by default. So, after creating the window object, we must always call .show() to make it visible. You can remove the .show() and run the app, but you'll have no way to quit it!

What is a window? - Holds the user-interface of your application - Every application needs at least one (...but can have more) - Application will (by default) exit when last window is closed

Finally, we call app.exec() to start up the event loop.

In PyQt5 you can also use app.exec_(). This was a legacy feature avoid a clash with the exec reserved word in Python 2. It has been removed in PyQt6.

What's the event loop?

Before getting the window on the screen, there are a few key concepts to introduce about how applications are organized in the Qt world. If you're already familiar with event loops you can safely skip to the next section.

The core of every Qt Applications is the QApplication class. Every application needs one — and only one — QApplication object to function. This object holds the event loop of your application — the core loop which governs all user interaction with the GUI.

The event loop in Qt.

Each interaction with your application — whether a press of a key, click of a mouse, or mouse movement — generates an event which is placed on the event queue. In the event loop, the queue is checked on each iteration and if a waiting event is found, the event and control is passed to the specific event handler for the event. The event handler deals with the event, then passes control back to the event loop to wait for more events. There is only one running event loop per application.

The QApplication class - QApplication holds the Qt event loop - One QApplication instance required - You application sits waiting in the event loop until an action is taken - There is only one event loop running at any time

QMainWindow

As we discovered in the last part, in Qt any widgets can be windows. For example, if you replace QtWidget with QPushButton. In the example below, you would get a window with a single push-able button in it.

python
import sys
from PyQt5.QtWidgets import QApplication, QPushButton

app = QApplication(sys.argv)

window = QPushButton("Push Me")
window.show()

app.exec()
python
import sys
from PyQt6.QtWidgets import QApplication, QPushButton

app = QApplication(sys.argv)

window = QPushButton("Push Me")
window.show()

app.exec()

This is neat, but not really very useful -- it's rare that you need a UI that consists of only a single control! But, as we'll discover later, the ability to nest widgets within other widgets using layouts means you can construct complex UIs inside an empty QWidget.

But, Qt already has a solution for you -- the QMainWindow. This is a pre-made widget which provides a lot of standard window features you'll make use of in your apps, including toolbars, menus, a statusbar, dockable widgets and more. We'll look at these advanced features later, but for now, we'll add a simple empty QMainWindow to our application.

python
import sys
from PyQt5.QtWidgets import QApplication, QMainWindow

app = QApplication(sys.argv)

window = QMainWindow()
window.show()

# Start the event loop.
app.exec()

python
import sys
from PyQt6.QtWidgets import QApplication, QMainWindow

app = QApplication(sys.argv)

window = QMainWindow()
window.show()

# Start the event loop.
app.exec()

Run it! You will now see your main window. It looks exactly the same as before!

So our QMainWindow isn't very interesting at the moment. We can fix that by adding some content. If you want to create a custom window, the best approach is to subclass QMainWindow and then include the setup for the window in the __init__ block. This allows the window behavior to be self contained. We can add our own subclass of QMainWindow — call it MainWindow to keep things simple.

python
import sys

from PyQt5.QtCore import QSize, Qt
from PyQt5.QtWidgets import QApplication, QMainWindow, QPushButton


# Subclass QMainWindow to customize your application's main window
class MainWindow(QMainWindow):
    def __init__(self):
        super().__init__()

        self.setWindowTitle("My App")
        button = QPushButton("Press Me!")

        # Set the central widget of the Window.
        self.setCentralWidget(button)


app = QApplication(sys.argv)

window = MainWindow()
window.show()

app.exec()

python
import sys

from PyQt6.QtCore import QSize, Qt
from PyQt6.QtWidgets import QApplication, QMainWindow, QPushButton


# Subclass QMainWindow to customize your application's main window
class MainWindow(QMainWindow):
    def __init__(self):
        super().__init__()

        self.setWindowTitle("My App")
        button = QPushButton("Press Me!")

        # Set the central widget of the Window.
        self.setCentralWidget(button)


app = QApplication(sys.argv)

window = MainWindow()
window.show()

app.exec()

For this demo we're using a QPushButton. The core Qt widgets are always imported from the QtWidgets namespace, as are the QMainWindow and QApplication classes. When using QMainWindow we use .setCentralWidget to place a widget (here a QPushButton) in the QMainWindow -- by default it takes the whole of the window. We'll look at how to add multiple widgets to windows in the layouts tutorial.

When you subclass a Qt class you must always call the super __init__ function to allow Qt to set up the object.

In our __init__ block we first use .setWindowTitle() to change the title of our main window. Then we add our first widget — a QPushButton — to the middle of the window. This is one of the basic widgets available in Qt. When creating the button you can pass in the text that you want the button to display.

Finally, we call .setCentralWidget() on the window. This is a QMainWindow specific function that allows you to set the widget that goes in the middle of the window.

Run it! You will now see your window again, but this time with the QPushButton widget in the middle. Pressing the button will do nothing, we'll sort that next.

Our QMainWindow with a single QPushButton on Windows, macOS and Linux. Our QMainWindow with a single QPushButton on Windows, macOS and Linux.

We'll cover more widgets in detail shortly but if you're impatient and would like to jump ahead you can take a look at the http://doc.qt.io/qt-5/widget-classes.html#basic-widget-classes[QWidget documentation]. Try adding the different widgets to your window!

Sizing windows and widgets

The window is currently freely resizable -- if you grab any corner with your mouse you can drag and resize it to any size you want. While it's good to let your users resize your applications, sometimes you may want to place restrictions on minimum or maximum sizes, or lock a window to a fixed size.

In Qt sizes are defined using a QSize object. This accepts width and height parameters in that order. For example, the following will create a fixed size window of 400x300 pixels.

python
import sys

from PyQt5.QtCore import QSize, Qt
from PyQt5.QtWidgets import QApplication, QMainWindow, QPushButton


# Subclass QMainWindow to customize your application's main window
class MainWindow(QMainWindow):
    def __init__(self):
        super().__init__()

        self.setWindowTitle("My App")

        button = QPushButton("Press Me!")

        self.setFixedSize(QSize(400, 300))

        # Set the central widget of the Window.
        self.setCentralWidget(button)


app = QApplication(sys.argv)

window = MainWindow()
window.show()

app.exec()
python
import sys

from PyQt6.QtCore import QSize, Qt
from PyQt6.QtWidgets import QApplication, QMainWindow, QPushButton


# Subclass QMainWindow to customize your application's main window
class MainWindow(QMainWindow):
    def __init__(self):
        super().__init__()

        self.setWindowTitle("My App")

        button = QPushButton("Press Me!")

        self.setFixedSize(QSize(400, 300))

        # Set the central widget of the Window.
        self.setCentralWidget(button)


app = QApplication(sys.argv)

window = MainWindow()
window.show()

app.exec()

Run it! You will see a fixed size window -- try and resize it, it won't work.

Our fixed-size window, notice that the _maximize_ control is disabled on Windows & Linux. On macOS you _can_ maximize the app to fill the screen, but the central widget will not resize. Our fixed-size window, notice that the _maximize control is disabled on Windows & Linux. On macOS you can maximize the app to fill the screen, but the central widget will not resize._

As well as .setFixedSize() you can also call .setMinimumSize() and .setMaximumSize() to set the minimum and maximum sizes respectively. Experiment with this yourself!

You can use these size methods on any widget.

In this section we've covered the QApplication class, the QMainWindow class, the event loop and experimented with adding a simple widget to a window. In the next section we'll take a look at the mechanisms Qt provides for widgets and windows to communicate with one another and your own code.

For more, see the complete PyQt5 tutorial.

October 20, 2021 09:00 AM UTC


Python Bytes

#255 Closember eve, the cure for Hacktoberfest?

<p><strong>Watch the live stream:</strong></p> <a href='https://www.youtube.com/watch?v=ICp8eD6uENI' style='font-weight: bold;'>Watch on YouTube</a><br> <br> <p><strong>About the show</strong></p> <p>Sponsored by <strong>us:</strong></p> <ul> <li>Check out the <a href="https://training.talkpython.fm/courses/all"><strong>courses over at Talk Python</strong></a></li> <li>And <a href="https://pythontest.com/pytest-book/"><strong>Brian’s book too</strong></a>!</li> </ul> <p>Special guest: <strong>Will McGugan</strong></p> <p><strong>Michael #1:</strong> <a href="https://azhpushkin.me/posts/cython-cpp-intro"><strong>Wrapping C++ with Cython</strong></a></p> <ul> <li>By <a href="https://azhpushkin.me/">Anton Zhdan-Pushkin</a></li> <li>A small series showcasing the implementation of a Cython wrapper over a C++ library.</li> <li>C library: <strong>yaacrl - Y</strong>et <strong>A</strong>nother <strong>A</strong>udio <strong>R</strong>ecognition <strong>L</strong>ibrary is a small Shazam-like library, which can recognize songs using a small recorded fragment.</li> <li>For Cython to consume yaacrl correctly, we need to “teach” it about the API using `cdef extern </li> <li>It is convenient to put such declarations in <code>*.pxd</code> files.</li> <li>One of the first features of Cython that I find extremely useful — aliasing. With aliasing, we can use names like <code>Storage</code> or <code>Fingerprint</code> for Python classes without shadowing original C++ classes.</li> <li><strong>Implementing a wrapper: pyaacrl</strong> - The most common way to wrap a C++ class is to use <a href="https://docs.python.org/3/extending/newtypes_tutorial.html">Extension types</a>. As an extension type a just a C struct, it can have an underlying C++ class as a field and act as a proxy to it.</li> <li>Cython documentation has a whole page dedicated to the pitfalls of <a href="https://cython.readthedocs.io/en/latest/src/userguide/wrapping_CPlusPlus.html">“Using C++ in Cython</a>.”</li> <li>Distribution is hard, but there is a tool that is designed specifically for such needs: scikit-build.</li> <li><a href="https://pybind11.readthedocs.io/en/stable/"><strong>PyBind11 too</strong></a></li> </ul> <p><strong>Brian #2:</strong> <a href="https://github.com/dmerejkowsky/tbump"><strong>tbump</strong></a> <a href="https://github.com/dmerejkowsky/tbump"></a><a href="https://github.com/dmerejkowsky/tbump"><strong>: bump software releases</strong></a></p> <ul> <li><a href="https://twitter.com/geosephi/status/1437697175987920896">suggested by Sephi Berry</a></li> <li>limits the manual process of updating a project version</li> <li><code>tbump init 1.2.2</code> initializes a <code>tbump.toml</code> file with customizable settings <ul> <li><code>--pyproject</code> will append to <code>pyproject.toml</code> instead</li> </ul></li> <li><code>tbump 1.2.3</code> will <ul> <li>patch files: wherever the version listed</li> <li>(optional) run configured commands before commit <ul> <li>failing commands stop the bump.</li> </ul></li> <li>commit the changes with a configurable message</li> <li>add a version tag</li> <li>push code</li> <li>push tag</li> <li>(optional) run post publish command</li> <li>Tell you what it’s going to do before it does it. (can opt out of this check)</li> </ul></li> <li>pretty much everything is customizable and configurable.</li> <li>I tried this on a flit based project. Only required one change</li> </ul> <pre><code> # For each file to patch, add a [[file]] config # section containing the path of the file, relative to the # tbump.toml location. [[file]] src = "pytest_srcpaths.py" search = '__version__ = "{current_version}"' </code></pre> <ul> <li>cool example of a pre-commit check:</li> </ul> <pre><code> # [[before_commit]] # name = "check changelog" # cmd = "grep -q {new_version} Changelog.rst" </code></pre> <p><strong>Will #3:</strong> <strong><a href="https://closember.org/">Closember</a></strong> by <strong>Matthias Bussonnier</strong></p> <p><strong>Michael #4:</strong> <a href="https://twitter.com/btskinn/status/1441597030934011909?s=12"><strong>scikit learn goes 1.0</strong></a></p> <ul> <li>via Brian Skinn</li> <li>The library has been stable for quite some time, releasing version 1.0 is recognizing that and signalling it to our users.</li> <li>Features:</li> <li>Keyword and positional arguments - To improve the readability of code written based on scikit-learn, now users have to provide most parameters with their names, as keyword arguments, instead of positional arguments.</li> <li>Spline Transformers - One way to add nonlinear terms to a dataset’s feature set is to generate spline basis functions for continuous/numerical features with the new <a href="https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.SplineTransformer.html#sklearn.preprocessing.SplineTransformer">SplineTransformer</a>.</li> <li>Quantile Regressor - Quantile regression estimates the median or other quantiles of Y conditional on X</li> <li>Feature Names Support - When an estimator is passed a <a href="https://pandas.pydata.org/docs/user_guide/dsintro.html#dataframe">pandas’ dataframe</a> during <a href="https://scikit-learn.org/stable/glossary.html#term-fit">fit</a>, the estimator will set a <code>feature_names_in_</code> attribute containing the feature names.</li> <li>A more flexible plotting API</li> <li>Online One-Class SVM</li> <li>Histogram-based Gradient Boosting Models are now stable</li> <li>Better docs</li> </ul> <p><strong>Brian #5:</strong> <a href="https://blog.jaraco.com/devpi-as-offline-pypi-cache/"><strong>Using devpi as an offline PyPI cache</strong></a></p> <ul> <li>Jason R. Coombs</li> <li>This is the devpi tutorial I’ve been waiting for.</li> <li>Single machine local server mirror of PyPI (mirroring needs primed), usable in offline mode.</li> </ul> <pre><code> $ pipx install devpi-server $ devpi-init $ devpi-server </code></pre> <ul> <li>now in another window, prime the cache by grabbing whatever you need, with the index redirected</li> </ul> <pre><code> (venv) $ export PIP_INDEX_URL=http://localhost:3141/root/pypi/ (venv) $ pip install pytest, ... </code></pre> <ul> <li>then you can restart the server anytime, or even offline</li> </ul> <pre><code> $ devpi-server --offline </code></pre> <ul> <li>tutorial includes examples, proving how simple this is.</li> </ul> <p><strong>Will #6:</strong> <strong><a href="https://github.com/wasi-master/pypi-command-line">PyPi command line</strong> </a></p> <p><strong>Extras</strong></p> <p>Brian:</p> <ul> <li>I’ve started using pyenv on my Mac just for downloading Python versions. Verdict still out if I like it better than just downloading from pytest.org. </li> <li>Also started using <a href="https://starship.rs/">Starship</a> with no customizations so far. I’d like to hear from people if they have nice Starship customizations I should try.</li> <li><a href="https://vscode.dev/">vscode.dev</a> is a thing, <a href="https://code.visualstudio.com/blogs/2021/10/20/vscode-dev">announcement</a> just today </li> </ul> <p>Michael:</p> <ul> <li>PyCascades <a href="https://bit.ly/pycascades2022cfp"><strong>Call for Proposals</strong></a> is currently open </li> <li>Got your <a href="https://www.apple.com/macbook-pro-14-and-16/"><strong>M1 Max</strong></a>?</li> <li>Prediction: Tools like <a href="https://www.codeweavers.com/crossover"><strong>Crossover for Windows apps</strong></a> will become more of a thing.</li> </ul> <p>Will: </p> <ul> <li>GIL removal <ul> <li><a href="https://docs.google.com/document/u/0/d/18CXhDb1ygxg-YXNBJNzfzZsDFosB5e6BfnXLlejd9l0/mobilebasic?urp=gmail_link">https://docs.google.com/document/u/0/d/18CXhDb1ygxg-YXNBJNzfzZsDFosB5e6BfnXLlejd9l0/mobilebasic?urp=gmail_link</a></li> <li><a href="https://lwn.net/SubscriberLink/872869/0e62bba2db51ec7a/">https://lwn.net/SubscriberLink/872869/0e62bba2db51ec7a/</a></li> </ul></li> <li><a href="https:///vscode.dev">vscode.dev</a></li> </ul> <p><strong>Joke:</strong> </p> <ul> <li><a href="https://devops.com/the-torture-never-stops/"><strong>The torture never stops</strong></a></li> <li><a href="http://www.zaphinath.com/wp-content/uploads/2015/11/ie-eats-glue.jpg"><strong>IE (“Safari”) Eating Glue</strong></a></li> </ul>

October 20, 2021 08:00 AM UTC


Python Anywhere

Our October system update

On 6 October we upgraded our EU-based systems to the latest version of our platform, and today, 20 October, we did the same upgrade on our US-based system. There are a bunch of changes to report!

October 20, 2021 12:00 AM UTC

October 19, 2021


PyCoder’s Weekly

Issue #495 (Oct. 19, 2021)

#495 – OCTOBER 19, 2021
View in Browser »

The PyCoder’s Weekly Logo


No-GIL Fork of CPython

This is a proof-of-concept implementation of CPython that supports multithreading without the global interpreter lock (GIL), from Facebook research. An overview of the design is described in the Python Multithreading without the GIL Google doc. Also see the related discussions on LWN and Hacker News.
GITHUB.COM/COLESBURY • Shared by Henry Schreiner

Why You Shouldn’t Invoke setup.py Directly

“The setuptools team no longer wants to be in the business of providing a command line interface and is actively working to become just a library for building packages. What you should do instead depends on your use case, but if you want some basic rules of thumb, there is a table in the summary section.”
PAUL GANSSLE

Data Elixir: Data Science Newsletter

alt

Data Elixir is an email newsletter that keeps you on top of the tools and trends in Data Science. Covers machine learning, data visualization, analytics, and strategy. Curated weekly with top picks from around the web →
DATA ELIXIR sponsor

Where Does All the Effort Go? Looking at Python Core Developer Activity

“One of the tasks given me by the Python Software Foundation as part of the Developer in Residence job was to look at the state of CPython as an active software development project. What are people working on? Which standard libraries require most work? Who are the active experts behind which libraries?”
ŁUKASZ LANGA

Cool New Features in Python 3.10

In this course, you’ll explore some of the coolest and most useful features in Python 3.10. You’ll appreciate more user-friendly error messages, learn about how you can handle complicated data structures with structural pattern matching, and explore new enhancements to Python’s type system.
REAL PYTHON course

PyCascades 2022 CFP Closes on Sunday (Oct 24)

PYCASCADES CONFERENCE

Announcing PSF Fellow Members for Q3 2021

PYTHON SOFTWARE FOUNDATION

Join the Python Developers Survey 2021

PYTHON SOFTWARE FOUNDATION

Psycopg 3.0 Released

PSYCOPG.ORG

PyPy 7.3.6 Released

PYPY.ORG

Python Jobs

Senior Python Engineer @ Moody's AI & ML Center of Excellence (New York, NY, USA)

Moody's

Senior Software Engineer (Washington D.C., DC, USA)

Quorum

Senior Backend Software Engineer (Anywhere)

Clay

Full Stack Developer (Anywhere)

Level 12

Software Engineer (Anywhere)

1Point21 Interactive

More Python Jobs >>>

Articles & Tutorials

Tests Aren’t Enough: Case Study After Adding Type Hints to urllib3

“Since Python 3.5 was released in 2015 including PEP 484 and the typing module type hints have grown from a nice-to-have to an expectation for popular packages. To fulfill this expectation our team has committed to shipping type hints for the v2.0 milestone. What we didn’t realize is the amount of value we’d derive from this project in terms of code correctness.”
SETH MICHAEL LARSON

Welcoming the CPython Developer in Residence

Earlier this year, the Python Software Foundation announced the creation of the Developer in Residence role. The first Visionary Sponsors of the PSF have provided funding for this new role for one year. What development responsibilities does this job address? This week on the show, Łukasz Langa talks about becoming the first CPython Developer in Residence.
REAL PYTHON podcast

Accelerate Your Python Apps With Apache Cassandra™ NoSQL. Register for an Astra DB Demo

alt

Scale data for your Django, Flask, FastAPI apps with our multi-cloud, serverless DBaaS–built on Apache Cassandra™. Painless APIs, free for developers. Get 80 Gigabytes of Storage Free Every Month. Explore Astra DB now →
DATASTAX sponsor

Python Assignment Expressions and Using the Walrus Operator

In this course, you’ll learn about assignment expressions and the walrus operator. The biggest change in Python 3.8 was the inclusion of the := operator, which you can use to assign variables in the middle of expressions. You’ll see several examples of how to take advantage of this new feature.
REAL PYTHON course

A Roadmap to XML Parsers in Python

In this tutorial, you’ll learn what XML parsers are available in Python and how to pick the right parsing model for your specific use case. You’ll explore Python’s built-in parsers as well as major third-party libraries.
REAL PYTHON

How APT Does Its Fancy Progress Bar

“Today while running an apt full-upgrade I asked myself how apt does this nice progress bar stuck at the bottom line while still writing scrolling text.” Python example code included.
JULIEN PALARD

Configuration Is an API, Not an SDK

Guidelines for config management in general and for Python apps in particular. Why “Configuration is just another API of your app” might be a good philosophy to adopt.
HERNAN LOZANO • Shared by Hernan Lozano

Python’s property(): Add Managed Attributes to Your Classes

In this step-by-step tutorial, you’ll learn how to create managed attributes, also known as properties, using Python’s property() in your custom classes.
REAL PYTHON

Receive a $5 Donation to the OSS of Your Choice When You Deploy Your Free Scout APM Trial Today

Scout is performance monitoring designed to provide the data insights necessary for any dev to become a performance pro. Find and fix observability issues before your customers notice by connecting your error reporting and APM data on one platform.
SCOUT APM sponsor

Pip vs Conda: A Comparison of Python’s Two Packaging Systems

“Python has two commonly used packaging systems, pip and Conda. Learn the differences between them so you can pick the right one for you.”
ITAMAR TURNER-TRAURING

Secure Password Handling in Python

Protect and secure your passwords and credentials in Python with help of these techniques and tips.
MARTIN HEINZ • Shared by Martin Heinz

Understanding np.where()

The NumPy where() function is like a vectorized switch that you can use to combine two arrays.
BEN COOK

Three Tools to Profile a Django App

KRACEKUMAR.COM

More Uses for functools.partial() in Django

ADAM JOHNSON

Projects & Code

kubernetes-client: Official Python Client Library for Kubernetes

GITHUB.COM/KUBERNETES-CLIENT

Lenia: Mathematical Life Forms Simulator

GITHUB.COM/CHAKAZUL

troposphere: Create AWS CloudFormation Descriptions

GITHUB.COM/CLOUDTOOLS

classyconf: Declarative and Extensible Library for Configuration & Code Separation

GITHUB.COM/HERNANTZ

Events

Weekly Real Python Office Hours Q&A (Virtual)

October 20, 2021
REALPYTHON.COM

Inland Empire Pyladies (CA, USA)

October 25, 2021
MEETUP.COM

Introduction to the Python Programming Language (In Persian)

October 26, 2021
INSTAGRAM.COM

Python Sheffield

October 26, 2021
GOOGLE.COM

Dominican Republic Python User Group

October 26 to October 27, 2021
PYTHON.DO

PyKla Monthly Meetup

October 27, 2021
MEETUP.COM

Python Meeting Düsseldorf

October 27, 2021
PYDDF.DE

Heidelberg Python Meetup

October 27, 2021
MEETUP.COM

PyData Global 2021

October 28 to October 31, 2021
PYDATA.ORG

deploy by DigitalOcean

November 16 to November 17, 2021
DIGITALOCEAN


Happy Pythoning!
This was PyCoder’s Weekly Issue #495.
View in Browser »

alt

[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]

October 19, 2021 07:30 PM UTC


Peter Bengtsson

How to string pad a string in Python with a variable

I just have to write this down because that's the rule; if I find myself googling something basic like this more than once, it's worth blogging about.

Suppose you have a string and you want to pad with empty spaces. You have 2 options:

>>> s = "peter"
>>> s.ljust(10)
'peter     '
>>> f"{s:<10}"
'peter     '

The f-string notation is often more convenient because it can be combined with other formatting directives.
But, suppose the number 10 isn't hardcoded like that. Suppose it's a variable:

>>> s = "peter"
>>> width = 11
>>> s.ljust(width)
'peter      '
>>> f"{s:<width}"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Invalid format specifier

Well, the way you need to do it with f-string formatting, when it's a variable like that is this syntax:

>>> f"{s:<{width}}"
'peter      '

October 19, 2021 04:44 PM UTC


PyCon

PyCon US 2022 Website and Sponsorship Program Launch!

With PyCon US 2022 planning underway, we are excited to be launching the conference website along with our sponsorship program.

Our team is planning to host the event in person with an online component. Head over to the PyCon US 2022 website for details about the conference and more information about our sponsorship program. You will not want to miss the opportunity to be part of this event! If your organization depends on the Python ecosystem, check out our prospectus online, and sign up today!
 
The PSF's comprehensive sponsorship program allows organizations to support community programs financially and delivers a variety of benefits that provide visibility across PyCon US, pypi.org, python.org, and much more: 

Sponsorships have a tremendous impact!

Most importantly, the Python Software Foundation operates off sponsorships it receives. The more funding the PSF has to work with, the more we can do to support Python and its community. In 2021 the PSF launched two new initiatives that will have a tremendous impact on all users:

  1. In July, the PSF hired the inaugural Developer-in-Residence (Łukasz Langa) to support CPython. A full-time, paid core developer supporting dozens of volunteers that make Python happen has been an enormous help. In addition to supporting volunteers, Łukasz is analyzing usage and workflows to better address sustainability and future funding.

  2. In August, the PSF hired Shamika Mohanan to support Python packaging. Shamika will handle outreach to Python users so the PSF can better understand the landscape, identify fundable initiatives, seek grants, oversee funded projects, and report on their progress and results to improve Python packaging for all users.

The above would not have been possible without the generous support of our sponsors over the years. Please help us ensure these initiatives have funding for years to come.

Sponsor the PSF today!

If you have any questions about sponsoring PyCon US and the PSF, please get in touch with us at sponsors@python.org

As we get closer to the event, the conference website is where you’ll find details for our call for proposals, registration process, venue information, and everything PyCon US related!

October 19, 2021 03:27 PM UTC


Quansight Labs Blog

An efficient method of calling C++ functions from numba using clang++/ctypes/rbc

The aim of this document is to explore a method of calling C++ library functions from Numba compiled functions --- Python functions that are decorated with numba.jit(nopython=True).

While there exist ways to wrap C++ codes to Python, calling these wrappers from Numba compiled functions is often not as straightforward and efficient as one would hope.

Read more… (5 min remaining to read)

October 19, 2021 02:00 PM UTC


Real Python

Python Assignment Expressions and Using the Walrus Operator

Each new version of Python adds new features to the language. For Python 3.8, the biggest change is the addition of assignment expressions. Specifically, the := operator gives you a new syntax for assigning variables in the middle of expressions. This operator is colloquially known as the walrus operator.

This course is an in-depth introduction to the walrus operator. You’ll learn some of the motivations for the syntax update and explore some examples where assignment expressions can be useful.

In this course, you’ll learn how to:


[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

October 19, 2021 02:00 PM UTC