Planet Python

Last update: October 23, 2021 10:40 AM UTC

October 22, 2021

Ben Cook

TorchVision Datasets: Getting Started

The TorchVision datasets subpackage is a convenient utility for accessing well-known public image and video datasets. You can use these tools to start training new computer vision models very quickly.

TorchVision Datasets Example

To get started, all you have to do is import one of the Dataset classes. Then, instantiate it and access one of the samples with indexing:

from torchvision import datasets

dataset = datasets.MNIST(root="./", download=True)
img, label = dataset[10]
img.size

# Expected result
# (28, 28)

You’ll get a tuple with a Pillow image and an integer label back:

The TorchVision datasets implement __len__() and __getitem__() methods, which means that in addition to getting specific elements by index, you can also get the number of samples with the len() function:

len(dataset)

# Expected result
# 60000

Additionally, DataLoader classes can use TorchVision Dataset objects to create automatic batches for training.

Since they mostly return Pillow images, you do need to pass in a transform to convert the image to a tensor:

import torch
from torchvision import transforms

dataset = datasets.MNIST(
    root="./",
    transform=transforms.ToTensor()
)
data_loader = torch.utils.data.DataLoader(dataset, batch_size=4)

x, y = next(iter(data_loader))

x.shape

# Expected result
# torch.Size([4, 1, 28, 28])

API

The interface for the TorchVision Dataset classes is somewhat inconsistent because every dataset has a slightly different set of constraints. For example, many of the datasets return (PIL.Image, int) tuples, but this obviously wouldn’t work for videos (TorchVision packs them into tensors).

But generally speaking, the constructors take the following arguments:

• root: where to download the raw dataset or where the Dataset class should expect to find a raw dataset that has already been downloaded.
• split: which holdout to use. This can be train, test, val, extra… best to look at the docs for the dataset you want to use.
• download: a boolean indicating whether TorchVision should download the raw data for you. Although setting this argument to true will raise an error for datasets like ImageNet. More on this below.
• transform: a TorchVision transform to apply to the input image or video.

A word about ImageNet

ImageNet is no longer available for small companies or independent researchers. This is a real shame because pre-trained classifiers in model zoos are almost always trained on ImageNet.

However, it is possible to download most of the ImageNet dataset from Academic Torrents. I cannot endorse this strategy because I don’t know if it’s allowed.

If you did want to download the train and validation sets from ImageNet 2012, here are some steps you could follow:

1. Launch an Amazon Linux EC2 instance with at least 200GB of storage. The whole process takes about 2 hours on a c5.xlarge instance.

2. Install the aria2c command-line tool (instructions here).

3. Download the tar files:

# Download the validation set
aria2c https://academictorrents.com/download/dfa9ab2528ce76b907047aa8cf8fc792852facb9.torrent

# Download the train set
aria2c https://academictorrents.com/download/a306397ccf9c2ead27155983c254227c0fd938e2.torrent

4. Make sure the files match the MD5 hashes (helpfully provided by the TorchVision team):

# Check the validation file
md5sum ILSVRC2012_img_val.tar

# Expected result
# 29b22e2961454d5413ddabcf34fc5622

# Check the train file
md5sum ILSVRC2012_img_train.tar

# Expected result
# 1d675b47d978889d74fa0da5fadfb00e

5. Upload the files to S3 — hosting the files costs a little over $3 per month.

6. Terminate the instance.

7. Send the Academic Torrents team some Bitcoin to say thank you.

Summary

And that’s all you need to know to get started with TorchVision Datasets. For production machine learning pipelines, you probably want to implement your own Dataset class, but the datasets that come out of the box with TorchVision are a great way to experiment quickly!

The post TorchVision Datasets: Getting Started appeared first on Sparrow Computing.

October 22, 2021 09:59 PM UTC

Python for Beginners

Compare two lists in Python

While programming in python, comparison has to be done very often for checking different conditions. We may need to compare two variables or two groups of variables for a single condition checking. In this article we will try different ways to compare two lists in python. While comparing, we will have to check if both the lists contain the same elements or not irrespective of the order in which the elements are present in the lists. Accordingly, we will have to print the result.

Compare two lists using sort() method

To check whether two lists contain the same elements or not, we can use  the sort() method to sort the elements of the lists first. Then, we can compare the two lists.

For comparison,first we will check if the length of the lists are equal or not. If the lengths are not equal, the lists will be automatically considered as different. 

If the length of the lists are the same, we will sort the two lists. Then we will compare the lists using the == operator to check whether the lists are equal or not. If the sorted lists are equal, it will establish that both the original lists contain the same elements. This can be implemented as follows.

# function to compare lists
def compare(l1, l2):
    # here l1 and l2 must be lists
    if len(l1) != len(l2):
        return False
    l1.sort()
    l2.sort()
    if l1 == l2:
        return True
    else:
        return False


list1 = [1, 2, 3, 4]
list2 = [1, 4, 3, 2]
list3 = [2, 3, 4, 5]
print("list1 is:",list1)
print("list2 is:",list2)
print("list3 is:",list3)

# comparing list1 and list 2
print("list1 and list2 contain same elements:",compare(list1, list2))
#comparing list2 and list3
print("list1 and list3 contain same elements:",compare(list1, list3))

Output:

list1 is: [1, 2, 3, 4]
list2 is: [1, 4, 3, 2]
list3 is: [2, 3, 4, 5]
list1 and list2 contain same elements: True
list1 and list3 contain same elements: False

Compare using sets in Python

To compare two lists in python, we can use sets. A set in python only allows unique values in it. We can use this property of sets to find if two lists have the same elements or not.

For comparison,first we will check if the length of the lists are equal or not. If the lengths are not equal, the lists will be automatically flagged as different.

After that, we will convert the lists into sets using set() constructor. We can compare the two sets using the == operator to check if both the sets are equal or not. If both sets are equal, it will be established that both the lists contain equal values. Following example illustrates this concept.

# function to compare lists
def compare(l1, l2):
    # here l1 and l2 must be lists
    if len(l1) != len(l2):
        return False
    set1 = set(l1)
    set2 = set(l2)
    if set1 == set2:
        return True
    else:
        return False


list1 = [1, 2, 3, 4]
list2 = [1, 4, 3, 2]
list3 = [2, 3, 4, 5]
print("list1 is:", list1)
print("list2 is:", list2)
print("list3 is:", list3)

# comparing list1 and list 2
print("list1 and list2 contain same elements:", compare(list1, list2))
# comparing list2 and list3
print("list1 and list3 contain same elements:", compare(list1, list3))

Output:

list1 is: [1, 2, 3, 4]
list2 is: [1, 4, 3, 2]
list3 is: [2, 3, 4, 5]
list1 and list2 contain same elements: True
list1 and list3 contain same elements: False

Compare two lists using frequency counter

We can also compare two lists without comparing their lengths. For this, first we will have to create a python dictionary for each list which will keep track of the frequency of the elements in the lists. After creating the dictionaries where elements of the lists are stored as keys and their frequency are stored as values, we can compare the frequency of each element in the two dictionaries. If the frequency of each elements becomes equal, it will be confirmed that both the lists contained equal elements.

For this task, we can use the counter() method. The counter() method, when invoked on a list, creates a python dictionary and stores the elements as keys and their frequency as values in it. After invoking the counter() method, we can compare the created dictionary using == operator to check whether the frequency of every element is equal or not. If the result is True, the lists contain equal elements. Otherwise not. This can be seen from the following example.

import collections
# function to compare lists
def compare(l1, l2):
    # here l1 and l2 must be lists
    if len(l1) != len(l2):
        return False
    counter1 = collections.Counter(l1)
    counter2=collections.Counter(l2)
    if counter1 == counter2:
        return True
    else:
        return False


list1 = [1, 2, 3, 4]
list2 = [1, 4, 3, 2]
list3 = [2, 3, 4, 5]
print("list1 is:", list1)
print("list2 is:", list2)
print("list3 is:", list3)

# comparing list1 and list 2
print("list1 and list2 contain same elements:", compare(list1, list2))
# comparing list2 and list3
print("list1 and list3 contain same elements:", compare(list1, list3))

Output:

list1 is: [1, 2, 3, 4]
list2 is: [1, 4, 3, 2]
list3 is: [2, 3, 4, 5]
list1 and list2 contain same elements: True
list1 and list3 contain same elements: False

Conclusion

In this article, we have seen three different ways to compare two lists in python and have checked if they contain the same elements without considering the position of the elements. To read more about lists, read this article on list comprehension.

In the compare() function used in the examples, it may be possible that user passes two other objects instead of lists. In such cases, the program may run into error. To avoid this, we can use exception handling using python try except to avoid runtime errors by applying type checking using type() method in try-except block to check if the objects passed as arguments are lists.

The post Compare two lists in Python appeared first on PythonForBeginners.com.

October 22, 2021 12:58 PM UTC

Real Python

The Real Python Podcast – Episode #83: Ready to Publish Your Python Packages?

Are you interested in sharing your Python project with the broader world? Would you like to make it easily installable using pip? How do you create Python packages that share your code in a scalable and maintainable way? This week on the show, Real Python author and former guest Dane Hillard returns to talk about his new book, "Publishing Python Packages."

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

October 22, 2021 12:00 PM UTC

Stack Abuse

Integrating Matplotlib Charts in a PDF in Python With borb

Introduction

The Portable Document Format (PDF) is not a WYSIWYG (What You See is What You Get) format. It was developed to be platform-agnostic, independent of the underlying operating system and rendering engines.

To achieve this, PDF was constructed to be interacted with via something more like a programming language, and relies on a series of instructions and operations to achieve a result. In fact, PDF is based on a scripting language - PostScript, which was the first device-independent Page Description Language.

In this guide, we'll be using borb - a Python library dedicated to reading, manipulating and generating PDF documents. It offers both a low-level model (allowing you access to the exact coordinates and layout if you choose to use those) and a high-level model (where you can delegate the precise calculations of margins, positions, etc to a layout manager).

Matplotlib is a data visualization library that fueled an entire generation of engineers to start visualizing data, and the engine behind many other popular libraries such as Seaborn.

Given how common PDF documents are for creating reports (which oftentimes include graphs), we'll take a look at how to integrate Matplotlib charts in a PDF document using borb.

Installing borb (and Matplotlib)

borb can be downloaded from source on GitHub, or installed via pip:

$ pip install borb

Matplotlib can be installed via pip:

$ pip install matplotlib

Integrating Matplotlib Charts in PDF Documents with borb

Before we can create a chart, such as a Pie Chart, we're going to write a small utility function that generates N colors, evenly distributed among the color-spectrum.

This will help us whenever we need to create a plot and color each section:

from borb.pdf.canvas.color.color import HSVColor, HexColor
from decimal import Decimal
import typing

def create_n_colors(n: int) -> typing.List[str]:
  # The base color is borb-blue
  base_hsv_color: HSVColor = HSVColor.from_rgb(HexColor("56cbf9"))
  # This array comprehension creates n HSVColor objects, transforms then to RGB, and then returns their hex string
  return [HSVColor(base_hsv_color.hue + Decimal(x / 360), Decimal(1), Decimal(1)).to_rgb().to_hex_string() for x in range(0, 360, int(360/n))]

Note: HSL (hue, saturation, lightness) and HSV/HSB (hue, saturation, value/hue, saturation, brightness) are alternative representations of the RGB color model.

HSL and HSV/HSB were designed in the 1970s by computer graphics researchers to more closely align with the way human vision perceives color-making attributes. In these models, colors of each hue are arranged in a radial slice, around a central axis of neutral colors which ranges from black at the bottom to white at the top:

Credits: Wikimedia (CC BY-SA 3.0) license

The advantage of using this representation for Color is that we can easily divide the color-spectrum in equal parts.

Now we can define a create_pie_chart() function (or a function for other types of plots):

# New import(s)
import matplotlib.pyplot as plt
from borb.pdf.canvas.layout.image.chart import Chart
from borb.pdf.canvas.layout.layout_element import Alignment

def create_piechart(labels: typing.List[str], data: typing.List[float]):

  # Symetric figure to ensure equal aspect ratio
  fig1, ax1 = plt.subplots(figsize=(4, 4))
  ax1.pie(
    data,
    explode=[0 for _ in range(0, len(labels))],
    labels=labels,
    autopct="%1.1f%%",
    shadow=True,
    startangle=90,
    colors=create_n_colors(len(labels)),
  )

  ax1.axis("equal")  # Equal aspect ratio ensures that pie is drawn as a circle.

  return Chart(
    plt.gcf(),
    width=Decimal(200),
    height=Decimal(200),
    horizontal_alignment=Alignment.CENTERED,
  )

Here, we've used Matplotlib to create a pie chart, via the pie() function.

If you'd like to learn more about creating Pie Charts, read our Guide to Matplotlib Pie Charts!

The gcf() function of the PyPlot instance returns the current figure (get current figure). This figure can be embedded into a PDF document, by injecting it in a Chart constructor, alongside your customization arguments such as the width, height and horizontal_alignment.

That's it! You just supply a Matplotlib figure to the Chart contructor.

Adding a Matplotlib Chart to a PDF Document

Now it's time to create our basic PDF Document and add content to it.

# New import(s)
from borb.pdf.document import Document
from borb.pdf.page.page import Page
from borb.pdf.pdf import PDF
from borb.pdf.canvas.layout.page_layout.multi_column_layout import MultiColumnLayout
from borb.pdf.canvas.layout.page_layout.page_layout import PageLayout
from borb.pdf.canvas.layout.text.paragraph import Paragraph

# Create empty Document
pdf = Document()

# Create empty Page
page = Page()

# Add Page to Document
pdf.append_page(page)

# Create PageLayout
layout: PageLayout = MultiColumnLayout(page)

# Write title
layout.add(Paragraph("About Lorem Ipsum", 
                     font_size=Decimal(20), 
                     font="Helvetica-Bold"))

We'll be using hyphenation in this PDF to ensure the text can be laid out even more smoothly. Hyphenation in borb is pretty straightforward:

# New import(s)
from borb.pdf.canvas.layout.hyphenation.hyphenation import Hyphenation

# Create hyphenation algorithm
hyphenation_algorithm: Hyphenation = Hyphenation("en-gb")

# Write paragraph
layout.add(Paragraph(
    """
    Lorem Ipsum is simply dummy text of the printing and typesetting industry. 
    Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, 
    when an unknown printer took a galley of type and scrambled it to make a type specimen book. 
    It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged. 
    It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, 
    and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum.
    """, text_alignment=Alignment.JUSTIFIED, hyphenation=hyphenation_algorithm))

Now we can add a piechart using the function we declared earlier;

# Write graph
layout.add(create_piechart(["Loren", "Ipsum", "Dolor"], 
                           [0.6, 0.3, 0.1]))

Next we're going to write three more Paragraph objects.
One of them is going to be more of a quote (border on the side, different font, etc).

# Write paragraph
layout.add(Paragraph(
    """
    Contrary to popular belief, Lorem Ipsum is not simply random text. 
    It has roots in a piece of classical Latin literature from 45 BC, making it over 2000 years old. 
    Richard McClintock, a Latin professor at Hampden-Sydney College in Virginia, looked up one of the more obscure Latin words, 
    consectetur, from a Lorem Ipsum passage, and going through the cites of the word in classical literature, 
    discovered the undoubtable source.
    """, text_alignment=Alignment.JUSTIFIED, hyphenation=hyphenation_algorithm))

# Write paragraph
layout.add(Paragraph(
    """
    Lorem Ipsum is simply dummy text of the printing and typesetting industry. 
    """, 
    font="Courier-Bold",
    text_alignment=Alignment.JUSTIFIED, 
    hyphenation=hyphenation_algorithm,
    border_color=HexColor("56cbf9"),
    border_width=Decimal(3),
    border_left=True,
    padding_left=Decimal(5),
    padding_bottom=Decimal(5),
))

# Write paragraph
layout.add(Paragraph(
    """
    Lorem Ipsum comes from sections 1.10.32 and 1.10.33 of "de Finibus Bonorum et Malorum" 
    (The Extremes of Good and Evil) by Cicero, written in 45 BC. 
    This book is a treatise on the theory of ethics, very popular during the Renaissance.
    """, text_alignment=Alignment.JUSTIFIED, hyphenation=hyphenation_algorithm))

Let's add another plot

# Write graph
layout.add(create_piechart(["Loren", "Ipsum", "Dolor", "Sit", "Amet"], 
                           [600, 30, 89, 100, 203]))

And one more Paragraph of content

# Write paragraph
layout.add(Paragraph(
    """
    It is a long established fact that a reader will be distracted by the readable content of a page when looking at its layout. 
    The point of using Lorem Ipsum is that it has a more-or-less normal distribution of letters, as opposed to using 'Content here, content here', 
    making it look like readable English. Many desktop publishing packages and web page editors now use Lorem Ipsum as their default model text, 
    and a search for 'lorem ipsum' will uncover many web sites still in their infancy. 
    Various versions have evolved over the years, sometimes by accident, sometimes on purpose (injected humour and the like).
    """, text_alignment=Alignment.JUSTIFIED, hyphenation=hyphenation_algorithm))

Finally, we can store the Document:

# Write to disk
with open("output.pdf", "wb") as pdf_file_handle:
  PDF.dumps(pdf_file_handle, pdf)

Running this code results in a PDF document that looks like this:

Conclusion

In this guide you've learned how to integrate Matplotlib charts in a PDF using borb. From here, the sky is the limit! The more creative you get with data visualization, the nicer your PDFs will be.

October 22, 2021 08:30 AM UTC

Test and Code

167: React, TypeScript, and the Joy of Testing - Paul Everitt

Paul has a tutorial on testing and TDD with React and TypeScript.
We discuss workflow and the differences, similarities between testing with React/TypeScript and Python.
We also discuss what lessons that we can bring from front end testing to Python testing.

Special Guest: Paul Everitt.

Sponsored By:

• Patreon Supporters: Help support the show with as little as $1 per month and be the first to know when new episodes come out.

Support Test & Code in Python

Links:

• React, TypeScript, and TDD — Paul Everitt's tutorial
• React Testing Library

October 22, 2021 04:15 AM UTC

October 21, 2021

Ben Cook

Development containers in VS Code: a quick start guide

If you’re building production ML systems, dev containers are the killer feature of VS Code. Dev containers give you full VS Code functionality inside a Docker container. This lets you unify your dev and production environments if production is a Docker container. But even if you’re not targeting a Docker deployment, running your code in a container prevents you from fighting with local Python environments across development machines: if you configure your dev container correctly, it will work exactly the same for every member of your team.

In this post, I’ll give you a quick look at how development containers work in VS Code so you can try them for yourself. There are other ways to go about this, but for a basic workflow, I recommend a few things:

• A Dockerfile in the root folder to define the development container
• A configuration file at .devcontainer/devcontainer.json
• Whatever project files you need

This means, for a new project, the folder structure will look something like the following:

.
├── .devcontainer
│   └── devcontainer.json
├── Dockerfile
├── LICENSE
├── README.md
├── main.py
├── requirements-dev.txt
└── requirements.txt

The Dockerfile can be very flexible. Typically you’ll want to install dependencies unless your Docker image already has them installed. Here’s a very basic one you can use to start:

FROM python:3.9

COPY requirements.txt .
RUN pip install requirements.txt

The devcontainer.json file is also highly customizable, but a few things are particularly important:

• The build definition
• Workspace settings for VS Code (things like the path to the Python interpreter, what linter to use, etc)
• Extensions to use
• Additional commands to run inside the container

Here’s what a basic devcontainer.json looks like:

{
  "name": "Python 3",
  "build": {
    "dockerfile": "../Dockerfile",
  },
  // Set *default* container specific settings.json values on container create.
  "settings": {
    "python.pythonPath": "/usr/local/bin/python",
    "python.languageServer": "Pylance",
    "python.linting.enabled": true,
    "python.formatting.blackPath": "/usr/local/bin/black",
    "python.formatting.provider": "black",
    "editor.formatOnSave": true,
  },
  // Add the IDs of extensions you want installed when the container is created.
  "extensions": [
    "ms-python.python",
    "ms-python.vscode-pylance"
  ],
  "postCreateCommand": "pip install -r requirements-dev.txt"
}

Once you have these files setup in your VS Code session, you can open the project in a container. Open the Command Palette with F1 and run Remote-Containers: Open Folder in Container... From there, you can open a terminal and run your code.

Notice if you modify print("hello world") to print('hello world') (with a single quotation) Black will automatically change it for you on save. Nifty!

One other note: the community seems to be switching to Poetry for dependency management in Python and I think this is a good trend. Version conflicts are a huge pain in the regular pip toolchain. I will save that for another post, but just know that you can swap out the pip requirements files for Poetry.

The post Development containers in VS Code: a quick start guide appeared first on Sparrow Computing.

October 21, 2021 09:42 PM UTC

Łukasz Langa

PEP 563 and PEP 649

As the author of PEP 563, I can summarize my position as follows: had PEP 563 never been implemented, it would be easy to accept PEP 649. However, in the current situation it’s not that clear because I find it very important to allow writing a single codebase that works on Python 3.7 - 3.11. If we can secure this behavior, I’m +1 to accepting PEP 649. If not, I have an alternative idea.

October 21, 2021 07:54 PM UTC

Reuven Lerner

Stop fearing Python decorators

If I mention “decorators,” even if you’ve been programming with Python for a while, what comes to mind?

To many, decorators are mysterious and powerful — and beyond the reach of mere mortals. Their @ signs pop up in all sorts of places, from “pytest” to Flask to properties… but it’s not really clear what they are, what they’re doing, how they work, or how they can be useful.

Here’s the thing: If you are comfortable writing functions and classes in Python, then you can write decorators. And once you start, you’ll wonder how you lived without them.

I’ve taught decorators to engineers at some of the world’s largest companies, and also at conferences like PyCon and Euro Python. And this Sunday, I’ll be teaching a live, online course all about decorators.

Once you’re done with the course, you’ll be able to write and understand a wide variety of decorators, from those that filter arguments to those that cache function results and monitor your code’s performance.

I’m running these class like I do my corporate training: With lots of time for Q&A, and a ton of hands-on exercises. Without slides, as I live-code into Jupyter. We’ll even have a private forum, so you can ask questions when the class is done. And of course, you’ll have access to the video recording.

If you want to finally understand decorators, then you want this class.  Sign up here: https://store.lerner.co.il/live-class-decorators

Questions? Comments? Eligible for a discount (for students, retirees/pensioners, or people living outside the world’s 30 richest countries)? E-mail me at reuven@lerner.co.il or get me on Twitter as @reuvenmlerner, and I’ll answer ASAP.

The post Stop fearing Python decorators appeared first on Reuven Lerner.

October 21, 2021 07:35 PM UTC

PyCharm

PyCharm 2021.3 EAP #3

The third PyCharm 2021.3 EAP build is out! Please give it a try, and don’t forget to share your feedback on Twitter (by mentioning @pycharm) or by reporting issues to our tracker!

Important:

• PyCharm EAP builds are not fully tested and might be unstable.
• Starting from this build, you’ll need to provide your JetBrains credentials to use the trial versions of PyCharm. You can read more about the change in this blog post.

DOWNLOAD PYCHARM 2021.3 EAP

Poetry support

One of the most popular issues in our tracker is getting closer to being marked as “fixed”, as this EAP build bundles Poetry support out of the box. We’ve also bundled the TOML plugin so you will also get code insight for your pyproject.toml file out of the box. Kudos to Koudai Aono, who developed the original plugin, which is now being fine-tuned and will become available for all PyCharm users by default.

Endpoints tool window for FastAPI and Flask projects

If you develop web applications using FastAPI and Flask, you likely work with endpoints. Contrary to the name, endpoints are where your work as a Python developer often begins. Although working with them is generally simple in minor projects, it can become cumbersome in larger ones.

Our team is always looking for ways to make you more productive and remove roadblocks from your workflow. This is why we’ve made the Endpoints tool window default for FastAPI and Flask projects. When you start working on a new or existing project, PyCharm will scan its routes and list them as endpoints in a dedicated tool window. You’ll also get code completion and refactoring capabilities for your URLs.

The tool window will present relevant documentation, and you’ll have full navigation capabilities, including the ability to go to an endpoint’s declaration and to find usages of it across your project. You’ll also have quick access to the HTTP Client, an ad hoc tool that allows you to create requests and receive their results quickly for testing purposes.

Bug fixes and improvements

• Fixed an issue that was causing the UI to freeze on macOS. [IDEA-274712]
• Fixed a problem that was causing tabs to close incorrectly when multiple tabs were open. [IDEA-274154]
• Fixed a bug that was causing the Pytest runner to miss a part of the test output when using assertEqual. [PY-30212]
• Fixed a bug that made it impossible for users to re-run parametrized pytests with keywords in a run configuration template. [PY-41848]

Ready to join the EAP?

Some ground rules

• EAP builds are free to use and expire 30 days after the build date.
• You can install an EAP build side by side with your stable PyCharm version.
• These builds are not fully tested and can be unstable.

How to download

You can download this EAP build from our website. Alternatively, you can use the JetBrains Toolbox App to stay up to date throughout the entire EAP. If you’re on Ubuntu 16.04 or later, you can use snaps to make sure you always have the latest PyCharm EAP build.

The PyCharm team

October 21, 2021 06:08 PM UTC

qutebrowser development blog

CVE-2021-41146: Arbitrary command execution in qutebrowser on Windows via URL handler

I'm happy to announce that I just released qutebrowser v2.4.0!

This release fixes a high-severity arbitrary command execution on Windows via URL handlers, see the security advisory and commit message for details.

Windows users are urged to update as soon as possible. For everyone else, this is a …

October 21, 2021 05:28 PM UTC

Ben Cook

NumPy Any: Understanding np.any()

The np.any() function tests whether any element in a NumPy array evaluates to true:

np.any(np.array([[1, 0], [0, 0]]))

# Expected result
# True

The input can have any shape and the data type does not have to be boolean (as long as it’s truthy). If none of the elements evaluate to true, the function returns false:

np.any(np.array([[0, 0], [0, 0]]))

# Expected result
# False

Passing in a value for the axis argument makes np.any() a reducing operation. Say we want to know which rows in a matrix have any truthy elements. We can do that by passing in axis=-1:

np.any(np.zeros((2, 3)), axis=-1)

# Expected result
# array([False, False])

There are two rows and for each of them, none of the elements evaluate to true. The -1 value here is shorthand for “the last axis”.

Easy enough! NumPy also has a function called np.all() which has the same API as np.any() but returns true when all of the elements evaluate to true.

The post NumPy Any: Understanding np.any() appeared first on Sparrow Computing.

October 21, 2021 02:42 PM UTC

TorchVision Transforms: Image Preprocessing in PyTorch

TorchVision, a PyTorch computer vision package, has a simple API for image pre-processing in its torchvision.transforms module. The module contains a set of common, composable image transforms and gives you an easy way to write new custom transforms. As you would expect, these custom transforms can be included in your pre-processing pipeline like any other transform from the module.

Let’s start with a common use case, preparing PIL images for one of the pre-trained TorchVision image classifiers:

import io

import requests
import torchvision.transforms as T

from PIL import Image

resp = requests.get('https://sparrow.dev/assets/img/cat.jpg')
img = Image.open(io.BytesIO(resp.content))

preprocess = T.Compose([
   T.Resize(256),
   T.CenterCrop(224),
   T.ToTensor(),
   T.Normalize(
       mean=[0.485, 0.456, 0.406],
       std=[0.229, 0.224, 0.225]
   )
])

x = preprocess(img)
x.shape

# Expected result
# torch.Size([3, 224, 224])

Here, we apply the following in order:

1. Resize a PIL image to (<height>, 256), where <height> is the value that maintains the aspect ratio of the input image.
2. Crop the (224, 224) center pixels.
3. Convert the PIL image to a PyTorch tensor (which also moves the channel dimension to the beginning).
4. Normalize the image by subtracting a known ImageNet mean and standard deviation.

Let’s go a notch deeper to understand exactly how these transforms work.

Transforms

TorchVision transforms are extremely flexible – there are just a few rules. In order to be composable, transforms need to be callables. That means you can actually just use lambdas if you want:

times_2_plus_1 = T.Compose([
    lambda x: x * 2,
    lambda x: x + 1,
])

x.mean(), times_2_plus_1(x).mean()

# Expected result
# (tensor(1.2491), tensor(3.4982))

But often, you’ll want to use callable classes because they give you a nice way to parameterize the transform at initialization. For example, if you know you want to resize images to have height of 256 you can instantiate the T.Resize transform with a 256 as input to the constructor:

resize_callable = T.Resize(256)

Any PIL image passed to resize_callable() will now get resized to (<height>, 256):

resize_callable(img).size

# Expected result
# (385, 256)

This behavior is important because you will typically want TorchVision or PyTorch to be responsible for calling the transform on an input. We actually saw this in the first example: the component transforms (Resize, CenterCrop, ToTensor, and Normalize) were chained and called inside the Compose transform. And the calling code would not have knowledge of things like the size of the output image you want or the mean and standard deviation for normalization.

Interestingly, there is no Transform base class. Some transforms have no parent class at all and some inherit from torch.nn.Module. This means that if you’re writing a transform class, the constructor can do whatever you want. The only requirement is that there must be a __call__() method to ensure the instantiated object is callable. Note: when transforms override the torch.nn.Module class, they will typically define the forward() method and then the base class takes care of __call__().

Additionally, there are no real constraints on the callable’s inputs or outputs. A few examples:

• T.Resize: PIL image in, PIL image out.
• T.ToTensor: PIL image in, PyTorch tensor out.
• T.Normalize: PyTorch tensor in, PyTorch tensor out.

NumPy arrays may also be a good choice sometimes.

Ok. Now that we know a little about what transforms are, let’s look at an example that TorchVision gives us out of the box.

Example Transform: Compose

The T.Compose transform takes a list of other transforms in the constructor and applies them sequentially to the input. We can take a look at the __init__() and __call__() methods from a recent commit hash to see how this works:

class Compose:
    def __init__(self, transforms):
        self.transforms = transforms

    def __call__(self, img):
        for t in self.transforms:
            img = t(img)
        return img

Very simple! You can pass the T.Compose constructor a list (or any other in-memory sequence) of callables and it will dutifully apply them to any input one at a time. And notice that the input img can be any type you want. In the first example, the input was PIL and the output was a PyTorch tensor. In the second example, the input and output were both tensors. T.Compose doesn’t care!

Let’s instantiate a new T.Compose transform that will let us visualize PyTorch tensors. Remember, we took a PIL image and generated a PyTorch tensor that’s ready for inference in a TorchVision classifier. Let’s take a PyTorch tensor from that transformation and convert it into an RGB NumPy array that we can plot with Matplotlib:

%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np

reverse_preprocess = T.Compose([
    T.ToPILImage(),
    np.array,
])

plt.imshow(reverse_preprocess(x));

The T.ToPILImage transform converts the PyTorch tensor to a PIL image with the channel dimension at the end and scales the pixel values up to int8. Then, since we can pass any callable into T.Compose, we pass in the np.array() constructor to convert the PIL image to NumPy. Not too bad!

Functional Transforms

As we’ve now seen, not all TorchVision transforms are callable classes. In fact, TorchVision comes with a bunch of nice functional transforms that you’re free to use. If you look at the torchvision.transforms code, you’ll see that almost all of the real work is being passed off to functional transforms.

For example, here’s the functional version of the resize logic we’ve already seen:

import torchvision.transforms.functional as F

F.resize(img, 256).size

# Expected result
# (385, 256)

It does the same work, but you have to pass additional arguments in when you call it. My advice: use functional transforms for writing custom transform classes, but in your pre-processing logic, use callable classes or single-argument functions that you can compose.

At this point, we know enough about TorchVision transforms to write one of our own.

Custom Transforms

Let’s write a custom transform that erases the top left corner of an image with the color of a randomly selected pixel. We’ll use the F.erase() function and we’ll allow the caller to specify what how many pixels they want to erase in both directions:

import torch

class TopLeftCornerErase:
    def __init__(self, n_pixels: int):
        self.n_pixels = n_pixels
    
    def __call__(self, img: torch.Tensor) -> torch.Tensor:
        all_pixels = img.reshape(3, -1).transpose(1, 0)
        idx = torch.randint(len(all_pixels), (1,))[0]
        random_pixel = all_pixels[idx][:, None, None]
        return F.erase(img, 0, 0, self.n_pixels, self.n_pixels, random_pixel)

In the constructor, all we do is take the number of pixels as a parameter from the caller. The magic happens in the __call__() method:

1. Create a reshaped view of the image tensor as a (n_pixels, 3) tensor
2. Randomly select a pixel index using torch.randint()
3. Add two dummy dimensions to the tensor. This is because F.erase() and to the image, which has these two dimensions.
4. Call and return F.erase(), which takes five arguments: the tensor, the i coordinate to start at, the j coordinate to start at, the height of the box to erase, the width of the box to erase and the random pixel.

We can apply this custom transform just like any other transform. Let’s use T.Compose to both apply this erase transform and then convert it to NumPy for plotting:

torch.manual_seed(1)

erase = T.Compose([
    TopLeftCornerErase(100),
    reverse_preprocess,
])

plt.imshow(erase(x));

We’ve seen this type of transform composition multiple times now. One thing that is important to point out is that you need to call torch.manual_seed() if you want a deterministic (and therefore reproducible) result for any TorchVision transform that has random behavior in it. This is new as of version 0.8.0.

And that’s about all there is to know about TorchVision transforms! They’re lightweight and flexible, but using them will make your image preprocessing code much easier to reason about.

The post TorchVision Transforms: Image Preprocessing in PyTorch appeared first on Sparrow Computing.

October 21, 2021 01:51 PM UTC

October 20, 2021

Ben Cook

Poetry for Package Management in Machine Learning Projects

When you’re building a production machine learning system, reproducibility is a proxy for the effectiveness of your development process. But without locking all your Python dependencies, your builds are not actually repeatable. If you work in a Python project without locking long enough, you will eventually get a broken build because of a transitive dependency (that is, a dependency of a dependency).

But a broken build isn’t the most dangerous problem. A transitive dependency can change in a way that affects an algorithm’s results without you ever knowing it. Imagine being unable to reproduce an algorithm result because an ancestor dependency that never got pinned changes in an unexpected way. Not good.

Enter Poetry. Poetry is an environment management tool for Python projects. Admittedly, there are a lot of environment management tools for Python and there’s no guarantee that Poetry will win. But the Python development community does seem to be slowly standardizing around Poetry.

What is Poetry?

Virtual environment tools like virtualenv and conda have been an important part of modern Python development for a while. But there are a few problems:

• Different tools and approaches don’t play nicely with each other
• The standards for configuration are inconsistent
• Dependency resolution can fail in mysterious ways

Ideally, you want all your builds to be precisely repeatable and you want a straightforward solution that you can use across all your projects. This is where Poetry shines.

Think of Poetry like npm for Python. It’s centered around pyproject.toml, a single config file that defines your entire Python environment and it comes with a lock file, poetry.lock, which pins all dependencies (including transitive dependencies).

Poetry configuration

Poetry extends the pyproject.toml file (introduced in PEP 517) to specify build requirements, project dependencies and development dependencies. It also includes project-level configuration (things like the name, description and license), scripts, extra dependencies and more.

Importantly, the Poetry spec for pyproject.toml works for both Python packages and Python applications, which prevents you from using a different approach across Python projects. This is important for machine learning development because reusable utilities like config and basic calculations like bounding box math often fit better in Python packages than in your actual machine learning application.

Lock file

The poetry.lock file is similar in concept to running pip freeze > requirements.txt every time you add or remove a package. It’s just that Poetry manages this for you when you add or remove a dependency.

And Poetry comes with a good dependency resolver. Since some packages on PyPI don’t specify all dependencies in their metadata, it can make adding a new package a little slower in Poetry. But it does ensure that you won’t get broken builds down the road as dependencies evolve, which is important.

Quick Start

To install the latest Poetry into your Python environment, run:

curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python

After installation, the script will print instructions to your screen to add poetry to your path.

To initialize a new project, run:

poetry init

This will guide you interactively through the process of setting up a new Python project with Poetry. You will add information about the project, the version of Python to use and (optionally) dependencies and development dependencies.

Say you want to add a dependency, like NumPy, after your project has been initialized. You can do that with the poetry add command:

poetry add numpy@~1.20

This will install NumPy with version >=1.20.0,<1.21.0. Alternatively, you could run poetry add numpy@^1.20 to install version >=1.20.0,<2.0.0. When you run this command Poetry does a few things:

1. Update pyproject.toml to specify the new dependency.
2. Use the dependency resolver to find the set of package versions that best fit the configuration. In this case, just numpy 1.20.3, but this will also include other packages if they’re specified.
3. Install all the packages found by the resolver. By default, these will be installed into a virtual environment managed by Poetry. But you can also tell Poetry to install dependencies directly to the system Python.
4. Freeze all dependencies and save them to poetry.lock so the exact build can be repeated in the future.

If you add the --dev flag to the poetry add command, Poetry treats the dependency as a development dependency. For example:

poetry add black --dev

This distinction is useful because for production builds, you can install all non-development dependencies by adding the --no-dev flag:

poetry install --no-dev

This prevents unnecessary bloat in your production environment where you don’t need tools like formatters and linters.

Finally, you can enter a shell with the virtual environment managed by Poetry activated:

poetry shell
python

Recommended Setup

As mentioned above, Poetry manages virtual environments for you and gives you a dedicated shell. But for production machine learning projects, I think it’s usually better to develop inside a Docker container.

In this case, I recommend turning off Poetry’s virtual environment management. You can do that with a config command:

poetry config virtualenvs.create false

Or you can use an environment variable, for example:

POETRY_VIRTUALENVS_CREATE=false poetry install

And now, you should know everything you need to get started using Poetry to manage your Python environment machine learning projects! Of course, there’s a ton more to learn about Poetry. Fortunately, their documentation is pretty good.

Happy ML engineering!

The post Poetry for Package Management in Machine Learning Projects appeared first on Sparrow Computing.

October 20, 2021 09:33 PM UTC

Real Python

Using the len() Function in Python

>>> help(len)
Help on built-in function len in module builtins:
len(obj, /)
    Return the number of items in a container.

>>> greeting = "Good Day!"
>>> len(greeting)
9

>>> office_days = ["Tuesday", "Thursday", "Friday"]
>>> len(office_days)
3

>>> london_coordinates = (51.50722, -0.1275)
>>> len(london_coordinates)
2

>>> len("")
0
>>> len([])
0
>>> len(())
0

>>> len(range(1, 20, 2))
10

Read the full article at https://realpython.com/len-python-function/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

October 20, 2021 02:00 PM UTC

Python for Beginners

Iterator in Python

You must have used different data structures like python dictionary, list, tuple and set while programming. We often need to access the elements of these data structures in a sequential manner. To iterate these data structures sequentially, we generally use for loops and index of elements. In this article, we will try to understand how we can access elements of a list or a tuple without using a for loop or  the indices of the elements. So, let’s dive into the concept of iterator and iterables in Python.

What is an iterable in Python? 

A container object like a list or set can contain a lot of elements. If we can access the member elements of the container objects one at a time then the container object is called an iterable. In our programs we use different iterables like list, tuple, set, or dictionary. 

Elements of any iterable can be accessed one at a time using a for loop as follows.

myList = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
print("Elements of the list are:")
for element in myList:
    print(element)

Output:

Elements of the list are:
1
2
3
4
5
6
7
8
9
10

In a for loop, we access all the elements from start till end. But, Consider a situation where we have to access the next element of the iterable only when some specific event occurs. 

For example, Let us take a scenario in which we have a list of 100 elements. We continually ask the user to input a number and whenever they enter an even number, we print the next element of the list. 

Now, this cannot be done using a for loop as we cannot predict when the user will enter an even number.  Having no order or sequence in the inputs stops us from using the for loop to iterate the list in our program. Iterators can be a handy tool to access the elements of iterables in such situations. So, Let’s learn what an iterator is and how we can create an iterator in python to access the elements of an iterable.

What is an iterator in Python?

An iterator is an object that can be iterated upon. In other words, we can  access all the elements in an iterable object using an iterator. 

In python, we can create an iterator for any container object using the iter() method. The iter() method takes an iterable object as input and returns an iterator for the same object.

myList = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
myIter = iter(myList)
print("list is:", myList)
print("Iterator for the list is:", myIter)

Output:

list is: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
Iterator for the list is: <list_iterator object at 0x7f4c21734070>

In the output, you can see that a list_iterator object has been created by passing a list to the iter() function.

How to traverse an iterator in Python?

The simplest way to traverse an iterator is by using a for loop. We can access each element in the iterator using a for loop as follows.

myList = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
myIter = iter(myList)
print("list is:", myList)
print("Elements in the iterator are:")
for element in myIter:
    print(element)

Output:

list is: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
Elements in the iterator are:
1
2
3
4
5
6
7
8
9
10

As discussed earlier, for loop won’t work if we have no order or sequence for traversing the iterator. For this, we can use two ways.

The first way to access an element of the iterator is by using the __next__() method. The __next__() method, when invoked on the iterator, returns an element next to the previously traversed element. It always retains the information about the element which was returned last time and whenever it is invoked, it returns only the next element which has not been traversed yet. We can understand this using the following example.

myList = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
myIter = iter(myList)
print("list is:", myList)
print("Elements in the iterator are:")
try:
    print(myIter.__next__())
    print(myIter.__next__())
    print(myIter.__next__())
    print(myIter.__next__())
    print(myIter.__next__())
    print(myIter.__next__())
    print(myIter.__next__())
    print(myIter.__next__())
    print(myIter.__next__())
    print(myIter.__next__())
    print(myIter.__next__())
    print(myIter.__next__())
except StopIteration as e:
    print("All elements in the iterator already traversed. Raised exception", e)

Output:

list is: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
Elements in the iterator are:
1
2
3
4
5
6
7
8
9
10
All elements in the iterator already traversed. Raised exception 

Another way to access the elements of the iterator is by using the next() function. The next() function takes the iterator as input and returns the next element that has not been traversed yet, just like the __next__() method. 

myList = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
myIter = iter(myList)
print("list is:", myList)
print("Elements in the iterator are:")
try:
    print(next(myIter))
    print(next(myIter))
    print(next(myIter))
    print(next(myIter))
    print(next(myIter))
    print(next(myIter))
    print(next(myIter))
    print(next(myIter))
    print(next(myIter))
    print(next(myIter))
    print(next(myIter))
except StopIteration as e:
    print("All elements in the iterator already traversed. Raised exception", e)

Output:

list is: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
Elements in the iterator are:
1
2
3
4
5
6
7
8
9
10
All elements in the iterator already traversed. Raised exception 

You can observe from the above two examples that the functioning of the __next__() method and next() function are almost similar. Also, both the next() function and the __next__() method raise an StopIteration error when all the elements of the iterator are already traversed. So, it is advised to use exception handling using python try except blocks.

Example use case of an iterator

Now let us take a closer look at the situation discussed above where we have to access the elements from a list only when the user gives an even number as input. 

Now, we have the next() function that can access the element from an iterator whenever we call the function. We can use it to access the elements from the list. For this, we will create an iterator for the list and then we will access the elements from the list using the next() function whenever the user gives an even number as input. We can implement a python function for this as follows.

myList = range(0, 101)
myIter = iter(myList)
while True:
    try:
        user_input = int(input("Input an even number to get an output, 0 to exit:"))
        if user_input == 0:
            print("Good Bye")
            break
        elif user_input % 2 == 0:
            print("Very good. Here is an output for you.")
            print(next(myIter))
        else:
            print("Input an even number.")
            continue
    except StopIteration as e:
        print("All the output has been exhausted.")
        

Output:

Input an even number to get an output, 0 to exit:1
Input an even number.
Input an even number to get an output, 0 to exit:2
Very good. Here is an output for you.
0
Input an even number to get an output, 0 to exit:4
Very good. Here is an output for you.
1
Input an even number to get an output, 0 to exit:3
Input an even number.
Input an even number to get an output, 0 to exit:6
Very good. Here is an output for you.
2
Input an even number to get an output, 0 to exit:0
Good Bye

Conclusion

In this article, we have discussed how we can access elements from different iterable objects using an iterator in Python. To learn more about python programming, you can read this article on list comprehension. You may also like this article on the linked list in Python.

The post Iterator in Python appeared first on PythonForBeginners.com.

October 20, 2021 01:41 PM UTC

PyCharm

PyCharm 2021.2.3 Is Out!

The third minor release of PyCharm 2021.2 contains multiple bug fixes:

• We fixed a bug where PEP 585 parameterized built-in classes were not recognized when using from __future__ import annotations. [PY-46602]
• We fixed a bug in Jupyter notebooks where the connection to the Python debugger failed when using ipykernel>=6.3.0. [PY-50575]
• We fixed a bug where custom external documentation URLs for the standard entries were getting removed upon IDE restart. [PY-36369]
• We fixed a bug where PyCharm was updating indexes on every startup if IDE was closed with OS shutdown [IDEA-275337]

Download PyCharm 2021.2.3

For the full list of issues addressed in PyCharm 2021.2.3, please see the release notes.
Found a bug? Please report it using our bug tracker.

October 20, 2021 01:36 PM UTC

Python Software Foundation

Announcing Python Software Foundation Fellow Members for Q3 2021! 🎉

The PSF is pleased to announced its third batch of PSF Fellows for 2021! Let us welcome the new PSF Fellows for Q3! The following people continue to do amazing things for the Python community:

Anthony Sottile

Twitch, YouTube, GitHub Sponsors, Twitter

Bernát Gábor

Twitter, Website, GitHub

Cristián Danilo Maureira-Fredes

Michael Iyanda

LinkedIn, GitHub

Nicolás Demarchi

Twitter, GitHub, LinkedIn

Thank you for your continued contributions. We have added you to our Fellow roster online.

The above members help support the Python ecosystem by being phenomenal leaders, sustaining the growth of the Python scientific community, maintaining virtual Python communities, maintaining Python libraries, creating educational material, organizing Python events and conferences, starting Python communities in local regions, and overall being great mentors in our community. Each of them continues to help make Python more accessible around the world. To learn more about the new Fellow members, check out their links above.

Let's continue recognizing Pythonistas all over the world for their impact on our community. The criteria for Fellow members is available online: https://www.python.org/psf/fellows/. If you would like to nominate someone to be a PSF Fellow, please send a description of their Python accomplishments and their email address to psf-fellow at python.org. We are accepting nominations for quarter 4 through November 20, 2021.

Are you a PSF Fellow and want to help the Work Group review nominations? Contact us at psf-fellow at python.org.

October 20, 2021 11:26 AM UTC

Python GUIs

[updated for PyQt6] Creating your first app with PyQt — A simple Hello World! application with Python and Qt5

PyQt is a Python library for creating GUI applications using the Qt toolkit. Created by Riverbank Computing, PyQt is free software (GPL licensed) and has been in development since 1999. The latest version PyQt6 -- based on Qt 6 -- was released in 2021 and the library continues to be updated.

There are two major versions currently in use: PyQt5 based on Qt5 and PyQt6 based on Qt6. Both versions are almost completely compatible aside from imports, and lack of support for some advanced modules in Qt6. PyQt6 also makes some changes to how namespaces and flags work, but these are easily manageable.

In this tutorial we'll learn how to use PyQt to create desktop applications with Python.

First we'll create a series of simple windows on your desktop to ensure that PyQt is working and introduce some of the basic concepts. Then we'll take a brief look at the event loop and how it relates to GUI programming in Python. Finally we'll look at Qt's QMainWindow which offers some useful common interface elements such as toolbars and menus. These will be explored in more detail in the subsequent tutorials.

This PyQt tutorial features both PyQt5 and PyQt6 code examples. You can use whichever you prefer, but currently there are more examples online for PyQt5.

Creating an application

Let's create our first application! To start create a new Python file — you can call it whatever you like (e.g. app.py) and save it somewhere accessible. We'll write our simple app in this file.

We'll be editing within this file as we go along, and you may want to come back to earlier versions of your code, so remember to keep regular backups.

The source code for the application is shown below. Type it in verbatim, and be careful not to make mistakes. If you do mess up, Python will let you know what's wrong.

from PyQt5.QtWidgets import QApplication, QWidget

# Only needed for access to command line arguments
import sys

# You need one (and only one) QApplication instance per application.
# Pass in sys.argv to allow command line arguments for your app.
# If you know you won't use command line arguments QApplication([]) works too.
app = QApplication(sys.argv)

# Create a Qt widget, which will be our window.
window = QWidget()
window.show()  # IMPORTANT!!!!! Windows are hidden by default.

# Start the event loop.
app.exec()


# Your application won't reach here until you exit and the event
# loop has stopped.

from PyQt6.QtWidgets import QApplication, QWidget

# Only needed for access to command line arguments
import sys

# You need one (and only one) QApplication instance per application.
# Pass in sys.argv to allow command line arguments for your app.
# If you know you won't use command line arguments QApplication([]) works too.
app = QApplication(sys.argv)

# Create a Qt widget, which will be our window.
window = QWidget()
window.show()  # IMPORTANT!!!!! Windows are hidden by default.

# Start the event loop.
app.exec()


# Your application won't reach here until you exit and the event
# loop has stopped.

First, launch your application. You can run it from the command line like any other Python script, for example --

python3 app.py

Run it! You will now see your window. Qt automatically creates a window with the normal window decorations and you can drag it around and resize it like any window.

What you'll see will depend on what platform you're running this example on. The image below shows the window as displayed on Windows, macOS and Linux (Ubuntu).

Our window, as seen on Windows, macOS and Linux.

Stepping through the code

Let's step through the code line by line, so we understand exactly what is happening.

First, we import the PyQt classes that we need for the application. Here we're importing QApplication, the application handler and QWidget, a basic empty GUI widget, both from the QtWidgets module.

from PyQt5.QtWidgets import QApplication, QWidget

from PyQt6.QtWidgets import QApplication, QWidget

The main modules for Qt are QtWidgets, QtGui and QtCore.

You could do from <module> import * but this kind of global import is generally frowned upon in Python, so we'll avoid it here.

Next we create an instance of QApplication, passing in sys.arg, which is Python list containing the command line arguments passed to the application.

app = QApplication(sys.argv)

If you know you won't be using command line arguments to control Qt you can pass in an empty list instead, e.g.

app = QApplication([])

Next we create an instance of a QWidget using the variable name window.

window = QWidget()
window.show()

In Qt all top level widgets are windows -- that is, they don't have a parent and are not nested within another widget or layout. This means you can technically create a window using any widget you like.

Widgets without a parent are invisible by default. So, after creating the window object, we must always call .show() to make it visible. You can remove the .show() and run the app, but you'll have no way to quit it!

What is a window? - Holds the user-interface of your application - Every application needs at least one (...but can have more) - Application will (by default) exit when last window is closed

Finally, we call app.exec() to start up the event loop.

In PyQt5 you can also use app.exec_(). This was a legacy feature avoid a clash with the exec reserved word in Python 2. It has been removed in PyQt6.

What's the event loop?

Before getting the window on the screen, there are a few key concepts to introduce about how applications are organized in the Qt world. If you're already familiar with event loops you can safely skip to the next section.

The core of every Qt Applications is the QApplication class. Every application needs one — and only one — QApplication object to function. This object holds the event loop of your application — the core loop which governs all user interaction with the GUI.

Each interaction with your application — whether a press of a key, click of a mouse, or mouse movement — generates an event which is placed on the event queue. In the event loop, the queue is checked on each iteration and if a waiting event is found, the event and control is passed to the specific event handler for the event. The event handler deals with the event, then passes control back to the event loop to wait for more events. There is only one running event loop per application.

The QApplication class - QApplication holds the Qt event loop - One QApplication instance required - You application sits waiting in the event loop until an action is taken - There is only one event loop running at any time

QMainWindow

As we discovered in the last part, in Qt any widgets can be windows. For example, if you replace QtWidget with QPushButton. In the example below, you would get a window with a single push-able button in it.

import sys
from PyQt5.QtWidgets import QApplication, QPushButton

app = QApplication(sys.argv)

window = QPushButton("Push Me")
window.show()

app.exec()

import sys
from PyQt6.QtWidgets import QApplication, QPushButton

app = QApplication(sys.argv)

window = QPushButton("Push Me")
window.show()

app.exec()

This is neat, but not really very useful -- it's rare that you need a UI that consists of only a single control! But, as we'll discover later, the ability to nest widgets within other widgets using layouts means you can construct complex UIs inside an empty QWidget.

But, Qt already has a solution for you -- the QMainWindow. This is a pre-made widget which provides a lot of standard window features you'll make use of in your apps, including toolbars, menus, a statusbar, dockable widgets and more. We'll look at these advanced features later, but for now, we'll add a simple empty QMainWindow to our application.

import sys
from PyQt5.QtWidgets import QApplication, QMainWindow

app = QApplication(sys.argv)

window = QMainWindow()
window.show()

# Start the event loop.
app.exec()

import sys
from PyQt6.QtWidgets import QApplication, QMainWindow

app = QApplication(sys.argv)

window = QMainWindow()
window.show()

# Start the event loop.
app.exec()

Run it! You will now see your main window. It looks exactly the same as before!

So our QMainWindow isn't very interesting at the moment. We can fix that by adding some content. If you want to create a custom window, the best approach is to subclass QMainWindow and then include the setup for the window in the __init__ block. This allows the window behavior to be self contained. We can add our own subclass of QMainWindow — call it MainWindow to keep things simple.

import sys

from PyQt5.QtCore import QSize, Qt
from PyQt5.QtWidgets import QApplication, QMainWindow, QPushButton


# Subclass QMainWindow to customize your application's main window
class MainWindow(QMainWindow):
    def __init__(self):
        super().__init__()

        self.setWindowTitle("My App")
        button = QPushButton("Press Me!")

        # Set the central widget of the Window.
        self.setCentralWidget(button)


app = QApplication(sys.argv)

window = MainWindow()
window.show()

app.exec()

import sys

from PyQt6.QtCore import QSize, Qt
from PyQt6.QtWidgets import QApplication, QMainWindow, QPushButton


# Subclass QMainWindow to customize your application's main window
class MainWindow(QMainWindow):
    def __init__(self):
        super().__init__()

        self.setWindowTitle("My App")
        button = QPushButton("Press Me!")

        # Set the central widget of the Window.
        self.setCentralWidget(button)


app = QApplication(sys.argv)

window = MainWindow()
window.show()

app.exec()

For this demo we're using a QPushButton. The core Qt widgets are always imported from the QtWidgets namespace, as are the QMainWindow and QApplication classes. When using QMainWindow we use .setCentralWidget to place a widget (here a QPushButton) in the QMainWindow -- by default it takes the whole of the window. We'll look at how to add multiple widgets to windows in the layouts tutorial.

When you subclass a Qt class you must always call the super __init__ function to allow Qt to set up the object.

In our __init__ block we first use .setWindowTitle() to change the title of our main window. Then we add our first widget — a QPushButton — to the middle of the window. This is one of the basic widgets available in Qt. When creating the button you can pass in the text that you want the button to display.

Finally, we call .setCentralWidget() on the window. This is a QMainWindow specific function that allows you to set the widget that goes in the middle of the window.

Run it! You will now see your window again, but this time with the QPushButton widget in the middle. Pressing the button will do nothing, we'll sort that next.

Our QMainWindow with a single QPushButton on Windows, macOS and Linux.

We'll cover more widgets in detail shortly but if you're impatient and would like to jump ahead you can take a look at the http://doc.qt.io/qt-5/widget-classes.html#basic-widget-classes[QWidget documentation]. Try adding the different widgets to your window!

Sizing windows and widgets

The window is currently freely resizable -- if you grab any corner with your mouse you can drag and resize it to any size you want. While it's good to let your users resize your applications, sometimes you may want to place restrictions on minimum or maximum sizes, or lock a window to a fixed size.

In Qt sizes are defined using a QSize object. This accepts width and height parameters in that order. For example, the following will create a fixed size window of 400x300 pixels.

import sys

from PyQt5.QtCore import QSize, Qt
from PyQt5.QtWidgets import QApplication, QMainWindow, QPushButton


# Subclass QMainWindow to customize your application's main window
class MainWindow(QMainWindow):
    def __init__(self):
        super().__init__()

        self.setWindowTitle("My App")

        button = QPushButton("Press Me!")

        self.setFixedSize(QSize(400, 300))

        # Set the central widget of the Window.
        self.setCentralWidget(button)


app = QApplication(sys.argv)

window = MainWindow()
window.show()

app.exec()

import sys

from PyQt6.QtCore import QSize, Qt
from PyQt6.QtWidgets import QApplication, QMainWindow, QPushButton


# Subclass QMainWindow to customize your application's main window
class MainWindow(QMainWindow):
    def __init__(self):
        super().__init__()

        self.setWindowTitle("My App")

        button = QPushButton("Press Me!")

        self.setFixedSize(QSize(400, 300))

        # Set the central widget of the Window.
        self.setCentralWidget(button)


app = QApplication(sys.argv)

window = MainWindow()
window.show()

app.exec()

Run it! You will see a fixed size window -- try and resize it, it won't work.

Our fixed-size window, notice that the _maximize control is disabled on Windows & Linux. On macOS you can maximize the app to fill the screen, but the central widget will not resize._

As well as .setFixedSize() you can also call .setMinimumSize() and .setMaximumSize() to set the minimum and maximum sizes respectively. Experiment with this yourself!

You can use these size methods on any widget.

In this section we've covered the QApplication class, the QMainWindow class, the event loop and experimented with adding a simple widget to a window. In the next section we'll take a look at the mechanisms Qt provides for widgets and windows to communicate with one another and your own code.

For more, see the complete PyQt5 tutorial.

October 20, 2021 09:00 AM UTC

Python Bytes

#255 Closember eve, the cure for Hacktoberfest?

<p><strong>Watch the live stream:</strong></p> <a href='https://www.youtube.com/watch?v=ICp8eD6uENI' style='font-weight: bold;'>Watch on YouTube</a><br> <br> <p><strong>About the show</strong></p> <p>Sponsored by <strong>us:</strong></p> <ul> <li>Check out the <a href="https://training.talkpython.fm/courses/all"><strong>courses over at Talk Python</strong></a></li> <li>And <a href="https://pythontest.com/pytest-book/"><strong>Brian’s book too</strong></a>!</li> </ul> <p>Special guest: <strong>Will McGugan</strong></p> <p><strong>Michael #1:</strong> <a href="https://azhpushkin.me/posts/cython-cpp-intro"><strong>Wrapping C++ with Cython</strong></a></p> <ul> <li>By <a href="https://azhpushkin.me/">Anton Zhdan-Pushkin</a></li> <li>A small series showcasing the implementation of a Cython wrapper over a C++ library.</li> <li>C library: <strong>yaacrl - Y</strong>et <strong>A</strong>nother <strong>A</strong>udio <strong>R</strong>ecognition <strong>L</strong>ibrary is a small Shazam-like library, which can recognize songs using a small recorded fragment.</li> <li>For Cython to consume yaacrl correctly, we need to “teach” it about the API using `cdef extern </li> <li>It is convenient to put such declarations in <code>*.pxd</code> files.</li> <li>One of the first features of Cython that I find extremely useful — aliasing. With aliasing, we can use names like <code>Storage</code> or <code>Fingerprint</code> for Python classes without shadowing original C++ classes.</li> <li><strong>Implementing a wrapper: pyaacrl</strong> - The most common way to wrap a C++ class is to use <a href="https://docs.python.org/3/extending/newtypes_tutorial.html">Extension types</a>. As an extension type a just a C struct, it can have an underlying C++ class as a field and act as a proxy to it.</li> <li>Cython documentation has a whole page dedicated to the pitfalls of <a href="https://cython.readthedocs.io/en/latest/src/userguide/wrapping_CPlusPlus.html">“Using C++ in Cython</a>.”</li> <li>Distribution is hard, but there is a tool that is designed specifically for such needs: scikit-build.</li> <li><a href="https://pybind11.readthedocs.io/en/stable/"><strong>PyBind11 too</strong></a></li> </ul> <p><strong>Brian #2:</strong> <a href="https://github.com/dmerejkowsky/tbump"><strong>tbump</strong></a> <a href="https://github.com/dmerejkowsky/tbump"></a><a href="https://github.com/dmerejkowsky/tbump"><strong>: bump software releases</strong></a></p> <ul> <li><a href="https://twitter.com/geosephi/status/1437697175987920896">suggested by Sephi Berry</a></li> <li>limits the manual process of updating a project version</li> <li><code>tbump init 1.2.2</code> initializes a <code>tbump.toml</code> file with customizable settings <ul> <li><code>--pyproject</code> will append to <code>pyproject.toml</code> instead</li> </ul></li> <li><code>tbump 1.2.3</code> will <ul> <li>patch files: wherever the version listed</li> <li>(optional) run configured commands before commit <ul> <li>failing commands stop the bump.</li> </ul></li> <li>commit the changes with a configurable message</li> <li>add a version tag</li> <li>push code</li> <li>push tag</li> <li>(optional) run post publish command</li> <li>Tell you what it’s going to do before it does it. (can opt out of this check)</li> </ul></li> <li>pretty much everything is customizable and configurable.</li> <li>I tried this on a flit based project. Only required one change</li> </ul> <pre><code> # For each file to patch, add a [[file]] config # section containing the path of the file, relative to the # tbump.toml location. [[file]] src = "pytest_srcpaths.py" search = '__version__ = "{current_version}"' </code></pre> <ul> <li>cool example of a pre-commit check:</li> </ul> <pre><code> # [[before_commit]] # name = "check changelog" # cmd = "grep -q {new_version} Changelog.rst" </code></pre> <p><strong>Will #3:</strong> <strong><a href="https://closember.org/">Closember</a></strong> by <strong>Matthias Bussonnier</strong></p> <p><strong>Michael #4:</strong> <a href="https://twitter.com/btskinn/status/1441597030934011909?s=12"><strong>scikit learn goes 1.0</strong></a></p> <ul> <li>via Brian Skinn</li> <li>The library has been stable for quite some time, releasing version 1.0 is recognizing that and signalling it to our users.</li> <li>Features:</li> <li>Keyword and positional arguments - To improve the readability of code written based on scikit-learn, now users have to provide most parameters with their names, as keyword arguments, instead of positional arguments.</li> <li>Spline Transformers - One way to add nonlinear terms to a dataset’s feature set is to generate spline basis functions for continuous/numerical features with the new <a href="https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.SplineTransformer.html#sklearn.preprocessing.SplineTransformer">SplineTransformer</a>.</li> <li>Quantile Regressor - Quantile regression estimates the median or other quantiles of Y conditional on X</li> <li>Feature Names Support - When an estimator is passed a <a href="https://pandas.pydata.org/docs/user_guide/dsintro.html#dataframe">pandas’ dataframe</a> during <a href="https://scikit-learn.org/stable/glossary.html#term-fit">fit</a>, the estimator will set a <code>feature_names_in_</code> attribute containing the feature names.</li> <li>A more flexible plotting API</li> <li>Online One-Class SVM</li> <li>Histogram-based Gradient Boosting Models are now stable</li> <li>Better docs</li> </ul> <p><strong>Brian #5:</strong> <a href="https://blog.jaraco.com/devpi-as-offline-pypi-cache/"><strong>Using devpi as an offline PyPI cache</strong></a></p> <ul> <li>Jason R. Coombs</li> <li>This is the devpi tutorial I’ve been waiting for.</li> <li>Single machine local server mirror of PyPI (mirroring needs primed), usable in offline mode.</li> </ul> <pre><code> $ pipx install devpi-server $ devpi-init $ devpi-server </code></pre> <ul> <li>now in another window, prime the cache by grabbing whatever you need, with the index redirected</li> </ul> <pre><code> (venv) $ export PIP_INDEX_URL=http://localhost:3141/root/pypi/ (venv) $ pip install pytest, ... </code></pre> <ul> <li>then you can restart the server anytime, or even offline</li> </ul> <pre><code> $ devpi-server --offline </code></pre> <ul> <li>tutorial includes examples, proving how simple this is.</li> </ul> <p><strong>Will #6:</strong> <strong><a href="https://github.com/wasi-master/pypi-command-line">PyPi command line</strong> </a></p> <p><strong>Extras</strong></p> <p>Brian:</p> <ul> <li>I’ve started using pyenv on my Mac just for downloading Python versions. Verdict still out if I like it better than just downloading from pytest.org. </li> <li>Also started using <a href="https://starship.rs/">Starship</a> with no customizations so far. I’d like to hear from people if they have nice Starship customizations I should try.</li> <li><a href="https://vscode.dev/">vscode.dev</a> is a thing, <a href="https://code.visualstudio.com/blogs/2021/10/20/vscode-dev">announcement</a> just today </li> </ul> <p>Michael:</p> <ul> <li>PyCascades <a href="https://bit.ly/pycascades2022cfp"><strong>Call for Proposals</strong></a> is currently open </li> <li>Got your <a href="https://www.apple.com/macbook-pro-14-and-16/"><strong>M1 Max</strong></a>?</li> <li>Prediction: Tools like <a href="https://www.codeweavers.com/crossover"><strong>Crossover for Windows apps</strong></a> will become more of a thing.</li> </ul> <p>Will: </p> <ul> <li>GIL removal <ul> <li><a href="https://docs.google.com/document/u/0/d/18CXhDb1ygxg-YXNBJNzfzZsDFosB5e6BfnXLlejd9l0/mobilebasic?urp=gmail_link">https://docs.google.com/document/u/0/d/18CXhDb1ygxg-YXNBJNzfzZsDFosB5e6BfnXLlejd9l0/mobilebasic?urp=gmail_link</a></li> <li><a href="https://lwn.net/SubscriberLink/872869/0e62bba2db51ec7a/">https://lwn.net/SubscriberLink/872869/0e62bba2db51ec7a/</a></li> </ul></li> <li><a href="https:///vscode.dev">vscode.dev</a></li> </ul> <p><strong>Joke:</strong> </p> <ul> <li><a href="https://devops.com/the-torture-never-stops/"><strong>The torture never stops</strong></a></li> <li><a href="http://www.zaphinath.com/wp-content/uploads/2015/11/ie-eats-glue.jpg"><strong>IE (“Safari”) Eating Glue</strong></a></li> </ul>

October 20, 2021 08:00 AM UTC

Python Anywhere

Our October system update

On 6 October we upgraded our EU-based systems to the latest version of our platform, and today, 20 October, we did the same upgrade on our US-based system. There are a bunch of changes to report!

October 20, 2021 12:00 AM UTC

October 19, 2021

PyCoder’s Weekly

Issue #495 (Oct. 19, 2021)

#495 – OCTOBER 19, 2021
View in Browser »

Data Elixir: Data Science Newsletter

Data Elixir is an email newsletter that keeps you on top of the tools and trends in Data Science. Covers machine learning, data visualization, analytics, and strategy. Curated weekly with top picks from around the web →
DATA ELIXIR sponsor

Python Jobs

Senior Python Engineer @ Moody's AI & ML Center of Excellence (New York, NY, USA)

Moody's

Senior Software Engineer (Washington D.C., DC, USA)

Quorum

Senior Backend Software Engineer (Anywhere)

Clay

Full Stack Developer (Anywhere)

Level 12

Software Engineer (Anywhere)

1Point21 Interactive

More Python Jobs >>>

Articles & Tutorials

Accelerate Your Python Apps With Apache Cassandra™ NoSQL. Register for an Astra DB Demo

Scale data for your Django, Flask, FastAPI apps with our multi-cloud, serverless DBaaS–built on Apache Cassandra™. Painless APIs, free for developers. Get 80 Gigabytes of Storage Free Every Month. Explore Astra DB now →
DATASTAX sponsor

Projects & Code

Events

Happy Pythoning!
This was PyCoder’s Weekly Issue #495.
View in Browser »

[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]

October 19, 2021 07:30 PM UTC

Peter Bengtsson

How to string pad a string in Python with a variable

I just have to write this down because that's the rule; if I find myself googling something basic like this more than once, it's worth blogging about.

Suppose you have a string and you want to pad with empty spaces. You have 2 options:

>>> s = "peter"
>>> s.ljust(10)
'peter     '
>>> f"{s:<10}"
'peter     '

The f-string notation is often more convenient because it can be combined with other formatting directives.
But, suppose the number 10 isn't hardcoded like that. Suppose it's a variable:

>>> s = "peter"
>>> width = 11
>>> s.ljust(width)
'peter      '
>>> f"{s:<width}"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Invalid format specifier

Well, the way you need to do it with f-string formatting, when it's a variable like that is this syntax:

>>> f"{s:<{width}}"
'peter      '

October 19, 2021 04:44 PM UTC

PyCon

PyCon US 2022 Website and Sponsorship Program Launch!

• Outreach & Networking: Access to 2,000+ attendees interested in your products and services to generate qualified leads.

• Recruiting: Access to qualified job candidates. If you’re hiring, there’s no better place to find Python developers than PyCon US.

• Marketing: Build brand awareness with an audience to the world's biggest and most prestigious Python conference. Additionally, a broad range of promotional opportunities across Python’s ecosystem includes a full 12 months, not just during PyCon US.

• Flexibility: The freedom and ease to customize your sponsorship benefits via our online application to meet your goals.

Sponsorships have a tremendous impact!

Most importantly, the Python Software Foundation operates off sponsorships it receives. The more funding the PSF has to work with, the more we can do to support Python and its community. In 2021 the PSF launched two new initiatives that will have a tremendous impact on all users:

1. In July, the PSF hired the inaugural Developer-in-Residence (Łukasz Langa) to support CPython. A full-time, paid core developer supporting dozens of volunteers that make Python happen has been an enormous help. In addition to supporting volunteers, Łukasz is analyzing usage and workflows to better address sustainability and future funding.

2. In August, the PSF hired Shamika Mohanan to support Python packaging. Shamika will handle outreach to Python users so the PSF can better understand the landscape, identify fundable initiatives, seek grants, oversee funded projects, and report on their progress and results to improve Python packaging for all users.

The above would not have been possible without the generous support of our sponsors over the years. Please help us ensure these initiatives have funding for years to come.

Sponsor the PSF today!

If you have any questions about sponsoring PyCon US and the PSF, please get in touch with us at sponsors@python.org. 

October 19, 2021 03:27 PM UTC

Quansight Labs Blog

An efficient method of calling C++ functions from numba using clang++/ctypes/rbc

October 19, 2021 02:00 PM UTC

Real Python

Python Assignment Expressions and Using the Walrus Operator

Each new version of Python adds new features to the language. For Python 3.8, the biggest change is the addition of assignment expressions. Specifically, the := operator gives you a new syntax for assigning variables in the middle of expressions. This operator is colloquially known as the walrus operator.

This course is an in-depth introduction to the walrus operator. You’ll learn some of the motivations for the syntax update and explore some examples where assignment expressions can be useful.

In this course, you’ll learn how to:

• Identify the walrus operator and understand its meaning
• Understand use cases for the walrus operator
• Avoid repetitive code by using the walrus operator
• Convert between code using the walrus operator and code using other assignment methods
• Understand the impacts on backward compatibility when using the walrus operator
• Use appropriate style in your assignment expressions

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

October 19, 2021 02:00 PM UTC

• • RSS feed
  • Titles Only
  • Powered by Planet!
• Other Python Planets
  • Python Summer of Code
  • Planet Python Francophone
  • Planet Python Argentina
  • Planet Python Brasil
  • Planet Python Poland
• Python Libraries
  • OLPC
  • PySoy
  • SciPy
  • SymPy
  • Twisted
• Python/Web Planets
  • CherryPy
  • Django Community
  • Plone
  • Turbogears
• Other Languages
  • Haskell
  • Lisp
  • Parrot
  • Perl
  • Ruby
• Databases
  • MySQL
  • PostgreSQL
• Subscriptions
  • [OPML feed]
  • "Mathspp Pydon'ts"
  • "Menno's Musings"
  • "Michael J.T. O'Kelly"
  • "Morphex's Blogologue"
  • "Speno's Pythonic Avocado"
  • "William's Journal"
  • 2degrees
  • A. Jesse Jiryu Davis
  • ABlog for Sphinx
  • Aahz
  • Abhijeet Pal
  • Abu Ashraf Masnun
  • Adam Pletcher
  • Agendaless Consulting
  • Ahmed Bouchefra
  • Al-Ahmadgaid Asaad
  • Alec Munro
  • Alex Grönholm
  • Alex Morozov
  • Alexander Limi
  • Alexandre Conrad
  • Alexandre Vassalotti
  • Alexey Evseev
  • Allison Kaptur
  • Amjith Ramanujam
  • AmvTek
  • Anarcat
  • Anatoly Techtonik
  • Andre Roberge
  • Andrea Grandi
  • Andrew Dalke
  • Andriy Kornatskyy
  • Andy Dustman
  • Andy R. Terrel
  • Anna Martelli Ravenscroft
  • Anthony Baxter
  • Anton Belyaev
  • Anton Bobrov
  • Anwesha Das
  • Armin Ronacher
  • Artem Golubin
  • Artem Rys
  • Ashish Vidyarthi
  • Astro Code School
  • Automating OSINT
  • Awesome Python Applications
  • Baiju Muthukadan
  • Bajusz Tamás
  • BeDjango
  • Ben Bass
  • Ben Cook
  • Ben Rousch
  • Benjamin Peterson
  • Benji York
  • Bertrand Mathieu
  • Bhavin Gandhi
  • Bhishan Bhandari
  • BioPython News
  • Bit of Cheese
  • Bojan Mihelac
  • Brandon Rhodes
  • BreadcrumbsCollector
  • Brendan Scott
  • Brett Cannon
  • Brian Okken
  • Bruno Oliveira
  • Bruno Rocha
  • Caktus Consulting Group
  • Calvin Cheng
  • Calvin Spealman
  • Carl Chenet
  • Carl Düvel
  • Carl Trachte
  • Carlos Eduardo de Paula
  • Casey Duncan
  • Catherine Devlin
  • Ching-Hwa Yu
  • Chris Hager
  • Chris Miles
  • Chris Mitchell
  • Chris Moffitt
  • Chris Rose
  • Chris Warrick
  • Christian Heimes
  • Christian Scholz
  • Christoph Zwerschke
  • CodeGrades
  • CodeSnipers
  • Codementor
  • CodersLegacy
  • Coding Diet
  • Corey Gallon
  • Corey Goldberg
  • Cormoran Project
  • Cross-Platform Command Line Tools
  • CubicWeb
  • Curtis Miller
  • DSPIllustrations.com
  • DaPythonista
  • Daily Tech Video (Python)
  • Dallas Fort Worth Pythoneers
  • Dan Crosta
  • Dan Stromberg
  • Dan Yeaw
  • Daniel Bader
  • Daniel Nouri
  • Daniel Roy Greenfeld
  • Data Community DC
  • Data School
  • Dave Beazley
  • David Caron
  • David Goodger
  • David Lindelof
  • David MacIver
  • David Malcolm
  • David Szotten
  • Davide Moro
  • Davy Mitchell
  • Davy Wybiral
  • Denis Kurov
  • Django Weblog
  • Django Weekly
  • Djangostars
  • Doing Math with Python
  • Doug Hellmann
  • Dougal Matthews
  • Duncan McGreggor
  • Ed Crewe
  • Edward K. Ream
  • Eli Bendersky
  • Eniram Ltd.
  • Eray Özkural (examachine)
  • Erik Marsja
  • Etienne Desautels
  • EuroPython
  • EuroPython Society
  • Evennia
  • Everyday Superpowers
  • Fabio Zadrozny
  • Filip Wasilewski
  • Filipe Saraiva
  • Flavio Coelho
  • Flavio Percoco
  • Floris Bruynooghe
  • Frank Wierzbicki
  • François Dion
  • François Marier
  • Frederik Rietdijk
  • From Python Import Podcast
  • Full Stack Python
  • Gaël Varoquaux
  • Georges Dubus
  • Ghaandee on IT
  • Giampaolo Rodola
  • Giulio Fidente
  • Glenn Franxman
  • Glyph Lefkowitz
  • Go Deh
  • Gocept Weblog
  • Godson Gera
  • Graeme Cross
  • Graham Dumpleton
  • Graham Wheeler
  • Grant Baillie
  • Grant Rettke
  • Greg Taylor
  • Grig Gheorghiu
  • Grzegorz Śliwiński
  • Guido van Rossum
  • Gustavo Narea
  • Gustavo Niemeyer
  • Gökhan Sever
  • Hernan Grecco
  • Hilary Mason
  • Holger Krekel
  • Holger Peters
  • Humberto Rocha
  • Hynek Schlawack
  • Ian Ozsvald
  • Ilian Iliev
  • Import Python
  • Inspired Python
  • Ionel Cristian Maries
  • IronPython-URLs
  • IslandT
  • Israel Fruchter
  • Itamar Turner Trauring
  • Ivan Velichko
  • J. Pablo Fernández
  • Jack Diederich
  • Jacob Perkins
  • Jahongir Rahmonov
  • Jaime Buelta
  • Jamal Moir
  • James Bennett
  • Janusworx
  • Jarrod Millman
  • Jean-Paul Calderone
  • Jeff Bisbee
  • Jeff Bradberry
  • Jeff Hinrichs
  • Jeff Shell
  • Jeremy Epstein
  • Jeremy Hylton
  • Jim Fulton
  • Joe Abbate
  • Joe Pitz
  • Johan Dahlin
  • John Burns
  • John Cook
  • John Jacobsen
  • John Ludhi/nbshare.io
  • Jon Parise
  • Jonathan Ellis
  • Jonathan Harrington
  • Jonathan Hartley
  • Jonathan Street
  • Jorgen Schäfer
  • Juan Manuel Contreras
  • Julien Danjou
  • Julien Tayon
  • Juri Pakaste
  • Just a little Python
  • Justin Mayer
  • Kai Lautaportti
  • Karim Elghamrawy
  • Kay Hayen
  • Kay Schluehr
  • Kelly Yancey
  • Kodnito
  • Kogan Dev
  • Konrad Delong
  • Koodaamo
  • Kristján Valur Jónsson
  • Kriti Godey
  • Kulbir Saini
  • Kumar McMillan
  • Kumar Vipin Yadav
  • Kushal Das
  • Károly Nagy
  • LAAC Technology
  • Laurent Szyster
  • Leigh Honeywell
  • Lennart Regebro
  • Leonhard Vogt
  • Lintel Technologies
  • ListenData
  • Logilab
  • Low Kian Seong
  • Luca Botti
  • Lucas Cimon
  • Ludovic Gasc
  • Ludvig Ericson
  • Luke Plant
  • Maciej Fijalkowsk
  • Made With Mu
  • Mahmoud Hashemi
  • Malthe Borch
  • Marc Richter
  • Marc-André Lemburg
  • Marcos Dione
  • Marius Gedminas
  • Mark Dufour
  • Mark McLoughlin
  • Mark McMahon
  • Martijn Pieters
  • Martin Fitzpatrick
  • Mats Kindahl
  • Matt Layman
  • Matthew Rocklin
  • Matthew Rollings
  • Matthew Wilson
  • Matthew Wright
  • Mattias Brändström
  • Mauveweb
  • Michael Becker
  • Michael Droettboom
  • Michael Foord
  • Michael Nelson
  • Michal Kwiatkowski
  • Michał Bultrowicz
  • Michele Simionato
  • Michy Alice
  • Mike C. Fletcher
  • Mike Driscoll
  • Mike Müller
  • Mikhail Korobov
  • Mikko Ohtamaa
  • Mirek Długosz
  • Mitchell Garnaat
  • Mitya Sirenef
  • Montreal Python User Group
  • Moshe Zadka
  • Moya Project
  • Mozilla Web Development
  • Muharem Hrnjadovic
  • Mycli
  • Nadav Samet
  • Naomi Ceder
  • Natan Zabkar
  • Ned Batchelder
  • Neil Schemenauer
  • Nick Coghlan
  • Nick Craig-Wood
  • Nick Efford
  • Nick Janetakis
  • Nicola Iarocci
  • Nicolas Dumazet
  • Nicolas Paris
  • Nigel Babu
  • Nikola
  • Nikolaos Diamantis
  • Not Invented Here
  • Obey the Testing Goat
  • Ofosos
  • Omaha Python Users Group
  • Ondřej Čertík
  • Paolo Amoroso
  • Paolo Melchiorre
  • Pathwright
  • Patrice Neff
  • Patrick Kennedy
  • Paul Everitt
  • Paul Harrison
  • Paul Redman
  • Paweł Fertyk
  • Pedro Lima
  • Peter Bengtsson
  • Peter Eisentraut
  • Peter Fankhänel
  • Peter Harkins
  • Peter Hoffmann
  • Phil Hassey
  • Philip Jenvey
  • Philipp von Weitershausen
  • Philippe Normand
  • Phillip J. Eby
  • Podcast.__init__
  • Polyglot.Ninja()
  • Possbility and Probability
  • Pradeep Gowda
  • Pranav Pandey
  • Praveen Gollakota
  • Programiz
  • Programming Ideas With Jake
  • Przemysław Kołodziejczyk
  • PyATL Bytecode
  • PyBites
  • PyCharm
  • PyCoder’s Weekly
  • PyCon
  • PyPy
  • PyTennessee
  • Python 4 Kids
  • Python 411 Podcast
  • Python Advocacy
  • Python Anywhere
  • Python Bytes
  • Python Celery - Weekly Celery Tutorials and How-tos
  • Python Circle
  • Python Data
  • Python Diary
  • Python Does What?!
  • Python Engineering at Microsoft
  • Python GUIs
  • Python Insider
  • Python Morsels
  • Python Piedmont Triad User Group
  • Python Pool
  • Python Software Foundation
  • Python Sweetness
  • Python User Groups
  • Python for Beginners
  • Python on Karan
  • Python with Myo
  • Python(x,y) News
  • PythonClub - A Brazilian collaborative blog about Python
  • PythonDebugging.com
  • Pythonicity
  • Pythonology
  • Python⇒Speed
  • Péter Szabó
  • Péter Zsoldos
  • Quansight Labs Blog
  • R David Murray
  • RMOTR
  • Ralph Bean
  • Ram Rachum
  • Randell Benavidez
  • Randle Taylor
  • Randy Zwitch
  • Raymond Hettinger
  • Read the Docs
  • Real Python
  • Red Hat Developers
  • Rene Dudfield
  • Reuven Lerner
  • Richard Tew
  • Richard Wall
  • Rickard Lindberg
  • Rob Galanakis
  • Robert Collins
  • Robert Zaremba
  • Robin Parmar
  • Robin Wilson
  • Rodrigo Araúj
  • RoseHosting Blog
  • Ruslan Spivak
  • Ryan Cox
  • S. Lott
  • S. R. Krishnan
  • SDJournal
  • SPE Weblog
  • STX Next
  • Salim Fadhley
  • Sandipan Dey
  • Sandro Tosi
  • Sean McGrath
  • Sebastian Pölsterl
  • Sebastian Witowski
  • Selena Deckelmann
  • Senthil Kumaran
  • Shannon -jj Behrens
  • ShiningPanda
  • Simeon Franklin
  • Simeon Visser
  • Simon
  • Simon Brunning
  • Simon Wittber
  • Simple is Better Than Complex
  • SoftFormance
  • Speed Matters
  • Spike ekipS
  • Spyder IDE
  • Stack Abuse
  • Stanislas Morbieu
  • Starzel.de
  • Stefan Behnel
  • Stefan Scherfke
  • Stein Magnus Jodal
  • Stephen Ferg
  • Steve Holden
  • Steven Klass
  • Stuart Gordon Reid
  • Stéphane Wirtel
  • Sumana Harihareswara - Cogito, Ergo Sumana
  • Suresh Dasari/Tutlane.com
  • Swisscom ICT
  • Talk Python to Me
  • Taylor Edmiston
  • TechBeamers Python
  • Techiediaries - Django
  • Terri Oda
  • Terry Jones
  • Test and Code
  • TestDriven.io
  • The Data Scientist
  • The Digital Cat
  • The No Title® Tech Blog
  • The Occasional Occurrence
  • The Open Sourcerer
  • The Parcon Blog
  • The Python Coding Blog
  • The Python Papers
  • The Three of Wands
  • Thibauld Nion
  • ThisHosting.Rocks
  • Thomas Guest
  • Thomas Vander Stichele
  • Tibo Beijen
  • Tim Arnold / reachtim
  • Tim Gilbert
  • Tim Knapp
  • Tim Lesher
  • Tobias Ivarsson
  • Tom Christie
  • Tomasz Ducin
  • Tomaž Muraus
  • Tomer Filiba
  • Tony Breyal
  • Toshio Kuratomi
  • Travis Oliphant
  • Trey Hunner
  • Tryton News
  • Turnkey Linux
  • Twisted Matrix Labs
  • V.S. Babu
  • Varun Nischal
  • Vasudev Ram
  • Vinay Sajip
  • Vinay Sajip (Logging)
  • Vinayak Mehta
  • Virgil Dupras
  • Vladimir Perić
  • Wayne Witzel
  • Weekly Python Chat
  • Weekly Python StackOverflow Report
  • Wes Mason
  • Wesley Chun
  • Will Kahn-Greene
  • Will McGugan
  • Will Pierce
  • William Minchin
  • William Reade
  • Wing Tips
  • Wingware
  • Wyatt Baldwin
  • Yaniv Aknin
  • Yann Larrivée
  • Yuval Greenfield
  • Zaki Akhmad
  • Zato Blog
  • Zero to Mastery
  • Zero-with-Dot (Oleg Żero)
  • ZeroDB
  • bottlepy-dev
  • codeboje
  • death and gravity
  • eGenix.com
  • hypothesis.works articles
  • kdev-python
  • meejah.ca
  • nl-project
  • pgcli
  • py.CheckIO
  • pythonwise
  • qutebrowser development blog
  • saaj/recollection
  • testmon
  • tryexceptpass
  • Éric Araujo
  • Łukasz Langa
  • To request addition or removal:
    Open an issue on github