Planet Python
Last update: April 19, 2024 07:44 PM UTC
April 19, 2024
Real Python
The Real Python Podcast – Episode #201: Decoupling Systems to Get Closer to the Data
What are the benefits of using a decoupled data processing system? How do you write reusable queries for a variety of backend data platforms? This week on the show, Phillip Cloud, the lead maintainer of Ibis, will discuss this portable Python dataframe library.
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
ListenData
Complete Guide to Visual ChatGPT
In this post, we will talk about how to run Visual ChatGPT in Python with Google Colab. ChatGPT has garnered huge popularity recently due to its capability of human style response. As of now, it only provides responses in text format, which means it cannot process, generate or edit images. Microsoft recently released a solution for the same to handle images. Now you can ask ChatGPT to generate or edit the image for you.
In the image below, you can see the final output of Visual ChatGPT - how it looks like.
To read this article in full, please click here
AutoGPT : Everything You Need To Know
In this post we have covered AutoGPT in detail. By end of this tutorial, you will not only understand how it works but also will be able to run it on your system. Auto-GPT has gained a significant amount of popularity in the media. It has become one of the most talked-about topics across various social media platforms after ChatGPT. It has not only captured the attention of people in Artifical Intelligence community but also people from other background. Media outlets across countries covered it and reported how it can automate everything ranging from simple to complex tasks.
What is AutoGPT?
AutoGPT is an experimental open-source project built on the latest ChatGPT model i.e GPT-4. It is not limited to ChatGPT as it can also do web search and try to find information from internet. When a client gives us a project with instructions on what to do. We, as analysts, perform tasks to fulfill the project requirements. In the same way, by assigning a project to AutoGPT, it will do on its own all the necessary tasks to meet the project's requirements.
Let's say you ask AutoGPT to do market research about different headphones in the market. It will do web search and try to find this information on internet and produce output for you in a cleaned and structured format. See the video below.
Complete Guide to Massively Multilingual Speech (MMS) Model
In this article we have covered everything about the latest multilingual speech model from the basics of how it works to the step-by-step implementation of the model in Python.
Meta, the company that owns Facebook, released a new AI model called Massively Multilingual Speech (MMS) that can convert text to speech and speech to text in over 1,100 languages. It is available for free. It will not only help academicians and researchers across the world but also language preservationists or activists to document and preserve endangered languages to prevent their extinction.
MMS is trained on a large dataset of text and audio in over 1,100 languages. Another best part about the model is that it generates audio which sounds very natural, like human speech. It is also able to identify more than 4,000 spoken languages.
To read this article in full, please click here
Transformers Agent: AI Tool That Automates Everything
We have a new AI tool in the market called Transformers Agent which is so powerful that it can automate just about any task you can think of. It can generate and edit images, video, audio, answer questions about documents, convert speech to text and do a lot of other things.
Hugging Face, a well-known name in the open-source AI world, released Transformers Agent that provides a natural language API on top of transformers. The API is designed to be easy to use. With a single line code, it provides a variety of tools for performing natural language tasks, such as question answering, image generation, video generation, text to speech, text classification, and summarization.
Talk Python to Me
#457: Software Supply Chain Security with Phylum
We've spoken previously about security and software supply chains and we are back at it this episode. We're diving in again with Charles Coggins. Charles works at a software supply chain company and is on to give us the insiders and defender's perspective on how to keep our Python apps and infrastructure safe.<br/> <br/> <strong>Episode sponsors</strong><br/> <br/> <a href='https://talkpython.fm/sentry'>Sentry Error Monitoring, Code TALKPYTHON</a><br> <a href='https://talkpython.fm/mailtrap'>Mailtrap</a><br> <a href='https://talkpython.fm/training'>Talk Python Courses</a><br/> <br/> <strong>Links from the show</strong><br/> <br/> <div><b>Pick a Python Lockfile and Improve Security</b>: <a href="https://blog.phylum.io/pick-a-python-lockfile-and-improve-security/" target="_blank" rel="noopener">blog.phylum.io</a><br/> <b>Bad Beat Poetry</b>: <a href="https://blog.phylum.io/bad-beat-poetry/" target="_blank" rel="noopener">blog.phylum.io</a><br/> <b>PEP 665 – A file format to list Python dependencies for reproducibility of an application</b>: <a href="https://peps.python.org/pep-0665/" target="_blank" rel="noopener">peps.python.org</a><br/> <b>PEP 517 – A build-system independent format for source trees</b>: <a href="https://peps.python.org/pep-0517/" target="_blank" rel="noopener">peps.python.org</a><br/> <b>PEP 518 – Specifying Minimum Build System Requirements for Python Projects</b>: <a href="https://peps.python.org/pep-0518/" target="_blank" rel="noopener">peps.python.org</a><br/> <b>Lockfiles should be committed on all projects</b>: <a href="https://classic.yarnpkg.com/blog/2016/11/24/lockfiles-for-all/" target="_blank" rel="noopener">classic.yarnpkg.com</a><br/> <b>An Overview of Software Supply Chain Security</b>: <a href="https://tldrsec.com/p/supply-chain-security-overview" target="_blank" rel="noopener">tldrsec.com</a><br/> <b>Typosquatting</b>: <a href="https://docs.phylum.io/analytics/typosquatting" target="_blank" rel="noopener">docs.phylum.io</a><br/> <b>Common Attack Pattern Enumeration and Classification</b>: <a href="https://capec.mitre.org/data/definitions/693.html" target="_blank" rel="noopener">capec.mitre.org</a><br/> <b>Dependency Confusion</b>: <a href="https://docs.phylum.io/analytics/dependency_confusion" target="_blank" rel="noopener">docs.phylum.io</a><br/> <b>Expired Author Domains</b>: <a href="https://docs.phylum.io/analytics/expired_author_domains" target="_blank" rel="noopener">docs.phylum.io</a><br/> <b>Unverifiable Dependency</b>: <a href="https://docs.phylum.io/analytics/odd_dependency" target="_blank" rel="noopener">docs.phylum.io</a><br/> <b>Repo Jacking: Hidden Danger in Broken Links</b>: <a href="https://blog.phylum.io/repojacking-software-supply-chain-vulnerability/" target="_blank" rel="noopener">blog.phylum.io</a><br/> <b>Software Libraries Are Terrifying</b>: <a href="https://medium.com/@dmrickert/software-libraries-are-terrifying-4875b6a74be6" target="_blank" rel="noopener">medium.com</a><br/> <b>phylum 0.43.0</b>: <a href="https://pypi.org/project/phylum/" target="_blank" rel="noopener">pypi.org</a><br/> <b>linguist</b>: <a href="https://github.com/github-linguist/linguist" target="_blank" rel="noopener">github.com</a><br/> <b>rich-codex ⚡️📖⚡️</b>: <a href="https://ewels.github.io/rich-codex/" target="_blank" rel="noopener">ewels.github.io</a><br/> <b>Phylum Community Discord</b>: <a href="https://discord.gg/Fe6pr5eW6p" target="_blank" rel="noopener">discord.gg</a><br/> <b>The dream is dead?</b>: <a href="https://mastodon.social/@tveskov/111289358585305218" target="_blank" rel="noopener">mastodon.social</a><br/> <b>When "Everything" Becomes Too Much: The npm Package Chaos of 2024</b>: <a href="https://socket.dev/blog/when-everything-becomes-too-much?utm_source=tldrnewsletter" target="_blank" rel="noopener">socket.dev</a><br/> <b>pip-tools</b>: <a href="https://github.com/jazzband/pip-tools" target="_blank" rel="noopener">github.com</a><br/> <b>Watch this episode on YouTube</b>: <a href="https://www.youtube.com/watch?v=uB-2nMphYBI" target="_blank" rel="noopener">youtube.com</a><br/> <b>Episode transcripts</b>: <a href="https://talkpython.fm/episodes/transcript/457/software-supply-chain-security-with-phylum" target="_blank" rel="noopener">talkpython.fm</a><br/> <br/> <b>--- Stay in touch with us ---</b><br/> <b>Subscribe to us on YouTube</b>: <a href="https://talkpython.fm/youtube" target="_blank" rel="noopener">youtube.com</a><br/> <b>Follow Talk Python on Mastodon</b>: <a href="https://fosstodon.org/web/@talkpython" target="_blank" rel="noopener"><i class="fa-brands fa-mastodon"></i>talkpython</a><br/> <b>Follow Michael on Mastodon</b>: <a href="https://fosstodon.org/web/@mkennedy" target="_blank" rel="noopener"><i class="fa-brands fa-mastodon"></i>mkennedy</a><br/></div>
April 18, 2024
Test and Code
218: Balancing test coverage with test costs - Nicole Tietz-Sokolskaya
Nicole is a software engineer and writer, and recently wrote about the trade-offs we make when deciding which tests to write and how much testing is enough.
We talk about:
- Balancing schedule vs testing
- How much testing is the right about of testing
- Should code coverage be measured and tracked
- Good refactoring can reduce code coverage
- Is it worth testing error conditions?
- Are rare error codes ok to just monitor?
- API drift and autospec
- Mitigating risk
- Deciding what to test and what not to test
- Focus testing on key money-making features
- If there's a bug in this part of the code, how much business impact is there?
- Performance testing needs to approximately match real world workloads
- Cost of a service breaking vs the cost of creating, maintaining, and running tests
- Keeping test suites quick to minimize getting distracted
Links:
- Too much of a good thing: the trade-off we make with tests
- Load testing is hard, and the tools are... not great. But why?
- Yet Another Rust Resource (YARR!)
- Goodhart's law - "When a measure becomes a target, it ceases to be a good measure"
Sponsored by Mailtrap.io
- An Email Delivery Platform that developers love.
- An email-sending solution with industry-best analytics, SMTP, and email API, SDKs for major programming languages, and 24/7 human support.
- Try for Free at MAILTRAP.IO
Sponsored by PyCharm Pro
- Use code PYTEST for 20% off PyCharm Professional at jetbrains.com/pycharm
- Now with Full Line Code Completion
- See how easy it is to run pytest from PyCharm at pythontest.com/pycharm
The Complete pytest Course
- For the fastest way to learn pytest, go to courses.pythontest.com
- Whether your new to testing or pytest, or just want to maximize your efficiency and effectiveness when testing.
April 17, 2024
Tryton News
Security Release for issue #13142
Cédric Krier has found that trytond accepts compressed content from unauthenticated requests which makes it vulnerable to zip bomb attacks.
Impact
- Attack Vector: Network
- Attack Complexity: Low
- Privileges Required: None
- User Interaction: None
- Scope: Unchanged
- Confidentiality: None
- Integrity: None
- Availability: Low
Workaround
A proxy can be deployed in front of the trytond server to forbid this kind of request.
Resolution
All affected users should upgrade trytond to the latest version.
Affected versions per series:
trytond:- 7.0: <= 7.0.9
- 6.8: <= 6.8.14
- 6.0: <= 6.0.44
Non affected versions per series:
trytond:- 7.0: >= 7.0.10
- 6.8: >= 6.8.15
- 6.0: >= 6.0.45
Reference
Concerns?
Any security concerns should be reported on the bug-tracker at https://bugs.tryton.org/ with the confidential checkbox checked.
2 posts - 1 participant
Mike Driscoll
Announcing The Python Logging Book & Course Kickstarter
What does every new developer do when they are first learning to program? They print out strings to their terminal. It’s how we learn! But printing out to the terminal isn’t what you do with most professional applications.
In those cases, you log into files. Sometimes, you log into multiple locations at once. These logs may serve as an audit trail for compliance purposes or help the engineers debug what went wrong.
Python Logging teaches you how to log in the Python programming language. Python is one of the most popular programming languages in the world. Python comes with a logging module that makes logging easy.
What You’ll Learn
In this book, you will learn how about the following:
- Logger objects
- Log levels
- Log handlers
- Formatting your logs
- Log configuration
- Logging decorators
- Rotating logs
- Logging and concurrency
- and more!
Book formats
The finished book will be made available in the following formats:
- paperback (at the appropriate reward level)
- epub
The paperback is a 6″ x 9″ book and is approximately 150 pages long.
You can support the book by clicking the button below:
The post Announcing The Python Logging Book & Course Kickstarter appeared first on Mouse Vs Python.
Real Python
How to Format Floats Within F-Strings in Python
You’ll often need to format and round a Python float to display the results of your calculations neatly within strings. In earlier versions of Python, this was a messy thing to do because you needed to round your numbers first and then use either string concatenation or the old string formatting technique to do this for you.
Since Python 3.6, the literal string interpolation, more commonly known as a formatted string literal or f-string, allows you to customize the content of your strings in a more readable way.
An f-string is a literal string prefixed with a lowercase or uppercase letter f and contains zero or more replacement fields enclosed within a pair of curly braces {...}. Each field contains an expression that produces a value. You can calculate the field’s content, but you can also use function calls or even variables.
While most strings have a constant value, f-strings are evaluated at runtime. This makes it possible for you to pass different data into the replacement fields of the same f-string and produce different output. This extensibility of f-strings makes them a great way to embed dynamic content neatly inside strings. However, even though f-strings have largely replaced the earlier methods, they do have their short-comings.
For example, one of the most common attacks performed on a relational database is a SQL injection attack. Often, users provide parameters to SQL queries, and if the query is formed within an f-string, it may be possible to damage a database by passing in rogue commands. F-strings can also be used in a denial-of-service attack by attacking Python’s logging module code.
In older versions of Python, f-strings had a number of other limitations that were only fixed with Python version 3.12. This version is used throughout this tutorial.
Take a look at the example below. It shows you how to embed the result of a calculation within an f-string:
>>> f"One third, expressed as a float is: {1 / 3}"
'One third, expressed as a float is: 0.3333333333333333'
Without any explicit rounding, once an expression has produced its value it’ll be inserted into the string using a default number of decimal places. Here, the result is shown to sixteen decimal places, which, in most cases, is more precision than you’ll ever need. So you’ll likely want to round your final answer to a more practical number of digits.
In this tutorial, you’ll learn how to use a Python f-string to format floats to meet your requirements.
Get Your Code: Click here to download the free sample code and exercise solutions you’ll use for learning how to format floats within f-strings in Python.
Take the Quiz: Test your knowledge with our interactive “Format Floats Within F-Strings” quiz. Upon completion you will receive a score so you can track your learning progress over time:
Interactive Quiz
Format Floats Within F-Strings
In this quiz, you'll test your understanding of how to format floats within f-strings in Python. This knowledge will let you control the precision and appearance of floating-point numbers when you incorporate them into formatted strings.
How to Format and Round a Float Within a Python F-String
To format a float for neat display within a Python f-string, you can use a format specifier. In its most basic form, this allows you to define the precision, or number of decimal places, the float will be displayed with.
The code below displays the same calculation as before, only it’s displayed more neatly:
>>> f"One third, rounded to two decimal places is: {1 / 3:.2f}"
'One third, rounded to two decimal places is: 0.33'
To use Python’s format specifiers in a replacement field, you separate them from the expression with a colon (:). As you can see, your float has been rounded to two decimal places. You achieved this by adding the format specifier .2f into the replacement field. The 2 is the precision, while the lowercase f is an example of a presentation type. You’ll see more of these later.
Note: When you use a format specifier, you don’t actually change the underlying number. You only improve its display.
Python’s f-strings also have their own mini-language that allows you to format your output in a variety of different ways. Although this tutorial will focus on rounding, this is certainly not the only thing you can use them for. As you’ll see later, their mini-language is also used in other string formatting techniques.
In addition to displaying the result of calculations, the precision part of a format specifier can also be applied directly to variables and the return values of function calls:
>>> def total_price(cost):
... return cost * 1.2
...
>>> cost_price = 1000
>>> tax = 0.2
>>> f"£{1000:,.2f} + £{cost_price * tax:,.2f} = £{total_price(cost_price):,.2f}"
'£1,000.00 + £200.00 = £1,200.00'
This time, you’ve used multiple replacement fields in the same string. The first one formats a literal number, the second formats the result of a calculation, while the third formats the return value from a function call. Also, by inserting a comma (,) before the decimal point (.) in the format specifier, you add a thousands separator to your final output.
In everyday use, you display numbers with a fixed amount of decimals, but when performing scientific or engineering calculations, you may prefer to format them using significant figures. Your results are then assumed to be accurate to the number of significant figures you display them with.
If you want to round numbers to significant figures, you use the lowercase letter g in the format specifier. You can also use an uppercase G, but this automatically switches the format to scientific notation for large numbers.
Suppose you have a circle with a radius of 10.203 meters. To work out the area, you could use this code:
Read the full article at https://realpython.com/how-to-python-f-string-format-float/ »
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
PyBites
OpenStreetMaps, Overpass API and Python
OpenStreetMaps (OSM) is known for being an open source project that allows people to browse the world map and to plan routes. However it is more than that. Among others it provides a read-only API that allows users to query for very diverse map data: Overpass API
Data structure of OSM
To understand the structure of the queries, we first need to understand how OSM saves its data. OSM uses basic data structures: node, way, and relation. Each element can have multiple tags, which consist of a key-value pair and they help specify the features of each element.
Nodes also save a latitude and longitude, indicating where the node is located on the world. Ways contain a list of the nodes contained in the way. Finally, relations are groups of nodes, ways and relations defining a logical relationship for this elements, for example being part of the same street, neighbourhood, city, postal code area, etc.
If we go to https://www.openstreetmap.org/, right-click on some point of the map and then click on Query Features, we get a list of the elements of the area. By clicking on one of them, we can explore it’s characteristics. As an example, we can explore the node 1918872432, which we can see on the left picture. Looking at the data we can see that it is located at longitude 52.5024783 and latitude 13.4155630. On the tags we can see it has the name Oranienplatz and among others it contains a bench and a bin. The key highway contains the value bus_stop, which tells us it is a bus stop.
If we explore the way 1228261444 (right picture) we can see a list of all the nodes it contains. In addition on the key highway we can see it’s of the type footway, telling us it’s a way intended for pedestrians. In addition we can see that it is a lit way.
Many keywords like highway are normalized. As an example, we can find a list of standardized values for highway here.
Overpass Queries
Querying a single element
We can query the previous information using Overpass QL. For the query to get the information about the way 1228261444 we use way(1228261444). The default output format is in xml, so if we want to have it in json, we need to add [out:json] in front of it. Other possible formats are csv, for which we need to define the list of columns for it to work. We mark the end of the query with an out, so the complete query is:
[out:json];
way(1228261444);
out;To execute the query we need to run it against an instance, thus the complete URL looks like this:
https://overpass-api.de/api/interpreter?data=[out:json];way(1228261444);out;We can copy the URL directly on the browser, or we can use wget to see the result:
wget -O my_file_name.osm "https://overpass-api.de/api/interpreter?data=[out:json];way(1228261444);out;"To query a single node or relation works in a parallel way.
Output format
{
"version": 0.6,
"generator": "Overpass API 0.7.61.5 4133829e",
"osm3s": {
"timestamp_osm_base": "2024-03-06T20:54:14Z",
"copyright": "The data included in this document is from www.openstreetmap.org. The data is made available under ODbL."
},
"elements": [
{
"type": "way",
"id": 1228261444,
"nodes": [
9010521154,
9010521155,
11390038869,
11390038870,
9394918823,
9394918824,
4613855449,
11390038871,
11390038872,
11390038873,
9394918822
],
"tags": {
"footway": "sidewalk",
"highway": "footway",
"lit": "yes",
"surface": "paving_stones"
}
}
]
}
Here we can see two main blocks: the first with metadata like the version or the timestamp, and the second one with a list of the elements found with the query. Even if we queried for one specific element of type way, we still get it as a set formatted as list with length 1.
Querying many elements
If we don’t want the information of one singe way, but the information of all ways in an area, we need to change the query a little bit:
[out:json];
way(52.49505089381484,13.401961459729343,52.509912981033516,13.411502705004548);
out;Here, instead of requesting one single way, we put 4 coordinates in the parenthesis: boundaries in the south, west, north and east.
This returns a set of all the ways contained in the bounding box defined by the given coordinates.
It is not always needed to know the coordinates to get information for an area. With the query
[out:json];
area[name="Zürich"];
out;we get all areas called Zürich. In this example we can see that we can use special characters without problem. Most interestingly are both square parentheses behind area [name=”Zürich”]. This is a filter where we only get the areas that contain a tag with key name and value Zürich.
We can use more than one filter, for example to get only the element representing the city Zürich, in contrast to the canton Zürich. In that case we can just place them one after another like this:
[out:json];
area[name=”Zürich”][place=”city”];
out;We can filter by elements containing a certain tag independently from it’s value.
[out:json];
area[name=”Zürich”][place];
out;This returns a set of all areas containing the tag place, which in this case contains the same element as above.
Subqueries
We can use the area we got in the previous query to get a set of certain elements inside said area. For example, if we want to get all public toilets in Zürich, we can use the following query:
[out:json];
area[name="Zürich"][place="city"];
node[amenity="toilets"](area);
out;First we get the area corresponding to the city Zürich, and then we get all nodes whose tag key amenity contains the value toilets. By passing the area between the round parenthesis we limit the set of nodes to the previous area.
The tag amenity describes a large number of important facilities from cafés, public bookcases, benches, vending machines, ATMs, etc.
Other useful tag names to find interesting places are tourism and historic.
TIL: there is also a value for public baking ovens, baking_oven. Apparently there are 2 of those which Zürich, which looks to me like a good reason to visit that city again. Seriously, how awesome is that?
Using overpass in Python
There are two ways to execute overpass queries with python that we will explore. One possibility is using the library and the other the library overpass-api-python-wrapper.
Requests
Example:
import requests
overpass_url = 'https://overpass-api.de/api/interpreter'
overpass_query = '''
[out:json];
area[name="Zürich"][place="city"];
node[amenity="toilets"](area);
out;
'''
response = requests.get(overpass_url, params={'data': overpass_query})This way we send to the overpass interpreter the query in overpass_query and save the result in the variable response. If no error happened, response.text contains the data as string so we can decode it with:
decoded_data =json.loads(response.text)If we had requested the information in xml or csv format, we would decode it accordingly.
Overpass-api-python-wrapper
The overpass-api-python-wrapper allows us to execute queries in a more concise and elegant way. First we need to initialize an overpass object.
import overpass
api = overpass.API(endpoint="https://overpass.myserver/interpreter", timeout=25)All parameters are optional and the default values are the ones shown here.
query = 'area[name="Zürich"][place="city"];node[amenity="toilets"](area);'
response = api.get(query, responseformat='json')By default, response contains a dictionary representing the returned json object, however we can define the output format with the parameter responseformat. With this query, the response is
{
"features": [
{
"geometry": {
"coordinates": [
8.534562,
47.350221
],
"type": "Point"
},
"id": 75743585,
"properties": {
"amenity": "toilets",
"capacity": "4",
"check_date:wheelchair": "2022-06-06",
"fee": "no",
"name": "Landiwiese",
"opening_hours": "24/7",
"operator": "Z\u00fcriWC",
"unisex": "yes",
"wheelchair": "no"
},
"type": "Feature"
},
etc
}If we use xml as format, response will be a string, and with csv it will be a list.
Other Useful tools:
We can find a complete manual on Overpass QL here, where we can read about more awesome possibilities that Overpass has to offer to us.
You can also try out queries with overpass-turbo. However it will only display the returned nodes on the map, but you can also get the returned response as text in the tab Data.
Finally, you can find another cool tutorial for using overpass with python here. In this case, the author shows us new possibilities with overpass, including how to plot the queried data in plots.
I hope you could awake your curiosity about the potential that OpenStreetMaps and Overpass offers. For me in meant not only to learn about a powerful tool, but while exploring it and playing with it it allowed me to learn more about the possibilities our cities and towns have to offer to us. Did you know that there are public showers, including in some train stations? Or that there are public ovens?
What I finally have to say is: keep calm and keep building cool stuff!
Seth Michael Larson
Microsoft supports urllib3 with FOSS Fund 2024
Microsoft supports urllib3 with FOSS Fund 2024
Back in January we announced that urllib3 would be fundraising to implement support for HTTP/2 in a backwards compatible way to urllib3 v2.x and to ensure the project's development remains sustainable in the long-term.
Microsoft has awarded urllib3 $5,000 USD as one of its FOSS Fund recipients for 2024 🥳 Thank you, Microsoft for supporting open source software!
Since announcing our fundraiser we have raised $7,275 USD in 3 months, many of the donations coming from individuals and long-time sponsors like Sourcegraph and Sentry.
Thanks Microsoft! ❤️
How to get started supporting your own dependencies?
Contribute directly to projects you depend on. Many of them already have funding mechanisms documented like Open Collective, GitHub Sponsors, or similar.
Subscribe to an organization like Tidelift to handle discovery of dependencies and fundable projects and to get additional guarantees like security and long-term support.
Thanks.dev provides a lightweight option to throw money in open sources' direction.
Thanks for reading! ♡ Did you find this article helpful and want more content like it? Get notified of new posts by subscribing to the RSS feed or the email newsletter.
This work is licensed under CC BY-SA 4.0 
April 16, 2024
Python Engineering at Microsoft
Glow up! A new look for Python Reference documentation on Microsoft Learn
Today, we’re excited to announce a new, improved experience for Python library reference documentation on Microsoft Learn, formerly Microsoft Docs. The new experience is now available for all Microsoft Python libraries on Learn, such as the Azure SDK for Python. In this blog post, we take a closer look at the specific changes made and how they improve the overall experience and accessibility of Python reference documentation on Learn.
Change Summary
The key changes made to Python reference documentation on Learn were improving the navigation experience with the table of contents (TOC), the organization of in-page content, and in-page navigation via the right rail. When combined, these three key changes greatly improve the experience of finding information in Python reference on Learn.
Table of Contents (TOC)
If you’ve used reference documentation before, you’ll know that the TOC serves as a uniquely pivotal tool for navigation between pages. Reference documentation, unlike conceptual articles and tutorials, rarely links together in an easy-to-follow way. This means that the TOC is often the only way for cross-page navigation in reference documentation, making it a key priority for us to get right.
TOC – Before
Previously, the table of contents for Python Reference documentation on Learn had a redundant layer of navigation for package names (in the screenshot below, the azure-storage-blob node), which added confusion for users looking to navigate deeper into the documentation. In addition, the table of contents had package prefixes attached to every item. This led to a cluttered experience, with some items in the TOC taking up two lines of space, further impacting readability and accessibility.
TOC – After
The new TOC design removes the package name overview page and trims the package prefix from TOC items, making it easier to scan and navigate. If you’ve bookmarked a package name overview page, don’t worry, as you’ll be redirected to the package overview page automatically.
In-Page Right Rail
The in-page right rail is a key part of reference documentation, allowing you to navigate in-page and quickly jump to different sections.
In-Page Right Rail – Before
Before, the right rail wasn’t “sticky”, meaning that once users scrolled on the page, the rail wouldn’t follow them, and the entire right side of the screen would be unused space. Additionally, the right rail links only covered H2 content on the page, resulting in a right rail that didn’t clarify any major details on the page. Finally, the right rail would be collapsed beyond four items, leading to extra effort from the user to see all of the H2 sections on the page. We recognize this design wasn’t consistent with the ways that other Python libraries design their documentation and made updates to better align with design patterns and expectations.
In-Page Right Rail – After
The enhanced experience adds an individually scrollable, sticky right rail with details down to H3s, giving much more detail into what content the page has. The right rail additionally has active highlighting for the current item on the screen, making it easier for you to understand your current position relative to other content.
Parameter, Return, and Exception Organization
While the first two major feature changes focus on both overall and in-page navigation, we also identified an opportunity to improve in-page organization as well. When you’re looking at our reference documentation, being able to easily identify key information such as parameters, return values, and exceptions is crucial.
In-Page Organization – Before
Before the updates, these details were listed but not clearly separated between classes, leading to a confusing experience where it’s not clear where items started or ended. Additionally, as a result of the formatting of these values, a significant amount of vertical space was used, leading to a lot of scrolling to find the information needed. Lastly, required parameters, default values, and keyword-only parameters weren’t clearly highlighted to follow accessibility best practices, resulting in extra effort to find crucial information.
In-Page Organization – After
The new experience organizes parameters, return values, and exceptions into easy-to-scan tables. By using tables, default values, required tags, and types are clearly indicated. The end result is a cleaner and easier-to-use experience for all developers when scrolling through our documentation.
Conclusion
In short, the new Python reference documentation experience on Learn is a major improvement for any developer using Microsoft’s Python libraries. The new TOC, in-page right rail, and in-page organization changes make finding information easier, with an added visual upgrade. We’re excited to continue optimizing the Python developer experience across all of our libraries and tools.
We hope you enjoy the changes! If you have any thoughts, you can leave feedback in the Azure SDK for Python GitHub repo.
The post Glow up! A new look for Python Reference documentation on Microsoft Learn appeared first on Python.
Philippe Normand
From WebKit/GStreamer to rust-av, a journey on our stack’s layers
In this post I’ll try to document the journey starting from a WebKit issue and ending up improving third-party projects that WebKitGTK and WPEWebKit depend on.
I’ve been working on WebKit’s GStreamer backends for a while. Usually some new feature needed on WebKit side would trigger work …
PyCoder’s Weekly
Issue #625 (April 16, 2024)
#625 – APRIL 16, 2024
View in Browser »
Pydantic: Simplifying Data Validation in Python
Discover the power of Pydantic, Python’s most popular data parsing, validation, and serialization library. In this hands-on tutorial, you’ll learn how to make your code more robust, trustworthy, and easier to debug with Pydantic.
REAL PYTHON
Best Python Security Practices for Web Developers
Coding on the web means you have to be more security conscious as everyone has access to your software. This article discusses key steps you can take to help make your code more secure.
ARJAN
Python Error and Performance Monitoring That Doesn’t Suck
With Sentry, you can trace issues from the frontend to the backend—detecting slow and broken code, to fix what’s broken faster. Installing the Python SDK is super easy and PyCoder’s Weekly subscribers get three full months of the team plan. Just use code “pycoder” on signup →
SENTRY sponsor
The State of Python in 2024
Michael of TalkPython fame gave this keynote at PyCon Philippines. He addresses topics such as Python trends in code, asyncio, Flask and Django, getting rid of the GIL and more.
MICHAEL KENNEDY video
Discussions
Articles & Tutorials
Optimizing Django’s System Check Framework
“Django’s system check framework provides fantastic protection for configuration mishaps. It’s like a targeted linter that runs when you start Django commands.” It also can be a bit slow for large projects. As part of the Django 5.1 release many optimizations have been applied. This blog posts discusses the work involved.
ADAM JOHNSON
A Lazy and Flippant Classification of Programming Languages
“When talking flippantly about programming languages, it’s sometimes useful to glob a bunch of them together based on characteristics, whether superficial or not. This started with jokes about common features of ‘P languages’ like Perl, PHP, Python, JavaScript and Ruby, but you can classify other languages too.”
BRYCE KERLEY
Generating QR Codes With Python
In this video course, you’ll learn how to use Python to generate QR codes, from your standard black-and-white QR codes to beautiful ones with your favorite colors. You’ll learn how to format QR codes, rotate them, and even replace the static background with moving images.
REAL PYTHON course
So Your Teacher Wants You to Do Open Source
Sometimes teachers or mentors ask students to contribute to an open source project, without the context of what that entails. This opinion piece covers just how much noise that causes for the projects and why you shouldn’t do it unless you truly mean to contribute.
DAVID LORD
Adventures in Import-Land
Ashlynn ran into a problem where her code was initializing a connection to the Google Cloud API before the credentials for it had been imported. This blog post covers the problem and how she got around it.
ASHLYNN ANTROBUS
503 Days Working Full-Time on FOSS: Lessons Learned
For a year and a half, Rodrigo worked at Textualize the company behind the popular open source Python projects Rich and Textual. This blog post talks about what he learned while he was there.
RODRIGO GIRÃO SERRÃO
Python F-String Codes I Use Every Day
Like many of us, Pawel uses f-strings a lot. Even with frequent use, it can be hard to remember how the formatting syntax works. This post covers some of the more common formats around.
PAWEL JASTRZEBSKI
Clean Code in Python
What’s it mean to write clean code? What is clean Python code, specifically? This article talks about how to write code that is easier to read and the tools you can use to get there.
NIK TOMAZIC
Predicting Solar Eclipses With Python
Were you in the path of the eclipse last week? Would you like to figure out if you’ll be in the next one? This article shows you how using the Astropy library.
ERIK BERNHARDSSON
Video Subtitling With Whisper, FFmpeg, and Python
This guide walks you through transcribing video using the OpenAI Whisper model and seamlessly adding subtitles with FFmpeg tool.
EDITFRAME
Projects & Code
Events
Weekly Real Python Office Hours Q&A (Virtual)
April 17, 2024
REALPYTHON.COM
PyData Bristol Meetup
April 18, 2024
MEETUP.COM
PyLadies Dublin
April 18, 2024
PYLADIES.COM
PyTexas 2024
April 19 to April 22, 2024
PYTEXAS.ORG
Django Girls Ekpoma Workshop
April 19 to April 21, 2024
DJANGOGIRLS.ORG
Chattanooga Python User Group
April 19 to April 20, 2024
MEETUP.COM
PyCon DE & PyData Berlin 2024
April 22 to April 25, 2024
PYCON.DE
Happy Pythoning!
This was PyCoder’s Weekly Issue #625.
View in Browser »
[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]
PyCon
Support PyLadies: Donate to the PyLadies Charity Auction at PyCon US 2024!
PyCon US 2024 is quickly approaching and we can’t wait to see all of you there! This year, we are reaching out to our community to help support one of our favorite events: the PyLadies Charity Auction. We know our wonderful community is bursting with creative, artsy, and generous folks. That’s why we are sending out a community wide call to donate items to this year’s auction. If you’d like to support PyLadies by making a donation, please fill out the PyLadies Charity Auction donation form. To learn more about the event and what to donate, read on!
What to Donate
Need inspiration? Over the years, we’ve seen that people love to bid on items that are artsy, nerdy, and Python-specific. Art, decor, customized tech, and vintage PyCon or Python related swag are all examples of things that will inspire a friendly bidding war. To get more inspiration search ‘PyLadies Auction’ on social media or check out the PyLadies Instagram.
Items that are personalized or have a story tend to attract bidders– DIYers, artists, crafters, and collectors, this is your time to shine! It's also worth thinking about what will show up well on stage. We have a magnifying camera, but smaller and less colorful items like jewelry and watches might be harder for bidders to see than a big piece of art or an adorable, bright colored snake stuffy!
Please note that we cannot accept donations of guns, alcohol, cars, real estate, or houses. We may decline other gifts that are inappropriate or impractical at the PSF’s discretion (for example, sorry, no real pythons!).
If you’ve got an item to contribute that you’d like to share in a sneak preview, feel free to post a photo and tag us on the social media platform of your choice!
Donation details
To donate an item, please fill out the PyLadies Charity Auction donation form. This form asks for required information around the person or organization donating, item specifics, and logistics. You will need to indicate if you will ship the item in advance or deliver it in person. You will receive shipping information once you complete the form. The deadline for donations is May 16th, 2024 (that’s the Thursday before the event!).
About the PyLadies Charity Auction
The twelfth PyLadies Charity Auction will be held in-person during PyCon US 2024 in Pittsburgh, Pennsylvania. It’s an evening of fun and refreshments while supporting the PyLadies community!
The Entry cover charge of $35.00 includes dinner and a drink ticket, while the Supporter cover charge of $50.00 includes dinner, a drink ticket, and a $15.00 contribution to PyLadies. Everyone who attends will also have access to the cash bar. To save your place at the Auction, add the PyLadies Auction to your PyCon US registration via your dashboard. Last year we sold out quickly, so if you’d like to go, reserve your ticket soon!
Python Morsels
Python Big O: the time complexities of different data structures in Python
The time complexity of common operations on Python's many data structures.
Table of contents
Time Complexity ⏱️
Time complexity is one of those Computer Science concepts that's scary in its purest form, but often fairly practical as a rough "am I doing this right" measurement.
In the words of Ned Batchelder, time complexity is all about "how your code slows as your data grows".
Time complexity is usually discussed in terms of "Big O" notation.
This is basically a way to discuss the order of magnitude for a given operation while ignoring the exact number of computations it needs.
In "Big O" land, we don't care if something is twice as slow, but we do care whether it's n times slower where n is the length of our list/set/slice/etc.
Here's a graph of the common time complexity curves:
Remember that these lines are simply about orders of magnitude.
If an operation is on the order of n, that means 100 times more data will slow things down about 100 times.
If an operation is on the order of n² (that's n*n), that means 100 times more data will slow things down 100*100 times.
I usually think about those curves in terms of what would happen if we suddenly had 1,000 times more data to work with:
O(1): no change in time (constant time!)O(log n): ~10 times slow downO(n): 1,000 times slow downO(n log n): 10,000 times slow downO(n²): 1,000,000 times slow down! 😲
With that very quick recap behind us, let's take a look at the relative speeds of all common operations on each of Python's data structures.
List 📋
Python's lists are similar to …
Read the full article: https://www.pythonmorsels.com/time-complexities/
Real Python
Using raise for Effective Exceptions
In your Python journey, you’ll come across situations where you need to signal that something is going wrong in your code. For example, maybe a file doesn’t exist, a network or database connection fails, or your code gets invalid input. A common approach to tackle these issues is to raise an exception, notifying the user that an error has occurred. That’s what Python’s raise statement is for.
Learning about the raise statement allows you to effectively handle errors and exceptional situations in your code. This way, you’ll develop more robust programs and higher-quality code.
In this video course, you’ll learn how to:
- Raise exceptions in Python using the
raisestatement - Decide which exceptions to raise and when to raise them in your code
- Explore common use cases for raising exceptions in Python
- Apply best practices for raising exceptions in your Python code
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
Talk Python to Me
#456: Building GPT Actions with FastAPI and Pydantic
Do you know what custom GPTs are? They're configurable and shareable chat experiences with a name, logo, custom instructions, conversation starters, access to OpenAI tools, and custom API actions. And, you can build them with Python! Ian Maurer has been doing just that and is here to share his experience building them.<br/> <br/> <strong>Episode sponsors</strong><br/> <br/> <a href='https://talkpython.fm/sentry'>Sentry Error Monitoring, Code TALKPYTHON</a><br> <a href='https://talkpython.fm/neo4j-notes'>Neo4j</a><br> <a href='https://talkpython.fm/training'>Talk Python Courses</a><br/> <br/> <strong>Links from the show</strong><br/> <br/> <div><b>Ian on Twitter</b>: <a href="https://twitter.com/imaurer" target="_blank" rel="noopener">@imaurer</a><br/> <br/> <b>Mobile Navigation</b>: <a href="https://openai.com/blog/introducing-gpts" target="_blank" rel="noopener">openai.com</a><br/> <b>What is a Custom GPT?</b>: <a href="https://www.imaurer.com/what-is-a-custom-gpt/" target="_blank" rel="noopener">imaurer.com</a><br/> <b>Mobile Navigation</b>: <a href="https://openai.com/blog/introducing-the-gpt-store" target="_blank" rel="noopener">openai.com</a><br/> <b>FuzzTypes: Pydantic library for auto-correcting types</b>: <a href="https://github.com/genomoncology/FuzzTypes" target="_blank" rel="noopener">github.com</a><br/> <b>pypi-gpt</b>: <a href="https://github.com/imaurer/pypi-gpt" target="_blank" rel="noopener">github.com</a><br/> <b>marvin</b>: <a href="https://github.com/prefecthq/marvin" target="_blank" rel="noopener">github.com</a><br/> <b>instructor</b>: <a href="https://github.com/jxnl/instructor" target="_blank" rel="noopener">github.com</a><br/> <b>outlines</b>: <a href="https://github.com/outlines-dev/outlines" target="_blank" rel="noopener">github.com</a><br/> <b>llamafile</b>: <a href="https://github.com/Mozilla-Ocho/llamafile" target="_blank" rel="noopener">github.com</a><br/> <b>llama-cpp-python</b>: <a href="https://github.com/abetlen/llama-cpp-python" target="_blank" rel="noopener">github.com</a><br/> <b>LLM Dataset</b>: <a href="https://llm.datasette.io/en/stable/index.html" target="_blank" rel="noopener">llm.datasette.io</a><br/> <b>Plugin directory</b>: <a href="https://llm.datasette.io/en/stable/plugins/directory.html" target="_blank" rel="noopener">llm.datasette.io</a><br/> <b>Data exploration at your fingertips.</b>: <a href="https://www.visidata.org/" target="_blank" rel="noopener">visidata.org</a><br/> <b>hottest new programming language is English</b>: <a href="https://twitter.com/karpathy/status/1617979122625712128" target="_blank" rel="noopener">twitter.com</a><br/> <b>OpenAI & other LLM API Pricing Calculator</b>: <a href="https://docsbot.ai/tools/gpt-openai-api-pricing-calculator" target="_blank" rel="noopener">docsbot.ai</a><br/> <b>Vector DB Comparison</b>: <a href="https://vdbs.superlinked.com/" target="_blank" rel="noopener">vdbs.superlinked.com</a><br/> <b>bpytop</b>: <a href="https://github.com/aristocratos/bpytop" target="_blank" rel="noopener">github.com</a><br/> <b>Source Graph</b>: <a href="https://about.sourcegraph.com/cody" target="_blank" rel="noopener">about.sourcegraph.com</a><br/> <b>Watch this episode on YouTube</b>: <a href="https://www.youtube.com/watch?v=FwmbJiKdAG0" target="_blank" rel="noopener">youtube.com</a><br/> <b>Episode transcripts</b>: <a href="https://talkpython.fm/episodes/transcript/456/building-gpt-actions-with-fastapi-and-pydantic" target="_blank" rel="noopener">talkpython.fm</a><br/> <br/> <b>--- Stay in touch with us ---</b><br/> <b>Subscribe to us on YouTube</b>: <a href="https://talkpython.fm/youtube" target="_blank" rel="noopener">youtube.com</a><br/> <b>Follow Talk Python on Mastodon</b>: <a href="https://fosstodon.org/web/@talkpython" target="_blank" rel="noopener"><i class="fa-brands fa-mastodon"></i>talkpython</a><br/> <b>Follow Michael on Mastodon</b>: <a href="https://fosstodon.org/web/@mkennedy" target="_blank" rel="noopener"><i class="fa-brands fa-mastodon"></i>mkennedy</a><br/></div>
Python Bytes
#379 Constable on the debugging case
<strong>Topics covered in this episode:</strong><br> <ul> <li><a href="https://stefaniemolin.com/articles/devx/pre-commit/setup-guide/">How to Set Up Pre-Commit Hooks A step-by-step guide to installing and configuring pre-commit hooks on your project</a>.</li> <li><a href="https://difftastic.wilfred.me.uk"><strong>difftastic</strong></a></li> <li><a href="https://quarto.org"><strong>Quarto</strong></a></li> <li><a href="https://github.com/saurabh0719/constable"><strong>constable</strong></a></li> <li><strong>Extras</strong></li> <li><strong>Joke</strong></li> </ul><a href='https://www.youtube.com/watch?v=4PoBtLFRWGU' style='font-weight: bold;'data-umami-event="Livestream-Past" data-umami-event-episode="379">Watch on YouTube</a><br> <p><strong>About the show</strong></p> <p>Sponsored by us! Support our work through:</p> <ul> <li>Our <a href="https://training.talkpython.fm/"><strong>courses at Talk Python Training</strong></a></li> <li><a href="https://courses.pythontest.com/p/the-complete-pytest-course"><strong>The Complete pytest Course</strong></a></li> <li><a href="https://www.patreon.com/pythonbytes"><strong>Patreon Supporters</strong></a></li> </ul> <p><strong>Connect with the hosts</strong></p> <ul> <li>Michael: <a href="https://fosstodon.org/@mkennedy"><strong>@mkennedy@fosstodon.org</strong></a></li> <li>Brian: <a href="https://fosstodon.org/@brianokken"><strong>@brianokken@fosstodon.org</strong></a></li> <li>Show: <a href="https://fosstodon.org/@pythonbytes"><strong>@pythonbytes@fosstodon.org</strong></a></li> </ul> <p>Join us on YouTube at <a href="https://pythonbytes.fm/stream/live"><strong>pythonbytes.fm/live</strong></a> to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too.</p> <p>Finally, if you want an artisanal, hand-crafted digest of every week of </p> <p>the show notes in email form? Add your name and email to <a href="https://pythonbytes.fm/friends-of-the-show">our friends of the show list</a>, we'll never share it.</p> <p><strong>Michael #1:</strong> <a href="https://stefaniemolin.com/articles/devx/pre-commit/setup-guide/">How to Set Up Pre-Commit Hooks A step-by-step guide to installing and configuring pre-commit hooks on your project</a>.</p> <ul> <li>by <a href="https://stefaniemolin.com/"><strong>Stefanie Molin</strong></a></li> <li>Pre-commit hooks are code checks that run as part of the “pre-commit” stage of the git commit process. </li> <li>If any of these checks fail, git aborts the commit</li> <li>Sometimes, we need to bypass the hooks temporarily. For these instances, we can pass the --no-verify option when we run git commit</li> </ul> <p><strong>Brian #2:</strong> <a href="https://difftastic.wilfred.me.uk"><strong>difftastic</strong></a></p> <ul> <li>Found this a couple years ago, but really using it a lot now.</li> <li>Excellent structurally diff tool that compares code based on syntax, not line by line.</li> </ul> <p><strong>Michael #3:</strong> <a href="https://quarto.org"><strong>Quarto</strong></a></p> <ul> <li>via Mathias Johansson</li> <li>An open-source scientific and technical publishing system</li> <li>Transforming a notebook into a pdf / HTML / MS Word / ePub with minimal effort, or even all formats at once.</li> <li>Author using <a href="https://jupyter.org/">Jupyter</a> notebooks or with plain text markdown in your favorite editor.</li> <li>Write using <a href="https://pandoc.org/">Pandoc</a> markdown, including equations, citations, crossrefs, figure panels, callouts, advanced layout, and more.</li> </ul> <p><strong>Brian #4:</strong> <a href="https://github.com/saurabh0719/constable"><strong>constable</strong></a></p> <ul> <li>“inserts print statements directly into the AST at runtime “</li> <li>“If you find yourself aimlessly adding print statements while debugging your code, this is for you. !”</li> <li>Add decorators like @constable.trace('a', 'b') to functions and you’ll get nice output showing when and how a and b changed.</li> <li>see also <a href="https://github.com/gruns/icecream">icecream</a> for another fun debugging with print project.</li> </ul> <p><strong>Extras</strong> </p> <p>Brian:</p> <ul> <li><a href="https://www.reddit.com/r/Python/comments/1bt7rnw/pointerspy_being_added_to_the_standard_library/"><strong>pointers being added to the standard library</strong></a> <ul> <li>A couple weeks old, but still worth covering</li> <li>Guido’s take on adding this, "Why the hell not?"</li> </ul></li> </ul> <p>Michael:</p> <ul> <li><a href="https://docs.python.org/release/3.12.3/whatsnew/changelog.html#python-3-12-2">Python 3.12.3 is out</a></li> </ul> <p><strong>Joke:</strong> <a href="https://twitter.com/hynek/status/1777377316269883420">Hugo SciFi Award</a></p>
April 15, 2024
Ned Batchelder
Try it: function/class coverage report
I’ve added experimental function and class coverage reports to coverage.py. I’d like feedback about whether they behave the way you want them to.
I haven’t made a PyPI release. To try the new reports, install coverage from GitHub. Be sure to include the hash:
$ python3 -m pip install git+https://github.com/nedbat/coveragepy@f10c455b7c8fd26352de#egg=coverage==0.0
Then run coverage and make an HTML report as you usually do. You should have two new pages, not linked from the index page (yet). “htmlcov/function_index.html” is the function coverage report, and the classes are in “htmlcov/class_index.html”.
I had to decide how to categorize nested functions and classes. Inner functions are not counted as part of their outer functions. Classes consist of the executable lines in their methods, but not lines outside of methods, because those lines run on import. Each file has an entry in the function report for all of the lines outside of any function, called “(no function)”. The class report has “(no class)” entries for lines outside of any classes.
The result should be that every line is part of one function, or the “(no function)” entry, and every line is part of one class, or the “(no class)” entry. This is what made sense to me, but maybe there’s a compelling reason to do it differently.
The reports have a sortable column for the file name, and a sortable column for the function or class. Where functions or classes are nested, the name is a dotted sequence, but is sorted by only the last component. Just like the original file listing page, the new pages can be filtered to focus on areas of interest.
You can look at some sample reports:
It would be helpful if you could give me feedback on the original issue about some questions:
- Is it useful to have “(no function)” and “(no class)” entries or is it just distracting pedantry? With the entries, the total is the same as the file report, but they don’t seem useful by themselves.
- Does the handling of nested functions and classes make sense?
- Should these reports be optional (requested with a switch) or always produced?
- Is it reasonable to produce one page with every function? How large does a project have to get before that’s not feasible or useful?
- And most importantly: do these reports help you understand how to improve your code?
This is only in the HTML report for now, but we can do more in the future. Other ideas about improvements are of course welcome. Thanks!
Real Python
Build a Blog Using Django, GraphQL, and Vue
Are you a regular Django user? Do you find yourself wanting to decouple your back end and front end? Do you want to handle data persistence in the API while you display the data in a single-page app (SPA) in the browser using a JavaScript framework like React or Vue?
If you answered yes to any of these questions, then you’re in luck. This tutorial will take you through the process of building a Django blog back end and a Vue front end, using GraphQL to communicate between them.
Projects are an effective way to learn and solidify concepts. This tutorial is structured as a step-by-step project so you can learn in a hands-on way and take breaks as needed.
In this tutorial, you’ll learn how to:
- Translate your Django models into a GraphQL API
- Run the Django server and a Vue application on your computer at the same time
- Administer your blog posts in the Django admin
- Consume a GraphQL API in Vue to show data in the browser
You can download all the source code you’ll use to build your Django blog application by clicking the link below:
Get Your Code: Click here to download the free sample code that you’ll use to build a blog using Django, GraphQL, and Vue.
Demo: A Django Blog Admin, a GraphQL API, and a Vue Front End
Blog applications are a common starter project because they involve create, read, update, and delete (CRUD) operations. In this project, you’ll use the Django admin to do the heavy CRUD lifting and you’ll focus on providing a GraphQL API for your blog data.
You’ll use Vue.js 3 and its composition API for the front end of your blog. Vue lets you create dynamic interfaces pretty smoothly, thanks to its reactive data binding and easy-to-manage components. Plus, since you’re dealing with data from a GraphQL API, you can leverage the Vue Apollo plugin.
Here’s a demonstration of the completed project in action:
Next, you’ll make sure you have all the necessary background information and tools before you dive in and build your blog application.
Project Overview
For this project, you’ll create a small blogging application with some rudimentary features:
- Authors can write many posts.
- Posts can have many tags and can be either published or unpublished.
You’ll build the back end of this blog in Django, complete with an admin for adding new blog content. Then you’ll expose the content data as a GraphQL API and use Vue to display that data in the browser.
You’ll accomplish this in several high-level steps. At the end of each step, you’ll find a link to the source code for that stage of the project.
If you’re curious about how the source code for each step looks, then you can click the link below:
Get Your Code: Click here to download the free sample code that you’ll use to build a blog using Django, GraphQL, and Vue.
Prerequisites
You’ll be best equipped for this tutorial if you already have a solid foundation in some web application concepts. You should understand how HTTP requests and responses and APIs work. You can check out Python & APIs: A Winning Combo for Reading Public Data to understand the details of using GraphQL APIs vs REST APIs.
Because you’ll use Django to build the back end for your blog, you’ll want to be familiar with starting a Django project and customizing the Django admin. If you haven’t used Django much before, you might also want to try building another Django-only project first. For a good introduction, check out Get Started with Django Part 1: Build a Portfolio App.
And because you’ll be using Vue on the front end, some experience with JavaScript will also help. If you’ve only used a JavaScript framework like jQuery in the past, the Vue introduction is a good foundation.
Familiarity with JSON is also important because GraphQL queries are JSON-like and return data in JSON format. You can read about Working with JSON Data in Python for an introduction.
Read the full article at https://realpython.com/python-django-blog/ »
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
Write Pythonic and Clean Code With namedtuple
In this quiz, you’ll test your understanding of
Python’s namedtuple().
By working through this quiz, you’ll revisit how to create namedtuple classes, identify and take advantage of the cool features of namedtuple, use namedtuple instances to write Pythonic code, and decide when to use a namedtuple over a similar data structure.
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
eGenix.com
Python Meeting Düsseldorf - 2024-04-17
The following text is in German, since we're announcing a regional user group meeting in Düsseldorf, Germany.
Ankündigung
Das nächste Python Meeting Düsseldorf findet an folgendem Termin statt:
17.04.2024, 18:00 Uhr
Raum 1, 2.OG im Bürgerhaus Stadtteilzentrum Bilk
Düsseldorfer Arcaden, Bachstr. 145, 40217 Düsseldorf
Programm
Bereits angemeldete Vorträge
- Marc-André Lemburg:
Advanced parsing structured data with Python's new match statement - Jens Diemer:
Anbindung von Tinkerforge in Home Assistant - Charlie Clark:
Eine kleine Datenanalyse - Detlef Lannert:
Überblick über CLI-Frameworks
Startzeit und Ort
Wir treffen uns um 18:00 Uhr im Bürgerhaus in den Düsseldorfer Arcaden.
Das Bürgerhaus teilt sich den Eingang mit dem Schwimmbad und befindet
sich an der Seite der Tiefgarageneinfahrt der Düsseldorfer Arcaden.
Über dem Eingang steht ein großes "Schwimm’ in Bilk" Logo. Hinter der Tür
direkt links zu den zwei Aufzügen, dann in den 2. Stock hochfahren. Der
Eingang zum Raum 1 liegt direkt links, wenn man aus dem Aufzug kommt.
>>> Eingang in Google Street View
Einleitung
Das Python Meeting Düsseldorf ist eine regelmäßige Veranstaltung in Düsseldorf, die sich an Python Begeisterte aus der Region wendet.
Einen guten Überblick über die Vorträge bietet unser PyDDF YouTube-Kanal, auf dem wir Videos der Vorträge nach den Meetings veröffentlichen.Veranstaltet wird das Meeting von der eGenix.com GmbH, Langenfeld, in Zusammenarbeit mit Clark Consulting & Research, Düsseldorf:
Format
Das Python Meeting Düsseldorf nutzt eine Mischung aus (Lightning) Talks und offener Diskussion.
Vorträge können vorher angemeldet werden, oder auch spontan während des Treffens eingebracht werden. Ein Beamer mit HDMI und FullHD Auflösung steht zur Verfügung.(Lightning) Talk Anmeldung bitte formlos per EMail an info@pyddf.de
Kostenbeteiligung
Das Python Meeting Düsseldorf wird von Python Nutzern für Python Nutzer veranstaltet.
Da Tagungsraum, Beamer, Internet und Getränke Kosten produzieren, bitten wir die Teilnehmer um einen Beitrag in Höhe von EUR 10,00 inkl. 19% Mwst. Schüler und Studenten zahlen EUR 5,00 inkl. 19% Mwst.
Wir möchten alle Teilnehmer bitten, den Betrag in bar mitzubringen.
Anmeldung
Da wir nur 25 Personen in dem angemieteten Raum empfangen können, möchten wir bitten, sich vorher anzumelden.
Meeting Anmeldung bitte per Meetup
Weitere Informationen
Weitere Informationen finden Sie auf der Webseite des Meetings:
https://pyddf.de/
Viel Spaß !
Marc-Andre Lemburg, eGenix.com
Zato Blog
Service-oriented API task scheduling
Service-oriented API task scheduling
An integral part of Zato, its scalable, service-oriented scheduler makes it is possible to execute high-level API integration processes as background tasks. The scheduler runs periodic jobs which in turn trigger services and services are what is used to integrate systems.
Integration process
In this article we will check how to use the scheduler with three kinds of jobs, one-time, interval-based and Cron-style ones.
What we want to achieve is a sample yet fairly common use-case:
- Periodically consult a remote REST endpoint for new data
- Store data found in Redis
- Push data found as an e-mail attachment
Instead of, or in addition to, Redis or e-mail, we could use SQL and SMS, or MongoDB and AMQP or anything else - Redis and e-mail are just example technologies frequently used in data synchronisation processes that we use to highlight the workings of the scheduler.
No matter the input and output channels, the scheduler works always the same - a definition of a job is created and the job's underlying service is invoked according to the schedule. It is then up to the service to perform all the actions required in a given integration process.
Python code
Our integration service will read as below:
# -*- coding: utf-8 -*-
# Zato
from zato.common.api import SMTPMessage
from zato.server.service import Service
class SyncData(Service):
name = 'api.scheduler.sync'
def handle(self):
# Which REST outgoing connection to use
rest_out_name = 'My Data Source'
# Which SMTP connection to send an email through
smtp_out_name = 'My SMTP'
# Who the recipient of the email will be
smtp_to = 'hello@example.com'
# Who to put on CC
smtp_cc = 'hello.cc@example.com'
# Now, let's get the new data from a remote endpoint ..
# .. get a REST connection by name ..
rest_conn = self.out.plain_http[rest_out_name].conn
# .. download newest data ..
data = rest_conn.get(self.cid).text
# .. construct a new e-mail message ..
message = SMTPMessage()
message.subject = 'New data'
message.body = 'Check attached data'
# .. add recipients ..
message.to = smtp_to
message.cc = smtp_cc
# .. attach the new data to the message ..
message.attach('my.data.txt', data)
# .. get an SMTP connection by name ..
smtp_conn = self.email.smtp[smtp_out_name].conn
# .. send the e-mail message with newest data ..
smtp_conn.send(message)
# .. and now store the data in Redis.
self.kvdb.conn.set('newest.data', data)
Now, we just need to make it run periodically in background.
Mind the timezone
In the next steps, we will use the Zato Dashboard to configure new jobs for the scheduler.
Keep it mind that any date and time that you enter in web-admin is always interepreted to be in your web-admin user's timezone and this applies to the scheduler too - by default the timezone is UTC. You can change it by clicking Settings and picking the right timezone to make sure that the scheduled jobs run as expected.
It does not matter what timezone your Zato servers are in - they may be in different ones than the user that is configuring the jobs.
Endpoint definitions
First, let's use web-admin to define the endpoints that the service uses. Note that Redis does not need an explicit declaration because it is always available under "self.kvdb" in each service.
- Configuring outgoing REST APIs
- Configuring SMTP e-mail
Now, we can move on to the actual scheduler jobs.
Three types of jobs
To cover different integration needs, three types of jobs are available:
- One-time - fires once only at a specific date and time and then never runs again
- Interval-based - for periodic processes, can use any combination of weeks, days, hours, minutes and seconds for the interval
- Cron-style - similar to interval-based but uses the syntax of Cron for its configuration
One-time
Select one-time if the job should not be repeated after it runs once.
Interval-based
Select interval-based if the job should be repeated periodically. Note that such a job will by default run indefinitely but you can also specify after how many times it should stop, letting you to express concepts such as "Execute once per hour but for the next seven days".
Cron-style
Select cron-style if you are already familiar with the syntax of Cron or if you have some Cron tasks that you would like to migrate to Zato.
Running jobs manually
At times, it is convenient to run a job on demand, no matter what its schedule is and regardless of what type a particular job is. Web-admin lets you always execute a job directly. Simply find the job in the listing, click "Execute" and it will run immediately.
Extra context
It is very often useful to provide additional context data to a service that the scheduler runs - to achieve it, simply enter any arbitrary value in the "Extra" field when creating or an editing a job in web-admin.
Afterwards, that information will be available as self.request.raw_request in the service's handle method.
Reusability
There is nothing else required - all is done and the service will run in accordance with a job's schedule.
Yet, before concluding, observe that our integration service is completely reusable - there is nothing scheduler-specific in it despite the fact that we currently run it from the scheduler.
We could now invoke the service from command line. Or we could mount it on a REST, AMQP, WebSocket or trigger it from any other channel - exactly the same Python code will run in exactly the same fashion, without any new programming effort needed.