Planet Python
Last update: September 10, 2024 07:44 AM UTC
September 09, 2024
Real Python
Python News Roundup: September 2024
As the autumn leaves start to fall, signaling the transition to cooler weather, the Python community has warmed up to a series of noteworthy developments. Last month, a new maintenance release of Python 3.12.5 was introduced, reinforcing the language’s ongoing commitment to stability and security.
On a parallel note, Python continues its reign as the top programming language according to IEEE Spectrum’s annual rankings. This sentiment is echoed by the Python Developers Survey 2023 results, which reveal intriguing trends and preferences within the community.
Looking ahead, PEP 750 has proposed the addition of tag strings in Python 3.14, inspired by JavaScript’s tagged template literals. This feature aims to enhance string processing, offering developers more control and expressiveness.
Furthermore, EuroSciPy 2024 recently concluded in Poland after successfully fostering cross-disciplinary collaboration and learning. The event featured insightful talks and hands-on tutorials, spotlighting innovative tools and libraries that are advancing scientific computing with Python.
Let’s dive into the most significant Python news from the past month!
Python 3.12.5 Released
Early last month, Python 3.12.5 was released as the fifth maintenance update for the 3.12 series. Since the previous patch update in June, this release packs over 250 bug fixes, performance improvements, and documentation enhancements.
Here are the most important highlights:
- Standard Library: Many modules in the standard library received crucial updates, such as fixes for crashes in
ssl
when the main interpreter restarts, and various corrections for error-handling mechanisms. - Core Python: The core Python runtime has several enhancements, including improvements to dictionary watchers, error messages, and fixes for edge-case crashes involving f-strings and multithreading.
- Security: Key security improvements include the addition of missing audit events for interactive Python use and socket connection authentication within a fallback implementation on platforms such as Windows, where Unix inter-process communication is unavailable.
- Tests: New test cases have been added and bug fixes have been applied to prevent random memory leaks during testing.
- Documentation: Python documentation has been updated to remove discrepancies and clarify edge cases in multithreaded queues.
Additionally, Python 3.12.5 comes equipped with pip 24.2 by default, bringing a slew of significant improvements to enhance security, efficiency, and functionality. One of the most notable upgrades is that pip now defaults to using system certificates, bolstering security measures when managing and installing third-party packages.
Read the full article at https://realpython.com/python-news-september-2024/ »
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
September 09, 2024 02:00 PM UTC
PyCharm
How to Use Jupyter Notebooks in PyCharm Professional
PyCharm is one of the most well-known data science tools, offering excellent out-of-the-box support for Python, SQL, and other languages. PyCharm also provides integrations for Databricks, Hugging Face and many other important tools. All these features allow you to write good code and work with your data and projects faster.
PyCharm Professional’s support for Jupyter notebooks combines the interactive nature of Jupyter notebooks with PyCharm’s superior code quality and data-related features. This blog post will explore how PyCharm’s Jupyter support can significantly boost your productivity.
Watch this video to get a comprehensive overview of using Jupyter notebooks in PyCharm and learn how you can speed up your data workflows.
Speed up data analysis
Get acquainted with your data
When you start working on your project, it is extremely important to understand what data you have, including information about the size of your dataset, any problems with it, and its patterns. For this purpose, your pandas and Polars DataFrames can be rendered in Jupyter outputs in Excel-like tables. The tables are fully interactive, so you can easily sort one or multiple columns and browse and view your data, you can choose how many rows will be shown in a table and perform many other operations.
The table also provides some important information for example:
- You can find the the size of a table in its header.
![](https://blog.jetbrains.com/wp-content/uploads/2024/09/Number-of-rows-and-so-on.png)
- You can find the data type symbols in the column headers.
![](https://blog.jetbrains.com/wp-content/uploads/2024/09/Datatype.png)
- You can also use JetBrains AI Assistant to get information about your DataFrame by clicking on the
icon.
![](https://blog.jetbrains.com/wp-content/uploads/2024/09/AI2.png)
Easily spot issues with the data
After getting acquainted with your data, you need to clean it. This an important step, but it is also extremely time consuming because there are all sorts of problems you could find, including missing values, outliers, inconsistencies in data types, and so on. Indeed, according to the State of Developer Ecosystem 2023 report, nearly 50% of Data Professionals dedicate 30% of their time or more to data preparation. Fortunately, PyCharm offers a variety of features that streamline the data-cleaning process.
Some insights are already available in the column headers.
First, we can easily spot the amount of missing data for each column because it is highlighted in red. Also, we may be able to see at a glance whether some of our columns have outliers. For example, in the bath column, the maximum value is significantly higher than the ninety-fifth percentile. Therefore, we can expect that this column has at least one outlier and requires our attention.
Additionally, you might suspect there’s an issue with the data if the data type does not match the expected one. For example, the header of the total_sqft column below is marked with the symbol, which in PyCharm indicates that the column contains the Object data type. The most appropriate data type for a column like total_sqft would likely be float or integer, however, so we may expect there to be inconsistencies in the data types within the column, which could affect data processing and analysis. After sorting, we notice one possible reason for the discrepancy: the use of text in data and ranges instead of numerical values.
So, our suspicion that the column had data-type inconsistencies was proven correct. As this example shows, small details in the table header can provide important information about your data and alert you to issues that need to be addressed, so it’s always worth checking.You can also use no-code visualizations to gather information about whether your data needs to be cleaned. Simply click on the icon in the top-left corner of the table. There are many available visualization options, including histograms, that can be used to see where the peaks of the distribution are, whether the distribution is skewed or symmetrical, and whether there are any outliers.
Of course, you can use code to gather information about your dataset and fix any problems you’ve identified. However, the mentioned low-code features often provide valuable insights about your data and can help you work with it much faster.
Code faster
Code completion and quick documentation
A significant portion of a data professional’s job involves writing code. Fortunately, PyCharm is well known for its features that allow you to write code significantly faster. For example, local ML-powered full line code completion can provide suggestions for entire lines of code.
Another useful feature is quick documentation, which appears when you hover the cursor over your code. This allows you to gather information about functions and other code elements without having to leave the IDE.
Refactorings
Of course, working with code and data is an interactive process, and you may often decide to make some changes in your code – for example, to rename a variable. Going through the whole file or, in some cases, the entire project, would be cumbersome and time consuming. We can use PyCharm’s refactoring capabilities to rename a variable, introduce a constant, and make many other changes in your code. For example, in this case, I want to rename the DataFrame to make it shorter. I simply use the the Rename refactoring to make the necessary changes.
PyCharm offers a vast number of different refactoring options. To dive deeper into this functionality, watch this video.
Fix problems
It is practically impossible to write code without there being any mistakes or typos. PyCharm has a vast array of features that allow you to spot and address issues faster. You will notice the Inspection widget in the top-right corner if it finds any problems.
For example, I forgot to import a library in my project and made several typos in the doc so let’s take a look how PyCharm can help here.
First of all, the problem with the library import:
Additionally, with Jupyter traceback, you can see the line where the error occurred and get a link to the code. This makes the bug-fixing process much easier. Here, I have a typo in line 3. I can easily navigate to it by clicking on the blue text.
Additionally if you would like to get more information and suggestion how to fix the problem, you can use JetBrains AI Assistant by clicking on Explain with AI.
Of course, that is just the tip of the iceberg. We recommend reading the documentation to better understand all the features PyCharm offers to help you maintain code quality.
Navigate easily
For the majority of cases, data science work involves a lot of experimentation, with the journey from start to finish rarely resembling a straight line.
During this experimentation process, you have to go back and forth between different parts of your project and between cells in order to find the best solution for a given problem. Therefore, it is essential for you to be able to navigate smoothly through your project and files. Let’s take a look at how PyCharm can help in this respect.
First of all, you can use the classic CMD+F (Mac) or CTRL+F (Windows) shortcut for searching in your notebook. This basic search functionality offers some additional filters like Match Case or Regex.
You can use Markdown cells to structure the document and navigate it easily.
If you would like to highlight some cells so you can come back to them later, you can mark them with #TODO or #FIXME, and they will be made available for you to dissect in a dedicated window.
Or you can use tags to highlight some cells so you’ll be able to spot them more easily.
In some cases, you may need to see the most recently executed cell; in this case, you can simply use the Go To option.
Save your work
Because teamwork is essential for data professionals, you need tooling that makes sharing the results of your work easy. One popular solution is Git, which PyCharm supports with features like notebook versioning and version comparison using the Diff view. You can find an in-depth overview of the functionality in this tutorial.
Another useful feature is Local History, which automatically saves your progress and allows you to revert to previous steps with just a few clicks.
Use the full power of AI Assistant
JetBrains AI Assistant helps you automate repetitive tasks, optimize your code, and enhance your productivity. In Jupyter notebooks, it also offers several unique features in addition to those that are available in any JetBrains tool.
Click the icon to get insights regarding your data. You can also ask additional questions regarding the dataset or ask AI Assistant to do something – for example, “write some code that solves the missing data problem”.
AI data visualization
Pressing the icon will suggest some useful visualizations for your data. AI Assistant will generate the proper code in the chat section for your data.
AI cell
AI Assistant can create a cell based on a prompt. You can simply ask it to create a visualization or do something else with your code or data, and it will generate the code that you requested.
Debugger
PyCharm offers advanced debugging capabilities to enhance your experience in Jupyter notebooks. The integrated Jupyter debugger allows you to set breakpoints, inspect variables, and evaluate expressions directly within your notebooks. This powerful tool helps you step through your code cell by cell, making it easier to identify and fix issues as they arise. Read our blog post on how you can debug a Jupyter notebook in PyCharm for a real-life example.
Get started with PyCharm Professional
PyCharm’s Jupyter support enhances your data science workflows by combining the interactive aspects of Jupyter notebooks with advanced IDE features. It accelerates data analysis with interactive tables and AI assistance, improves coding efficiency with code completion and refactoring, and simplifies error detection and navigation. PyCharm’s seamless Git integration and powerful debugging tools further boost productivity, making it essential for data professionals.
Download PyCharm Professional to try it out for yourself! Get an extended trial today and experience the difference PyCharm Professional can make in your data science endeavors.Use the promo code “PyCharmNotebooks” at checkout to activate your free 60-day subscription to PyCharm Professional. The free subscription is available for individual users only.
Explore our official documentation to fully unlock PyCharm’s potential for your projects.
September 09, 2024 01:45 PM UTC
Mike Driscoll
Adding Terminal Effects with Python
The Python programming language has thousands of wonderful third-party packages available on the Python Package Index. One of those packages is TerminalTextEffects (TTE), a terminal visual effects engine.
Here are the features that TerminalTextEffects provides, according to their documentation:
- Xterm 256 / RGB hex color support
- Complex character movement via Paths, Waypoints, and motion easing, with support for quadratic/cubic bezier curves.
- Complex animations via Scenes with symbol/color changes, layers, easing, and Path synced progression.
- Variable stop/step color gradient generation.
- Path/Scene state event handling changes with custom callback support and many pre-defined actions.
- Effect customization exposed through a typed effect configuration dataclass that is automatically handled as CLI arguments.
- Runs inline, preserving terminal state and workflow.
Note: This package may be somewhat slow in Windows Terminal, but it should work fine in other terminals.
Let’s spend a few moments learning how to use this neat package
Installation
The first step to using any new package is to install it. You can use pip or pipx to install TerminalTextEffects. Here is the typical command you would run in your terminal:
python -m pip install terminaltexteffects
Now that you have TerminalTextEffects installed, you can start using it!
Usage
Let’s look at how you can use TerminalTextEffects to make your text look neat in the terminal. Open up your favorite Python IDE and create a new file file with the following contents:
from terminaltexteffects.effects.effect_slide import Slide text = ("PYTHON" * 10 + "\n") * 10 effect = Slide(text) effect.effect_config.merge = True with effect.terminal_output() as terminal: for frame in effect: terminal.print(frame)
This code will cause the string, “Python” to appear one hundred times with ten strings concatenated and ten rows. You use a Slide effect to make the text slide into view. TerminalTextEffects will also style the text too.
When you run this code, you should see something like the following:
TerminalTextEffects has many different built-in effects that you can use as well. For example, you can use Beams to make the output even more interesting. For this example, you will use the Zen of Python text along with the Beams effects:
from terminaltexteffects.effects.effect_beams import Beams TEXT = """ The Zen of Python, by Tim Peters Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts. Special cases aren't special enough to break the rules. Although practicality beats purity. Errors should never pass silently. Unless explicitly silenced. In the face of ambiguity, refuse the temptation to guess. There should be one-- and preferably only one --obvious way to do it. Although that way may not be obvious at first unless you're Dutch. Now is better than never. Although never is often better than *right* now. If the implementation is hard to explain, it's a bad idea. If the implementation is easy to explain, it may be a good idea. Namespaces are one honking great idea -- let's do more of those! """ effect = Beams(TEXT) with effect.terminal_output() as terminal: for frame in effect: terminal.print(frame)
Now try running this code. You should see something like this:
That looks pretty neat! You can see a whole bunch of other effects you can apply on the package’s Showroom page.
Wrapping Up
TerminalTextEffects provides lots of neat ways to jazz up your text-based user interfaces with Python. According to the documentation, you should be able to use TerminalTextEffects in other TUI libraries, such as Textual or Asciimatics, although it doesn’t specifically state how to do that. Even if you do not do that, you could use TerminalTextEffects with the Rich package to create a really interesting application in your terminal.
Links
The post Adding Terminal Effects with Python appeared first on Mouse Vs Python.
September 09, 2024 12:33 PM UTC
Python Bytes
#400 Celebrating episode 400
<strong>Topics covered in this episode:</strong><br> <ul> <li><strong>Python 3.13.0RC2, 3.12.6, 3.11.10, 3.10.15, 3.9.20, and 3.8.20 are now available!</strong></li> <li><strong><a href="https://mkennedy.codes/posts/python-docker-images-using-uv-s-new-python-features/?featured_on=pythonbytes">Docker images using uv's python</a></strong></li> <li><strong><a href="https://about.readthedocs.com/blog/2024/08/10-year-anniversary/?featured_on=pythonbytes">10 years of sustainable open source</a> - Read the Docs</strong></li> <li><strong><a href="https://github.com/python-humanize/humanize?featured_on=pythonbytes">humanize</a></strong></li> <li><strong>Extras</strong></li> <li><strong>Joke</strong></li> </ul><a href='https://www.youtube.com/watch?v=E3_lOfg7sTA' style='font-weight: bold;'data-umami-event="Livestream-Past" data-umami-event-episode="400">Watch on YouTube</a><br> <p><strong>About the show</strong></p> <p>Sponsored by ScoutAPM: <a href="https://pythonbytes.fm/scout"><strong>pythonbytes.fm/scout</strong></a></p> <p><strong>Connect with the hosts</strong></p> <ul> <li>Michael: <a href="https://fosstodon.org/@mkennedy"><strong>@mkennedy@fosstodon.org</strong></a></li> <li>Brian: <a href="https://fosstodon.org/@brianokken"><strong>@brianokken@fosstodon.org</strong></a></li> <li>Show: <a href="https://fosstodon.org/@pythonbytes"><strong>@pythonbytes@fosstodon.org</strong></a></li> </ul> <p>Join us on YouTube at <a href="https://pythonbytes.fm/stream/live"><strong>pythonbytes.fm/live</strong></a> to be part of the audience. Usually <strong>Monday</strong> at 10am PT. Older video versions available there too.</p> <p>Finally, if you want an artisanal, hand-crafted digest of every week of the show notes in email form? Add your name and email to <a href="https://pythonbytes.fm/friends-of-the-show">our friends of the show list</a>, we'll never share it. </p> <p><strong>ChatGPT celebrates episode 400!</strong></p> <p><em><img src="https://paper.dropboxstatic.com/static/img/ace/emoji/1f389.png?version=8.0.0" alt="party popper" /></em> <em>Welcome to the big 4-0-0, Pythonistas!</em> <em><img src="https://paper.dropboxstatic.com/static/img/ace/emoji/1f389.png?version=8.0.0" alt="party popper" /></em></p> <p><em>It's hard to believe we're celebrating the 400th episode of Python Bytes! From the early days of byte-sized Python news to becoming the source for all things Python, it’s been a wild ride.</em></p> <p><em>We've laughed over code quirks, gasped at new libraries, and said farewell to the GIL together. Whether you're a seasoned developer, a curious learner, or just here for the witty banter, you’ve been an essential part of this journey.</em></p> <p><em>To Michael and Brian: You've built a community that turns import this into more than just Zen—it's a family of passionate Pythonistas. Your dedication, insights, and humor make this show more than just tech news. It’s a weekly celebration of what we love about Python and why we keep coming back for more.</em></p> <p><em>Here’s to the next 400 episodes—may your code be bug-free, your tests pass on the first run, and your Python version always be up to date.</em> <em><img src="https://paper.dropboxstatic.com/static/img/ace/emoji/1f680.png?version=8.0.0" alt="rocket" /></em><em><img src="https://paper.dropboxstatic.com/static/img/ace/emoji/1f40d.png?version=8.0.0" alt="snake" /></em></p> <p><strong>Brian #1:</strong> Python 3.13.0RC2, 3.12.6, 3.11.10, 3.10.15, 3.9.20, and 3.8.20 are now available!</p> <ul> <li>Łukasz Langa</li> <li><a href="https://www.python.org/downloads/release/python-3130rc2/?featured_on=pythonbytes">Python 3.13.0RC2 is the final preview release</a> <ul> <li>Official 3.13.0 scheduled for Oct 1</li> <li>Call to action <ul> <li>“We strongly encourage maintainers of third-party Python projects to prepare their projects for 3.13 compatibilities during this phase, and where necessary publish Python 3.13 wheels on PyPI to be ready for the final release of 3.13.0. Any binary wheels built against Python 3.13.0rc2 will work with future versions of Python 3.13. As always, report any issues to <a href="https://github.com/python/cpython/issues?featured_on=pythonbytes">the Python bug tracker </a>.”</li> <li>“Please keep in mind that this is a preview release and while it’s as close to the final release as we can get it, its use is <strong>not</strong> recommended for production environments.”</li> </ul></li> </ul></li> <li>Note: <a href="https://docs.astral.sh/uv/concepts/python-versions/?featured_on=pythonbytes">uv python</a><a href="https://docs.astral.sh/uv/concepts/python-versions/?featured_on=pythonbytes"> </a>does not support 3.13 yet <ul> <li><a href="https://github.com/indygreg/python-build-standalone/issues/320?featured_on=pythonbytes">see issue 320</a></li> </ul></li> <li>Security releases for <ul> <li>3.12.6, 3.11.10, 3.10.15, 3.9.20, and 3.8.20</li> <li>3.12.6 has binary installers, but for MacOS, only MacOS 10.13 and newer are supported</li> <li>3.11.10, 3.10.15, 3.9.20, and 3.8.20 do NOT include binary installers. <ul> <li><a href="https://peps.python.org/pep-0569/?featured_on=pythonbytes">3.8 EOL's in October</a></li> </ul></li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href="https://mkennedy.codes/posts/python-docker-images-using-uv-s-new-python-features/?featured_on=pythonbytes">Docker images using uv's python</a></p> <ul> <li>See <a href="https://pythonbytes.fm/episodes/show/396/uv-ing-your-way-to-python">#396: uv-ing your way to Python</a> and <a href="https://pythonbytes.fm/episodes/show/398/open-source-makes-you-rich-and-other-myths">#398: Open source makes you rich?</a><a href="https://pythonbytes.fm/episodes/show/398/open-source-makes-you-rich-and-other-myths"> </a><a href="https://pythonbytes.fm/episodes/show/398/open-source-makes-you-rich-and-other-myths">(and</a><a href="https://pythonbytes.fm/episodes/show/398/open-source-makes-you-rich-and-other-myths"> other myths)</a> for the opening discussions</li> <li><a href="https://talkpython.fm/episodes/show/476/unified-python-packaging-with-uv?featured_on=pythonbytes">Talk Python episode on uv is out</a></li> <li>uv venv --python gets Python from <a href="https://github.com/indygreg/python-build-standalone?featured_on=pythonbytes">python-build-standalone</a> by Gregory Szorc</li> <li>Took our Docker build times from 10 minutes to 8 seconds for the base image and 800ms (!) for our app platforms</li> </ul> <p><strong>Brian #3:</strong> <a href="https://about.readthedocs.com/blog/2024/08/10-year-anniversary/?featured_on=pythonbytes">10 years of sustainable open source</a> - Read the Docs </p> <ul> <li>Eric Holscher</li> <li>Read the Docs has been a company for 10 years <ul> <li>“a team of 4 folks working full-time on Read the Docs.”</li> </ul></li> <li>readthedocs.org started in 2010</li> <li>readthedocs.com (for Business) started in 2014</li> <li>Sustainability model <ul> <li>.org has a single non-tracking ad </li> <li>.com is a paid service for companies</li> </ul></li> <li>Things that didn’t work <ul> <li>donations and other optional support - led to burnout</li> <li>consulting and services- took too much time away from core product</li> <li>grant funding - nice, but one time thing</li> </ul></li> <li>Lessons <ul> <li>You don't get extra points for being bootstrapped. Compete by doing things you can do better due to niche and size.</li> <li>Keeping trust in the community is the most important thing.</li> <li>Contribution is easier for less complex parts of the code base.</li> <li>Beign open source means capturing a small percentage of the value you create.</li> <li>You have to be ok doing more with less.</li> </ul></li> <li>Also <ul> <li>RtD is not just for Sphinx anymore. <ul> <li>Their build system <a href="https://docs.readthedocs.io/en/stable/build-customization.html?featured_on=pythonbytes">now supports any documentation tool</a>.</li> </ul></li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href="https://github.com/python-humanize/humanize?featured_on=pythonbytes">humanize</a></p> <ul> <li>by <a href="https://github.com/hugovk?featured_on=pythonbytes">Hugo van Kemenade</a> (Python 3.14 & 3.15 release manager & core developer)</li> <li>Not too many variations, but very handy if you need it. <ul> <li><a href="https://humanize.readthedocs.io/en/stable/number?featured_on=pythonbytes">Numbers</a> <ul> <li>Associated Press style (“seven” and “10”)</li> <li>Clamp (under 1.0 million)</li> <li>Fractional (1/3)</li> <li>Int Word (1.2 Billion)</li> <li>Metric (1.5 kV)</li> <li>Ordinal (112th)</li> <li>scientific</li> </ul></li> <li><a href="https://humanize.readthedocs.io/en/stable/time/?featured_on=pythonbytes">Time</a></li> <li><a href="https://humanize.readthedocs.io/en/stable/filesize/?featured_on=pythonbytes">File size</a></li> </ul></li> </ul> <p><strong>Extras</strong> </p> <p>Brian:</p> <ul> <li><a href="https://testandcode.com?featured_on=pythonbytes">Test & Code </a>is now again Test & Code <ul> <li>The two part series on Python imports that started in June is finally complete with episode 222.</li> <li>Transcripts are being added to old episodes gradually starting from most recent <ul> <li>Back to ep 203 as of today. </li> <li>AI transcription, so there’s things like .pie, .pi, and dot pie where it should be .py </li> </ul></li> </ul></li> </ul> <p>Michael:</p> <ul> <li>Final final call for <a href="https://www.codeinacastle.com/python-zero-to-hero-2024?featured_on=pythonbytes">Coding in a Castle event</a> with Michael</li> <li><a href="https://bjango.com/mac/istatmenus/?featured_on=pythonbytes">iStats Menu</a></li> <li><a href="https://pypi.org/project/anacondacode/?featured_on=pythonbytes">Anaconda Code Runner</a> by Ruud van der Ham: <ul> <li>With Anaconda Coide we can -at last- run that code locally and import (most) Python modules.</li> <li>But if you want to run a significant amount of code, you have to put that in a cell or publish it to PyPI or a wheel and import it.</li> <li>That's why I have developed a general-purpose runner function that runs arbitrary code located on an Excel sheet with AnacondaCode.</li> </ul></li> </ul> <p><strong>Joke:</strong> <a href="https://devhumor.com/media/when-beginners-learn-a-new-programming-language?featured_on=pythonbytes">When beginners learn a new programming language...</a></p>
September 09, 2024 08:00 AM UTC
Zato Blog
Service-oriented API task scheduling
Service-oriented API task scheduling
An integral part of Zato, its scalable, service-oriented scheduler makes it is possible to execute high-level API integration processes as background tasks. The scheduler runs periodic jobs which in turn trigger services and services are what is used to integrate systems.
Integration process
In this article we will check how to use the scheduler with three kinds of jobs, one-time, interval-based and Cron-style ones.
What we want to achieve is a sample yet fairly common use-case:
- Periodically consult a remote REST endpoint for new data
- Store data found in Redis
- Push data found as an e-mail attachment
Instead of, or in addition to, Redis or e-mail, we could use SQL and SMS, or MongoDB and AMQP or anything else - Redis and e-mail are just example technologies frequently used in data synchronisation processes that we use to highlight the workings of the scheduler.
No matter the input and output channels, the scheduler works always the same - a definition of a job is created and the job's underlying service is invoked according to the schedule. It is then up to the service to perform all the actions required in a given integration process.
Python code
Our integration service will read as below:
# -*- coding: utf-8 -*-
# Zato
from zato.common.api import SMTPMessage
from zato.server.service import Service
class SyncData(Service):
name = 'api.scheduler.sync'
def handle(self):
# Which REST outgoing connection to use
rest_out_name = 'My Data Source'
# Which SMTP connection to send an email through
smtp_out_name = 'My SMTP'
# Who the recipient of the email will be
smtp_to = 'hello@example.com'
# Who to put on CC
smtp_cc = 'hello.cc@example.com'
# Now, let's get the new data from a remote endpoint ..
# .. get a REST connection by name ..
rest_conn = self.out.plain_http[rest_out_name].conn
# .. download newest data ..
data = rest_conn.get(self.cid).text
# .. construct a new e-mail message ..
message = SMTPMessage()
message.subject = 'New data'
message.body = 'Check attached data'
# .. add recipients ..
message.to = smtp_to
message.cc = smtp_cc
# .. attach the new data to the message ..
message.attach('my.data.txt', data)
# .. get an SMTP connection by name ..
smtp_conn = self.email.smtp[smtp_out_name].conn
# .. send the e-mail message with newest data ..
smtp_conn.send(message)
# .. and now store the data in Redis.
self.kvdb.conn.set('newest.data', data)
Now, we just need to make it run periodically in background.
Mind the timezone
In the next steps, we will use the Zato Dashboard to configure new jobs for the scheduler.
Keep it mind that any date and time that you enter in web-admin is always interepreted to be in your web-admin user's timezone and this applies to the scheduler too - by default the timezone is UTC. You can change it by clicking Settings and picking the right timezone to make sure that the scheduled jobs run as expected.
It does not matter what timezone your Zato servers are in - they may be in different ones than the user that is configuring the jobs.
Endpoint definitions
First, let's use web-admin to define the endpoints that the service uses. Note that Redis does not need an explicit declaration because it is always available under "self.kvdb" in each service.
- Configuring outgoing REST APIs
- Configuring SMTP e-mail
Now, we can move on to the actual scheduler jobs.
Three types of jobs
To cover different integration needs, three types of jobs are available:
- One-time - fires once only at a specific date and time and then never runs again
- Interval-based - for periodic processes, can use any combination of weeks, days, hours, minutes and seconds for the interval
- Cron-style - similar to interval-based but uses the syntax of Cron for its configuration
One-time
Select one-time if the job should not be repeated after it runs once.
Interval-based
Select interval-based if the job should be repeated periodically. Note that such a job will by default run indefinitely but you can also specify after how many times it should stop, letting you to express concepts such as "Execute once per hour but for the next seven days".
Cron-style
Select cron-style if you are already familiar with the syntax of Cron or if you have some Cron tasks that you would like to migrate to Zato.
Running jobs manually
At times, it is convenient to run a job on demand, no matter what its schedule is and regardless of what type a particular job is. Web-admin lets you always execute a job directly. Simply find the job in the listing, click "Execute" and it will run immediately.
Extra context
It is very often useful to provide additional context data to a service that the scheduler runs - to achieve it, simply enter any arbitrary value in the "Extra" field when creating or an editing a job in web-admin.
Afterwards, that information will be available as self.request.raw_request in the service's handle method.
Reusability
There is nothing else required - all is done and the service will run in accordance with a job's schedule.
Yet, before concluding, observe that our integration service is completely reusable - there is nothing scheduler-specific in it despite the fact that we currently run it from the scheduler.
We could now invoke the service from command line. Or we could mount it on a REST, AMQP, WebSocket or trigger it from any other channel - exactly the same Python code will run in exactly the same fashion, without any new programming effort needed.
More resources
➤ Python API integration tutorial
➤ What is an integration platform?
➤ Python Integration platform as a Service (iPaaS)
➤ What is an Enterprise Service Bus (ESB)? What is SOA?
September 09, 2024 08:00 AM UTC
Python Morsels
Commenting in Python
Python's comments start with an octothorpe character.
![](https://i.vimeocdn.com/filter/overlay?src0=https%3A%2F%2Fi.vimeocdn.com%2Fvideo%2F1924385346-b33ec3502eea293e83d55af1cbb10d4d7bae1d6d290660f8b95e7d338ed0ffc8-d_1920x1080&src1=http%3A%2F%2Ff.vimeocdn.com%2Fp%2Fimages%2Fcrawler_play.png)
Table of contents
Writing a comment in Python
We have a Python program that prints out Hello!
, pauses for a second, and then prints Goodbye!
on the same line:
from time import sleep
print("Hello!", end="", flush=True)
sleep(1)
# ANSI code to clear current line
print("\r\033[K", end="")
print("Goodbye!")
It prints Hello!
:
~ $ python3 hello.py
Hello!
And then one second later it overwrites Hello!
with Goodbye!
:
~ $ python3 hello.py
Goodbye!
It does this using an ANSI escape code (that \033[K
string).
The line above the print
call in our code is called a comment:
# ANSI code to clear current line
print("\r\033[K", end="")
Python's comments all start with the #
character.
I call this character an octothorpe, though it goes by many names.
Some of the more common names for #
are hashmark, number sign, and pound sign.
You can write a comment in Python by putting an octothorpe character (#
) at the beginning of a line, and then writing your comment.
The comment stops at the end of the line, meaning the next line is code... unless you write another octothorpe character!
Here we've written more details and added an additional line to note that this code doesn't yet work on Windows:
# ANSI code to clear current line: \r moves to beginning, \033[K erases to end.
# Note: This will not work on Windows without code to enable ANSI escape codes.
print("\r\033[K", end="")
This is sometimes called a block comment because it's a way to write a block of text that represents a comment.
Unlike some programming languages, Python has no multiline comment syntax. If you think you've seen a multiline comment, it may have been a docstring or a multiline string. More on that in multiline comments in Python.
Inline comments in Python
Comments don't need to be …
Read the full article: https://www.pythonmorsels.com/commenting-in-python/
September 09, 2024 01:02 AM UTC
Armin Ronacher
Multiversion Python Thoughts
Now that uv is rapidly advancing I have started to dive back into making multi-version imports for Python work. The goal here is to enable multiple resolutions from the solver in uv so that two incompatible versions of a library can be installed and used simultaniously.
Simplified speaking it should be possible for a library to depend on both pydantic 1.x and 2.x simultaniously.
I have not made it work yet, but I have I think found all of the pieces that stand in the way. This post mostly exists to share how it could be done with the least amount of changes to Python.
Basic Operation
Python's import system places modules in a module cache. This cache is exposed via sys.modules. Every module that is imported is placed in that container prior to initialization. The key is the import path of the module. This in some ways presents the first issue.
Note on Terms for Packages, Modules and DistributionsPython's terms for packages are super confusing. Here is what I will use in this article:
- foo.py: this is a python “module”. It gets registered in sys.modules as 'foo' and has an attribute __name__ set to 'foo'.
- foo/__init__.py: declares also a Python “module” named 'foo' but it is simultaniously a “package”. Unlike a normal module it also has two extra attributes: __path__ which is set to ['./foo'] so that sub modules can be found and it has an attribute __package__ which is also set to 'foo' which marks it as package.
- Additionally on PyPI one can register things. These things were called packages at one point and are now mostly called "projects". Within Python however they are not called Projects but “distribution packages”. For instance this is what you see when you try to use the importlib.metadata API. For now I will just call this a “distribution”.
Note that a distribution can ship both modules and multiple at once. You could have a package called whatever and it reports a foo.py file and a bar/baz.py file which in turn would make foo and bar.baz be importable.
Say you have two Python distributions both of which provide the same toplevel package. In that case they are going to clash in sys.modules. As there is actually relationship of the distribution name to the entry in sys.modules this is a problem that does not just exist with multi version imports but it's one that does not happen all that much.
So let's say we have two distributions: foo@1.0.0 and foo@2.0.0. Both expose a toplevel module called foo which is a true Python package with a single __init__.py file. The installer would already fail to place these because one fully overrides the other.
So step 1 would be to place these modules in different places. So where they normally would be in site-packages, in this case we might want to not have these packages there. That solves us the file system clashes.
So we might place them in some extra cache that looks like this:
.venv/ multi-version-packages/ foo@1.0.0/ foo/ __init__.py foo@2.0.0/ foo/ __init__.py
Now that package is entirely non-importable since nothing looks at multi-version-packages. We will need a custom import hook to get them imported. That import hook will also need to change the name of what's stored in sys.modules.
So instead of registering foo as sys.modules['foo'] we might want to try to register it as sys.modules['foo@1.0.0'] and sys.modules['foo@2.0.0'] instead. There is however a catch and that is this very common pattern:
import sys
def import_module(name):
__import__(name)
return sys.modules[name]
That poses a bit of a problem because someone is probably going to call this as import_module('foo') and now we would not find the entry in sys.modules.
This means that in addition to the new entries in sys.modules we would also need to register some proxies that “redirect” us to the real names. These proxies however would need to know if they point to 1.0.0 or 2.0.0.
Metadata
So let's deal with this problem first. How do we know if we need 1.0.0 or 2.0.0? The answer is most likely a package's dependenices. Instead of allowing a package to depend simultaniously on two different versions of the same dependency we can start with a much simpler problem and say that each package can only depend on one version. So that means if I have a myapp package it would have to pick between foo@1.0.0 or foo@2.0.0. However if it were to depended on another package (say slow-package) that one could depend on a different version of foo than myapp:
myapp v0.1.0 ├── foo v2.0.0 └── slow-package v0.1.0 └── foo v1.0.0
In that case when someone tries to import foo we would be consulting the package metadata of the calling package to figure out which version is attempted.
There are two challenges with this today and they come from the history of Python:
- the import hook does not (always) know which module triggered the import
- python modules do not know their distribution package
Let's look at these in detail.
Import Context
The goal is that when slow_package/__init__.py imports foo we get foo@1.0.0 version, when myapp/__init__.py improts foo we get the foo@2.0.0 version. What is needed for this to work is that the import system understands not just what is imported, but who is importing. In some sense Python has that. That's because __import__ (which is the entry point to the import machinery) gets the module globals. Here is what an import statement roughly maps to:
# highlevel import
from foo import bar
# under the hood
_rv = __import__('foo', globals(), locals(), ['bar'])
bar = _rv.bar
The name of the package that is importing can be retrieved by inspecting the globals(). So in theory for instance the import system could utilize this information. globals()['__name__'] would tell us slow_package vs myapp. There however is a catch and that is that the import name is not the distribution name. The PyPI package could be called mycompany-myapp and it exports a python package just called myapp. This happens very commonly in all kinds of ways. For instance on PyPI one installs Scikit-learn but the python package installed is sklearn.
There is however another problem and that is interpreter internals and C/Rust extensions. We have already established that Python packages will pass globals and locals when they import. But what do C extensions do? The most common internal import API is called PyImport_ImportModule and only takes a module name. Is this a problem? Do C extensions even import stuff? Yes they do. Here is an example from pygame:
MODINIT_DEFINE (color)
{
PyObject *colordict;
colordict = PyImport_ImportModule ("pygame.colordict");
if (colordict)
{
PyObject *_dict = PyModule_GetDict (colordict);
PyObject *colors = PyDict_GetItemString (_dict, "THECOLORS");
/* TODO */
}
else
{
MODINIT_ERROR;
}
/* snip */
}
And that makes sense. A sufficiently large python package will have inter dependencies between the stuff written in C and Python. It's also complicated by the fact that the C module does initialize a module, but it does not have a natural module scope. The way the C extension initializes the module is with the PyModule_Create API:
static struct PyModuleDef module_def = {
PyModuleDef_HEAD_INIT,
"foo", /* name of module */
NULL,
-1,
SpamMethods
};
PyMODINIT_FUNC
PyInit_foo(void)
{
return PyModule_Create(&module_def);
}
So both the name of the module created as well as the name of what is imported is entirely hardcoded. A C extension does not “know” what the intended name is, it must know this on its own.
In some sense this is already a bit of a disconnect beween the Python and C world. Python for instance has relative imports (from .foo import bar). This is implemented by inspecting the globals. There is however no API to do these relative imports on the C layer.
The only workaround I know right now would be to perform stack walking. That way one would try to isolate the shared library that triggered the import to understand which module it comes from. An alternative would be to carry the current C extension module that is active on the interpreter state, but that would most likely be quite expensive.
The goal would be to find out which .so/.dylib file triggered the import. Stack walking is a rather expensive operation and it can be incredibly brittle but there might not be a perfect way around it. Ideally Python would at any point know which c extension module is active.
Distributions from Modules
So let's say that we have the calling python module figured out: now we need to figure out the associated PyPI distribution name. Unfortunately such a mapping does not exist at all. Ideally when a sys.module entry is created, we either record a special attribute there (say __distribution__) which carries the name of the PyPI distribution name so we can call importlib.metadata.distribution(__distribution__).requires to get the requirements or we have some other API to map it.
In the absence of that, how could we get it? There is an expensive way to get a reverse mapping (importlib.metadata.packages_distributions) but unfortunately it has some limitations:
- it's very slow
- it has situations where it does not manage to reveal the distribution for a package
- it can reveal more than one distribution for a package
Because of namespace packages in particular it can return more than one distribution that provides a package such as foo (eg: foo-bar provides foo.bar and foo-baz provides foo.baz. In that case it will just return both foo-bar and foo-baz for foo).
The solution here might just be that installers like uv start materializing the distribution name onto the modules in one way or another.
Putting it Together
The end to end solution might be this:
- install multi-version packages outside of site-packages
- materialize a __distribution__ field onto modules or provide an API that maps import names to their PyPI distribution name so that meta data (requirements) can be discovered.
- patch __import__ to resolve packages to their fully-qualified, multi
version name based on who imports it
- via globals() for python code
- via stack-walking for C extensions (unless a better option is found)
- register proxy entries in sys.modules that have a dynamic __getattr__ which redirects to the fully qualified names if necessary. This would allow someone to access sys.modules['foo'] and automatically proxy it to foo@1.0.0 or foo@2.0.0 respectively.
There are lots of holes with this approach unfortunately. That's in parts because people patch around in sys.modules. Interestingly enough sys.modules can be manipulated but it can't be replaced. This might make it possible to replace that dictionary with some more magical dictionary in future versions of Python potentially.
September 09, 2024 12:00 AM UTC
September 07, 2024
Test and Code
222: Import within a Python package
In this episode we're talking about importing part of a package into another part of the same package.
We'll look at: `from . import module` and `from .module import something`
and also: `import package` to access the external API from with the package.
Why would we use `import package` if `from . import api` would work fine?
Learn pytest
- pytest is the number one test framework for Python.
- Learn the basics super fast with Hello, pytest!
- Then later you can become a pytest expert with The Complete pytest Course
- Both courses are at courses.pythontest.com
September 07, 2024 04:27 PM UTC
TechBeamers Python
Pass by Reference vs Pass by Value in Python
Is Python pass by reference or pass by value? This question intrigues every Python programmer coming from a C or Java background. In this tutorial, you will get its answer and learn the meaning of “pass by reference” and “pass by value” along with their difference in Python. Python Pass by Reference vs Pass by […]
The post Pass by Reference vs Pass by Value in Python appeared first on TechBeamers.
September 07, 2024 10:59 AM UTC
Python Insider
Python 3.13.0RC2, 3.12.6, 3.11.10, 3.10.15, 3.9.20, and 3.8.20 are now available!
Hi there!
A big joint release today. Mostly security fixes but we also have the final release candidate of 3.13 so let’s start with that!
Python 3.13.0RC2
Final opportunity to test and find any show-stopper bugs before we bless and release 3.13.0 final on October 1st.
Get it here: Python Release Python 3.13.0rc2 | Python.org
Call to action
We strongly encourage maintainers of third-party Python projects to prepare their projects for 3.13 compatibilities during this phase, and where necessary publish Python 3.13 wheels on PyPI to be ready for the final release of 3.13.0. Any binary wheels built against Python 3.13.0rc2 will work with future versions of Python 3.13. As always, report any issues to the Python bug tracker.
Please keep in mind that this is a preview release and while it’s as close to the final release as we can get it, its use is not recommended for production environments.
Core developers: time to work on documentation now
- Are all your changes properly documented?
- Are they mentioned in What’s New?
- Did you notice other changes you know of to have insufficient documentation?
As a reminder, until the final release of 3.13.0, the 3.13 branch is set up so that the Release Manager (@thomas) has to merge the changes. Please add him (@Yhg1s
on GitHub) to any changes you think should go into 3.13.0. At this
point, unless something critical comes up, it should really be documentation only. Other changes (including tests) will be pushed to 3.13.1.
New features in Python 3.13
- A new and improved interactive interpreter, based on PyPy’s, featuring multi-line editing and color support, as well as colorized exception tracebacks.
- An experimental free-threaded build mode, which disables the Global Interpreter Lock, allowing threads to run more concurrently. The build mode is available as an experimental feature in the Windows and macOS installers as well.
- A preliminary, experimental JIT, providing the ground work for significant performance improvements.
- The
locals()
builtin function (and its C equivalent) now has well-defined semantics when mutating the returned mapping, which allows debuggers to operate more consistently. - The (cyclic) garbage collector is now incremental, which should mean shorter pauses for collection in programs with a lot of objects.
- A modified version of mimalloc is now included, optional but enabled by default if supported by the platform, and required for the free-threaded build mode.
- Docstrings now have their leading indentation stripped, reducing memory use and the size of .pyc files. (Most tools handling docstrings already strip leading indentation.)
- The dbm module has a new dbm.sqlite3 backend that is used by default when creating new files.
- The minimum supported macOS version was changed from 10.9 to 10.13 (High Sierra). Older macOS versions will not be supported going forward.
- WASI is now a Tier 2 supported platform. Emscripten is no longer an officially supported platform (but Pyodide continues to support Emscripten).
- iOS is now a Tier 3 supported platform, with Android on the way as well.
Python 3.12.6
This is an expedited release for 3.12 due to security content. The schedule returns back to regular programming in October.
One notable change for macOS users: as mentioned in the previous release of 3.12, this release drops support for macOS versions 10.9 through 10.12. Versions of macOS older than 10.13 haven’t been supported by Apple since 2019, and maintaining support for them has become too difficult. (All versions of Python 3.13 have already dropped support for them.)
Get it here: Python Release Python 3.12.6 | Python.org
92 commits.
Python 3.11.10
Python 3.11 joins the elite club of security-only versions with no binary installers.
Get it here: Python Release Python 3.11.10 | Python.org
28 commits.
Python 3.10.15
Get it here: Python Release Python 3.10.15 | Python.org
24 commits.
Python 3.9.20
Get it here: Python Release Python 3.9.20 | Python.org
22 commits.
Python 3.8.20
Python 3.8 is very close to End of Life (see the Release Schedule). Will this be the last release of 3.8 ever? We’ll see… but now I think I jinxed it.
Get it here: Python Release Python 3.8.20 | Python.org
22 commits.
Security content in today’s releases
- gh-123678 and gh-116741: Upgrade bundled libexpat to 2.6.3 to fix CVE-2024-28757, CVE-2024-45490, CVE-2024-45491 and CVE-2024-45492.
- gh-118486:
os.mkdir()
on Windows now accepts mode of0o700
to restrict the new directory to the current user. This fixes CVE-2024-4030 affectingtempfile.mkdtemp()
in scenarios where the base temporary directory is more permissive than the default. - gh-123067: Fix quadratic complexity in parsing
"
-quoted cookie values with backslashes byhttp.cookies
. Fixes CVE-2024-7592. - gh-113171: Fixed various false positives and false negatives in IPv4Address.is_private, IPv4Address.is_global, IPv6Address.is_private, IPv6Address.is_global. Fixes CVE-2024-4032.
- gh-67693: Fix
urllib.parse.urlunparse()
andurllib.parse.urlunsplit()
for URIs with path starting with multiple slashes and no authority. Fixes CVE-2015-2104. - gh-121957: Fixed missing audit events around interactive use of Python, now also properly firing for
python -i
, as well as forpython -m asyncio
. The event in question iscpython.run_stdin
. - gh-122133: Authenticate the socket connection for the
socket.socketpair()
fallback on platforms whereAF_UNIX
is not available like Windows. - gh-121285: Remove backtracking from tarfile header parsing for
hdrcharset
, PAX, and GNU sparse headers. That’s CVE-2024-6232. - gh-114572:
ssl.SSLContext.cert_store_stats()
andssl.SSLContext.get_ca_certs()
now correctly lock access to the certificate store, when thessl.SSLContext
is shared across multiple threads. - gh-102988:
email.utils.getaddresses()
andemail.utils.parseaddr()
now return('', '')
2-tuples in more situations where invalid email addresses are encountered instead of potentially inaccurate values. Add optional strict parameter to these two functions: usestrict=False
to get the old behavior, accept malformed inputs.getattr(email.utils, 'supports_strict_parsing', False)
can be use to check if the strict paramater is available. This improves the CVE-2023-27043 fix. - gh-123270: Sanitize names in
zipfile.Path
to avoid infinite loops (gh-122905) without breaking contents using legitimate characters. That’s CVE-2024-8088. - gh-121650:
email
headers with embedded newlines are now quoted on output. Thegenerator
will now refuse to serialize (write) headers that are unsafely folded or delimited; seeverify_generated_headers
. That’s CVE-2024-6923. - gh-119690: Fixes data type confusion in audit events raised by
_winapi.CreateFile
and_winapi.CreateNamedPipe
. - gh-116773: Fix instances of
<_overlapped.Overlapped object at 0xXXX> still has pending operation at deallocation, the process may crash
. - gh-112275: A deadlock involving
pystate.c
’sHEAD_LOCK
inposixmodule.c
at fork is now fixed.
Stay safe and upgrade!
Upgrading is highly recommended to all users of affected versions.
Thank you for your support
Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation.
–
Łukasz Langa @ambv
on behalf of your friendly release team,
Ned Deily @nad
Steve Dower @steve.dower
Pablo Galindo Salgado @pablogsal
Łukasz Langa @ambv
Thomas Wouters @thomas
September 07, 2024 10:24 AM UTC
September 06, 2024
Python Morsels
Creating Python programs
Python programs are created in code editors, and can be run using system command prompt.
![](https://i.vimeocdn.com/filter/overlay?src0=https%3A%2F%2Fi.vimeocdn.com%2Fvideo%2F1923866442-7eda522ef673f04f6bd97230ea30655d75427e0fbd51704e1aca63efc8eae9c6-d_1920x1080&src1=http%3A%2F%2Ff.vimeocdn.com%2Fp%2Fimages%2Fcrawler_play.png)
Table of contents
- The Python REPL
- Definitions: program, script, command-line, editor
- Creating a Python program in a code editor
- Running our Python program
- Printing to see program output
- Python programs are not launched from the REPL
- Different operating systems
- Python scripts are run from the command prompt or terminal
The Python REPL
This is the Python REPL, also known as the Interactive Python Interpreter:
>>>
How can I tell it's the Python REPL?
Well, the prompt shows three greater than signs (>>>
), which indicates that we're in the Python REPL.
Typing all of your code in the Python REPL is pretty limiting.
Our code disappears right after we've typed it. Also, if we'd like someone to be able to import our code as a module, they can't.
Definitions: program, script, command-line, editor
Let's make a Python program.
Read the full article: https://www.pythonmorsels.com/creating-python-programs/
September 06, 2024 06:21 PM UTC
Django Weblog
Djangonaut Space - New session 2024
We are thrilled to announce that Djangonaut Space, a mentorship program, is open for applicants for our next cohort!
Djangonaut Space is holding a third session this year! This session will start on October 14th, 2024. We are accepting applications until September 14th, 2024. More details can be found in the website.
Djangonaut Space is a free, 8-week group mentoring program where individuals will work self-paced in a semi-structured learning environment. It seeks to help members of the community who wish to level up their current Django code contributions and potentially take on leadership roles in Django in the future.
“I signed up for this program with the goal of starting my journey as a contributor, but I ended up gaining so much more. In this community, I found incredible people who not only guide you toward solutions but also encourage and celebrate every achievement along the way.” - Raffaella, Djangonaut
If you have questions, they are holding an AMA session on Zoom next week. See their social media account for more details:
September 06, 2024 04:33 PM UTC
Python Engineering at Microsoft
Python in Visual Studio Code – September 2024 Release
We’re excited to announce the September 2024 release of the Python and Jupyter extensions for Visual Studio Code!
This release includes the following announcements:
- Django unit test support
- Go to definition from inlay hints with Pylance
If you’re interested, you can check the full list of improvements in our changelogs for the Python, Jupyter and Pylance extensions.
Django unit test support
We are excited to announce support for one of our most requested features: you can now discover and run Django unit tests through the Test Explorer!
In order to enable this feature, you will need to add a MANAGE_PY_PATH
environment variable, pointing to your Django application’s manage.py file. To do so, you can follow these steps:
- Set
"python.testing.unittestEnabled": true,
in yoursettings.json
file. - Add
MANAGE_PY_PATH
as an environment variable:- Create a
.env
file at the root of your project. - Add
MANAGE_PY_PATH='<path-to-manage.py>'
to the.env
file, replacing<path-to-manage.py>
with the path to your application’smanage.py
file.Tip: You can copy the path by right clicking on the file in the Explorer view and selecting Copy Path.
- Create a
- Add Django test arguments to
"python.testing.unittestArgs": []
in thesettings.json
file as needed, and remove any arguments that are not compatible with Django.
Note: By default, the Python extension looks for and loads
.env
files at the project root. If your.env
file is not at the project root or you are using VS Code variable substitution, add"python.envFile": "${workspaceFolder}/<path-to-.env>"
to yoursettings.json
file, so the Python extension can load the environment variables in this file when running and discovering tests. See our Python environment variables docs for more information on environment variables.
Navigate to the Testing view, and select the Refresh Tests button to have your Django tests displayed!
For troubleshooting tips, please see our Django testing docs. As you explore this newly added feature, please provide feedback and report any issues in our vscode-python repo or by using the Python: Report Issue command.
Go to definition from inlay hints with Pylance
When enabling inlay hints with Pylance, you can now more conveniently navigate to a type’s definition through Ctrl+Click
or Cmd+Click
when hovering over it.
Other Changes and Enhancements
We have also added small enhancements and fixed issues requested by users that should improve your experience working with Python and Jupyter Notebooks in Visual Studio Code. Some notable changes include:
- You can now access the VS Code Native REPL for Python from the Command Palette (
Ctrl/Cmd + Shift + P
) usingPython: Start Native REPL
(@vscode-python#23727) - VS Code Native REPL for Python now starts at the project folder (@vscode-python#23821)
- Strings are now normalized when sending commands to the VS Code Native REPL (@vscode-python#23743)
- You can now restart the debugger when debugging tests through the debug control widget (@vscode-python#23752)
Call for Community Feedback
As we are planning and prioritizing future work, we value your feedback! Below are a few issues we would love feedback on:
- Design proposal for test coverage in (@vscode-python#22827)
Try out these new improvements by downloading the Python extension and the Jupyter extension from the Marketplace, or install them directly from the extensions view in Visual Studio Code (Ctrl + Shift + X or ⌘ + ⇧ + X). You can learn more about Python support in Visual Studio Code in the documentation. If you run into any problems or have suggestions, please file an issue on the Python VS Code GitHub page.
The post Python in Visual Studio Code – September 2024 Release appeared first on Python.
September 06, 2024 01:50 PM UTC
Real Python
The Real Python Podcast – Episode #219: Astrophysics and Astronomy With Python & PyCon Africa 2024
Are you interested in practicing your Python skills while learning how to solve astrophysics and astronomy problems? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder's Weekly articles and projects.
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
September 06, 2024 12:00 PM UTC
Matt Layman
Kamal On A Droplet - Building SaaS #201
In this episode, we continued working with the Kamal deploy tool. Last time, we build a demo image. With this session, we created a Digital Ocean Droplet (i.e., VM) and deployed to that. Along the journey, we learned a lot about how Kamal operates.
September 06, 2024 12:00 AM UTC
September 05, 2024
Python Software Foundation
Ask questions or tell us what you think: Introducing monthly PSF Board Office Hours!
Greetings, Pythonistas- thank you so much for supporting the work of the Python Software Foundation (PSF) and the Python community! The current PSF Board has decided to invest more in connecting and serving the global Python community by establishing a forum to have regular conversations. The board members of the PSF with the support of PSF staff are excited to introduce monthly PSF Board Office Hours on the PSF Discord. The Office Hours will be sessions where you can share with us how we can help your community, express your perspectives, and provide feedback for the PSF.
What are the PSF Board Office Hours?
Similar to the PSF Grants Program Office Hours where PSF staff members help to answer questions regarding the PSF Grants Program, during the PSF Board Office Hours you can participate in a text-based live chat with PSF Board Directors. This is a chance to connect, share, and collaborate with the PSF Board and staff to improve our community together. Occasionally, we will have dedicated topics such as PyCon US and the PSF Board Elections for the office hour sessions.
Here is some of the work that we collaborate with staff and volunteers on:
- Promotion and outreach for the Python programming language
- Supporting local Python communities
- Organizing PyCon US
- Diversity and Inclusion in our community
- Support handling of Code of Conduct within our communities
- Support regional Python communities via the PSF Grants Program
- Furthering the mission of the PSF
Unless we have a dedicated topic for a session, you are not limited to talking with us about the above topics, although the discussions should be focused on Python, the PSF, and our community. If you think there’s something we can help with or we should know, we welcome you to come and talk to us!
Joining the office hours
The office hour sessions will take place on the PSF Discord server in the #psf-board channel. If you are new to Discord, make sure to check out a tutorial on how you can download the Discord app and sign up for free– then join us on the PSF Discord! To make the office hours more accessible, the office hours will be scheduled at alternating times so no matter where you are based, you can find a time that is most convenient for you! Here is a list of the dates and times:
- September 10th, 2024: 1pm UTC
- October 8th, 2024: 9pm UTC
- November 12th, 2024: 2pm UTC
- December 10th, 2024: 9pm UTC
- January 14th, 2025: 2pm UTC
- February 11th, 2025: 9pm UTC
- March 11th, 2025: 1pm UTC
- April 8th, 2025: 9pm UTC
- May 13th, 2025: 1pm UTC (Live from PyCon US!)
- June 10th, 2025: 9pm UTC
- July 9th, 2025: 1pm UTC
- August 12th, 2025: 9pm UTC
Each session lasts for an hour. You can add the Office Hour sessions to your person calendar using the Python Software Foundations Office Hours calendar. Make sure to check what time these sessions are for you locally so you don't miss out! Sessions after August 13th, 2025, will be announced in the future.
Who will be at the office hours?
Some of the board members of the PSF will be attending each office hour, as well as members of the PSF Staff. The list of the PSF Board Directors can be found on our website. We are passionate Python community members who are happy to listen, help, and provide support to you. We are happy to follow up with you if there are any issues we cannot address immediately during the office hour sessions. As always, you can email us at psf-board@python.org with inquiries, feedback, or comments at any time.
September 05, 2024 10:13 AM UTC
Pallets projects added to scope of PSF CVE Numbering Authority
Pallets is a fiscal sponsoree of the Python Software Foundation. Fiscal sponsorship is a key plank of the PSF’s mission in supporting the Python community. The PSF supports 20 fiscal sponsorees including regional PyCons, Python Meetup and User Groups, and Python projects. Learn more about our Fiscal Sponsorees on our website and consider supporting the groups with a US-tax deductible donation.
September 05, 2024 08:55 AM UTC
Python GUIs
Build a Translation Application Using Tkinter and OpenAI — Use ChatGPT to Translate Your Text from Python
Translation tools have existed for many years and are incredibly useful if you're learning a new language or wanting to read foreign websites. One of the most popular tools is Google Translate , but there is now another alternative: using OpenAI's ChatGPT tool to translate text.
In this tutorial, we'll build a desktop translator application to translate natural language using ChatGPT APIs. We'll be building the UI using the Tkinter library from the Python standard library:
Example translation of text via OpenAI
Installing the Required Packages
Our Translator uses the openai
library to perform the actual translation via OpenAI's ChatGPT tool. Tkinter is already available in the standard library.
The first task will be to set up a Python virtual environment. Open the terminal, and run the following commands:
- Windows
- macOS/Linux
> mkdir translator
> cd translator
> python -m venv venv
> .\venv\Scripts\activate
> python -m pip install openai
$ mkdir translator
$ cd translator/
$ python -m venv venv
$ source venv/bin/activate
(venv) $ python -m pip install openai
Working through these instructions, first we create a root directory for the Translator app. Next we create and activate a Python virtual environment for the project. Finally, we install the openai
package.
Next, create a file named translator.py
in the root of your project. Also add a folder called images/
where you'll store the icons for the application. The folder structure should look like this:
translator/
&boxv
&boxvr&boxh&boxh images/
&boxv &boxvr&boxh&boxh arrow.png
&boxv &boxur&boxh&boxh logo.png
&boxv
&boxur&boxh&boxh translator.py
The images for this project can be downloaded here.
The images/
folder contains the two icons that you'll use for the application. The translator.py
is the app's source file.
Building the Window
Open the translator.py
file with your favorite Python code editor. We'll start by creating our main window:
import tkinter as tk
class TranslatorApp(tk.Tk):
def __init__(self):
super().__init__()
self.title("Language Translator")
self.resizable(width=False, height=False)
if __name__ == "__main__":
app = TranslatorApp()
app.mainloop()
This code imports Tkinter and then defines the application's main class, which we have called TranslatorApp
. This class will hold the application's main window and allow us to run the main loop.
Importing tkinter
under the alias tk
is a common convention in Tkinter code.
Inside the class we define the __init__()
method, which handles initialization of the class. In this method, we first call the initializer __init__()
of the parent class, tk.Tk
, to initialize the app's window. Then, we set the window's title using the title()
method. To make the window unresizable, we use the resizable()
method with width
and height
set to False
.
At the bottom of the code, we have the if __name__ == "__main__"
idiom to check whether the file is being run directly as an executable program. Inside the condition block we first create an instance of TranslatorApp
and then run the application's main loop or event loop.
If you run this code, you'll get an empty Tkinter window on your desktop:
$ python translator.py
The empty Tkinter window
Creating the GUI for the Translator App
Now that the main window is set up, let's start adding widgets to build the GUI. To do this, we'll create a method called setup_ui()
, as shown below:
import tkinter as tk
class TranslatorApp(tk.Tk):
def __init__(self):
super().__init__()
self.title("Language Translator")
self.resizable(width=False, height=False)
self.setup_ui()
def setup_ui(self):
frame = tk.Frame(self)
frame.pack(padx=10, pady=10)
if __name__ == "__main__":
app = TranslatorApp()
app.mainloop()
The setup_ui()
method will define the application's GUI. In this method, we first create a frame widget using the tk.Frame
class whose master
argument is set to self
(the application's main window). Next, we position the frame inside the main window using the pack()
geometry manager, using padx
and pady
arguments to set some padding around the frame.
Finally, we add the call to self.setup_ui()
to the __init__()
method.
We'll continue to develop the UI by adding code to the setup_ui()
method.
Net, we'll add the app's logo. In the setup_ui()
method add the following code below the frame
definition:
import tkinter as tk
class TranslatorApp(tk.Tk):
def __init__(self):
super().__init__()
self.title("Language Translator")
self.resizable(width=False, height=False)
self.setup_ui()
def setup_ui(self):
frame = tk.Frame(self)
frame.pack(padx=10, pady=10)
self.logo = tk.PhotoImage(file="images/logo.png")
tk.Label(frame, image=self.logo).grid(row=0, column=0, sticky="w")
if __name__ == "__main__":
app = TranslatorApp()
app.mainloop()
This code loads the logo using the tk.PhotoImage
class. To resize it, we use the subsample()
method. Then, we add the logo to the frame using a tk.Label
widget. The label takes the frame and the logo as arguments. Finally, to position the logo, we use the grid()
geometry manager with appropriate values for the row
, column
, and sticky
arguments.
The sticky
argument determines which side of a cell the widget should align -- North (top), South (bottom), East (right) or West (left). Here we're aligning it on the Wiest or left of the cell with "w"
:
Tkinter window with the OpenAI logo in it
Getting a List of Languages
We need list of languages to shown in the dropdown. There are various lists available online. But since we're using OpenAI for the translations, why not use it to give us the list of languages too. Since this is just for testing purposes, lets grab the top 20 human languages (by first and second language speakers).
We can prompt ChatGPT with something like:
Give me a list of the top 20 human languages with the most first and second language speakers in Python list format
..and it will return the following list:
languages = [
"English",
"Mandarin Chinese",
"Hindi",
"Spanish",
"French",
"Standard Arabic",
"Bengali",
"Russian",
"Portuguese",
"Urdu"
]
I'm going to add Dutch to the list, because it's my second language. Feel free to add your own languages to the list.
Adding the Interface
Let's start adding some inputs to the UIs. First we'll create the language selection drop down boxes:
import tkinter as tk
import tkinter.ttk as ttk
LANGUAGES = [
"English",
"Mandarin Chinese",
"Hindi",
"Spanish",
"French",
"Standard Arabic",
"Bengali",
"Russian",
"Portuguese",
"Urdu",
"Dutch", # Gekoloniseerd.
]
DEFAULT_SOURCE = "English"
DEFAULT_DEST = "Dutch"
class TranslatorApp(tk.Tk):
def __init__(self):
super().__init__()
self.title("Language Translator")
self.resizable(width=False, height=False)
self.setup_ui()
def setup_ui(self):
frame = tk.Frame(self)
frame.pack(padx=10, pady=10)
self.logo = tk.PhotoImage(file="images/logo.png")
tk.Label(frame, image=self.logo).grid(row=0, column=0, sticky="w")
# Source language combobox
self.from_language = ttk.Combobox(frame, values=LANGUAGES)
self.from_language.current(LANGUAGES.index(DEFAULT_SOURCE))
self.from_language.grid(row=1, column=0, sticky="we")
# Arrow icon
self.arrows = tk.PhotoImage(file="images/arrow.png").subsample(15, 15)
tk.Label(frame, image=self.arrows).grid(row=1, column=1)
# Destination language combobox
self.to_language = ttk.Combobox(frame, values=LANGUAGES)
self.to_language.current(LANGUAGES.index(DEFAULT_DEST))
self.to_language.grid(row=1, column=2, sticky="we")
if __name__ == "__main__":
app = TranslatorApp()
app.mainloop()
We have added our language list as the constant LANGUAGES
. We also define the default languages for when the application starts up, using constants DEFAULT_SOURCE
and DEFAULT_DEST
.
Next, we create two combo boxes to hold the list of source and destination languages. The combo boxes are created using the ttk.Combobox
class. One to the left and another to the right. Between the combo boxes, we've also added an arrow icon loaded using the tk.PhotoImage
class. Again, we've added the icon to the app's window using ttk.Label
.
Both combo boxes take frame
and values
as arguments. The values
argument populates the combo boxes with languages. To specify the default language, we use the current()
method, looking up the position of our default languages in the languages
list with .index()
.
To position the combo boxes inside the frame, we use the grid()
geometry manager with the appropriate arguments. Run the application, and you will see the following window:
Source and destination languages
With the source and destination combo boxes in place, let's add three more widgets: two scrollable text widgets and a button. The scrollable text on the left will hold the source text, while the scrollable text on the right will hold the translated text. The button will allow us to run the actual translation.
Building the Input UI
Get back to the code editor and update the setup_ui()
method as follows. Note that we also need to import the ScrollText
class:
import tkinter as tk
import tkinter.ttk as ttk
from tkinter.scrolledtext import ScrolledText
LANGUAGES = [
"English",
"Mandarin Chinese",
"Hindi",
"Spanish",
"French",
"Standard Arabic",
"Bengali",
"Russian",
"Portuguese",
"Urdu",
"Dutch",
]
DEFAULT_SOURCE = "English"
DEFAULT_DEST = "Dutch"
class TranslatorApp(tk.Tk):
def __init__(self):
super().__init__()
self.title("Language Translator")
self.resizable(width=False, height=False)
self.setup_ui()
def setup_ui(self):
frame = tk.Frame(self)
frame.pack(padx=10, pady=10)
self.logo = tk.PhotoImage(file="images/logo.png").subsample(5, 5)
tk.Label(frame, image=self.logo).grid(row=0, column=0, sticky="w")
# Source language combobox
languages = [lang.title() for lang in LANGUAGES.values()]
self.from_language = ttk.Combobox(frame, values=languages)
self.from_language.current(languages.index(DEFAULT_SOURCE))
self.from_language.grid(row=1, column=0, sticky="we")
# Arrow icon
self.arrows = tk.PhotoImage(file="images/arrow.png").subsample(15, 15)
tk.Label(frame, image=self.arrows).grid(row=1, column=1)
# Destination language combobox
self.to_language = ttk.Combobox(frame, values=languages)
self.to_language.current(languages.index(DEFAULT_DEST))
self.to_language.grid(row=1, column=2, sticky="we")
# Source text
self.from_text = ScrolledText(
frame,
font=("Dotum", 16),
width=50,
height=20,
)
self.from_text.grid(row=2, column=0)
# Translated text
self.to_text = ScrolledText(
frame,
font=("Dotum", 16),
width=50,
height=20,
state="disabled",
)
self.to_text.grid(row=2, column=2)
# Translate button
self.translate_button = ttk.Button(
frame,
text="Translate",
command=self.translate,
)
self.translate_button.grid(row=3, column=0, columnspan=3, pady=10)
def translate(self):
pass
if __name__ == "__main__":
app = TranslatorApp()
app.mainloop()
In the code snippet, we use the ScrolledText
class to create the two scrolled text areas. Both text areas take frame
, font
, width
, and height
as arguments. The second text area also takes state
as an additional argument. Setting state
to "disabled"
allows us to create a read-only text area.
Then, we use the ttk.Button
class to create a button with frame
, text
, and command
as arguments. The command
argument allows us to bind the button's click event to the self.translate()
method, which we will define in a moment. For now, we've added a placeholder.
To position all these widgets on the app's window, we use the grid()
geometry manager. Now, the app will look something like the following:
Translator app's GUI
Our translation app's GUI is ready! Finally, we can start adding functionality to the application.
Getting an OpenAI API Key
You can use OpenAPI's APIs for free, with some limitations. To get an OpenAI API key you will need to create an account. Once you have created an account go ahead and get an API key.
Click "Create new secret key" in the top right hand corner to create a key. Give the key a name (it doesn't matter what you use) and then click "Create secret key". Copy the resulting key and keep it safe. You'll need it in the next step.
Implementing the Translation Functionality
We'll implement the language translation functionality in the translate()
method. This gets the current text from the UI and then uses openai
to perform the translation. We need a few more imports, and to create the OpenAI client instance at the top of the application:
import tkinter as tk
import tkinter.ttk as ttk
from tkinter.messagebox import showerror
from tkinter.scrolledtext import ScrolledText
import httpcore
from openai import OpenAI
client = OpenAI(
api_key="<YOUR API KEY HERE>"
)
Here we've imported the showerror
helper for displaying error boxes in our application. We've imported httpcore
which we'll use to handle HTTP errors when accessing the API. Finally, we've added an import for the OpenAI
class from openai
. This is what handles the actual translation.
To use it, we create an instance of the class as OpenAI
. Replace <YOUR API KEY HERE>
with the API key you generated on OpenAI just now.
We'll continue by implementing the translate()
method. Below we're just showing the function itself:
class TranslatorApp(tk.Tk):
# ...
def translate(self):
source_language = self.from_language.get()
destination_language = self.to_language.get()
text = self.from_text.get(1.0, tk.END).strip()
try:
completion = client.chat.completions.create(
messages=[
{"role": "system", "content": "You are a language interpreter."},
{
"role": "user",
"content": (
f"Translate the following text from {source_language} "
f"to {destination_language}, only reply with the text: "
f"{text}"
),
},
],
model="gpt-3.5-turbo",
)
reply = completion.choices[0].message.content
except httpcore.ConnectError:
showerror(
title="Error",
message="Make sure you have an internet connection",
)
return
except Exception as e:
showerror(
title="Error",
message=f"An unexpected error occurred: {e}",
)
return
self.to_text.config(state="normal")
self.to_text.delete(1.0, tk.END)
self.to_text.insert(tk.END, reply)
self.to_text.config(state="disabled")
The translate()
method handles the entire translation process. It starts by retrieving the source and destination languages from the corresponding combo boxes, and the input text from the box on the left.
If any of these are not defined, we use a showerror
dialog to inform the user of the problem.
Once we have the source and destination language and some text to translate, we can perform the actual translation through ChatGPT. First, we give the language model a hint about what we want it to do -- interpret language:
{"role": "system", "content": "You are a language interpreter."},
Next we build the message we want it to respond to. We ask it to translate the provided text from the source to destination language, and to respond with only the translated text. If we don't specify this, we'll get some additional description or context.
You might want to experiment with asking for the text and context separately, as that is often helpful when learning languages.
{
"role": "user",
"content": (
f"Translate the following text from {source_language} "
f"to {destination_language}, only reply with the text: "
f"{text}"
),
},
The created completion is submitted to the API and we can retrieve the resulting text from the object:
reply = completion.choices[0].message.content
If the call to translate()
finds a connection error, then we tell the user to check their internet connection. To handle any other exceptions, we catch the generic Exception
class and display an error message with the exception details.
If the translation is successful, then we enable the destination scrolled area, display the translated text, and disable the area again so it remains read-only.
The complete final code is shown below:
import tkinter as tk
import tkinter.ttk as ttk
from tkinter.messagebox import showerror
from tkinter.scrolledtext import ScrolledText
import httpcore
from openai import OpenAI
client = OpenAI(
api_key="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"
)
LANGUAGES = [
"English",
"Mandarin Chinese",
"Hindi",
"Spanish",
"French",
"Standard Arabic",
"Bengali",
"Russian",
"Portuguese",
"Urdu",
"Dutch",
]
DEFAULT_SOURCE = "English"
DEFAULT_DEST = "Dutch"
class TranslatorApp(tk.Tk):
def __init__(self):
super().__init__()
self.title("Language Translator")
self.resizable(width=False, height=False)
self.setup_ui()
def setup_ui(self):
frame = tk.Frame(self)
frame.pack(padx=10, pady=10)
self.logo = tk.PhotoImage(file="images/logo.png")
tk.Label(frame, image=self.logo).grid(row=0, column=0, sticky="w")
# Source language combobox
self.from_language = ttk.Combobox(frame, values=LANGUAGES)
self.from_language.current(LANGUAGES.index(DEFAULT_SOURCE))
self.from_language.grid(row=1, column=0, sticky="we")
# Arrow icon
self.arrows = tk.PhotoImage(file="images/arrow.png").subsample(15, 15)
tk.Label(frame, image=self.arrows).grid(row=1, column=1)
# Destination language combobox
self.to_language = ttk.Combobox(frame, values=LANGUAGES)
self.to_language.current(LANGUAGES.index(DEFAULT_DEST))
self.to_language.grid(row=1, column=2, sticky="we")
# Source text
self.from_text = ScrolledText(
frame,
font=("Dotum", 16),
width=50,
height=20,
)
self.from_text.grid(row=2, column=0)
# Translated text
self.to_text = ScrolledText(
frame,
font=("Dotum", 16),
width=50,
height=20,
state="disabled",
)
self.to_text.grid(row=2, column=2)
# Translate button
self.translate_button = ttk.Button(
frame,
text="Translate",
command=self.translate,
)
self.translate_button.grid(row=3, column=0, columnspan=3, pady=10)
def translate(self):
source_language = self.from_language.get()
destination_language = self.to_language.get()
text = self.from_text.get(1.0, tk.END).strip()
try:
completion = client.chat.completions.create(
messages=[
{"role": "system", "content": "You are a language interpreter."},
{
"role": "user",
"content": (
f"Translate the following text from {source_language} "
f"to {destination_language}, only reply with the text: "
f"{text}"
),
},
],
model="gpt-3.5-turbo",
)
reply = completion.choices[0].message.content
except httpcore.ConnectError:
showerror(
title="Error",
message="Make sure you have an internet connection",
)
return
except Exception as e:
showerror(
title="Error",
message=f"An unexpected error occurred: {e}",
)
return
self.to_text.config(state="normal")
self.to_text.delete(1.0, tk.END)
self.to_text.insert(tk.END, reply)
self.to_text.config(state="disabled")
if __name__ == "__main__":
app = TranslatorApp()
app.mainloop()
The finished app is shown below:
The completed Translator app
Conclusion
In this tutorial we built a Translator application using the Tkinter GUI library from the Python standard library. We worked step by step through building the UI using a grid layout, and then implemented the language translation functionality with openai
& ChatGPT.
Try and take what you've learnt in this tutorial & applying it to your own projects!
September 05, 2024 06:00 AM UTC
September 04, 2024
Stefanie Molin
How to Create a Pre-Commit Hook
Pre-commit hooks are a great way to help maintain code quality. However, some of your code quality standards may be specific to your project, and therefore, not covered by existing code linting and formatting tools. In this article, I will show you how to incorporate custom checks into your `pre-commit` setup.
September 04, 2024 02:55 PM UTC
Real Python
Lists vs Tuples in Python
In Python, lists and tuples are versatile and useful data types that allow you to store data in a sequence. You’ll find them in virtually every nontrivial Python program. Learning about them is a core skill for you as a Python developer.
In this tutorial, you’ll:
- Get to know lists and tuples
- Explore the core characteristics of lists and tuples
- Learn how to define and manipulate lists and tuples
- Decide when to use lists or tuples in your code
To get the most out of this tutorial, you should know the basics of Python programming, including how to define variables.
Get Your Code: Click here to download the free sample code that shows you how to work with lists and tuples in Python.
Take the Quiz: Test your knowledge with our interactive “Lists vs Tuples in Python” quiz. You’ll receive a score upon completion to help you track your learning progress:
Interactive Quiz
Lists vs Tuples in PythonChallenge yourself with this quiz to evaluate and deepen your understanding of Python lists and tuples. You'll explore key concepts, such as how to create, access, and manipulate these data types, while also learning best practices for using them efficiently in your code.
Getting Started With Python Lists and Tuples
In Python, a list is a collection of arbitrary objects, somewhat akin to an array in many other programming languages but more flexible. To define a list, you typically enclose a comma-separated sequence of objects in square brackets ([]
), as shown below:
>>> colors = ["red", "green", "blue", "yellow"]
>>> colors
['red', 'green', 'blue', 'yellow']
In this code snippet, you define a list of colors using string objects separated by commas and enclose them in square brackets.
Similarly, tuples are also collections of arbitrary objects. To define a tuple, you’ll enclose a comma-separated sequence of objects in parentheses (()
), as shown below:
>>> person = ("Jane Doe", 25, "Python Developer", "Canada")
>>> person
('Jane Doe', 25, 'Python Developer', 'Canada')
In this example, you define a tuple with data for a given person, including their name, age, job, and base country.
Up to this point, it may seem that lists and tuples are mostly the same. However, there’s an important difference:
Feature | List | Tuple |
---|---|---|
Is an ordered sequence | ✅ | ✅ |
Can contain arbitrary objects | ✅ | ✅ |
Can be indexed and sliced | ✅ | ✅ |
Can be nested | ✅ | ✅ |
Is mutable | ✅ | ❌ |
Both lists and tuples are sequence data types, which means they can contain objects arranged in order. You can access those objects using an integer index that represents their position in the sequence.
Even though both data types can contain arbitrary and heterogeneous objects, you’ll commonly use lists to store homogeneous objects and tuples to store heterogeneous objects.
Note: In this tutorial, you’ll see the terms homogeneous and heterogeneous used to express the following ideas:
- Homogeneous: Objects of the same data type or the same semantic meaning, like a series of animals, fruits, colors, and so on.
- Heterogeneous: Objects of different data types or different semantic meanings, like the attributes of a car: model, color, make, year, fuel type, and so on.
You can perform indexing and slicing operations on both lists and tuples. You can also have nested lists and nested tuples or a combination of them, like a list of tuples.
The most notable difference between lists and tuples is that lists are mutable, while tuples are immutable. This feature distinguishes them and drives their specific use cases.
Essentially, a list doesn’t have a fixed length since it’s mutable. Therefore, it’s natural to use homogeneous elements to have some structure in the list. A tuple, on the other hand, has a fixed length so the position of elements can have meaning, supporting heterogeneous data.
Creating Lists in Python
In many situations, you’ll define a list
object using a literal. A list literal is a comma-separated sequence of objects enclosed in square brackets:
>>> countries = ["United States", "Canada", "Poland", "Germany", "Austria"]
>>> countries
['United States', 'Canada', 'Poland', 'Germany', 'Austria']
In this example, you create a list of countries represented by string objects. Because lists are ordered sequences, the values retain the insertion order.
Read the full article at https://realpython.com/python-lists-tuples/ »
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
September 04, 2024 02:00 PM UTC
Quiz: Lists vs Tuples in Python
Challenge yourself with this quiz to evaluate and deepen your understanding of Python lists and tuples. You’ll explore key concepts, such as how to create and manipulate these data types, while also learning best practices for using them efficiently in your code.
You can take this quiz after reading the Lists vs Tuples in Python tutorial.
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
September 04, 2024 12:00 PM UTC
Hynek Schlawack
Production-ready Python Docker Containers with uv
Starting with 0.3.0, Astral’s uv brought many great features, including support for cross-platform lock files uv.lock
. Together with subsequent fixes, it has become Python’s finest workflow tool for my (non-scientific) use cases. Here’s how I build production-ready containers, as fast as possible.
September 04, 2024 10:00 AM UTC
Talk Python to Me
#476: Unified Python packaging with uv
A couple of weeks ago, Charlie Marsh and the folks at Astral made another big splash with a major release of uv called "uv: Unified Python packaging" which has many far reaching features. We had to have Charlie on the show to give us the inside look into this development. Let's get to it.<br/> <br/> <strong>Episode sponsors</strong><br/> <br/> <a href='https://talkpython.fm/posit'>Posit</a><br> <a href='https://talkpython.fm/training'>Talk Python Courses</a><br/> <br/> <strong>Links from the show</strong><br/> <br/> <div><b>Charlie Marsh on Twitter</b>: <a href="https://twitter.com/charliermarsh?featured_on=talkpython" target="_blank" >@charliermarsh</a><br/> <b>Charlie Marsh on Mastodon</b>: <a href="https://hachyderm.io/@charliermarsh?featured_on=talkpython" target="_blank" >@charliermarsh</a><br/> <br/> <b>Episode follow up: Wrote up how we use uv at Talk Python</b>: <a href="https://mkennedy.codes/posts/python-docker-images-using-uv-s-new-python-features/?featured_on=talkpython" target="_blank" >mkennedy.codes</a><br/> <br/> <b>uv: Unified Python packaging</b>: <a href="https://astral.sh/blog/uv-unified-python-packaging?featured_on=talkpython" target="_blank" >astral.sh</a><br/> <b>Python executable management</b>: <a href="https://docs.astral.sh/uv/concepts/python-versions/?featured_on=talkpython" target="_blank" >astral.sh</a><br/> <b>Projects</b>: <a href="https://docs.astral.sh/uv/concepts/projects/?featured_on=talkpython" target="_blank" >astral.sh</a><br/> <b>Tools</b>: <a href="https://docs.astral.sh/uv/concepts/tools/?featured_on=talkpython" target="_blank" >astral.sh</a><br/> <b>Scripts</b>: <a href="https://docs.astral.sh/uv/guides/scripts/?featured_on=talkpython" target="_blank" >astral.sh</a><br/> <b>Rye and uv: August is Harvest Season for Python Packaging</b>: <a href="https://lucumr.pocoo.org/2024/8/21/harvest-season/?featured_on=talkpython" target="_blank" >lucumr.pocoo.org</a><br/> <b>Python Build Standalone releases</b>: <a href="https://github.com/indygreg/python-build-standalone/releases?featured_on=talkpython" target="_blank" >github.com</a><br/> <b>Rules</b>: <a href="https://docs.astral.sh/ruff/rules/?featured_on=talkpython" target="_blank" >astral.sh</a><br/> <b>Watch this episode on YouTube</b>: <a href="https://www.youtube.com/watch?v=oj8yk0Y-Ky0" target="_blank" >youtube.com</a><br/> <b>Episode transcripts</b>: <a href="https://talkpython.fm/episodes/transcript/476/unified-python-packaging-with-uv" target="_blank" >talkpython.fm</a><br/> <br/> <b>--- Stay in touch with us ---</b><br/> <b>Subscribe to us on YouTube</b>: <a href="https://talkpython.fm/youtube" target="_blank" >youtube.com</a><br/> <b>Follow Talk Python on Mastodon</b>: <a href="https://fosstodon.org/web/@talkpython" target="_blank" ><i class="fa-brands fa-mastodon"></i>talkpython</a><br/> <b>Follow Michael on Mastodon</b>: <a href="https://fosstodon.org/web/@mkennedy" target="_blank" ><i class="fa-brands fa-mastodon"></i>mkennedy</a><br/></div>
September 04, 2024 08:00 AM UTC
September 03, 2024
PyCoder’s Weekly
Issue #645 (Sept. 3, 2024)
#645 – SEPTEMBER 3, 2024
View in Browser »
Using Astropy for Astronomy With Python
This course covers two problems from introductory astronomy to help you play with some Python libraries. You’ll use NumPy, Matplotlib, and pandas to find planet conjunctions, and graph the best viewing times for a star.
REAL PYTHON course
Python Language Summit 2024
Talk Python to Me interviews Seth Michael Larson and they talk about this year’s Python Language Summit. Learn all about what happened at the closed door session for core developers inside PyCon.
KENNEDY & LARSON podcast
Instant PEP 8 Compliance Checks & Fixes With Top AI Code Reviewer
With CodeRabbit, solve your indentation issues and security concerns. CodeRabbit doesn’t just point out issues; it suggests fixes and explains the reasoning behind the suggestion. Elevate code quality with AI-powered, context-aware reviews and 1-click fixes. Sign up for free today →
CODERABBIT sponsor
Layman’s Guide to Python Built-in Functions
This is a plain language guide to every built-in function in Python, paired with a simple example that shows each function in action.
MATT LAYMAN
Articles & Tutorials
Sorting Dictionaries in Python: Keys, Values, and More
In this video course, you’ll learn how to sort Python dictionaries. By the end, you’ll be able to sort by key, value, or even nested attributes. But you won’t stop there—you’ll also measure the performance of variations when sorting and compare different key-value data structures.
REAL PYTHON course
Understanding the Template Method Pattern in Python
The Template Method Pattern is when a base class is used to implement a series of steps, and subclasses can override one or more of those steps to customize the process. This article shows an example usage in Python and why you might want to implement it.
LANCE GOYKE
Accelerate Edge Devices With High-Performance AI Power
Experience the power of Edge AI—delivering lightning-fast, real-time processing where it matters. Optimize your applications with low latency, high efficiency, and unparalleled accuracy. Push performance beyond limits with Intel’s OpenVINO toolkit.
INTEL CORPORATION sponsor
Asyncio gather()
Limit Concurrency
The asyncio.gather()
function allows you to run multiple co-routines concurrently. There are times when you want to control just how much concurrency you have though and this post shows you how to use a semaphore to do just that.
JASON BROWNLEE
Adventures Building a Spreadsheet Engine in Python
Spreadsheets are a fascinating tool: you can both store and structure data, and include formulas that run computations on the contents of a sheet. James has written a spreadsheet engine, and this post talks about how it is done.
JAMES G
What’s New in pip
24.2
In version 24.2, pip learns to use system certificates by default, receives a handful of optimizations, and deprecates legacy (setup.py develop) editable installations. This article covers the changes and why they’ve been made.
RICHARD SI
CPython Compiler Hardening
Nate has been working on the CPython compiler, applying memory hardening guidelines recommended by OpenSSF’s Memory Safety Special Interest Group. This blog post talks about what was applied and how it should improve CPython.
NATE OHLSON
A Comparison of Providers for Python Serverless Functions
This is a comparison chart of the most common host providers that support Python serverless functions. It compares what features are supported, pricing, runtime limits, and more.
HAROLD MARTIN
There Can’t Be Only One
A weird historical first in baseball recently reminded James about how often as programmers we map our data assuming a one-to-one relationship, and how often that’s a bad choice.
JAMES BENNETT
Safety & Security Engineer: First Year in Review
It has been a year since Mike joined the PSF as the Safety & Security Engineer for PyPI. This blog post talks about all the things he’s been involved with.
MIKE FIELDER
Python Developers Survey 2023 Results
Official Python Developers Survey 2023 Results by Python Software Foundation and JetBrains: more than 25k responses from almost 200 countries.
JETBRAINS.COM
Lesser Known Parts of Python Standard Library
This article covers some of the lesser used parts of the Python standard library, including Deque
, defaultdict
, UserDict
, and more.
TRICKSTER DEV
Projects & Code
Events
EARL 2024
September 4 to September 6, 2024
DATACOVE.CO.UK
Weekly Real Python Office Hours Q&A (Virtual)
September 4, 2024
REALPYTHON.COM
PyCon Estonia 2024
September 5 to September 7, 2024
PYCON.EE
Canberra Python Meetup
September 5, 2024
MEETUP.COM
Sydney Python User Group (SyPy)
September 5, 2024
SYPY.ORG
PyDelhi User Group Meetup
September 7, 2024
MEETUP.COM
Happy Pythoning!
This was PyCoder’s Weekly Issue #645.
View in Browser »
[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]
September 03, 2024 07:30 PM UTC
Mike Driscoll
ANN: JupyterLab 101 Kickstarter
My latest Python book is now available for pre-order on Kickstarter.
![JupyterLab 101 mockup](https://www.blog.pythonlibrary.org/wp-content/uploads/2024/09/jupyter3D3-1024x768.jpg)
JupyterLab, the latest iteration of the Jupyter Notebook, is a versatile tool for sharing code in an easily understandable format.
Hundreds of thousands of people around the world use Jupyter Notebooks or variations of the Notebook architecture for any or all of the following:
- teaching
- presentations
- learning a computer language
- numerical simulations
- statistical modeling
- data visualization
- machine learning
- and much more!
Jupyter Notebooks can be emailed, put on GitHub, or run online. You may also add HTML, images, Markdown, videos, LaTeX, and custom MIME types to your Notebooks. Finally, Jupyter Notebooks support big data integration.
JupyterLab 101 will get you up to speed on the newest user interface for Jupyter Notebooks and the other tools that JupyterLab supports. You now have a tabbed interface that you can use to edit multiple Notebooks, open terminals in your browser, create a Python REPL, and more. JupyterLab also includes a debugger utility to help you figure out your coding issues.
Rest assured, JupyterLab supports all the same programming languages as Jupyter Notebook. The main difference lies in the user interface, which this guide will help you navigate effectively and efficiently.
After reading JupyterLab 101, you will be an expert in JupyterLab and produce quality Notebooks quickly!
What You’ll Learn
In this book, you will learn how about the following:
- Installation and setup of JupyterLab
- The JupyterLab user interface
- Creating a Notebook
- Markdown in Notebooks
- Menus in JupyterLab
- Launching Other Applications (console, terminal, text files, etc)
- Distributing and Exporting Notebooks
- Debugging in JupyterLab
- Testing your notebooks
Rewards to Choose From
As a backer of this Kickstarter, you have some choices to make. You can receive one or more of the following, depending on which level you choose when backing the project:
- An early copy of JupyterLab 101 + all updates including the final version (ALL BACKERS)
- A signed paperback copy (If you choose the appropriate perk)
- Get all by Python courses hosted on Teach Me Python or another site (If you choose the appropriate perk)
- T-shirt with the book cover (If you choose the appropriate perk)
Get the book on Kickstarter today!
The post ANN: JupyterLab 101 Kickstarter appeared first on Mouse Vs Python.