skip to navigation
skip to content

Planet Python

Last update: November 22, 2019 10:48 AM UTC

November 22, 2019

Programiz

Python CSV

In this tutorial, we will learn how to read and write into CSV files in Python with the help of examples.

November 22, 2019 08:41 AM UTC

November 21, 2019

Python Anywhere

System update on 21 November 2019

This morning's system update went smoothly; some websites did take a bit longer than we expected to start up afterwards, but all is well now.

There are two big new features that we have introduced which are going through some final post-deploy tests before they go live -- a new system image (called fishnchips) to support Python 3.8 and to add on a number of extra OS packages that people have been asking us for, and an update to our virtualization system that will fix a number of problems with specific programs. We'll be posting more about those when they're ready to go live.

But there were a number of other things we added:

Happy coding!

November 21, 2019 05:40 PM UTC

PyCharm

PyCharm 2019.3 Release Candidate

The release of PyCharm 2019.3 is right around the corner and we’re excited to announce we now have available a release candidate version. Check it out by downloading it from our website!

Improvements in this version

Read the release notes to learn more.

Interested?

Download the RC from our website. Alternatively, you can use the JetBrains Toolbox App to stay up to date.

If you’re on Ubuntu 16.04 or later, you can use snap to get PyCharm RC versions, and stay up to date. You can find the installation instructions on our website.

The release candidate (RC) is not an early access program (EAP) build, and does not bundle an EAP license. If you get PyCharm Professional Edition RC, you will either need a currently active PyCharm subscription, or you will receive a 30-day free trial.

November 21, 2019 05:07 PM UTC

py.CheckIO

11 Ways How To Make Home Education More Effective

November 21, 2019 04:06 PM UTC

NumFOCUS

Now Hiring: Matplotlib Research Software Engineering Fellow

The post Now Hiring: Matplotlib Research Software Engineering Fellow appeared first on NumFOCUS.

November 21, 2019 03:15 PM UTC

The Digital Cat

Punch 2.0.0 is out

Punch 2.0.0 is out!

This is the latest release of the project that I started to replace bumpversion. Update your version while having a drink!

Punch is a configurable version updater, and you can use to automate the management of your project’s version number.

Changes:

Read the full documentation here

November 21, 2019 03:00 PM UTC

IslandT

Sort list alphabetically with python

You will be given a vector of string(s). You must sort it alphabetically (case-sensitive!!) and then return the first value.

The returned value must be a string and have “***” between each of its letters.

You should not remove or add elements from/to the array.

Above is another problem in codewars, besides asking us to sort the array list and returning the first value in that list, we also need to insert stars within the characters.

def two_sort(array):
    
    array.sort()
    first_string = array[0]
    first_star_string = ''
    limit = len(first_string)

    for i in range(0, limit):
        if i == 0:
            first_star_string += first_string[i]
        else:
            first_star_string +='***'+first_string[i]
            
    return first_star_string

The python solution above is straight forward but needs further improvement if possible, do write down your own answer in the comment box below this post.

November 21, 2019 12:38 PM UTC

Andre Roberge

Abolishing SyntaxError: invalid syntax ...

... and other cryptic messages.

Do you remember when you first started programming (possibly with Python) and encountered an error message that completely baffled you? For some reason, perhaps because you were required to complete a formal course or because you were naturally persistent, you didn't let such messages discourage you entirely and you persevered. And now, whenever you see such cryptic error messages, you can almost immediately decipher them and figure out what causes them and fix the problem.

Congratulations, you are part of an elite group! Even a large number of people who claim that they can program are almost certainly less capable than you are.

Given your good fortune, would you mind donating 5 to 10 minutes of your time to help countless beginners that are struggling in trying to understand Python error messages?  All you need to do is:


  1. Glance through of exceptions on this page and use your experience to find a case not covered. Note that this excludes SyntaxError cases, some of which are included here, but would require more of your time.
  2. Fire up your favourite Python REPL and write some simple code that generates an exception not already covered.  Perhaps, something like ValueError: could not convert string to float: 'a'
  3. Create a new issue with the error message as the title, including the code that generated the exception in the description of the issue together with a simple explanation (in a couple of sentences) of what the error message means.  Imagine that you are writing an explanation for the twelve year old child of your best friend who has expressed some interest in learning how to program.  This simple explanation is the most important part ... however, do not worry about getting it absolutely perfect as it will likely be improved upon based on feedback from future "real beginners".
  4. Go back to whatever you were doing before, knowing that the few minutes you have invested will cumulatively save many hours to future generation of programmers that encounter the exception you wrote about.
It should go without saying that contributions that require more time and effort that what is described above are also very welcome!  If you feel particularly ambitious, you can certainly improve the existing code that currently analyses cases of SyntaxError: invalid syntax, which currently handles only a few cases, and should be seen more as a prototype/proof-of-concept.

Future plans for friendly-traceback


Friendly-traceback is being written so that it could be easily incorporated into editors or IDEs that are designed for beginners. I intend to ensure that it can be easily added to Python's IDLE, as well as Mu and Thonny. I hasten to add that Thonny already includes an excellent tool (its "Assistant") which provides amazing feedback to beginners in some cases. Thonny's assistant uses a complementary approach to that of Friendly-traceback and it is quite likely that a future version of Friendly-traceback will include and expand upon the type of analysis performed by Thonny's assistant to help beginners. However, for the moment, the current development of Friendly-traceback is focused on breadth of coverage (i.e. increasing the number of exceptions included), providing a single most-likely explanation for each exception, rather than looking at multiple possible causes for a given exception as is done by Thonny's Assistant.

November 21, 2019 07:46 AM UTC

Wingware Blog

Navigating Python Code with Wing Pro 7 (part 2 of 3)

Last week we looked at goto-definition, find uses, and project-wide search as tools for navigating Python code in Wing 7. This time, we'll take a look at the code indices that Wing provides.

Code Index Menus

A quick way to navigate code in the current Python file is to use the source index menus shown at the top of the editor:

/images/blog/code-navigation/index-menus.png

Depending on the location of the caret in the editor, Wing may show multiple menus, as in the above example: One for the contents of the top level of the file, one for the contents of the current top-level scope, and additional menus for each sub-scope. Clicking on any of these provides an index and selecting an item jumps to that place in the source code.

Source Browser

In Wing Pro only, the Source Browser in the Tools menu provides another way to view an index of your source code, either for the current module, all project modules, or all classes:

/images/blog/code-navigation/source-browser.png

The scope being browsed and the types of symbols shown may be selected by clicking on the menus at the top of the tool. Double-clicking on items displays them in the editor.

Browsing all project modules or classes assume that you have used Add Existing Directory in the Project menu to add your source code to your project. Typically the project should contain the code you are actively working on. Packages that your code uses can be left out of the project, unless you anticipate often wanting to open or search files in them. Wing will still be able to find them through the Python Path configured by Python or in Wing's Project Properties.



That's it for now! We'll be back next week to conclude this Wing Tips mini-series on navigating Python code with Wing.

As always, please don't hesitate to email support@wingware.com if you run into problems or have any questions.

November 21, 2019 01:00 AM UTC

November 20, 2019

NumFOCUS

NumFOCUS Summit 2019

The post NumFOCUS Summit 2019 appeared first on NumFOCUS.

November 20, 2019 11:24 PM UTC

Codementor

Scraping dynamic websites using Scraper API and Python

Learn how to efficiently and easily scrape modern Javascript enabled websites or Single Page Applications without installing a headless browser and Selenium

November 20, 2019 05:30 PM UTC

TechBeamers Python

Top 25 Python Libraries for Data Science Projects

This post is attempting to enlighten you about the most useful and popular Python libraries used by data scientists. And why only Python, because it has been the leading programming language for solving real-time data science problems. These libraries have been tested to give excellent results in various areas like Machine Learning (ML), Deep Learning, Artifical Intelligence (AI), and Data Science challenges. Hence, you can confidently induct any of these without putting too much time and effort in R&D. In every data science project, programmers, even architects, use to spend considerable time researching the Python libraries that can be the

The post Top 25 Python Libraries for Data Science Projects appeared first on Learn Programming and Software Testing.

November 20, 2019 05:24 PM UTC

Real Python

Invalid Syntax in Python: Common Reasons for SyntaxError

Python is known for its simple syntax. However, when you’re learning Python for the first time or when you’ve come to Python with a solid background in another programming language, you may run into some things that Python doesn’t allow. If you’ve ever received a SyntaxError when trying to run your Python code, then this guide can help you. Throughout this tutorial, you’ll see common examples of invalid syntax in Python and learn how to resolve the issue.

By the end of this tutorial, you’ll be able to:

Free Bonus: 5 Thoughts On Python Mastery, a free course for Python developers that shows you the roadmap and the mindset you'll need to take your Python skills to the next level.

Invalid Syntax in Python

When you run your Python code, the interpreter will first parse it to convert it into Python byte code, which it will then execute. The interpreter will find any invalid syntax in Python during this first stage of program execution, also known as the parsing stage. If the interpreter can’t parse your Python code successfully, then this means that you used invalid syntax somewhere in your code. The interpreter will attempt to show you where that error occurred.

When you’re learning Python for the first time, it can be frustrating to get a SyntaxError. Python will attempt to help you determine where the invalid syntax is in your code, but the traceback it provides can be a little confusing. Sometimes, the code it points to is perfectly fine.

Note: If your code is syntactically correct, then you may get other exceptions raised that are not a SyntaxError. To learn more about Python’s other exceptions and how to handle them, check out Python Exceptions: An Introduction.

You can’t handle invalid syntax in Python like other exceptions. Even if you tried to wrap a try and except block around code with invalid syntax, you’d still see the interpreter raise a SyntaxError.

SyntaxError Exception and Traceback

When the interpreter encounters invalid syntax in Python code, it will raise a SyntaxError exception and provide a traceback with some helpful information to help you debug the error. Here’s some code that contains invalid syntax in Python:

 1 # theofficefacts.py
 2 ages = {
 3     'pam': 24,
 4     'jim': 24
 5     'michael': 43
 6 }
 7 print(f'Michael is {ages["michael"]} years old.')

You can see the invalid syntax in the dictionary literal on line 4. The second entry, 'jim', is missing a comma. If you tried to run this code as-is, then you’d get the following traceback:

$ python theofficefacts.py
File "theofficefacts.py", line 5
    'michael': 43
            ^
SyntaxError: invalid syntax

Note that the traceback message locates the error in line 5, not line 4. The Python interpreter is attempting to point out where the invalid syntax is. However, it can only really point to where it first noticed a problem. When you get a SyntaxError traceback and the code that the traceback is pointing to looks fine, then you’ll want to start moving backward through the code until you can determine what’s wrong.

In the example above, there isn’t a problem with leaving out a comma, depending on what comes after it. For example, there’s no problem with a missing come after 'michael' in line 5. But once the interpreter encounters something that doesn’t make sense, it can only point you to the first thing it found that it couldn’t understand.

Note: This tutorial assumes that you know the basics of Python’s tracebacks. To learn more about the Python traceback and how to read them, check out Understanding the Python Traceback.

There are a few elements of a SyntaxError traceback that can help you determine where the invalid syntax is in your code:

In the example above, the file name given was theofficefacts.py, the line number was 5, and the caret pointed to the closing quote of the dictionary key michael. The SyntaxError traceback might not point to the real problem, but it will point to the first place where the interpreter couldn’t make sense of the syntax.

There are two other exceptions that you might see Python raise. These are equivalent to SyntaxError but have different names:

  1. IndentationError
  2. TabError

These exceptions both inherit from the SyntaxError class, but they’re special cases where indentation is concerned. An IndentationError is raised when the indentation levels of your code don’t match up. A TabError is raised when your code uses both tabs and spaces in the same file. You’ll take a closer look at these exceptions in a later section.

Common Syntax Problems

When you encounter a SyntaxError for the first time, it’s helpful to know why there was a problem and what you might do to fix the invalid syntax in your Python code. In the sections below, you’ll see some of the more common reasons that a SyntaxError might be raised and how you can fix them.

Misusing the Assignment Operator (=)

There are several cases in Python where you’re not able to make assignments to objects. Some examples are assigning to literals and function calls. In the code block below, you can see a few examples that attempt to do this and the resulting SyntaxError tracebacks:

>>>
>>> len('hello') = 5
  File "<stdin>", line 1
SyntaxError: can't assign to function call

>>> 'foo' = 1
  File "<stdin>", line 1
SyntaxError: can't assign to literal

>>> 1 = 'foo'
  File "<stdin>", line 1
SyntaxError: can't assign to literal

The first example tries to assign the value 5 to the len() call. The SyntaxError message is very helpful in this case. It tells you that you can’t assign a value to a function call.

The second and third examples try to assign a string and an integer to literals. The same rule is true for other literal values. Once again, the traceback messages indicate that the problem occurs when you attempt to assign a value to a literal.

Note: The examples above are missing the repeated code line and caret (^) pointing to the problem in the traceback. The exception and traceback you see will be different when you’re in the REPL vs trying to execute this code from a file. If this code were in a file, then you’d get the repeated code line and caret pointing to the problem, as you saw in other cases throughout this tutorial.

It’s likely that your intent isn’t to assign a value to a literal or a function call. For instance, this can occur if you accidentally leave off the extra equals sign (=), which would turn the assignment into a comparison. A comparison, as you can see below, would be valid:

>>>
>>> len('hello') == 5
True

Most of the time, when Python tells you that you’re making an assignment to something that can’t be assigned to, you first might want to check to make sure that the statement shouldn’t be a Boolean expression instead. You may also run into this issue when you’re trying to assign a value to a Python keyword, which you’ll cover in the next section.

Misspelling, Missing, or Misusing Python Keywords

Python keywords are a set of protected words that have special meaning in Python. These are words you can’t use as identifiers, variables, or function names in your code. They’re a part of the language and can only be used in the context that Python allows.

There are three common ways that you can mistakenly use keywords:

  1. Misspelling a keyword
  2. Missing a keyword
  3. Misusing a keyword

If you misspell a keyword in your Python code, then you’ll get a SyntaxError. For example, here’s what happens if you spell the keyword for incorrectly:

>>>
>>> fro i in range(10):
  File "<stdin>", line 1
    fro i in range(10):
        ^
SyntaxError: invalid syntax

The message reads SyntaxError: invalid syntax, but that’s not very helpful. The traceback points to the first place where Python could detect that something was wrong. To fix this sort of error, make sure that all of your Python keywords are spelled correctly.

Another common issue with keywords is when you miss them altogether:

>>>
>>> for i range(10):
  File "<stdin>", line 1
    for i range(10):
              ^
SyntaxError: invalid syntax

Once again, the exception message isn’t that helpful, but the traceback does attempt to point you in the right direction. If you move back from the caret, then you can see that the in keyword is missing from the for loop syntax.

You can also misuse a protected Python keyword. Remember, keywords are only allowed to be used in specific situations. If you use them incorrectly, then you’ll have invalid syntax in your Python code. A common example of this is the use of continue or break outside of a loop. This can easily happen during development when you’re implementing things and happen to move logic outside of a loop:

>>>
>>> names = ['pam', 'jim', 'michael']
>>> if 'jim' in names:
...     print('jim found')
...     break
...
  File "<stdin>", line 3
SyntaxError: 'break' outside loop

>>> if 'jim' in names:
...     print('jim found')
...     continue
...
  File "<stdin>", line 3
SyntaxError: 'continue' not properly in loop

Here, Python does a great job of telling you exactly what’s wrong. The messages "'break' outside loop" and "'continue' not properly in loop" help you figure out exactly what to do. If this code were in a file, then Python would also have the caret pointing right to the misused keyword.

Another example is if you attempt to assign a Python keyword to a variable or use a keyword to define a function:

>>>
>>> pass = True
  File "<stdin>", line 1
    pass = True
         ^
SyntaxError: invalid syntax

>>> def pass():
  File "<stdin>", line 1
    def pass():
           ^
SyntaxError: invalid syntax

When you attempt to assign a value to pass, or when you attempt to define a new function called pass, you’ll get a SyntaxError and see the "invalid syntax" message again.

It might be a little harder to solve this type of invalid syntax in Python code because the code looks fine from the outside. If your code looks good, but you’re still getting a SyntaxError, then you might consider checking the variable name or function name you want to use against the keyword list for the version of Python that you’re using.

The list of protected keywords has changed with each new version of Python. For example, in Python 3.6 you could use await as a variable name or function name, but as of Python 3.7, that word has been added to the keyword list. Now, if you try to use await as a variable or function name, this will cause a SyntaxError if your code is for Python 3.7 or later.

Another example of this is print, which differs in Python 2 vs Python 3:

Version print Type Takes A Value
Python 2 keyword no
Python 3 built-in function yes

print is a keyword in Python 2, so you can’t assign a value to it. In Python 3, however, it’s a built-in function that can be assigned values.

You can run the following code to see the list of keywords in whatever version of Python you’re running:

import keyword
print(keyword.kwlist)

keyword also provides the useful keyword.iskeyword(). If you just need a quick way to check the pass variable, then you can use the following one-liner:

>>>
>>> import keyword; keyword.iskeyword('pass')
True

This code will tell you quickly if the identifier that you’re trying to use is a keyword or not.

Missing Parentheses, Brackets, and Quotes

Often, the cause of invalid syntax in Python code is a missed or mismatched closing parenthesis, bracket, or quote. These can be hard to spot in very long lines of nested parentheses or longer multi-line blocks. You can spot mismatched or missing quotes with the help of Python’s tracebacks:

>>>
>>> message = 'don't'
  File "<stdin>", line 1
    message = 'don't'
                   ^
SyntaxError: invalid syntax

Here, the traceback points to the invalid code where there’s a t' after a closing single quote. To fix this, you can make one of two changes:

  1. Escape the single quote with a backslash ('don\'t')
  2. Surround the entire string in double-quotes instead ("don't")

Another common mistake is to forget to close string. With both double-quoted and single-quoted strings, the situation and traceback are the same:

>>>
>>> message = "This is an unclosed string
  File "<stdin>", line 1
    message = "This is an unclosed string
                                        ^
SyntaxError: EOL while scanning string literal

This time, the caret in the traceback points right to the problem code. The SyntaxError message, "EOL while scanning string literal", is a little more specific and helpful in determining the problem. This means that the Python interpreter got to the end of a line (EOL) before an open string was closed. To fix this, close the string with a quote that matches the one you used to start it. In this case, that would be a double quote (").

Quotes missing from statements inside an f-string can also lead to invalid syntax in Python:

 1 # theofficefacts.py
 2 ages = {
 3     'pam': 24,
 4     'jim': 24,
 5     'michael': 43
 6 }
 7 print(f'Michael is {ages["michael]} years old.')

Here, the reference to the ages dictionary inside the printed f-string is missing the closing double quote from the key reference. The resulting traceback is as follows:

$ python theofficefacts.py
  File "theofficefacts.py", line 7
    print(f'Michael is {ages["michael]} years old.')
         ^
SyntaxError: f-string: unterminated string

Python identifies the problem and tells you that it exists inside the f-string. The message "unterminated string" also indicates what the problem is. The caret in this case only points to the beginning of the f-string.

This might not be as helpful as when the caret points to the problem area of the f-string, but it does narrow down where you need to look. There’s an unterminated string somewhere inside that f-string. You just have to find out where. To fix this problem, make sure that all internal f-string quotes and brackets are present.

The situation is mostly the same for missing parentheses and brackets. If you leave out the closing square bracket from a list, for example, then Python will spot that and point it out. There are a few variations of this, however. The first is to leave the closing bracket off of the list:

# missing.py
def foo():
    return [1, 2, 3

print(foo())

When you run this code, you’ll be told that there’s a problem with the call to print():

$ python missing.py
  File "missing.py", line 5
    print(foo())
        ^
SyntaxError: invalid syntax

What’s happening here is that Python thinks the list contains three elements: 1, 2, and 3 print(foo()). Python uses whitespace to group things logically, and because there’s no comma or bracket separating 3 from print(foo()), Python lumps them together as the third element of the list.

Another variation is to add a trailing comma after the last element in the list while still leaving off the closing square bracket:

# missing.py
def foo():
    return [1, 2, 3,

print(foo())

Now you get a different traceback:

$ python missing.py
  File "missing.py", line 6

                ^
SyntaxError: unexpected EOF while parsing

In the previous example, 3 and print(foo()) were lumped together as one element, but here you see a comma separating the two. Now, the call to print(foo()) gets added as the fourth element of the list, and Python reaches the end of the file without the closing bracket. The traceback tells you that Python got to the end of the file (EOF), but it was expecting something else.

In this example, Python was expecting a closing bracket (]), but the repeated line and caret are not very helpful. Missing parentheses and brackets are tough for Python to identify. Sometimes the only thing you can do is start from the caret and move backward until you can identify what’s missing or wrong.

Mistaking Dictionary Syntax

You saw earlier that you could get a SyntaxError if you leave the comma off of a dictionary element. Another form of invalid syntax with Python dictionaries is the use of the equals sign (=) to separate keys and values, instead of the colon:

>>>
>>> ages = {'pam'=24}
  File "<stdin>", line 1
    ages = {'pam'=24}
                 ^
SyntaxError: invalid syntax

Once again, this error message is not very helpful. The repeated line and caret, however, are very helpful! They’re pointing right to the problem character.

This type of issue is common if you confuse Python syntax with that of other programming languages. You’ll also see this if you confuse the act of defining a dictionary with a dict() call. To fix this, you could replace the equals sign with a colon. You can also switch to using dict():

>>>
>>> ages = dict(pam=24)
>>> ages
{'pam': 24}

You can use dict() to define the dictionary if that syntax is more helpful.

Using the Wrong Indentation

There are two sub-classes of SyntaxError that deal with indentation issues specifically:

  1. IndentationError
  2. TabError

While other programming languages use curly braces to denote blocks of code, Python uses whitespace. That means that Python expects the whitespace in your code to behave predictably. It will raise an IndentationError if there’s a line in a code block that has the wrong number of spaces:

 1 # indentation.py
 2 def foo():
 3     for i in range(10):
 4         print(i)
 5   print('done')
 6 
 7 foo()

This might be tough to see, but line 5 is only indented 2 spaces. It should be in line with the for loop statement, which is 4 spaces over. Thankfully, Python can spot this easily and will quickly tell you what the issue is.

There’s also a bit of ambiguity here, though. Is the print('done') line intended to be after the for loop or inside the for loop block? When you run the above code, you’ll see the following error:

$ python indentation.py
  File "indentation.py", line 5
    print('done')
                ^
IndentationError: unindent does not match any outer indentation level

Even though the traceback looks a lot like the SyntaxError traceback, it’s actually an IndentationError. The error message is also very helpful. It tells you that the indentation level of the line doesn’t match any other indentation level. In other words, print('done') is indented 2 spaces, but Python can’t find any other line of code that matches this level of indentation. You can fix this quickly by making sure the code lines up with the expected indentation level.

The other type of SyntaxError is the TabError, which you’ll see whenever there’s a line that contains either tabs or spaces for its indentation, while the rest of the file contains the other. This might go hidden until Python points it out to you!

If your tab size is the same width as the number of spaces in each indentation level, then it might look like all the lines are at the same level. However, if one line is indented using spaces and the other is indented with tabs, then Python will point this out as a problem:

 1 # indentation.py
 2 def foo():
 3     for i in range(10):
 4         print(i)
 5     print('done')
 6 
 7 foo()

Here, line 5 is indented with a tab instead of 4 spaces. This code block could look perfectly fine to you, or it could look completely wrong, depending on your system settings.

Python, however, will notice the issue immediately. But before you run the code to see what Python will tell you is wrong, it might be helpful for you to see an example of what the code looks like under different tab width settings:

$ tabs 4 # Sets the shell tab width to 4 spaces
$ cat -n indentation.py
     1   # indentation.py
     2   def foo():
     3       for i in range(10)
     4           print(i)
     5       print('done')
     6   
     7   foo()

$ tabs 8 # Sets the shell tab width to 8 spaces (standard)
$ cat -n indentation.py
     1   # indentation.py
     2   def foo():
     3       for i in range(10)
     4           print(i)
     5           print('done')
     6   
     7   foo()

$ tabs 3 # Sets the shell tab width to 3 spaces
$ cat -n indentation.py
     1   # indentation.py
     2   def foo():
     3       for i in range(10)
     4           print(i)
     5      print('done')
     6   
     7   foo()

Notice the difference in display between the three examples above. Most of the code uses 4 spaces for each indentation level, but line 5 uses a single tab in all three examples. The width of the tab changes, based on the tab width setting:

When you run the code, you’ll get the following error and traceback:

$ python indentation.py
  File "indentation.py", line 5
    print('done')
                ^
TabError: inconsistent use of tabs and spaces in indentation

Notice the TabError instead of the usual SyntaxError. Python points out the problem line and gives you a helpful error message. It tells you clearly that there’s a mixture of tabs and spaces used for indentation in the same file.

The solution to this is to make all lines in the same Python code file use either tabs or spaces, but not both. For the code blocks above, the fix would be to remove the tab and replace it with 4 spaces, which will print 'done' after the for loop has finished.

Defining and Calling Functions

You might run into invalid syntax in Python when you’re defining or calling functions. For example, you’ll see a SyntaxError if you use a semicolon instead of a colon at the end of a function definition:

>>>
>>> def fun();
  File "<stdin>", line 1
    def fun();
             ^
SyntaxError: invalid syntax

The traceback here is very helpful, with the caret pointing right to the problem character. You can clear up this invalid syntax in Python by switching out the semicolon for a colon.

In addition, keyword arguments in both function definitions and function calls need to be in the right order. Keyword arguments always come after positional arguments. Failure to use this ordering will lead to a SyntaxError:

>>>
>>> def fun(a, b):
...     print(a, b)
...
>>> fun(a=1, 2)
  File "<stdin>", line 1
SyntaxError: positional argument follows keyword argument

Here, once again, the error message is very helpful in telling you exactly what is wrong with the line.

Changing Python Versions

Sometimes, code that works perfectly fine in one version of Python breaks in a newer version. This is due to official changes in language syntax. The most well-known example of this is the print statement, which went from a keyword in Python 2 to a built-in function in Python 3:

>>>
>>> # Valid Python 2 syntax that fails in Python 3
>>> print 'hello'
  File "<stdin>", line 1
    print 'hello'
                ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print('hello')?

This is one of the examples where the error message provided with the SyntaxError shines! Not only does it tell you that you’re missing parenthesis in the print call, but it also provides the correct code to help you fix the statement.

Another problem you might encounter is when you’re reading or learning about syntax that’s valid syntax in a newer version of Python, but isn’t valid in the version you’re writing in. An example of this is the f-string syntax, which doesn’t exist in Python versions before 3.6:

>>>
>>> # Any version of python before 3.6 including 2.7
>>> w ='world'
>>> print(f'hello, {w}')
  File "<stdin>", line 1
    print(f'hello, {w}')
                      ^
SyntaxError: invalid syntax

In versions of Python before 3.6, the interpreter doesn’t know anything about the f-string syntax and will just provide a generic "invalid syntax" message. The problem, in this case, is that the code looks perfectly fine, but it was run with an older version of Python. When in doubt, double-check which version of Python you’re running!

Python syntax is continuing to evolve, and there are some cool new features introduced in Python 3.8:

If you want to try out some of these new features, then you need to make sure you’re working in a Python 3.8 environment. Otherwise, you’ll get a SyntaxError.

Python 3.8 also provides the new SyntaxWarning. You’ll see this warning in situations where the syntax is valid but still looks suspicious. An example of this would be if you were missing a comma between two tuples in a list. This would be valid syntax in Python versions before 3.8, but the code would raise a TypeError because a tuple is not callable:

>>>
>>> [(1,2)(2,3)]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'tuple' object is not callable

This TypeError means that you can’t call a tuple like a function, which is what the Python interpreter thinks you’re doing.

In Python 3.8, this code still raises the TypeError, but now you’ll also see a SyntaxWarning that indicates how you can go about fixing the problem:

>>>
>>> [(1,2)(2,3)]
<stdin>:1: SyntaxWarning: 'tuple' object is not callable; perhaps you missed a comma?
Traceback (most recent call last):   
  File "<stdin>", line 1, in <module>    
TypeError: 'tuple' object is not callable

The helpful message accompanying the new SyntaxWarning even provides a hint ("perhaps you missed a comma?") to point you in the right direction!

Conclusion

In this tutorial, you’ve seen what information the SyntaxError traceback gives you. You’ve also seen many common examples of invalid syntax in Python and what the solutions are to those problems. Not only will this speed up your workflow, but it will also make you a more helpful code reviewer!

When you’re writing code, try to use an IDE that understands Python syntax and provides feedback. If you put many of the invalid Python code examples from this tutorial into a good IDE, then they should highlight the problem lines before you even get to execute your code.

Getting a SyntaxError while you’re learning Python can be frustrating, but now you know how to understand traceback messages and what forms of invalid syntax in Python you might come up against. The next time you get a SyntaxError, you’ll be better equipped to fix the problem quickly!

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

November 20, 2019 02:00 PM UTC

PyCon

Registration for PyCon US 2020 is open!


We are excited to announce the opening of PyCon US 2020 registration.  The registration site has been updated, tweaked, and tested all in the effort to provide you a seamless experience.

The new system will allow you to access, view, and add to your current registration. You can book and view hotel reservations and request changes if needed right through your dashboard.

Where do I go to register?

Head over to us.pycon.org and create an account.  Once you are logged in access the registration page via your dashboard.

Registration costs

The early bird pricing is $550 for corporate, $350 for individuals, and $100 for students. Once we sell the first 800 tickets, regular prices will go into effect. Regular pricing will be $700 for corporate, $400 for individuals, and $125 for students.

PyCon will take place April 14-23, 2020 in Pittsburgh, PA. The conference starts with 2 days of Tutorials, Education Summit and Sponsor Workshops. Followed by 3 days of keynotes, talks, lightning talks, hatchery program and much more. Ending with 4 days of sprints.

Over 3,000 people from across the globe in one place to learn from and share experiences with.

Join in hallway conversations, participate in special events, visit with our many sponsors in the expo hall, and enjoy the many talks available with beginner to advanced content offered.

Tutorials

Tutorials will be presented Wednesday April 14, 2020 and Thursday April 15, 2020. We are accepting proposals for tutorials through Friday, November 22, 2019. Find more information and submit a proposal here. Once our tutorial committee has scheduled the selected tutorials, you will be able to add them to your conference registration for an additional fee. Watch for tutorial registration to launch in February 2020.

Education Summit

The Education Summit is held on Thursday April 15, 2020. The Education Summit requires you to be registered due to capacity limits. Please only register if you plan to attend as this is a popular event. If you register and are unable to attend please let us know by emailing pycon-reg@python.org. We want to be sure the room is full and those that are able to attend have the chance.

Evening Dinners

There are two evening dinners that require additional registration and capacity is limited. The Gateway Clipper Dinner Cruise on Friday April 16, 2020 and the Trivia Night Dinner with host Brandon Rhodes being held on Sunday April 19, 2020. If you register for the dinners, please be sure you are able to attend. These events do sell out and we want all those that want to attend to have the opportunity. If you register to attend and your plans change, please let us know by emailing pycon-reg@python.org.

Cancellation Fees

Registration cancellations must be submitted in writing and received by April 19, 2019 in order to receive a refund minus the $50 cancellation fee ($25 for students). No refunds will be granted for cancellations received after April 19, 2019.

In lieu of cancellation you are able to transfer your registration to another person. For details about transferring your registration, visit the registration information page.

Attendees traveling to PyCon internationally are encouraged to review our International Travel Refund Policy. This is especially important for recipients of Financial Aid applicants attending from abroad. PyCon strives to support the Python community in attending, no matter where they are traveling from.

Hotel

PyCon has contracted special rates with nearby hotels. When you complete your registration for PyCon US 2020, you will be able to book a hotel reservation through our official housing bureau. This is the only way to get the conference rates. More information can be found on the Venue and Hotels page.

Note: Beware of Housing Pirates! PyCon or our official housing bureau, VisitPittsburgh, will not be calling delegates to sell rooms. If you are contacted by an agency other than VisitPittsburgh offering to make your hotel reservations, we urge you to not use their services. We cannot protect you against them if you do book a reservation.
Looking for a roommate? Check out PyCon’s Room Sharing page.

Childcare

PyCon is proud to announce that we will be once again offering Childcare during the main conference days, April 16-19, 2020. Space is limited, so be sure to sign-up soon.

Financial Aid

Check out the Financial Aid page to learn more about the support we provide for travel, hotel, registration, and childcare to ensure that everyone has an opportunity to attend PyCon.

More Information

Head to the registration information page for more details!

November 20, 2019 11:32 AM UTC

Robin Wilson

A couple of handy zsh/bash functions for Python programmers

Just a quick post today, to tell you about a couple of simple zsh functions that I find handy as a Python programmer.

First, pyimp – a very simple function that tries to import a module in Python and displays the output. If there is no output then the import succeeded, otherwise you’ll see the error. This saves constantly going into a Python interpreter and trying to import something, making that ‘has it worked or not’ cycle a bit quicker when installing a tricky package.

The function is defined as

function pyimp() { python -c "import $1" }

This just calls Python with the -c flag which tells it to execute the code you’ve given on the command line – which in this case is just an import command.

You can see below that it returns nothing for a module which is importable, but returns the error for anything which fails:

$ pyimp numpy
$ pyimp blah
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'blah'

The second is pycd which changes directory to the folder where a particular module is defined. This can be useful if you want to inspect the code of the module in depth, or if you’ve installed the module in ‘develop mode’ and want to actually edit the code.

It’s defined as:

function pycd () {
    pushd python -c "import os.path, $1; print(os.path.dirname($1.__file__))";
}

It just changes to the directory that modulename.file is located in – again, fairly simple but quite useful.

As you’ve read to here, I’ll drop in a bonus function to display the column names of a CSV file:

function csvcols() { head -n 1 $1 | tr , \\n }

This combines the unix head tool to get the line of a file and the tr tool to convert commas to newlines, to make a handy little command.

November 20, 2019 11:28 AM UTC

Janusworx

#100DaysOfCode, Day 001 – Dates & Times

Watched videos for the tiny projects to do over the next two days.

We begin with a date/time project.
Python has objects (primitives) to deal with dates and times.
They are part of the datetime module, which is part of the Python Standard Library.

I learnt that I could add and subtract and otherwise modify dates, easily if my date data was in this format.

Will play more with this tomorrow.

November 20, 2019 09:48 AM UTC

Python Bytes

#157 Oh hai Pandas, hold my hand?

November 20, 2019 08:00 AM UTC

November 19, 2019

PyCoder’s Weekly

Issue #395 (Nov. 19, 2019)

#395 – NOVEMBER 19, 2019
View in Browser »

The PyCoder’s Weekly Logo

Threading in Python

Learn how to use threading in your Python programs. You’ll see how to create threads, how to coordinate and synchronize them, and how to handle common problems that arise in threading.
REAL PYTHON video

Reduce Pandas Memory Usage by Loading Less Data

How to reduce the memory your DataFrame uses at the time it is initially loaded: by dropping columns, lower-range numerical dtypes, and categoricals.
ITAMAR TURNER-TRAURING

Debug Your Python Applications Fast With Datadog APM and Distributed Tracing

alt

Use detailed flame graphs to identify bottlenecks and latency, and correlate log and trace data for individual requests to get the necessary context to debug critical errors. Plus, the Datadog tracing client auto-instruments popular frameworks and libraries like Flask, Tornado, Django, and more →
DATADOG sponsor

The Return of Python Dictionary “Addition”

An interesting summary of discussions around PEP 584 (“Add + and += operators to the built-in dict class”).
JAKE EDGE

Why Is the Migration to Python 3 Taking So Long?

Related discussion on Hacker News.
VICKI BOYKIS

Django 3.0 RC 1 Now Available

DJANGOPROJECT.COM

PyCon 2020 Tutorial Proposal Deadline Approaching

PYCON.BLOGSPOT.COM

Discussions

Don’t Use property() for Computationally Intensive Tasks

“To a user, attribute-style access is implicitly assumed to be fast. In contrast, a well-named method can communicate workload: raymondh.family_members vs raymondh.scan_entire_social_graph()“
RAYMOND HETTINGER

I Was About to Learn Django. Should I Wait a Bit for 3.0 to Have Some Tutorials Out?

REDDIT

Automated Tasks at Work… Boss Found Out

REDDIT

Python Jobs

Senior Python Engineer (Munich, Germany)

Stylight GmbH

Senior Python/Django Developer (Eindhoven, Netherlands)

Sendcloud

Django Full Stack Web Developer (Austin, TX, USA)

Zeitcode

Full Stack Developer (Toronto, ON, Canada)

Beanfield Metroconnect

More Python Jobs >>>

Articles & Tutorials

Getting Started With Python IDLE

See how to use the development environment included with your Python installation. Python IDLE is a small program that packs a big punch! You’ll learn how to use Python IDLE to interact with Python directly, work with Python files, and improve your development workflow.
REAL PYTHON

When Your Data Doesn’t Fit in Memory: The Basic Techniques

Python techniques for processing “bigger than RAM” data on a single computer, with minimal setup, and as much as possible using the same libraries you’re already using.
ITAMAR TURNER-TRAURING • Shared by ITAMAR TURNER-TRAURING

Python Developers Are in Demand on Vettery

alt

Vettery is an online hiring marketplace that’s changing the way people hire and get hired. Ready for a bold career move? Make a free profile, name your salary, and connect with hiring managers from top employers today →
VETTERY sponsor

Unconventional Secure and Asynchronous RESTful APIs Using SSH

How to build secure asynchronous APIs in Python using Korv and AsyncSSH that listen on TCP sockets over SSH sessions.
TRYEXCEPTPASS.ORG • Shared by Cristian Medina

Pandas GroupBy: Your Guide to Grouping Data in Python

Learn how to work adeptly with the Pandas GroupBy facility while mastering ways to manipulate, transform, and summarize data. You’ll work with real-world datasets and chain GroupBy methods together to get data in an output that suits your purpose.
REAL PYTHON

Jupyter on Your Phone (Or Anywhere, Really)

How to host Jupyter notebooks on your own server so you can access them anywhere, even with your mobile phone.
UMAR KHAN

How to Port an awk Script to Python

MOSHE ZADKA

Easily Build Beautiful Video Experiences Into Your Python App

Mux Video is an API-first platform, powered by data and designed by video experts. Test it out to build video for your Python app that streams beautifully, everywhere.
MUX sponsor

Profiling Django With DTrace and cProfile

SEBASTIAN WIEDENROTH

PyCon Africa 2019 (Recap)

REAL PYTHON

“int” Strips Strings of Whitespace

REUVEN LERNER

Projects & Code

modin: Speed Up Your Pandas Workflows

GITHUB.COM/MODIN-PROJECT

viewflow: Reusable Workflow Library for Django

GITHUB.COM/VIEWFLOW

django-fsm: Friendly Finite State Machines for Django

GITHUB.COM/VIEWFLOW

django-river: Django Workflow Library

GITHUB.COM/JAVRASYA

BrachioGraph: The Cheapest, Simplest Possible Pen-Plotter

BRACHIOGRAPH.ART

adtk: Unsupervised Anomaly Detection in Time Series

GITHUB.COM/ARUNDO

slim4py: Ruby Slim as a Template Engine for Python

GITHUB.COM/MULTIVERSECODER

Makefile.venv: Manage Python Virtual Environments With a Makefile

GITHUB.COM/SIO

jenkinsapi: Python API for Jenkins CI

PYPI.ORG

pythonji: Write Python With Emojis

GITHUB.COM/GAHJELLE

Events

PyData Bristol Meetup

November 21, 2019
MEETUP.COM

Python Northwest

November 21, 2019
PYNW.ORG.UK

PyLadies Dublin

November 21, 2019
PYLADIES.COM

MadPUG

November 21, 2019
MEETUP.COM

PyDelhi User Group Meetup

November 23, 2019
MEETUP.COM

Happy Pythoning!
This was PyCoder’s Weekly Issue #395.
View in Browser »

alt

[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]

November 19, 2019 07:30 PM UTC

Stack Abuse

Using cURL in Python with PycURL

Introduction

In this tutorial, we are going to learn how to use PycURL, which is an interface to the cURL library in Python. cURL is a tool used for transferring data to and from a server and for making various types of data requests. PycURL is great for testing REST APIs, downloading files, and so on. Some developers prefer using Postman for testing APIs but PycURL is another suitable option to do so as it supports multiple protocols like FILE, FTPS, HTTPS, IMAP, POP3, SMTP, SCP, SMB, etc. Moreover, PycURL comes in handy when a lot of concurrent, fast, and reliable connections are required.

As mentioned above, PycURL is an interface to the libcURL library in Python; therefore PycURL inherits all the capabilities of libcURL. PycURL is extremely fast (it is known to be much faster than Requests, which is a Python library for HTTP requests), has multiprotocol support, and also contains sockets for supporting network operations.

Pre-requisites

Before you go ahead with this tutorial, please note that there are a few prerequisites. You should have a basic understanding of Python's syntax, and/or have at least beginner-level programming experience in some other language. Furthermore, you should have a good understanding of common networking concepts like protocols and their types, and the client-server mode of communication. Familiarity with these concepts is essential to understand the PycURL library.

Installation

The installation process for PycURL is fairly simple and straightforward for all operating systems. You just need to have libcURL installed on your system in order to use PycURL.

Mac/Linux OS

For Mac OS and Linux, PycURL installation is the simplest as it has no dependencies, and libcURL is installed by default. Simply run the following command in your terminal and the installation will be completed:

Installation via pip
$ pip install pycurl
Installation via easy_install
$ easy_install pycurl

Windows OS

For Windows, however, there are a few dependencies that need to be installed before PyCURL can be used in your programs. If you are using an official distribution of Python (i.e. you've downloaded a Python version from the official website https://www.python.org) as well as pip, you simply need to run the following command in your command line and the installation will be done:

$ pip install pycurl

If you are not using pip, EXE and MSI installers are available at PycURL Windows. You can download and install them directly from there, like any other application.

Basic Code Examples

In this section, we are going to cover some PycURL coding examples demonstrating the different functionalities of the interface.

As mentioned in the introduction section, PycURL supports many protocols and has a lot of sophisticated features. However, in our examples, we will be working with the HTTP protocol to test REST APIs using HTTP's most commonly used methods: GET, POST, PUT and DELETE, along with a few other examples. We will write the syntax for declaring them in Python 3, as well as explain what they do.

So lets start!

Example 1: Sending an HTTP GET Request

A simple network operation of PycURL is to retrieve information from a given server using its URL. This is called a GET request as it is used to get a network resource.

A simple GET request can be performed using PycURL by importing the BytesIO module and creating its object. A CURL object is created to transfer data and files over URLs.

The desired URL is set using the setopt() function, which is used as setopt(option, value). The option parameter specifies which option to set, e.g. URL, WRITEDATA, etc., and the value parameter specifies the value given to that particular option.

The data retrieved from the set URL is then written in the form of bytes to the BytesIO object. The bytes are then read from the BytesIO object using the getvalue() function and are subsequently decoded to print the HTML to the console.

Here is an example of how to do this:

import pycurl
from io import BytesIO 

b_obj = BytesIO() 
crl = pycurl.Curl() 

# Set URL value
crl.setopt(crl.URL, 'https://wiki.python.org/moin/BeginnersGuide')

# Write bytes that are utf-8 encoded
crl.setopt(crl.WRITEDATA, b_obj)

# Perform a file transfer 
crl.perform() 

# End curl session
crl.close()

# Get the content stored in the BytesIO object (in byte characters) 
get_body = b_obj.getvalue()

# Decode the bytes stored in get_body to HTML and print the result 
print('Output of GET request:\n%s' % get_body.decode('utf8'))

Output:

Output of GET request:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<meta http-equiv="X-UA-Compatible" content="IE=Edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta http-equiv = "Content-Type" content = "text/html; charset = utf-8">
<meta name="robots" content="index,nofollow">

<title>BeginnersGuide - Python Wiki</title>
<script type="text/javascript" src = "/wiki/common/js/common.js" ></script>

<script type = "text/javascript" >
<!--
var search_hint = "Search";
//-->
</script>
.
.
.

Example 2: Examining GET Response Headers

You can also retrieve the response headers of a website with the help of PycURL. Response headers can be examined for several reasons, for example, to find out what encoding has been sent with the response and whether that is according to the encoding provided by the server.

In our example, we'll be examining the response headers simply to find out various attribute names and their corresponding values.

In order to examine the response headers, we first need to extract them, and we do so using the HEADERFUNCTION option and display them using our self-defined function (display_header() in this case).

We provide the URL of the site whose response headers we wish to examine; HEADERFUNCTION sends the response headers to the display_header() function where they are appropriately formatted. The response headers are decoded according to the specified standard and are split into their corresponding names and values. The whitespaces between the names and values are stripped and they are then converted to lowercase.

The response headers are then written to the BytesIO object, are transferred to the requester and are finally displayed in the proper format.

from io import BytesIO
import pycurl

headers = {}

def display_header(header_line):
    header_line = header_line.decode('iso-8859-1')

    # Ignore all lines without a colon
    if ':' not in header_line:
        return

    # Break the header line into header name and value
    h_name, h_value = header_line.split(':', 1)

    # Remove whitespace that may be present
    h_name = h_name.strip()
    h_value = h_value.strip()
    h_name = h_name.lower() # Convert header names to lowercase
    headers[h_name] = h_value # Header name and value.

def main():
    print('**Using PycURL to get Twitter Headers**')
    b_obj = BytesIO()
    crl = pycurl.Curl()
    crl.setopt(crl.URL, 'https://twitter.com')
    crl.setopt(crl.HEADERFUNCTION, display_header)
    crl.setopt(crl.WRITEDATA, b_obj)
    crl.perform()
    print('Header values:-')
    print(headers)
    print('-' * 20)
    
main()

Output:

**Using PycURL to get Twitter Headers**
Header values:-
{'cache-control': 'no-cache, no-store, must-revalidate, pre-check=0, post-check=0', 'content-length': '303055', 'content-type': 'text/html;charset=utf-8', 'date': 'Wed, 23 Oct 2019 13:54:11 GMT', 'expires': 'Tue, 31 Mar 1981 05:00:00 GMT', 'last-modified': 'Wed, 23 Oct 2019 13:54:11 GMT', 'pragma': 'no-cache', 'server': 'tsa_a', 'set-cookie': 'ct0=ec07cd52736f70d5f481369c1d762d56; Max-Age=21600; Expires=Wed, 23 Oct 2019 19:54:11 GMT; Path=/; Domain=.twitter.com; Secure', 'status': '200 OK', 'strict-transport-security': 'max-age=631138519', 'x-connection-hash': 'ae7a9e8961269f00e5bde67a209e515f', 'x-content-type-options': 'nosniff', 'x-frame-options': 'DENY', 'x-response-time': '26', 'x-transaction': '00fc9f4a008dc512', 'x-twitter-response-tags': 'BouncerCompliant', 'x-ua-compatible': 'IE=edge,chrome=1', 'x-xss-protection': '0'}
--------------------

In cases where we have multiple headers with the same name, only the last header value will be stored. To store all values in multi-valued headers, we can use the following piece of code:

if h_name in headers:
    if isinstance(headers[h_name], list):
        headers[name].append(h_value)
    else:
        headers[h_name] = [headers[h_name], h_value]
else:
    headers[h_name] = h_value

Example 3: Sending Form Data via HTTP POST

A POST request is the one that sends data to a web server by enclosing it in the body of the HTTP request. When you upload a file or submit a form, you are basically sending a POST request to the designated server.

A POST request can be performed using PycURL by firstly setting the URL to send the form data to through the setopt function. The data to be submitted is first stored in the form of a dictionary (in key value pairs) and is then URL-encoded using the urlencode function found in the urllib.parse module.

We use the POSTFIELDS option in sending form data as it automatically sets the HTTP request method to POST, and it handles our pf data as well.

from urllib.parse import urlencode
import pycurl

crl = pycurl.Curl()
crl.setopt(crl.URL, 'https://www.code-learner.com/post/')
data = {'field': 'value'}
pf = urlencode(data)

# Sets request method to POST,
# Content-Type header to application/x-www-form-urlencoded
# and data to send in request body.
crl.setopt(crl.POSTFIELDS, pf)
crl.perform()
crl.close()

Note: If you wish to specify another request method, you can use the CUSTOMREQUEST option to do so. Just write the name of the request method of your choice in the empty inverted commas following crl.CUSTOMREQUEST.

crl.setopt(crl.CUSTOMREQUEST, '')

Example 4: Uploading Files with Multipart POST

There are several ways in which you can replicate how a file is uploaded in a HTML form using PycURL:

  1. If the data to be sent via POST request is in a file on your system, you need to firstly set the URL where you wish to send the data. Then you specify your request method as HTTPPOST and use the fileupload option to upload the contents of the desired file.
import pycurl

crl = pycurl.Curl()
crl.setopt(crl.URL, 'https://www.code-learner.com/post/')

crl.setopt(crl.HTTPPOST, [
    ('fileupload', (
        # Upload the contents of the file
        crl.FORM_FILE, './my-resume.doc',
    )),
])
crl.perform()
crl.close()

Note: If you wish to change the name and/or the content type of the file, you can do so by making slight modifications to the above code:

crl.setopt(crl.HTTPPOST, [
    ('fileupload', (
        # Upload the contents of this file
        crl.FORM_FILE, './my-resume.doc',
        # Specify a file name of your choice
        crl.FORM_FILENAME, 'updated-resume.doc',
        # Specify a different content type of upload
        crl.FORM_CONTENTTYPE, 'application/msword',
    )),
])
  1. For file data that you have in memory, all that varies in the implementation of the POST request is the FORM_BUFFER and FORM_BUFFERPTR in place of FORM_FILE as these fetch the data to be posted, directly from memory.
import pycurl

crl = pycurl.Curl()
crl.setopt(crl.URL, 'https://www.code-learner.com/post/')

crl.setopt(crl.HTTPPOST, [
    ('fileupload', (
        crl.FORM_BUFFER, 'contact-info.txt',
        crl.FORM_BUFFERPTR, 'You can reach me at billy@example.com',
    )),
])

crl.perform()
crl.close()

Example 5: Uploading a File with HTTP PUT

PUT request is similar in nature to POST request, except for the fact that it can be used to upload a file in the body of the request. You use a PUT request when you know the URL of the object you want to create or overwrite. Basically PUT replaces whatever currently exists at the target URL with something else.

If the desired data to be uploaded is located in a physical file, you first need to set the target URL, then you upload the file and open it. It's important for the file to be kept open while the cURL object is using it. Then the data is read from the file using READDATA.

Finally, the file transfer (upload) is performed using the perform function and the cURL session is then ended. Lastly, the file that was initially opened for the CURL object is closed.

import pycurl

crl = pycurl.Curl()
crl.setopt(crl.URL, 'https://www.code-learner.com/post/')

dat_file = open('data.txt')

crl.setopt(crl.UPLOAD, 1)
crl.setopt(crl.READDATA, dat_file)

crl.perform()
crl.close()
dat_file.close()

If the file data is located in a buffer, the PycURL implementation is pretty much the same as that of uploading data located in a physical file, with slight modifications. The BytesIO object encodes the data using the specified standard. This is because READDATA requires an IO-like object and encoded data is essential for Python 3. That encoded data is stored in a buffer and that buffer is then read. The data upload is carried out and upon completing the upload, the cURL session is ended.

import pycurl
crl = pycurl.Curl()
crl.setopt(crl.URL, 'https://www.code-learner.com/post/')

data = '{"person":{"name":"billy","email":"billy@example.com"}}'
buffer = BytesIO(data.encode('utf-8'))

crl.setopt(crl.UPLOAD, 1)
crl.setopt(crl.READDATA, buffer)

crl.perform()
crl.close()

Example 6: Sending an HTTP DELETE Request

Another important and much used HTTP method is DELETE. The DELETE method requests that the server deletes the resource identified by the target URL. It can be implemented using the CUSTOMREQUEST function, as can be seen in the code sample below:

import pycurl

crl = pycurl.Curl()
crl.setopt(crl.URL, "http://api.example.com/user/148951")
crl.setopt(crl.CUSTOMREQUEST, "DELETE")
crl.perform()
crl.close()

Example 7: Writing to a File

PycURL can also be used to save a response to a file. We use the open function to open the file and response is returned as a file object. The open function is of the form: open(file, mode). The file parameter represents the path and name of the file to be opened and mode represents the mode in which you want to open the file. In our example, it is important to have the file opened in binary mode (i.e. wb) in order to avoid the encoding and the decoding the response.

import pycurl

file = open('pycurl.md','wb')

crl = pycurl.Curl()
crl.setopt(crl.URL, 'https://wiki.python.org/moin/BeginnersGuide')
crl.setopt(crl.WRITEDATA, file)
crl.perform()
crl.close()

Conclusion

In this tutorial, we learnt about the PycURL interface in Python. We started off by talking about some of the general functions of PycURL and its relevance with the libcURL library in Python. We then saw the PycURL's installation process for different operating systems.

Lastly, we went through some of PycURL's general examples which demonstrated the various functionalities offered by PycURL, like the HTTP GET, POST, PUT, and DELETE methods. After following this tutorial, you should be able to fetch objects identified by a URL within a Python program with ease.

November 19, 2019 05:39 PM UTC

PyCharm

PyCharm 2019.2.5

A new version for PyCharm 2019.2 is now available!

New in this Version

TypedDict support for Python 3.8

Following our continuous efforts to keep up with Python latests features, this time we added support for PEP-0589. With this, expect PyCharm now to comply against typed dictionaries definitions and let you know through type hints whether or not you’re implementing a dictionary properly against its typed keys.

Screenshot 2019-11-19 at 11.34.16 AM

Further fixes

Getting the New Version

You can update PyCharm by choosing Help | Check for Updates (or PyCharm | Check for Updates on macOS) in the IDE. PyCharm will be able to patch itself to the new version, there should no longer be a need to run the full installer.

If you’re on Ubuntu 16.04 or later, or any other Linux distribution that supports snap, you should not need to upgrade manually, you’ll automatically receive the new version.

November 19, 2019 05:01 PM UTC

Sumana Harihareswara - Cogito, Ergo Sumana

My New Title, Improving pip, Availability For Work, And SSL (No, The Other One)

A few professional announcements.

Seeking developers for paid contract on pip; apply by Nov. 22

One is that I helped the Packaging Working Group of the Python Software Foundation get funding for a long-needed improvement to pip. I led the writing of a few proposals -- grantwriting, to oversimplify -- and, starting possibly as soon as next month, contractors will start work. As Dustin Ingram explains:

Big news: the Python Packaging Working Group has secured >$400K in grants from multiple funders (TBA) to improve one of the most fundamental parts of pip: its dependency resolver. https://pyfound.blogspot.com/2019/11/seeking-developers-for-paid-contract.html

The dependency resolver is the algorithm which takes multiple constrained requirements (e.g. "some_package>=1.0,=2.0") and finds a version of all dependencies (and sub-dependencies) which satisfy all the constraints.
https://pip.pypa.io/en/stable/user_guide/#requirements-files

Right now, pip's resolver mostly works for most use cases... However the algorithm it uses is naïve, and isn't always guaranteed to produce an optimal (or correct) result.

.....

These funds will pay multiple developers to work on completing the design, implementation and rollout of this new dependency resolver for pip, finally closing issue #988.

Not only will this give pip a better resolver, but it will "enable us to untangle pip’s internals from the resolver, enabling pip to share code for dependency resolution with other packaging tooling". https://pradyunsg.me/blog/2019/06/23/oss-update-1/

This is great news for pip and Python packaging in general. Huge shout out to @pradyunsg for his existing work on the resolver issue and guidance here, and to @brainwane for all her tireless work acquiring and directing funding for Python projects.

If you or your organization is interested in participating in this project, we've just posted the RFP, which includes instructions for submitting proposals, evaluation criteria and scope of work.
https://github.com/python/request-for/blob/master/2020-pip/RFP.md

If you're interested, please apply by 22 November.

NYU, Secure Systems Lab, and my new title

Working at the new space on NYU Tandon's campus, left to right: Sumana Harihareswara, a volunteer with the PSF's Packaging Working Group, a contracted project manager for the Python Packaging Index, and a visiting scholar in NYU Tandon Professor Justin Cappos's Secure Systems Lab; Stephanie Whited, communications director for the Tor Project and visiting researcher in the Secure System Lab; and Santiago Torres, a computer science doctoral candidate working in the Secure Systems Lab. Photo by NYU publicity.In further news: I am now a visiting scholar in Professor Justin Cappos's Secure Systems Lab at New York University's Tandon School of Engineering. And I get to use an office with a door, shelves, whiteboards, and so on (per the picture at right). If you contribute to Python packaging/distribution tools and live in/near or sometimes visit New York City, let me know and perhaps we could cowork a bit?

The Secure Systems Lab stewards The Update Framework (TUF) and related projects, and works to improve the security of the software supply chain. The Python Package Index is likely going to implement TUF to add cryptographic signatures to packages on PyPI, and so I've gotten to give TUF's developers some advice to help that work move along. (I won't be the manager on that project but I'll be watching with great interest.) PyPA may also choose to use more of SSL's work in implementing further security improvements to the package distribution toolchain, and I'm learning more to work out whether and how that could happen. Also, Cappos's research on backtracking dependency resolvers has been helpful to the pip resolver work.

Edited 19 Nov 2019 to clarify role.

PSF projects

I'm grateful to get to help connect the Python Software Foundation with more resources and volunteers. Changeset's current and recent projects have mostly been for the PSF. Last month we finished accessibility, security, and internationalization work on PyPI that was funded by the Open Technology Fund, and Changeset's work on communicating about the sunsetting of Python 2.x continues and will go through April 2020.

Availability for one-day engagements in San Francisco in February

But I am interested in taking on new clients for short engagements starting in February 2020. In particular, I will be in the San Francisco Bay Area in mid- to late February. If you're in SF or nearby, I could offer you a one-day engagement doing one of the following:

I'd spend a little time talking with you, then sit in your office and finish the document before leaving that afternoon. (Photo at right provides a sample of how I look while sitting.) Drop me a line for a free initial 30-minute chat and we can talk pricing.

November 19, 2019 03:15 PM UTC

Real Python

Threading in Python

Python threading allows you to have different parts of your program run concurrently and can simplify your design. If you’ve got some experience in Python and want to speed up your program using threads, then this course is for you!

In this article, you’ll learn:

This course assumes you’ve got the Python basics down pat and that you’re using at least version 3.6 to run the examples. If you need a refresher, you can start with the Python Learning Paths and get up to speed.

If you’re not sure if you want to use Python threading, asyncio, or multiprocessing, then you can check out Speed Up Your Python Program With Concurrency.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

November 19, 2019 02:00 PM UTC

Catalin George Festila

Python 3.7.5 : Display a file in the hexadecimal and binary output.

This is an example with a few python3 modules that display a file in the hexadecimal and binary output: import sys import os.path import argparse parser = argparse.ArgumentParser() parser.add_argument("FILE", help="the file that you wish to dump to hexadecimal", type=str) parser.add_argument("-b", "--binary", help="display bytes in binary format instead of hexadecimal") args = parser.parse_args(

November 19, 2019 10:43 AM UTC

Codementor

Lesson 101: Everything You Need To Learn About Programming Guidance

Myhomeworkhelponline Experts provides the most trusted and reliable online Programming assignment help . Programming is one of the most widely taught subjects across the universities.

November 19, 2019 09:45 AM UTC

Python Software Foundation

Python Software Foundation Fellow Members for Q3 2019

We are happy to announce our newest PSF Fellow Members for Q3!

Q3 2019


Abigail Mesrenyame Dogbe
Twitter

Anton Caceres
Twitter, Website

Bruno Oliveira
GitHub

Gautier Hayoun
Twitter, Personal company website

Mahmoud Hashemi
GitHub, Blog, Twitter

Manabu Terada
Twitter, GitHub

Mannie Young
Twitter, GitHub

Michael Young
Twitter

Noah Alorwu
Twitter, GitHub

Paul Kehrer
Twitter, GitHub

Tom Viner
Twitter, Blog

Valentin Dombrovsky
Personal company website, LinkedIn


Congratulations! Thank you for your continued contributions. We have added you to our Fellow roster online.

The above members have contributed to the Python ecosystem by teaching Python, maintaining popular libraries/tools such as cryptography and pytest, helping document on packaging.python.org, organizing Python events, starting Python communities in their home countries, and overall being great mentors in our community. Each of them continues to help make Python more accessible around the world. To learn more about the new Fellow members, check out their links above.

Let's continue to recognize Pythonistas all over the world for their impact on our community. Here's the criteria our Work Group uses to review nominations:

  • For those who have served the Python community by creating and/or maintaining various engineering/design contributions, the following statement should be true:
    • Nominated Person has served the Python community by making available code, tests, documentation, or design, either in a Python implementation or in a Python ecosystem project, that 1) shows technical excellence, 2) is an example of software engineering principles and best practices, and 3) has achieved widespread usage or acclaim.
  • For those who have served the Python community by coordinating, organizing, teaching, writing, and evangelizing, the following statement should be true:
    • Nominated Person has served the Python community through extraordinary efforts in organizing Python events, publicly promoting Python, and teaching and coordinating others. Nominated Person's efforts have shown leadership and resulted in long-lasting and substantial gains in the number and quality of Python users, and have been widely recognized as being above and beyond normal volunteering.
  • If someone is not accepted to be a fellow in the quarter they were nominated for, they will remain an active nominee for 1 year for future consideration.
  • It is suggested/recommended that the nominee have wide Python community involvement. Examples would be (not a complete list - just examples):
    • Someone who has received a Community Service Award or Distinguished Service Award
    • A developer that writes (more than one) documentation/books/tutorials for wider audience
    • Someone that helps translate (more than one) documentation/books/tutorials for better inclusivity
    • An instructor that teaches Python related tutorials in various regions
    • Someone that helps organize local meet ups and also helps organize a regional conference
  • Nominees should be aware of the Python community’s Code of Conduct and should have a record of fostering the community.
  • Sitting members of the PSF Board of Directors can be nominated if they meet the above criteria.
If you would like to nominate someone to be a PSF Fellow, please send a description of their Python accomplishments and their email address to psf-fellow at python.org. We are accepting nominations for quarter 4 through November 20, 2019. More information is available at: https://www.python.org/psf/fellows/.

November 19, 2019 08:42 AM UTC