skip to navigation
skip to content

Planet Python

Last update: April 25, 2017 10:48 AM

April 25, 2017

Django Weblog

DjangoCon Europe 2017 in retrospect

DjangoCon Europe 2017 upheld all the traditions established by previous editions: a volunteer-run event, speakers from all sections of the community and a commitment to stage a memorable, enjoyable conference for all attendees.

Held in a stunning Art Deco cinema in the centre of the city, this year's edition was host to over 350 Djangonauts.

The team of always-smiling and willing volunteers, led by Emanuela Dal Mas and Iacopo Spalletti under the auspices of the Fuzzy Brains association, created a stellar success on behalf of all the community.

Of note in this year's conference was an emphasis on inclusion, as expressed in the conference's manifesto. The organisers' efforts to expand the notion of inclusion was visible in the number of attendees from Africa and south Asia, nearly all of whom were also given a platform at the event. This was made possible not only by the financial assistance programme but also through the considerable logistical help the organisers were able to offer.

The conference's opening keynote talk by Anna Makarudze and Humphrey Butau on the growing Python community in Zimbabwe, and an all-woman panel discussing their journeys in technology, were just two examples of a commitment to making more space for voices and stories that are less often heard.

DjangoCon Europe continues to thrive and sparkle in the hands of the people who care about it most, and who step forward each year as volunteers who commit hundreds of hours of their time to make the best possible success of it. Once again, this care has shone through.

On behalf of the whole Django community, the Django Software Foundation would like to thank the entire organising team and all the other volunteers of this year's DjangoCon Europe, for putting on a superb and memorable production.

The next DjangoCons in Europe

The DSF Board is considering bids for DjangoCon Europe 2018-2020. If you're interested in hosting the event in one of these years, we'd like to hear from you as soon as possible.

April 25, 2017 09:05 AM


How to Write a Simple Slack Bot to Monitor Your Brand on Twitter

In this article I show you how to monitor Twitter and post alerts to a Slack channel. We built a nice tool to monitor whenever our domain gets mentioned on Twitter. The slacker and twython modules made this pretty easy. We also use configparser and logging.

April 25, 2017 09:00 AM

Daniel Bader

Sets and Multisets in Python

Sets and Multisets in Python

How to implement mutable and immutable set and multiset (bag) data structures in Python using built-in data types and classes from the standard library.

A set is an unordered collection of objects that does not allow duplicate elements. Typically sets are used to quickly test a value for membership in the set, to insert or delete new values from a set, and to compute the union or intersection of two sets.

In a “proper” set implementation, membership tests are expected to run in O(1) time. Union, intersection, difference, and subset operations should take O(n) time on average. The set implementations included in Python’s standard library follow these performance characteristics.

Just like dictionaries, sets get special treatment in Python and have some syntactic sugar that makes it easier to create sets. For example, the curly-braces set expression syntax and set comprehensions allow you to conveniently define new set instances:

vowels = {'a', 'e', 'i', 'o', 'u'}
squares = {x * x for x in range(10)}

Careful: To create an empty set you’ll need to call the set() constructor, as using empty curly-braces ({}) is ambiguous and will create a dictionary instead.

Python and its standard library provide the following set implementations:

✅ The set Built-in

The built-in set implementation in Python. The set type in Python is mutable and allows the dynamic insertion and deletion of elements. Python’s sets are backed by the dict data type and share the same performance characteristics. Any hashable object can be stored in a set.

>>> vowels = {'a', 'e', 'i', 'o', 'u'}
>>> 'e' in vowels

>>> letters = set('alice')
>>> letters.intersection(vowels)
{'a', 'e', 'i'}

>>> vowels.add('x')
>>> vowels
{'i', 'a', 'u', 'o', 'x', 'e'}

✅ The frozenset Built-in

An immutable version of set that cannot be changed after it was constructed. Frozensets are static and only allow query operations on their elements (no inserts or deletions.) Because frozensets are static and hashable they can be used as dictionary keys or as elements of another set.

>>> vowels = frozenset({'a', 'e', 'i', 'o', 'u'})
>>> vowels.add('p')
AttributeError: "'frozenset' object has no attribute 'add'"

✅ The collections.Counter Class

The collections.Counter class in the Python standard library implements a multiset (or bag) type that allows elements in the set to have more than one occurrence. This is useful if you need to keep track not only if an element is part of a set but also how many times it is included in the set.

>>> from collections import Counter
>>> inventory = Counter()

>>> loot = {'sword': 1, 'bread': 3}
>>> inventory.update(loot)
>>> inventory
Counter({'bread': 3, 'sword': 1})

>>> more_loot = {'sword': 1, 'apple': 1}
>>> inventory.update(more_loot)
>>> inventory
Counter({'bread': 3, 'sword': 2, 'apple': 1})

Read the full “Fundamental Data Structures in Python” article series here. This article is missing something or you found an error? Help a brother out and leave a comment below.

April 25, 2017 12:00 AM

April 24, 2017

Andre Roberge

Easily modifiable Python

You want to try out some new syntactic construction in Python but do NOT want to have to go through the trouble of 1. modifying Python's grammar file 2. modifying Python's lexer 3. modifying Python's parser 4. modifying Python's compiler 5. re-compile all the sources  so as to create a new Python interpreter? (These steps are described in these two blog  posts.) It can be done much more easily

April 24, 2017 10:47 PM

François Dion

Meet Eliza #AI

I will be presenting and directing a discussion on artificial intelligence, from various angles including the arts, Tuesday April 25th at Wake Forest in Winston Salem, NC.

Details here:

Francois Dion

April 24, 2017 09:48 PM


Moore Foundation gives grant to support NumFOCUS Diversity & Inclusion in Scientific Computing initiatives

As part of our mission to support and promote better science through support of the open source scientific software community, NumFOCUS champions technical progress through diversity. NumFOCUS recognizes that the open source data science community is currently highly homogenous. We believe that diverse contributors and community members produce better science and better projects. NumFOCUS strives […]

April 24, 2017 07:23 PM

Weekly Python Chat

Python Oddities

I tweeted one Python oddity each week in 2016. Let's chat about them.

April 24, 2017 05:15 PM

Ned Batchelder

Shell = Maybe

A common help Python question: how do I get Python to run this complicated command line program? Often, the answer involves details of how shells work. I tried my hand at explaining it what a shell does, why you want to avoid them, how to avoid them from Python, and why you might want to use one: Shell = Maybe.

April 24, 2017 03:38 PM


Anyone Can Do Astronomy with Python and Open Data

Ole Moeller-Nilsson, CTO at Pivigo, was kind enough to share his insights on how a beginner can easily get started exploring astronomy using Python. This blog post grew out of a presentation he gave at PyData London meetup on March 7th. Python is a great language for science, and specifically for astronomy. The various packages […]

April 24, 2017 03:00 PM

Doug Hellmann

zlib — GNU zlib Compression — PyMOTW 3

The zlib module provides a lower-level interface to many of the functions in the zlib compression library from the GNU project. Read more… This post is part of the Python Module of the Week series for Python 3. See for more articles from the series.

April 24, 2017 01:00 PM

Mike Driscoll

PyDev of the Week: Honza Král

This week we welcome Honza Král (@HonzaKral) as our PyDev of the Week! Honza is one of the core developers of the Django web framework. He is also the maintainer of the official Python client to Elasticsearch. You can see some of the projects he is interested in or working on over at Github. Let’s spend some time getting to know Honza better!

Can you tell us a little about yourself (hobbies, education, etc):

I grew up in the Old Town of Prague, Czech Republic where I also went to school, including University where I studied computer science (which I didn’t finish). During my studies I discovered Python and immediately fell in love, first with the language and later, after going to my first Pycon in 2008, with the community.

I became a part of the Django community which was (and still is to this day) very welcoming. I became a part of it to learn and, hopefully, contribute something back. For my part it worked amazingly well – I got all my recent jobs through the community and even met my fiancee at a EuroPython conference!

Nowadays I work for Elastic, the company behind Elasticsearch where I do consulting – traveling around the world helping people be successful with open source – and also maintain the official Python client.

Why did you start using Python / Django?

I started in university for a project. I had my eye on Python for some time then and a perfect opportunity presented itself to use it for one of our team projects – we decided it would be a web-based application and Django was the shiny new thing that caught my attention. I introduced it to the rest of the team and have been using it ever since.

What other programming languages do you know and which is your favorite?

My other favorite, besides Python of course :), would be shell – I still enjoy writing both simple (or a bit more complex) one-liners as well as longer scripts in shell. I like the variety of it where you have so many possibilities on how to accomplish things with a seemingly limited language. I like how it’s very unlike other languages I am used to working in as it forces you to think more in terms of pipelines and “just” passing off problems to another app/language. It definitely helped my Python skills as well I think – thinking of what I need to keep in a list/variable vs where a simple generator would suffice (as would a pipeline in shell) etc.

Right now I am looking at rust and hoping that a suitable project will come along for me to give it a serious try.

What projects are you working on now?

Currently I am working on elasticsearch-dsl which is an ORM-like abstraction on top of elasticsearch. It is an ambitious project since I wanted to expose all the power of elasticsearch’s query DSL while also trying to make it more accessible to non-experts and generally friendlier. While I think that in some areas I might have achieved this, I still have a long way to go in others; and for some I don’t even see the clear path of what needs to happen which makes it interesting – gives me something to ponder 🙂

Which Python libraries are your favorite (core or 3rd party)?

As a 3rd party library I really like click because of how well it fulfills its purpose. From the standard library I am a huge fan of itertools, collections and other “helper” libraries that don’t just offer some functionality but also, and often more importantly, show how a real pythonic code should look like and makes your code more pythonic when you use them properly – the same quality I always liked about Django where it guides you in the right direction and makes the right-way to things so obvious that it trains you to recognize it and use it also outside of your Django projects.

Another library I find really interesting is py.test – I love the feature set and how easy and flexible it makes testing but I am also a bit wary of the “magic” that happens there. To me that is a perfect example of the pragmatism in the python community – I think we can all accept a little magic as long as it is clearly defined and serves its purpose.

Is there anything else you’d like to say?

I want to say a huge Thank you to the global Python community out there! I have been privileged enough to see a huge part of it and it still manages to catch me off guard regularly with the amount of awesomeness and caring that make for a great environment. I interact with developers all over the world as part of my job and really appreciate the amount effort and thought the people in the Python community are putting into making sure it is a thriving and welcoming one.

Thanks so much for doing the interview!

April 24, 2017 12:30 PM

Import Python

Import Python Weekly Issue 122 - Django ORM vs SQLAlchemy, Type annotation, websockets and more

Worthy Read

Recently I started using SQLAlchemy and am very impressed with it. I have used Django ORM a lot in the past. This post compares achieving same result using Django and with SQLAlchemy. Let’s see which looks more intuitive.

Users finding bugs? Searching logs for errors? Find + fix broken code fast! You can find the Python integration here -

In this part, we will implement graph data structure using classes and interfaces, and discuss when it’s worth overruling type hints.
type annotation

I recently I had the opportunity to speed up some Websocket code that was a major bottleneck. The final solution was 60X (!) faster than the first pass, and an interesting exercise in optimizing inner loops.

This trouble to visualise the relationship between entities in a Relational Database is a great reason to introduce the concept of graph. Graph is a data structure formed by a set of vertices V and a set of edges E. It can be represented graphically (where the vertices are shown as circles and edges are shown as lines) or mathematically in the form G = (V, E).
graph databases

This week I raised my first pull-request to the CPython core project, which was declined :-( but as to not completely waste my time I’m writing my findings on how CPython works and show you how easy it is to modify the Python syntax.

Nchan makes writing realtime web based pub/sub applications a breeze. In this article, we will build a simple systems monitoring dashboard which displays process information in realtime similar to what you would see when you run the unix top or htop commands.

Suppose you generate several sets on the fly, and you want to find the elements that are in all the sets. That's easy, it's the intersection of sets.

I was working recently on writing Python code using TDD. So every time I change code I wanted to run command that will test my code if it pass unit tests. In order to do that I needed some service/app that will monitor for file changes, and if it changes execute my batch file that runs unit tests.

We help companies like Airbnb, Pfizer, and Artsy find great developers. Let us find your next great hire. Get started today.



In this session you will learn how SQL Server 2017 takes in-database analytics to the next level with support for both Python and R; delivering unparalleled scalability and speed with new deep learning algorithms built in.





algorithms: - 2112 Stars, 174 Fork
Minimal examples of data structures and algorithms in Python

jokekappa - 74 Stars, 7 Fork
A library for delivering one-line programming jokes

cashier - 53 Stars, 2 Fork
Persistent caching for python functions

crickbuzz_cricket_score - 13 Stars, 2 Fork
A Terminal based program to follow live cricket score by scraping

MigrateGitlabToGogs - 3 Stars, 0 Fork
Migrate repositories from Gitlab to Gogs or Gitea

venvdetect - 3 Stars, 1 Fork
Detect available Python virtual environments in your current directory

April 24, 2017 12:11 PM

Django Weekly

Django Weekly 35 - Signals, Django Rest Framework, Djangocon 2017

Worthy Read

In many cases when there is a modification in a model’s instance we need execute some action. Django provides us an elegant way to handle with these situations. The signals are utilities that allow us to associate events with actions. We can develop a function that will run when a signal calls it.

Users finding bugs? Searching logs for errors? Find + fix broken code fast!. You can find the Python integration here -

Sort of a learning path made up of curated articles.

One of our tables grew to over 25 million rows. Which, while not a large number, makes the most frequent queries slow. A direct effect it had was on our Admin panel. The change_list form for the model started taking ages to load.

I love DjangoCon. I've been going to it almost every year since I arrived in Europe back in 2010. Sure, a considerable portion of my career has been based on Django, but it's more than that: the community is stuffed full of amazing people who genuinely want us all to succeed and that just makes the conference all the more exciting.

We help companies like Airbnb, Pfizer, and Artsy find great developers. Let us find your next great hire. Get started today.


DjangoCon US is a six-day international conference for the community by the community, held each year in North America, about the Django web framework. Early bird tickets avialable.


django-webpacker - 18 Stars, 1 Fork
A django compressor tool that bundles css, js files to a single css, js file with webpack and updates your html files with respective css, js file path.

django-starter-kit - 12 Stars, 0 Fork
Opinionated template for Django projects on Python 3 and PostgreSQL.

mnm - 9 Stars, 0 Fork
Mastodon Network Monitoring: track and display browsable stats about Mastodon network (instances, toots, users...).

django-multicore - 4 Stars, 0 Fork
A framework that makes it easy to parallelize Django code

django-freeradio - 4 Stars, 4 Fork
A Django project for managing a radio station for a community radio station. The code was originally based on the Django website for Brum Radio, and was open sourced as part of a sprint session at DjangoCon Europe 2017.

django-confirmation-mixin - 3 Stars, 0 Fork
Django ConfirmationMixin for CreateView and UpdateView

April 24, 2017 12:09 PM

PyDDF Python Spring Sprint 2017

The following text is in German, since we're announcing a Python sprint in Düsseldorf, Germany.


PyDDF Python Frühlings-Sprint 2017 in

Samstag, 06.05.2017, 10:00-18:00 Uhr
Sonntag, 07.05.2017, 10:00-18:00 Uhr

trivago GmbH,  Karl-Arnold-Platz 1A,  40474 Düsseldorf


Das Python Meeting Düsseldorf (PyDDF) veranstaltet mit freundlicher Unterstützung der trivago GmbH ein Python Sprint Wochenende im Mai.

Der Sprint findet am Wochenende 6./7.5.2017 in der trivago Niederlassung am Karl-Arnold-Platz 1A statt (nicht am Bennigsen-Platz 1). Folgende Themengebiete haben wir als Anregung angedacht:
Openpyxl ist eine Python Bibliothek, mit der man Excel 2010+ Dateien lesen und schreiben kann.

Charlie Clark ist Co-Maintainer des Pakets.

Telegram ist eine Chat-Anwendung, die von vielen Nutzern verwendet wird. Telegram unterstützt das Registrieren von sogenannten Bots - kleinen Programmen, die man vom Chat aus ansteuern kann, um z.B. Informationen zu bekommen.

Im Sprint wollen wir versuchen, einen Telegram-Bot in Python zu schreiben.

Natürlich kann jeder Teilnehmer weitere Themen vorschlagen, z.B.

Anmeldung und weitere Infos

Alles weitere und die Anmeldung findet Ihr auf der Sprint Seite:

Teilnehmer sollten sich zudem auf der PyDDF Liste anmelden, da wir uns dort koordinieren:

Über das Python Meeting Düsseldorf

Das Python Meeting Düsseldorf ist eine regelmäßige Veranstaltung in Düsseldorf, die sich an Python Begeisterte aus der Region wendet.

Einen guten Überblick über die Vorträge bietet unser PyDDF YouTube-Kanal, auf dem wir Videos der Vorträge nach den Meetings veröffentlichen.

Veranstaltet wird das Meeting von der GmbH, Langenfeld, in Zusammenarbeit mit Clark Consulting & Research, Düsseldorf.

Viel Spaß !

Marc-Andre Lemburg,

April 24, 2017 11:00 AM


EuroPython 2017: How you can sponsor us

Check out our brand new brochure with all details about our sponsor packages, options and lounge events, available on our website.

Sponsoring EuroPython guarantees you highly targeted visibility and the opportunity to present yourself and your company in a professional and innovative environment. 


With more than 1200 participants, 8 days conference program, international keynotes, more than 200 talks, training sessions for beginners, advanced and experts, EuroPython creates an excellent environment to look for new job-opportunities, recruit people, connect, share and learn. 

New Attractive Location for 2017

This year we move to the Palacongressi in Rimini, the largest convention centre in Italy, unique in its design and technology, with 11,000 m² exhibit space, in addition to 39 conference rooms. Check out EuroPython demographics, refresh your mind with our beautiful location and consult the different opportunities we have created for you

Available Sponsor Options

We offer 7 different sponsor packages, with different levels of engagement.

Would you rather have more flexibility? You can separately book special events and choose from a list of very prominent advertisement options: have a table in the training session or get your logo to be by the ice-cream stall; sponsor our laser-tag and party events. 

Do you have further ideas? We can help you develop a tailored sponsorship package that fits your specific targets. If you would like to engage and sponsor EuroPython 2017, please contact us by e-mail.

Don’t miss the chance to become part of the largest Python conference in Europe!


EuroPython 2017 Team

April 24, 2017 08:54 AM

April 23, 2017

PythonClub - A Brazilian collaborative blog about Python

Configurando OpenShift com Python 3.5 + Flask + Gunicorn

Configurando OpenShift com Python 3.5


O OpenShift é uma plataforma de PasS que possibilita aos desenvolvedores "subir" aplicações na nuvem de uma maneira simples e rápida. Ele funciona a partir de gears(engrenagens) que representam máquinas que irão rodar as aplicações. Dentro de cada gear é possível instalar serviços, os são chamados de "cartridges".

Existem 3 planos:

Um problema que me deparei ao utilizar o Openshift é que ele não possui um cartridge com Python3.5. Porém existe uma forma um pouco mais complicada de resolver esse problema.

Após fazer seu cadastro no OpenShift e instalar o client tools que contém as ferramentas necessárias para configurar nossa aplicação.

Após tudo isso vamos colocar a mão na massa, abra seu terminal e vamos lá.

Criando a aplicação

rhc app create <app-name> diy-0.1

Substituindo "<app-name>" pelo nome de sua aplicação. O arquivo manifest.yml criado por Changaco(github) e "forkeado" por Grief(github) contém as configurações de um cartridge customizado que contém o python 3.5.

Para os curiosos o conteúdo do arquivo

Name: python
Cartridge-Short-Name: PYTHON
Display-Name: Only Python
Description: 'An embedded cartridge that provides only python, nothing else.'
Version: '3.5.0'
Versions: ['3.5.0', '2.7.11']
License: The Python License
Cartridge-Version: 0.0.2
Cartridge-Vendor: praisebetoscience
- service
- python
- embedded
  Developer Center:
- python
    Type: ENV:*
    Required: false
    Type: STRING:urlpath
    Required: false
  Min: 1
  Max: -1
    Display-Name: Python 2.7
    License: The Python License, version 2.7
    - python-2.7
    - python
    - python(version) = 2.7
    Display-Name: Python 3.5
    License: The Python License, version 3.5
    - python-3.5
    - python
    - python(version) = 3.5

Após isso sua aplicação já estárá executando, caso deseje acessar o endereço da mesma deverá ser http://<app-name> Você verá que a página do seu projeto não é nada mais do que o diy (Dot It Yourself), que é uma aplicação Ruby de exemplo que você pode alterar, e é o que vamos fazer.

Se você acessar o diretório do seu projeto verá que existe um diretório ".openshift", dentro desse diretório existe um outro diretório chamado "action_hooks", e dentro desse diretório existem dois arquivos "start" e "stop".

Os dois arquivos são respectivamente os comandos para "subir" e "pausar" sua aplicação.


Vamos criar um projeto de exemplo, bem simples, que apenas nos retorne a versão do python utilizada. Primeiramente vamos criar nosso requirements.txt, com gunicorn e o flask.



Depois disso vamos criar o arquivo que conterá nossa aplicação.


import sys
from flask import Flask

app = Flask(__name__)

def index():
    return sys.version

Após isso basta fazer o commit de suas alterações.

shell git add . git commit -am 'Minhas alterações'

Após isso você verá que sua aplicação não está rodando, pois ainda não alteramos os arquivos "start" e "stop".

Configurando o Gunicorn no Start e Stop

O projeto diy do openshift nos deixa uma variável de ambiente $OPENSHIFT_DIY_IP com o IP da máquina, dessa forma podemos passar a variável e porta ao gunicorn.


nohup $HOME/python/usr/bin/pip3 install -r $OPENSHIFT_REPO_DIR/requirements.txt
nohup $HOME/python/usr/bin/gunicorn app:app --bind=$OPENSHIFT_DIY_IP:8080 |& /usr/bin/logshifter -tag diy &

A primeira linha é o Shebang, o que significa que esse arquivo será executado pelo bash. Na segunda linha vemos nohup, que executa os comandos em uma sessão separada, vemos logo apóes vemos o uma chamada ao pip para instalar nossas dependências. Na terceira linha vemos o nohup, e depois o gunicorn inicializa nossa aplicação flask.

Isso só funciona pois o cartridge customizado instala o python3.5 dentro da pasta home do servidor do openshift.



# The logic to stop your application should be put in this script.
if [ -z "$(ps -ef | grep gunicorn | grep -v grep)" ]
    client_result "Application is already stopped"
    kill `ps -ef | grep gunicorn | grep -v grep | awk '{ print $2 }'` > /dev/null 2>&1

Podemos ver que o comando ps procura algum processo do gunicorn, caso ele exista o kill será chamado.

Após isso, é só fazer o commit das alterações e você verá sua aplicação rodando.

Espero que o post ajude a quem quer subir alguma aplicação com python3.5 e só tem o heroku como opção.


April 23, 2017 11:37 PM


The luxury of a creative community

For this blog post I want to focus on the series of very nice pull requests coming in from a growing cadre of contributors over the last few months.

Contributed goodness

People have put in a lot of good work to boost Evennia, both by improving existing things and by adding new features. Thanks a lot everyone (below is just a small selection)!
About that Evennia 0.7 ...

So while PRs are popping up left and right in master I've been working in the devel branch towards what will be the Evennia 0.7 release. The branch is not ready for public consumption and testing yet But tentatively it's about halfway there as I am slowly progressing through the tickets. Most of the upcoming features were covered in the previous blog post so I'll leave it at that.

I just want to end by saying that it's a very luxurious (and awesome) feeling for me to see master-branch Evennia expand with lots of new stuff "without me" so to speak. The power of Open Source indeed!

Image from, released as public domain.

April 23, 2017 11:10 PM


Code Challenge 16 - Query Your Favorite API

Hi Pythonistas, a new week, a new 'bite' of Python coding! This week we'll let you play with Web APIs. They are fun to fiddle with and great for learning and building cool things. Enjoy!

April 23, 2017 10:01 PM

Twitter digest 2017 week 16

Every weekend we share a curated list of 15 cool things (mostly Python) that we found / tweeted throughout the week.

April 23, 2017 06:04 PM

Intuitive Explanation of OFDM

An intuitive explanation on how OFDM works

This article is about the fundamental principle of OFDM. Without going into the mathematical details, I'm going to explain, why the commonly used OFDM waveform is designed as it is. In a previous article, we have already shown a source code example of OFDM.

Let us start with the assumption that we face a static multipath channel (static means, the channel coefficients do not change over all times, i.e. it is time-invariant). Such a channel is a linear, time-invariant (LTI) system, and hence it is described solely by its impulse response h(t). The output signal y(t) of this system is the convolution of the input signal x(t) with the impulse response h(t):

Eigenfunctions of the time-invariant channel

The most important property of LTI systems is, that complex exponentials of any frequency are eigenfunctions of the system. Literally, if the input to an LTI system is a complex exponential, its output will be a complex exponential of the same frequency, but with a different amplitude. Let's verify ...

April 23, 2017 12:00 PM

Weekly Python StackOverflow Report

(lxx) stackoverflow python report

These are the ten most rated questions at Stack Overflow last week.
Between brackets: [question score / answers count]
Build date: 2017-04-23 07:50:35 GMT

  1. What is python .. ("dot dot") notation syntax? - [112/4]
  2. Numpy dot too clever about symmetric multiplications - [17/2]
  3. No supported color terminal library Python/Jupyter - [13/4]
  4. Python is reading past the end of the file. Is this a security risk? - [9/0]
  5. Bar Chart with multiple labels - [8/2]
  6. Unexpected 32-bit integer overflow in pandas/numpy int64 (python 3.6) - [8/1]
  7. Difference between Linear Regression Coefficients between Python and R - [7/2]
  8. Pandas: Fill NaNs with next non-NaN / # consecutive NaNs - [7/1]
  9. Python default params confusion - [6/3]
  10. format value that could be number and/or string in python 3 - [6/3]

April 23, 2017 07:50 AM

April 22, 2017

Will McGugan

PyFilesystem 2.0.3 Released

I've just released version 2.0.3 of PyFilesystem.

PyFilesystem is an abstraction layer for filesystems (or anything that resembles a filesystem). See this post for more details.

New in this version is a TarFS filesystem contributed by Martin Larralde, which compliments ZipFS nicely.

Contributed by gpcimino, the copy module was extended new functionality to selectively copy only new files from one filesystem to another. Very useful for backups.

This is in addition to bugfixes, and improved documentation.

April 22, 2017 05:56 PM

Roberto Alsina

How I Learned to Stop Worrying and Love JSON Schema


This post operates on a few shared assumptions. So, we need to explicitly state them, or otherwise you will read things that are more or less rational but they will appear to be garbage.

  • APIs are good
  • Many APIs are web APIs
  • Many web APIs consume and produce JSON
  • JSON is good
  • JSON is better if you know what will be in it

So, JSON Schema is a way to increase the number of times in your life that JSON is better in that way, therefore making you happier.

So, let's do a quick intro on JSON Schema. You can always read a much longer and surely better one from which I stole most examples at Understanding JSON Schema. later (or right now, it's your time, lady, I am not the boss of you).


So, a JSON Schema describes your data. Here is the simplest schema, that matches anything:

{ }

Scary, uh? Here's a more restrictive one:

  "type": "string"

That means "a thing, which is a string." So this is valid: "foo" and this isn't 42 Usually, on APIs you exchange JSON objects (dictionaries for you pythonistas), so this is more like you will see in real life:

  "type": "object",
  "properties": {
    "street_address": { "type": "string" },
    "city": { "type": "string" },
    "state": { "type": "string" }
  "required": ["street_address", "city", "state"]

That means "it's an object", that has inside it "street_address", "city" and "state", and they are all required.

Let's suppose that's all we need to know about schemas. Again, before you actually use them in anger you need to go and read Understanding JSON Schema. for now just assume there is a thing called a JSON Schema, and that it can be used to define what your data is supposed to look like, and that it's defined something like we saw here, in JSON. Cool?

Using schemas

Of course schemas are useless if you don't use them. You will use them as part of the "contract" your API promises to fulfill. So, now you need to validate things against it. For that, in python, we can use jsonschema

It's pretty simple! Here is a "full" example.

import jsonschema

schema = {
  "type": "object",
  "properties": {
    "street_address": {"type": "string"},
    "city": {"type": "string"},
    "state": {"type": "string"},
  "required": ["street_address", "city", "state"]

    "street_address": "foo",
    "city": "bar",
    "state": "foobar"
}, schema)

If the data doesn't validate, jsonchema will raise an exception, like this:

>>> jsonschema.validate({
...     "street_address": "foo",
...     "city": "bar",
... }, schema)
Traceback (most recent call last):
  File "<stdin>", line 4, in <module>
  File "jsonschema/", line 541, in validate
    cls(schema, *args, **kwargs).validate(instance)
  File "jsonschema/", line 130, in validate
    raise error
jsonschema.exceptions.ValidationError: 'state' is a required property

Failed validating 'required' in schema:
    {'properties': {'city': {'type': 'string'},
                    'state': {'type': 'string'},
                    'street_address': {'type': 'string'}},
     'required': ['street_address', 'city', 'state'],
     'type': 'object'}

On instance:
    {'city': 'bar', 'street_address': 'foo'}

Hey, that is a pretty nice description of what is wrong with that data. That is how you use a JSON schema. Now, where would you use it?

Getting value out of schemas

Schemas are useless if not used. They are worthless if you don't get value out of using them.

These are some ways they add value to your code:

  • You can use them in your web app endpoint, to validate things.
  • You can use them in your client code, to validate you are not sending garbage.
  • You can use a fuzzer to feed data that is technically valid to your endpoint, and make sure things don't explode in interesting ways.

But here is the most value you can extract of JSON schemas:

You can discuss the contract between components in unambiguous terms and enforce the contract once it's in place.

We are devs. We discuss via branches, and comments in code review. JSON Schema turns a vague argument about documentation into a fact-based discussion of data. And we are much, much better at doing the latter than we are at doing the former. Discuss the contracts.

Since the document describing (this part of) the contract is actually used as part of the API definitions in the code, that means the document can never be left behind. Every change in the code that changes the contract is obvious and requires an explicit renegotiation. You can't break API by accident, and you can't break API and hope nobody will notice. Enforce the contracts.

Finally, you can version the contract. Use that along with API versioning and voilá, you know how to manage change! Version your contracts.

  • Discuss your contracts
  • Enforce your contracts
  • Version your contracts

So now you can stop worrying and love JSON Schema as well.

April 22, 2017 03:32 PM


Code Challenge 15 - Create a Simple Flask App - Review

It's end of the week again so we review the code challenge of this week. It's never late to sign up, just fork our challenges repo and start coding.

April 22, 2017 03:20 PM

April 21, 2017


NumFOCUS Welcomes SunPy, Our Newest Fiscally Sponsored Project

​NumFOCUS is pleased to announce the addition of SunPy to our fiscally sponsored projects. SunPy is a community-developed, free and open-source software library for solar physics based on Python. The aim of the SunPy project is to provide the software tools necessary so that anyone can analyze solar data. SunPy is written using the Python programming language […]

April 21, 2017 05:04 PM