skip to navigation
skip to content

Planet Python

Last update: August 02, 2016 01:49 PM

August 02, 2016


Python Software Foundation

"Avoiding the Curse of Knowledge": Community Service Award Recipient Ned Batchelder

Ned Batchelder didn't mean to get himself nominated for an award. He'd simply encouraged his Twitter followers to "nominate someone who made a difference for a PSF Community Service Award." A friend who saw his tweet thought, "You know, the person who best exemplifies the community spirit and community service of Python is Ned."

In the second quarter of 2016, the Python Software Foundation recognized Batchelder with a Community Service Award for his tireless work helping run the Boston Python user group, being a regular speaker at conferences, maintaining coverage.py, and being a friendly face for the community on IRC and elsewhere.

Coverage.py and ByteRun

Batchelder maintains coverage.py, a tool for measuring Python programs' code coverage. It monitors a program under test, to report which lines of source are executed and which are not. Coverage.py fills a vital niche in the Python ecosystem, ensuring our code is thoroughly tested, and Batchelder has developed it for well over a decade.

As an offshoot of his coverage.py work, Batchelder began experimenting with a project called ByteRun in 2013. His most prominent contributor is Allison Kaptur, who wrote a chapter on the project for The Architecture of Open Source Applications. "ByteRun is a Python interpreter written in Python," she says. "It’s not like a real one. It does now run most Python code that you throw at it, but Ned's original motivation was to explore instruction-level coverage for coverage.py. There are lots of cases where line-level coverage does not give the whole picture; he wanted to see if he could do better. That was his motivation, but I joined the project because I have an inexplicable love of byte code that I cannot justify."

Kaptur loved working on a codebase with Batchelder—despite his long experience, he is patient with contributors and enthusiastic about their work. When she reported a significant bug in his implementation of stack frames and scopes in ByteRun, he celebrated the discovery. "Wow, this is amazing! I'm humbled to learn that I had the data stack wrong."

Boston Python User Group

When Batchelder first attended the Boston Python User Group a decade ago it was already going strong. The group draws from Boston's big tech community, and it meets in Cambridge's Kendall Square, the epicenter of the tech industry there.

Under Batchelder's leadership the group has grown to over 6000 members. It was for this achievement that his friend and colleague David Baumgold, when he read Batchelder's tweet, nominated him for a Community Service Award. Baumgold says, "He is very, very good at getting people to open up about what they’re working on. He calls it pulling lightning talks out of people. He says, 'If you don’t think you have a lightning talk in you, come and talk to me. In five minutes I'll find your lightning talk.'"

Jessica McKellar, too, helped cultivate Boston Python for several years. Batchelder admits, "I’m not great at bringing in other organizers and giving them lots of control. But she was a big help in creating new events for the group." The workshops for women that McKellar started unlocked a pent-up demand in the area. McKellar created a follow-up event, the monthly Project Night, where workshop participants could advance further. Batchelder says, "On project nights people come and do what they want to do. When you run one of those every month, you get a consistent rhythm going and people continue to show up."

Another reason for the group's size is Batchelder's stick–to–itiveness. Organizing events for seven years and more isn't easy. There are nights he asks himself, "Really? I’m going to Boston Python again? That’s what I’m doing tonight?"

But it's worth it: helping people use computers, for Batchelder, is as fun as using computers himself. Even better is when he connects a newcomer to a project or person that provides precisely what they need. "When I can do that, I think—wow, I really nailed it. That was a good night."

PyCon Talks

When PyCon's call for proposals opens each year, Batchelder considers what question he's heard most often that year, and tries to come up with a "very good answer" that is the core of a PyCon talk.

But how does he compose the best answer he can?

"There’s all these topics that lead to each other in a dense graph, but I'm going to have to linearize it and speak one sentence after the other for 25 minutes." To trim that tree of knowledge to its trunk, Batchelder seeks the principles that lead from the question to the solution. "I try to lay bare the mechanisms that link them together."

His Unicode Sandwich talk in 2012, for example, distilled a painfully complex question into a simple answer. Programmers can handle Unicode correctly in their apps with his three concise tips. And his talk on looping and iteration in Python covers topics so fundamental that most Python programmers couldn't explain them much better than a mole could explain digging. David Baumgold says, "Ned is very good at avoiding the curse of knowledge. He's simultaneously very knowledgeable and also very accessible to beginners."

Juggling

Ewa Jodlowska, the PSF Director of Operations, says that every Python programmer she meets has a hidden talent. She was sitting at a table with Batchelder at PyCon Montréal last year when he picked up a few pieces of fruit and juggled them.

Batchelder has been juggling for so long he cannot remember how he first learned; perhaps from his father. Naturally, he teaches it too. At his workplace, Open edX, "There’s a bunch of people who learned how to juggle because they saw all the props on my desk. Whenever we have company outings there always kids that want to learn."

At PyCon this year he held an Open Space for juggling. Jodlowska recalls it was so popular it spilled out of the room. Batchelder stood in the hallway juggling his pins, while the crowd surrounding him cheered him on.

Photo by Max Batchelder

August 02, 2016 08:00 AM

August 01, 2016


Continuum Analytics News

Powering Down the Anaconda Build CI System

Posted Monday, August 1, 2016

We are reaching out to our committed base of Anaconda Cloud users to let you know we’ve made the decision to deprecate the Anaconda Build service within Anaconda Cloud as of September 1, 2016. Please note that Anaconda Build is not to be confused with conda-build, which will continue to be supported and maintained as a vital community project. The remaining functionality within Anaconda Cloud will continue to be available to the Anaconda user community to host, share and collaborate via conda packages, environments and notebooks.

Anaconda Build is a continuous integration service that is part of the Anaconda platform (specifically, Anaconda Cloud and Anaconda Repository). Anaconda Build helps conda users and data scientists automatically build and upload cross-platform conda packages as part of their project’s development and release workflow.

Anaconda Build was initially launched [in effect as a beta product] to support internal needs of the Anaconda platform teams and to explore the combination of a continuous integration service with the flexibility of conda packages and conda-build. While we were developing and supporting Anaconda Build, it was gratifying to see significant community effort around Conda-Forge to solve issues around cross-platform continuous integration workflows with conda and the formation of a community-driven collection of conda recipes to serve the open data science community. The resulting conda package builds from Conda-Forge are hosted on Anaconda Cloud.

We feel that the Anaconda community efforts around Conda-Forge (along with Travis CI, CircleCI, and Appveyor) are in a better position to deliver reliable continuous integration services compared to our internal efforts with Anaconda Build. For both Anaconda community users and Anaconda platform subscribers, alternative solutions for automated conda package builds can be developed using the previously mentioned continuous integration services.

The documentation for Anaconda Build will continue to be available in the Anaconda Cloud documentation for a short period of time after Anaconda Build is deprecated but will be removed in the near future.

If you have any questions about the deprecation of Anaconda Build or how to transition your conda package build workflow to Conda-Forge or other continuous integration services, please post on the Anaconda Cloud issue tracker on Github.

-The Anaconda Platform Product Team

August 01, 2016 07:51 PM


Django Weblog

Django 1.10 released

The Django team is happy to announce the release of Django 1.10.

As always, the release notes cover the panoply of new features in detail, but a few highlights are:

You can get Django 1.10 from our downloads page or from the Python Package Index. The PGP key ID used for this release is Tim Graham: 1E8ABDC773EDE252.

Django 1.10 will receive fixes for security issues, data loss bugs, crashing bugs, major functionality bugs in newly-introduced features, and regressions from older versions of Django for eight months until April 2017. Fixes for security issues and data loss bugs will be provided for another eight months until December 2017.

With the release of Django 1.10, Django 1.9 has reached the end of mainstream support. The final minor bugfix release (1.9.9) was issued today. Django 1.9 will receive security and data loss fixes for another eight months until April 2017. See the downloads page for a table of supported versions and the future release schedule.

August 01, 2016 06:52 PM


Graham Dumpleton

Testing out deployment of Python based Opal health care framework.

When I was working on mod_wsgi, but also in a previous job where I was working on web application performance monitoring tools, I was always after good sample Python web applications to test with. Unlike other programming languages for the web there weren’t many end user applications written in Python that you could quickly download and get running. Most of what existed out there were incomplete

August 01, 2016 04:57 PM


PyCon Australia

PyConAU 2016 sells out!

We are thrilled to announce that PyCon Australia 2016 has SOLD OUT!

Not in the bad way, of course -- we still have our principles, that beautiful is better than ugly, and explicit is better than implicit, and the rest. Just in the way that 500 of our best Pythonista mates will be joining us in Melbourne in a mere two weeks for a huge PyConAU #7!

For would-be attendees who left it a little too late, you can email us with the subject line "waiting list" and we will put you in the queue in case there are cancellations. However, this is not a promising queue. Another option is to purchase a miniconf-only ticket that lets you attend on Friday only. If that interests you, email us with the subject line "miniconf-only ticket". There is a limited number of these tickets available too so don't hesitate any longer.

For attendees who did register and pay in time, if you forgot to buy a dinner ticket, you can still purchase one before 6 August. Or, if you want to upgrade your ticket to become a Contributor✨, we would be thrilled to hear from you. Email away!

Finally, if you are a company feeling a little left out of all this Pythontastic fabulousness, never fear - it is not too late to become a sponsor and we would love to hear from you!

August 01, 2016 01:37 PM


Doug Hellmann

enum – Enumeration Type — PyMOTW 3

The enum module defines an enumeration type with iteration and comparison capabilities. It can be used to create well-defined symbols for values, instead of using literal integer or strings. Read more… This post is part of the Python Module of the Week series for Python 3. See PyMOTW.com for more articles from the series.

August 01, 2016 01:00 PM


Mike Driscoll

PyDev of the Week: Cory Benfield

This week we welcome Cory Benfield (@lukasaoz) as our PyDev of the Week! Cory is a core developer of the Python language, specifically on urllib3 as well as a core developer of the requests package. He is also the lead maintainer of the Hyper Project which is a set of related projects that provide HTTP/2 functionality to Python projects. Let’s spend some time getting to know Cory better!

Can you tell us a little about yourself (hobbies, education, etc):

I studied Physics at university in Scotland, originally with a bit of an eye towards ending up somewhere on the more practical end of professional physics: maybe doing something like medical physics. Before my degree, I’d never written a program in my life. However, I’d been a computer “power user” for a long time: I’d owned PCs and Macs and was comfortable in Linux as well as Windows and OS X, and I’ve been building PCs since I was 15. So when I found myself in a computational physics module staring at a Mathematica screen, something clicked in my brain and I realised that this was something I could be really happy doing. So I graduated with my masters degree in Physics and with a job offer to write software for the telecommunications industry.

When I’m not writing software, I like doing lots of different things to relax. I play video games (like most of the world now does), but I love watching great movies and TV shows. I also greatly enjoy cooking, reading, and spending time with people. Sometimes I even write: I’m not good at the discipline required to write regularly, but I think my writing is ok!

Why did you start using Python?

I’d just started using Mathematica for my Physics work in college, and while Mathematica is great it’s not really a useful tool for building utilities. So I glanced around for other languages, and it didn’t take long playing with C and Perl before Python seemed like the most obvious thing in the world to me. So I wrote some web scraping tools in Python, and that dragged me into the Python open source community. I’ve never looked back!

What other programming languages do you know and which is your favourite?

I’m the kind of person who likes to pick up and play with new languages for fun, so “know” gets a bit tricky here. I’d say I am most comfortable with Python, and then in roughly descending order of how idiomatic I am with the language: C, Go, Rust, Javascript, Swift, C#, Java, Objective C. I have dabbled a bit in the strongly functional languages, like Haskell, but wouldn’t say I “know” Haskell yet because I haven’t used it in anger yet. I also like Lisps, though again I haven’t written much Lisp code in anger: with that said, Hy is obviously my favourite lisp because it lets me use all the Python libraries I’ve already gotten used to!

What projects are you working on now?

Right now I’m spread pretty thin, but my major focus is HTTP and HTTP/2 in Python. That means that my time is spread roughly amongst maintaining several popular Python libraries: Requests and urllib3 being the most notable and the biggest drains on my time. Alongside those I maintain the only HTTP/2 libraries in Python, including hyper-h2, which is used as the basis for a number of HTTP/2 implementations in Python. Finally, I’m tangentially involved in a lot of other OSS Python projects, like the Python Cryptographic Authority and Twisted.

Which Python libraries are your favorite (core or 3rd party)?

Excluding my own, of course!

In the Python standard library I really like itertools: generators have a real appeal to me and itertools makes it possible to work with them in a really flexible way. Outside of the standard library, I think CFFI is awesome: being able to call into C regardless of how your VM is implemented is really powerful, and it removes one of the largest barriers to using PyPy, which I’d argue is one of the greatest assets of the Python community.

Where do you see Python going as a programming language?

It’s common in the Python community to worry about whether languages like Go are eating our lunch. I don’t buy into that rhetoric. Python is a great language: it’s clean, it’s easy to read, and when combined with tools like PyPy it’s more than fast enough for the majority of use-cases.

For now, I see Python consolidating, and building out tools that allow Python to integrate more tightly with other languages. Moving towards CFFI and away from writing explicit C extensions is a big part of this, as it allows Python programmers to combine the best of Python (e.g. PyPy) with the best of other languages (by calling into compiled DLLs without worrying about the specific Python implementation being used). Applications in the future will increasingly involve smaller components calling between each other, either via RPC mechanisms like HTTP APIs, or via direct function calls using things like CFFI. Python is well suited to be a part of that environment.

What is your take on the current market for Python programmers?

As far as I can tell, Python programmers are extremely employable. Python is a well established language that many companies would be willing to bet their businesses on, and that gives Python programmers a great deal of power and flexibility in the industry. Python skill is a great notch in the belt for anyone job hunting.

Is there anything else you’d like to say?

Only that if people are interested in getting involved in Open Source software, Python is almost certainly the language to do it in: there are lots of great projects floating around that would love more contribution from the community, and getting involved is a great way to give back.

Thanks for doing the interview!

August 01, 2016 12:30 PM


Hynek Schlawack

Please Fix Your Decorators

If your Python decorator unintentionally changes the signatures of my callables or doesn’t work with class methods, it’s broken and should be fixed. Sadly most decorators are broken because the web is full of bad advice.

August 01, 2016 12:00 PM


Alexey Evseev

Django: signal or model method?

Django: signal or model method?

When I needed to implement some functionality on model saving, I always asked a question to myself - where to place it. In signal or in model method save()? Let's see, what and when is more applicable.

When use model methods save(), delete()?

To my mind class methods are more usable, if logic concerns exclusively current model. For example, fill some field on model saving according to data from other fields.

Some people say, that ...

August 01, 2016 11:29 AM


Graeme Cross

PyCon AU 2016 runs (not sprints)

PyCon AU 2016 starts next week for five days of mini-confs, the main conference and two days of sprints.

With all that sitting around, we want you to pack your running gear and join us for a run each morning at 7am.

The runs are intended to be inclusive and the pace is civilised enough to chat along the way — these are groups runs, not races!

All of the details, including the routes with maps, are on the conference wiki.

We hope to see you there!

August 01, 2016 01:19 AM


Emmanuel Leblond

MongoDB at the EuroPython and a new sync/async ODM: μMongo

Coming back from the EuroPython where I could share feelings about the MongoDB ODMs in Python. I've learned few things there:

People only talk about Mongoengine

Mongoengine is the most well known ODM out there, probably because it is one of the oldest and most starred on Github. Beside it's the only one on the 1st Google page of a python mongodb search. So the alternatives to Mongoengine seems forgotten even if they are pretty numerous: humbledb, mongoalchemy, minimongo, ming, nanomongo etc.

My guess is people who come to MongoDB go with the bigger thing, if it's good they stick with it, if it's bad they don't want to do another poor choice and start relying on themselves :-(

People don't like Mongoengine

Asking what people think about Mongoengine was my little funny moment: First they always complained about how bad it is, then I announced I'm part of the Mongoengine team and finally I could see people turning white, red or laughing depending on how deeply they shared there painful relationship with Mongoengine.

But honestly I can't blame them. I think Mongoengine trouble comes from it inception: It was created a long time ago when MongoDB was just out.

My guess is back in those dark times they was not enough experience with this tech and the original author wanted to create and ORM for mongo to mimic what you would do with an SQL database.

For example Mongoengine allows you to add a reverse_delete_rule to a reference field. This way, if the referenced document is destroyed Mongoengine can update the reference to null or even do cascade deletion, feels pretty SQL isn't it ? A bit too much !

The trouble is this operation is not atomic at all given it is done by Mongoengine on top of a regular pymongo driver (and MongoDB cannot do that anyway). So you end up with concurrency issues, inconsistent data if your application stops during the processing of this operation etc.

I intended a talk on the MongoDB Driver during the week, at the end someone asked the talker what does he think about Mongoengine:

In a nutshell: people think of Mongoengine like a way to do SQL stuff in Mongo !

People think they don't need an ODM (but they do !)

After putting people into embarrassment with my Mongoengine question, I try to relieve them by asking them about the ODM they use. The answer was "Nothing, just plain pymongo !" pretty often (the "don't want to two poor choices in a row" effect I guess).

However, after digging a bit, I realize those people created they own validation system around pymongo, and so ended up with a custom ODM when their project grown up and they tried to refactor it...

If you go further in the upper video, that's precisely what says the next talking person: plenty of people use Mongoengine because they want a way to validate they documents.

AsyncIO is "the next big thing"™

The number of talks about asyncIO was crazy high, the nodejs community has proved Twisted was right a decade ago (well it's not the silver bullet, but I think it goes along pretty well with MongoDB big data credo). Now If you want to do asynchronous mongoDB, there is drivers. But you run short on ODM: I could only find motorengine, which is a fork of... Mongoengine !

What's from there ?

So here is μMongo, a simple ("μ" used to stand for "micro" but given it is now pretty feature-full it is now only the letter "mu") MongoDB ODM compatible with the majors drivers (both sync and async !) and relying on Marshamllow for all the validation/serialization stuff (why reinvent the wheel when a first class library is available ?).

August 01, 2016 12:00 AM


Thomas Guest

Negative Sequence Indices in Python

Supply a negative index when accessing a sequence and Python counts back from the end. So, for example, my_list[-2] is the penultimate element of my_list, which is much better than my_list[len(my_list)-2] or even *(++my_list.rbegin()).

That final example uses one of C++’s reverse iterators. It gets the penultimate element of a collection by advancing an iterator from the reversed beginning of that collection. If you’re addicted to negative indices you can use them with C++ arrays, sort of.

Negative array indices in C++
#include <iostream>

int main()
{
    char const * domain = "wordaligned.org";
    char const * end = domain + strlen(domain);
    std::cout << end[-3] << end[-2] << end[-1] << '\n';
    return 0;
}

Compiling and running this program outputs the string “org”.

Going back to Python, the valid indices into a sequence of length L are -L, -L+1, … , 0, 1, … L-1. Whenever you write code to calculate an index used for accessing a sequence, and especially if you’re catching any resulting IndexErrors, it’s worth checking if the result of the calculation can be negative, and if — in this case — you really do want the from-the-back indexing behaviour.

 
 
 
 
0
1
2
3
4
5
6
7
8
9
10
11
12
13
 
 
 
 
W
O
R
D
A
L
I
G
N
E
D
 
 
 
 
-14
-13
-12
-11
-10
-9
-8
-7
-6
-5
-4
-3
-2
-1
 
 
 
 

The power of negative indices increases with slicing. Take a slice of a sequence by supplying begin and end indices.

>>> domain = 'wordaligned.org'
>>> domain[4:9]
'align'
>>> domain[4:-4]
'aligned'
>>> digits = list(range(10))
>>> digits
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> digits[3:4]
[3]
>>> digits[1:-1]
[1, 2, 3, 4, 5, 6, 7, 8]

Omitting an index defaults it to the end of the sequence. Omit both indices and both ends of the sequence are defaulted, giving a sliced copy.

>>> domain[-3:]
'org'
>>> domain[:4]
'word'
>>> digits[:]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

I prefer the list(digits) form for copying digits but you should certainly be familiar with the digits[:] version.

You can supply any indices as slice limits, even ones which wouldn’t be valid for item access. Imagine laying your sequence out on an indexed chopping board, slicing it at the specified points, then taking whatever lies between these points.

>>> digits[-1000000]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: list index out of range
>>> digits[1000000]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: list index out of range
>>> digits[-1000000:1000000]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

Sequence slicing also takes a step parameter.

>>> digits[::2]
[0, 2, 4, 6, 8]
>>> digits[1::2]
[1, 3, 5, 7, 9]

This parameter too can be negative. The sign of the step affects which limiting values the begin and end slice parameters default to. It’s subtle behaviour, but you soon get used to it.

>>> digits[0:10:-2]
[]
>>> digits[::-2]
[9, 7, 5, 3, 1]
>>> digits[-2::-2]
[8, 6, 4, 2, 0]

How do you reverse a string? Slice it back to front!

>>> domain[::-1]
'gro.dengiladrow'

To recap: the default slice limits are the start and end of the sequence, 0 and -1, or -1 and 0 if the step is negative. The default step is 1 whichever way round the limits are. When slicing, s[i:j:k], i and j may take any integer value, and k can take any integer value except 0.

The zero value creates another interesting edge case. Here’s a function to return the last n items of a sequence.

>>> def tail(xs, n)
...     return xs[-n:]
...

It fails when n is 0.

>>> tail(digits, 3)
[7, 8, 9]
>>> tail(digits, 2)
[8, 9]
>>> tail(digits, 1)
[9]
>>> tail(digits, 0)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

By the way, we’ve already seen slicing working well with lists and strings. It also works nicely with range objects.

>>> r = range(10)
>>> r[::2]
range(0, 10, 2)
>>> r[1::2]
range(1, 10, 2)

August 01, 2016 12:00 AM

July 31, 2016


Hynek Schlawack

Testing & Packaging

How to ensure that your tests run code that you think they are running, and how to measure your coverage over multiple tox runs (in parallel!).

July 31, 2016 07:00 PM


Weekly Python Chat

Python Code Style

Are you concerned about code readability but your team argues too much about code style? Join this chat to discuss Python code style norms, line wrapping, naming things, explicit code style guides, and times when PEP 8 isn't quite opinionated enough.

July 31, 2016 05:00 PM


Krzysztof Żuraw

Design by contract in python- part three

I covered py.contracts and zope.interface, now it's time to write about abc module from python standard library.

I wanted to write about another library dpcontrancts, but unfortunately, I wasn't able to download it from PyPi.

When I was reading and reviewing material for previous posts I found out that there is a way to use python standard library abc.ABCMeta for contracts- I decided to give a try.

First, you have to know what are metaclasses: understanding python metaclasses and what are they needed for: Why use Abstract Base Classes in Python?.

After this introduction let's jump straight to the code:

import abc

class ExternalAPIPortMetaclass(object, metaclass=abc.ABCMeta):

    @abc.abstractmethod
    def search(self, query, *args, **kwargs):
        if not isinstance(query, str):
            raise ValueError('Query should be string')


class ExternalAPIPort(ExternalAPIPortMetaclass):

    def __init__(self, adapter):
        self.adapter = adapter

    def search(self, query, *args, **kwargs):
        super(ExternalAPIPort, self).search(query, *args, **kwargs)
        return self.adapter.search(query, *args, **kwargs)

What is happening here? I defined ExternalAPIPortMetaclass as a metaclass that inherits from abc.ABCMeta (This code snippet is valid for python 3). Then I decided to make abstractmethod called search so all instances of that metaclass will have to provide such function. Inside this code, I check whether provided query is a string or not. In ExternalAPIPort which inherits from previously defined I have to call super for ExternalAPIPortMetaclass search method. Thanks to that I can make a validation of query. Right after that I simply return search query.

What I don't like there is fact that I need to add additional line of code inside ExternalAPIPort.search with super just for checking contract which can trick others. That's why I think that metaclasses and contracts are two different topic besides that they have some pieces in common (both are designed for telling: here I make contract that you must obey).

To sum up this whole series I belive contracts are usefull for telling others that I made agrement that this function has to take and return certain value. In python word where is so called ducktyping and I don't think they are necessary in every case but designing by contracts can be helpfull as I shown in examples. In my day to day work I use contracts the same way to make agrements on ports methods in ports and adapters design pattern.

Thank you for reading! Feel free to comment!

Other blog posts in this series:

Edits (01.08.2016):

  • Add link to other posts with same series (thanks to alb1)

Cover image under CC0.

July 31, 2016 08:00 AM


Itamar Turner Trauring

Write test doubles you can trust using verified fakes

When you're writing tests for your code you often encounter some complex object that impedes your testing. Let's say some of your code uses a client library to talk to Twitter's API. You don't want your tests to have to talk to Twitter's servers: your tests would be slower, flakier, harder to setup, and require a working network connection.

The usual solution is to write a test double of some sort, an object that pretends to be the Twitter client but is easier to use in a test. Terminology varies slightly across programming communities, but you're probably going to make a fake, mock or stub Twitter client for use in your tests.

As you can tell from names like "fake" and "mock", there's a problem here: if your tests are running against a fake object, how do you know they will actually work against the real thing? You will often find the fake is insufficiently realistic, or just plain wrong, which means your tests using it were a waste of time. I've personally had Python mock objects cause completely broken code to pass its tests, because the mock was a bit too much of a mockup.

There's a better kind of test double, though, known as a "verified fake". The key point about a verified fake is that, unlike regular fakes, you've actually proven it has the same behavior as the real thing. That means that when you use the verified fake for a Twitter client in a test you can trust that the fake will behave the same as real Twitter client. And that means you can trust your tests are not being misled by a broken fake.

Read more...

July 31, 2016 04:00 AM


tryexceptpass

Engineering the Tests that Matter

Going beyond the checkbox

July 31, 2016 03:49 AM

July 30, 2016


Podcast.__init__

Episode 68 - Test Engineering with Cris Medina

Summary

We all know that testing is an important part of software and systems development. The problem is that as our systems and applications grow, the amount of testing necessary increases at an exponential rate. Cris Medina joins us this week to talk about some of the problems and approaches associated with testing these complex systems and some of the ways that Python can help.

Brief Introduction

Linode Sponsor Banner

Use the promo code podcastinit20 to get a $20 credit when you sign up!

sentry-horizontal-black.png

Stop hoping your users will report bugs. Sentry’s real-time tracking gives you insight into production deployments and information to reproduce and fix crashes. Use the code podcastinit at signup to get a $50 credit!

Hired Logo

On Hired software engineers & designers can get 5+ interview requests in a week and each offer has salary and equity upfront. With full time and contract opportunities available, users can view the offers and accept or reject them before talking to any company. Work with over 2,500 companies from startups to large public companies hailing from 12 major tech hubs in North America and Europe. Hired is totally free for users and If you get a job you’ll get a $2,000 “thank you” bonus. If you use our special link to signup, then that bonus will double to $4,000 when you accept a job. If you’re not looking for a job but know someone who is, you can refer them to Hired and get a $1,337 bonus when they accept a job.

Interview with Cris Medina

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Summary We all know that testing is an important part of software and systems development. The problem is that as our systems and applications grow, the amount of testing necessary increases at an exponential rate. Cris Medina joins us this week to talk about some of the problems and approaches associated with testing these complex systems and some of the ways that Python can help.Brief IntroductionHello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable. For details on how to support the show you can visit our site at pythonpodcast.comLinode is sponsoring us this week. Check them out at linode.com/podcastinit and get a $20 credit to try out their fast and reliable Linux virtual servers for your next projectWe are also sponsored by Sentry this week. Stop hoping your users will report bugs. Sentry's real-time tracking gives you insight into production deployments and information to reproduce and fix crashes. Check them out at getsentry.comHired has also returned as a sponsor this week. If you're looking for a job as a developer or designer then Hired will bring the opportunities to you. Sign up at hired.com/podcastinit to double your signing bonus.The O'Reilly Velocity conference is coming to New York this September and we have a free ticket to give away. If you would like the chance to win it then just sign up for our newsletter at pythonpodcast.comTo help other people find the show you can leave a review on iTunes, and tell your friends and co-workersJoin our community! Visit discourse.pythonpodcast.com for your opportunity to find out about upcoming guests, suggest questions, and propose show ideas.Your hosts as usual are Tobias Macey and Chris PattiToday we're interviewing Cris Medina about test engineering for large and complex systems. Use the promo code podcastinit20 to get a $20 credit when you sign up! Stop hoping your users will report bugs. Sentry's real-time tracking gives you insight into production deployments and information to reproduce and fix crashes. Use the code podcastinit at signup to get a $50 credit! On Hired software engineers designers can get 5+ interview requests in a week and each offer has salary and equity upfront. With full time and contract opportunities available, users can view the offers and accept or reject them before talking to any company. Work with over 2,500 companies from startups to large public companies hailing from 12 major tech hubs in North America and Europe. Hired is totally free for users and If you get a job you’ll get a $2,000 “thank you” bonus. If you use our special link to signup, then that bonus will double to $4,000 when you accept a job. If you’re not looking for a job but know someone who is, you can refer them to Hired and get a $1,337 bonus when they accept a job.Interview with Cris MedinaIntroductionsHow did you get introduced to Python? - ChrisTo get us started can you share your definition of test engineering and how it differs from the types of testing that your average developer is used to? - TobiasWhat are some common industries or situations where this kind of test engineering becomes necessary? - TobiasHow and where does Python fit into the kind of testing that becomes necessary when dealing with these complex systems? - TobiasHow do you determine which areas of a system to test and how can Python help in that discovery process? - TobiasWhat are some of your favorite tools and libraries for this kind of work? - TobiasWhat are some of the areas where the existing Python tooling falls short? - TobiasGiven the breadth of concerns that are encompassed with testing the various components of these large systems, what are some ways that a test engineer can get a high-level view of the overall state? - TobiasHow can that information be distilled for presentation to other areas of the business? - TobiasCould

July 30, 2016 07:21 PM


Davy Wybiral

WebGL terrain tutorial



Today I put together a quick tutorial on how to build a 3d terrain in Javascript, from setting up three.js to loading a heightmap and texture. More will be added later (controls and collision).

Each step is organized as a release in the Github repo so you can download them all individually or browse through the commit diffs to see each change being made.

The Github project is here: https://github.com/wybiral/terrain

And, if course, there's a demo of it here: https://wybiral.github.io/terrain/

July 30, 2016 10:53 AM


Weekly Python StackOverflow Report

(xxx) stackoverflow python report

These are the ten most rated questions at Stack Overflow last week.
Between brackets: [question score / answers count]
Build date: 2016-07-30 09:25:51 GMT


  1. rounding errors in Python floor division - [20/5]
  2. 0 is 0 == 0 (#evaluates to True?) - [16/2]
  3. Type hint that a function never returns - [9/0]
  4. Python: How to delete rows ending in certain characters? - [8/3]
  5. Pycharm import RuntimeWarning after updating to 2016.2 - [8/1]
  6. Combination of two lists while keeping the order - [7/5]
  7. Convert Python sequence to NumPy array, filling missing values - [7/5]
  8. Python- np.mean() giving wrong means? - [7/1]
  9. Python: get every possible combination of weights for a portfolio - [6/6]
  10. Removing duplicate edges from graph in Python list - [6/4]

July 30, 2016 09:26 AM


Full Stack Python

Python for Entrepreneurs

Python for Entrepreneurs is a new video course by the creators of Talk Python to Me and Full Stack Python.

We are creating this course and running a Kickstarter for it based on feedback that it's still too damn difficult to turn basic Python programming knowledge into a business to generate income as a side or full time project. Both Michael and I have been able to make that happen for ourselves and we want to share every difficult lesson we've learned through this course.

The Python for Entrepreneurs videos and content will dive into building and deploying a real-world web application, marketing it to prospective customers, handling search engine optimization, making money through credit card payments, getting help from part-time contractors for niche tasks and scaling up to meet traffic demands.

If this course hits the mark for what you want to do with Python, check out the Kickstarter - we've set up steep discounts for early backers.

If you have any questions, please reach out to Michael Kennedy or me, Matt Makai.

July 30, 2016 04:00 AM

July 29, 2016


Brian Okken

20: Talk Python To Me host Michael Kennedy

I talk with Michael about: Episodes of his show having to do with testing. His transition from employee to podcst host and online training entrepreneur. His Python training courses. The Pyramid Web framework. Givaway We’re giving away courses from Talk Python To Me, enter by signing up for my mailing list at pythontesting.net/subscribe Courses by […]

The post 20: Talk Python To Me host Michael Kennedy appeared first on Python Testing.

July 29, 2016 06:58 PM


Jamal Moir

[Video Series] Taking Your Python Skills to the Next Level With Pythonic Code – Hacking Python’s Memory With __slots__

This is the second post in a series covering Pythonic code written by Michael Kennedy of Talk Python To Me. Be sure to catch the whole series with 5 powerful Pythonic recommendations and over 45 minutes of video examples. What if I told you there was a simple technique you can apply to your custom classes that would dramatically decrease […]

The post [Video Series] Taking Your Python Skills to the Next Level With Pythonic Code – Hacking Python’s Memory With __slots__ appeared first on Data Dependence.

July 29, 2016 05:00 PM


بايثون العربي

مقدمة لمكتبة urllip

إن وحدة urllip في بايثون 3 عبارة عن عن محموعة من الوحدات التي يمكننا إستخدامها للتعامل مع الروابط ، وإذا كنت قادم من بايثون 2 ستجد أن بايثون 2 لديها urllip و urllip2  أما في بايثون 3 أصبحوا عبارة عن حزمة واحدة تحت إسم urllip ويتكون الإصدار الحالي من :

سنقوم بتغطية كل جزء ماعدا urllib.error 

urllib.request

تستخدم هذه الوحدة أساسا لفتح وجلب الروابط ، دعونا نقوم ببعض الأمثلة لنرى مالذي يمكننا فعله مع وظيفة urlopen


>>>import urllib.request
>>> url = urllib.request.urlopen('https://www.google.com/')
 >>>url.geturl()
'https://www.google.com/'
>>>url.info()
lt;http.client.HTTPMessage object at 0x7fddc2de04e0
>>> header = url.info()
>>>header.as_string()
('Date: Fri, 24 Jun 2016 18:21:19 GMT\n'
'Expires: -1\n'
'Cache-Control: private, max-age=0\n'
'Content-Type: text/html; charset=ISO-8859-1\n'
'P3P: CP=&quot;This is not a P3P policy! See '
'https://www.google.com/support/accounts/answer/151657?hl=en for more info.&quot;\n'
'Server: gws\n'
'X-XSS-Protection: 1; mode=block\n'
'X-Frame-Options: SAMEORIGIN\n'
'Set-Cookie: '
'NID=80=tYjmy0JY6flsSVj7DPSSZNOuqdvqKfKHDcHsPIGu3xFv41LvH_Jg6LrUsDgkPrtM2hmZ3j9V76pS4K_
cBg7pdwueMQfr0DFzw33SwpGex5qzLkXUvUVPfe9g699Qz4cx9ipcbU3HKwrRYA; '
'expires=Sat, 24-Dec-2016 18:21:19 GMT; path=/; domain=.google.com; HttpOnly\n'
'Alternate-Protocol: 443:quic\n'
'Alt-Svc: quic=&quot;:443&quot;; ma=2592000; v=&quot;34,33,32,31,30,29,28,27,26,25&quot;\n'
'Accept-Ranges: none\n'
'Vary: Accept-Encoding\n'
'Connection: close\n'
'\n')
>>> url.getcode()
200

قمنا بإستدعاء وجدتنا وقمنا أيضا بفتح موقع google وأصبح لدينا كائن HTTPResponse نستطيع التعامل معه ، الأمر الأول الذي قمنا به هو تنفيذ geturl للحصول على رابط وهي مفيدة لمعرفة ما إذا تم إعادة توجيهنا إلى رابط أخر أم لا .

بعد ذلك قمنا بإستدعاء  info والذي سيعرض بيانات وصفية حول الصفحة مثل معلومات حول headers وهكذا يمكننا تعيين نتيجة هذا الأخير لمتغير headers ثم نقوم بمناداة as_string وهذا سيقوم بعرض header المستقبل من موقع قوقل ، يمكننا أيضا الحصول على رمز الإستجابة الخاص ب HTTP بإستدعائنا ل getcode وفي حالتنا هذه هو عبارة عن 200 وهذا يعني نجاح العملية .

إذا كنت تريد الإطلاع على كود HTML الخاص بالصفحة يمكنك مناداة طريقة READ على متغير url الذي قمنا بإنشاءه، لن أقوم بعرض الناتج لأنه سيكون كبير نوعا ما ولكن يمكنك تجربته لتتأكد.

يرجى أيضا الـتأكد من أن كائن الطلب الإفتراضي هو GET مالم يتم تحديد خيارات البيانات .

تحميل ملف 

أهم إستخدامات حزمة  urllib هو تحميل الملفات فدعونا نارى بعض الأمثلة لتحقيق ذلك


>>> import urllib.request
>>>url = 'http://www.pyarab.com/wp-content/uploads/2016/07/n.zip'
>>>response = urllib.request.urlopen(url)
>>> data = response.read()
>>> with open('/home/mike/Desktop/test.zip', 'wb') as fobj:
... fobj.write(data)
...

هنا قمنا بفتح الرابط الذي يقودنا إلى الملف المطلوب للتحميل ثم نقوم بقراءة البيانات وبالتالي تتم عملية التحميل .

هناك طريقة أخرى لتحقيق نفس العملية وهي بإستخدام urlretrieve :


>>> import urllib.request
>>> url = 'http://www.pyarab.com/wp-content/uploads/2016/07/n.zip'
>>> urllib.request.urlretrieve(url, '/home/mike/Desktop/blog.zip')
('/home/mike/Desktop/blog.zip',
 <http.client.HTTPMessage object at 0x7fddc21c2470>)

هذه الطريقة ستقوم بنسخ كائن الشبكة إلى الملف المحلي ويتم تسمية الملف المنسوخ عشوائيا ويتم نسخه على دليل مؤقت إلا في حالة إستخدام خيارات خاصة لتجعله يقوم بحفظ الملف في دليل معين :


>>> import urllib.request
>>> url = 'http://www.blog.pythonlibrary.org/wp-content/uploads/2012/06/wxDbViewer.zip'
>>> tmp_file, header = urllib.request.urlretrieve(url)
>>> with open('/home/mike/Desktop/test.zip', 'wb') as fobj:
...     with open(tmp_file, 'rb') as tmp:
...         fobj.write(tmp.read())

وكما ترى فإنه في الأخير يقوم بعرض عنوان الدليل التي حفظ الملف عليه بالإضافة إلى معلومات رأس الطلب .

تحديد وكيل المستخدم الخاص بك

عندما تقوم بزيارة موقع ما من خلال المتصفح فإن هذالإخير يخبر الموقع من هو ، وهذا مايسمى بسلسلة user-agent ، مكتبة بايثون urllip تقوم بتحديد نفسها ك : Python-urllib/x.y حيث أن X و Y رقم إصدار بايثون الرئيسي والثانوي ، بعض المواقع لن تتعرف على سلسلة وكيل المستخدم وستتصرف بطريقة غريبة أو لن تعمل على الإطلاق .

لحسن الحظ يمكن إعداد سلسلة وكيل المستخدم خاص بك :


>>> import urllib.request
>>> user_agent = ' Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:47.0) Gecko/20100101 Firefox/47.0'
>>> url = 'http://www.pyarab.com/'
>>> headers = {'User-Agent': user_agent}
>>> request = urllib.request.Request(url, headers=headers)
>>> with urllib.request.urlopen(request) as response:
...     with open('/home/mdriscoll/Desktop/user_agent.html', 'wb') as out:
...         out.write(response.read())

هنا قمنا بتعيين وكيل مستخدم لموزيلا فايرفوكس وقمنا بتعيين الموقع الذي نريد منه إختبار وكيل المستخدم الخاص بنا ، بعد ذلك قمنا بتعيين طلب بإستخدام رابطنا والرأس ومررناه إلى urlopen وأخير قمنا بحفظ الناتج وإذا قمت بفتح ملف الذي يحتوي على الناتج ستجد أننا قمنا بتغيير سلسلة وكيل المستخدك بنجاح .

حاول أن تجرب تغيير وكيل المستخدم عدة مرات مع هذا الكود حتى تفهم ألية العمل .

urllib.parse

تعتبر مكتبة urllib.parse هي الواجهة القياسية لتفكيك سلاسل الروابط و إعادة ربطهم مرة أخرى ويمكن أيضا إستخدامها لتحويل الروابط النسبية إلى روابط مطلقة ، دعونا نأخذ مثال عن تحليل الرابط يحتوي على علامة إستفهام


>>> from urllib.parse import urlparse
>>> result = urlparse('https://duckduckgo.com/?q=python+stubbing&t=canonical&ia=qa')
>>> result
ParseResult(scheme='https', netloc='duckduckgo.com', path='/', params='', query='q=python+stubbing&t=canonical&ia=qa', fragment='')
>>> result.netloc
'duckduckgo.com'
>>> result.geturl()
'https://duckduckgo.com/?q=python+stubbing&t=canonical&ia=qa'
>>> result.port
None

قمنا بإستدعاء وظيفة urlparse ومررناها إلى رابط يحتوي على علامة إستفهام والتي نسأل من خلالها على python stubbing وكان الناتج عبارة عن كائن ParseResult والذي يكمن إستخدامه للمزيد من المعلومات عن الرابط (رقم المنفذ،مكان السبكة،المسار إلخ…).

تقديم نموذج ويب 

هذه الوحدة أيضا تحتوي على طريقة urlencode وهي جيدة لتمرير البيانات إلى عناوين الروابط ، والإستخدام الشائع لمكتبة urllib.parse هو تقديم نماذج الويب ، فدعونا نكتشف كيفية عمل محرك بحث ل duckduckgo خاص ببايثون


>>> import urllib.request
>>> import urllib.parse
>>> data = urllib.parse.urlencode({'q': 'Python'})
>>> data
'q=Python'
>>> url = 'http://duckduckgo.com/html/'
>>> full_url = url + '?' + data
>>> response = urllib.request.urlopen(full_url)
>>> with open('/home/mike/Desktop/results.html', 'wb') as f:
...     f.write(response.read())

مثال واضح وصريح ففي الأساس نريد تقديم إستعلام ل  duckduckgo بإنفسنا بإستخدام بايثون بدلا من المتصفح ولفعل ذلك نحتاج إلى بناء سلسلة إستفهام بإستخدام urlencode ثم نقوم بجمع كل ذلك لنشكل رابط مؤهل تماما ثم نقوم بإستخدام urllib.request لتقديم نموذج الويب بعد ذلك نقوم بالإستيلاء على النتيجة وحفظها .

 

إلى هنا نكون قد وصلنا إلى مرحلة ينبغي لك أن تكون قادرا على التعامل مع حزمة  urllib ، فقد تعلمنا كيفية تحميل ملف ، تقديم نموذج ويب ، تغيير عميل المستخدم وهذا ا يعني ان المكتبة تقوم بهذه الوظائف فقط بل على العكس هناك العديد من المهام التي يمكن لها أن تقوم بها .

July 29, 2016 11:44 AM


Simon

udatetime a fast RFC3339 compliant date-time Python library

Working with date-time formats can be pretty upsetting because of the variate of different formats people can come up with. date-times are used everywhere not just only logging or meta data in database entries and are pretty important. That’s why I encourage developers in using the ISO 8601 derived RFC3339 standard for their projects.

RFC3339 date-time: 2016-07-18T12:58:26.485897 +02:00

The RFC3339 specification offers the following advantages:

Having a date-time standard is nice, but using Python’s datetime library to parse/format a RFC3339 date-time string or even create a datetime object in UTC or local timezone can be painful and slowwwww. That’s why I decided to implement a Python 2 library to deal with such tasks. The library is called udatetime and available on github or PyPI.

$ pip install udatetime

The goal of the library is to be fast and handy with RFC3339 date-time formatted strings. The average performance increase of udatetime compared to the equivalent datetime code is 76%. Due to the usage of Python2 CPython API and POSIX features the library is currently only supported on POSIX systems and not Python3 or Pypy compatible. I’m working on cross-platform and Pypy support. Support in working on the library is greatly appreciated.

Benchmark

The benchmark setup is the following.

from datetime import datetime
import udatetime

RFC3339_DATE = '2016-07-18'
RFC3339_TIME = '12:58:26.485897+02:00'
RFC3339_DATE_TIME = RFC3339_DATE + 'T' + RFC3339_TIME
RFC3339_DATE_TIME_DTLIB = RFC3339_DATE_TIME[:-6]  # datetime can't parse timezones through strptime
DATE_TIME_FORMAT = '%Y-%m-%dT%H:%M:%S.%f'
DATETIME_OBJ = datetime.strptime(RFC3339_DATE_TIME_DTLIB, DATE_TIME_FORMAT)


def benchmark_parse():
    def datetime_strptime():
        datetime.strptime(RFC3339_DATE_TIME_DTLIB, DATE_TIME_FORMAT)

    def udatetime_parse():
        udatetime.from_string(RFC3339_DATE_TIME)

    return (datetime_strptime, udatetime_parse)


def benchmark_format():
    def datetime_strftime():
        DATETIME_OBJ.strftime(DATE_TIME_FORMAT)

    def udatetime_format():
        udatetime.to_string(DATETIME_OBJ)

    return (datetime_strftime, udatetime_format)


def benchmark_utcnow():
    def datetime_utcnow():
        datetime.utcnow()

    def udatetime_utcnow():
        udatetime.utcnow()

    return (datetime_utcnow, udatetime_utcnow)


def benchmark_now():
    def datetime_now():
        datetime.now()

    def udatetime_now():
        udatetime.now()

    return (datetime_now, udatetime_now)


def benchmark_utcnow_to_string():
    def datetime_utcnow_to_string():
        datetime.utcnow().strftime(DATE_TIME_FORMAT)

    def udatetime_utcnow_to_string():
        udatetime.utcnow_to_string()

    return (datetime_utcnow_to_string, udatetime_utcnow_to_string)


def benchmark_now_to_string():
    def datetime_now_to_string():
        datetime.now().strftime(DATE_TIME_FORMAT)

    def udatetime_now_to_string():
        udatetime.now_to_string()

    return (datetime_now_to_string, udatetime_now_to_string)

If you like you can run the benchmark yourself by running the bench.py script from the repository.

The results of 1 million executions and 3 repeats look like this.

benchmark_parse

datetime.strptime(RFC3339_DATE_TIME_DTLIB, DATE_TIME_FORMAT)
vs
udatetime.from_string(RFC3339_DATE_TIME)

Benchmark results benchmark_parse

benchmark_format

DATETIME_OBJ.strftime(DATE_TIME_FORMAT)
vs
udatetime.to_string(DATETIME_OBJ)

Benchmark results benchmark_format

benchmark_now

datetime.now()
vs
udatetime.now()

Benchmark results benchmark_now

benchmark_utcnow

datetime.utcnow()
vs
udatetime.utcnow()

Benchmark results benchmark_utcnow

benchmark_now_to_string

datetime.now().strftime(DATE_TIME_FORMAT)
vs
udatetime.now_to_string()

Benchmark results benchmark_now_to_string

benchmark_utcnow_to_string

datetime.utcnow().strftime(DATE_TIME_FORMAT)
vs
udatetime.utcnow_to_string()

Benchmark results benchmark_utcnow_to_string

July 29, 2016 09:29 AM