Aha! Preview this week’s live, online Python courses

In just 48 hours, I’ll be starting my latest round of live, online courses. Wondering what it’s like to take an online course from me? Or perhaps you’re wondering what sorts of topics I’ll discuss in my “Python dictionaries” and “Python functions” courses? Well, wonder no more; here’s a short preview of my teaching style, and the sorts of things I intend to demonstrate in my courses:

Preview: Reuven’s October 2017 live, online courses about Python and Git from Reuven Lerner on Vimeo.

If you are a beginning or intermediate Python developer, then you’ll become a more effective and fluent developer — good not only for your current employer, but for your career — thanks to my courses. And we’ll have lots of fun along the way. There will be plenty of time for exercises, questions, and comments, to ensure that you understand these technologies well. You’ll return to work the next day able to do more, and more quickly, than before.

Any questions? Just send me e-mail , and I’ll be happy to answer.

I look forward to seeing you this week (for my Python courses) and next week (for my two-day “Understanding Git” course)!

Reuven

Announcing: Three new live courses, to level up your Python and Git skills

  • Confused by Python dicts, or wondering how you can take advantage of them in your programs?
  • Do you wonder how Python functions work, and how you can make them more “Pythonic,” and easier to maintain?
  • Do you wonder why everyone raves about Git, when it seems impossibly hard to understand?  Cloning, pulling, and pushing mostly work… but when they don’t, Git seems like magic, and not the good kind.

If any (or all) of the above is true, then you’ll likely be interested in one or more of the live, online classes I’m teaching later this month:

  1. Python dictionaries, on Wednesday, October 25
  2. Python functions, on Thursday, October 26
  3. Understanding Git, on Tuesday, October 31 and Wednesday, November 1

Each of these classes is live, with tons of live-coding demos, exercises, and time for Q&A. My goal is for you to understand these technologies, how they work, and (most importantly) how you can use them effectively in your work.

Previous classes have been small and highly interactive. These are the same classes I give to some of the best-known companies in the world, such as Apple, Cisco, IBM, PayPal, VMWare, and Western Digital; I’m sure that you’ll enjoy yourself, and come out a better engineer.

Better yet: Buy a ticket by this Friday, and you’ll get a substantial (20%) discount on the ticket price.

This is not a recorded class (although recordings will be available later on).  I’ll be speaking and interacting the entire time, giving you a chance to get your questions answered.  I want to make sure you really understand what’s going on, and will answer any questions you have!

Speaking of which: If you have questions, just e-mail me at reuven@lerner.co.il, and I’ll do my best to answer.

And if you’re a student, ask me for a coupon code that will give you a substantial discount off of the ticket price.

I hope that you can join me for one or more of these classes!

Level up your Python, with three new courses

Back in July, I gave three live, online courses: Object-oriented Python, functional Python, and Python decorators.  I have long found that all three subjects are misunderstood by many Python developers, and I wanted to help people to understand how and when to use each one.

I’m pleased to announce that recordings from all three courses are now available for sale.

These are the same courses that I give all over the world to engineers at such companies as Apple, Cisco, Ericsson, IBM, VMWare, and Western Digital.

If you have ever wanted to level up your understanding of these topics, I think that my courses will really help you out.  Not only do I explain what’s going on, but I also ask you to do exercises to help cement these ideas in your mind. Of course, I go over every exercise as well, and provide the Jupyter notebooks and files that I created when doing so.

Want to buy learn more? Just click on the appropriate link for each one:

If you’re a student, then e-mail me for a discount code good on any (or all!) of these courses.

If you have any questions about the courses, just reply to this e-mail, or contact me at reuven@lerner.co.il.

I’ll be offering some more live courses later this month (October); I’ll be posting dates and topics on Monday of next week, so stay tuned for even more Python goodness!

My favorite terrible Python error message

Students in my Python classes occasionally get the following error message:

TypeError: object() takes no parameters

This error message is technically true, as I’ll explain in a moment. But it’s surprising and confusing for people who are new to Python, because it doesn’t point to the source of the actual problem.

Here’s the basic idea: Python methods are attributes, which means that when we invoke methods, Python needs to search for the attribute we’ve named. In other words, if I invoke:

o.m()

then Python will first look for the “m” attribute on the “o” object. If “o” has an attribute named “m” (i.e., if hasattr(o, ‘m’) returns True) then it retrieves the attribute’s value, and tries to call it.

However, Python methods aren’t defined on individual objects. They’re defined on classes. Which means that in almost all cases, if “m” is an actual method that can be invoked on “o”, there won’t be any “m” attribute on “o”.  Instead, we’ll need to look at type(o), the class to which “o” belongs, and look there.

And indeed, that’s how attributes work in Python: First search on the named object. If the attribute isn’t there, then look at the object’s class.  So we look for “m” on o’s class.  If the attribute is there, then it is invoked.  That’s what happens in normal method calls.

But say that the attribute isn’t on the class, either. What then? Python continues its search, looking next at the class from which type(o) inherits — which is located on the attribute type(o).__bases__.  This is a tuple, because Python classes can inherit from more than one parent; let’s ignore that for now.

Most classes inherit from the base object in the Python universe, known as “object”.  In Python 3, if you don’t specify “object” as the base from which you inherit, then it’s done for you automatically. In Python 2, failing to specify that a class inherits from “object” means that you have an “old-style class,” which will operate differently. I continue to specify “object” in my Python 3 classes, partly out of habit, partly because I think it looks nicer, and partly because I want my code to be compatible across versions as much as possible.

What happens if the attribute doesn’t exist on “object”?  Then we get an “attribute error,” with Python telling us that the attribute doesn’t exist.

However, this isn’t what happens in the case of the error message I showed:

TypeError: object() takes no parameters

This error message happens when you try to create a new instance of a class. For example:

class Foo(object):
    pass

If I say

f = Foo()

then I don’t get any error message. But if I say

f = Foo(10)

then I get the TypeError.  Why?

Because Python objects are created in two stages: First, the object is created in the __new__ method. This is a method that we almost never want to write; let Python take care of the allocation and creation of new objects.

However, __new__ doesn’t immediately return the object that it has created. Rather, it first searches for an __init__ method, whose job is to add new attributes to the newly created object. How does it look for (and then invoke) __init__?  It turns to the new object, which I’ll call “o” here, and invokes

o.__init__()

So, what happens now? Python looks for “__init__” on “o”, but doesn’t find it.  It looks for “__init__” on type(o), aka the “Foo” class, and doesn’t find it.  So it keeps searching, and looks on “object” for an “__init__” attribute.

Good news: object.__init__ exists!  Moreover, it’s a method!  So Python tries to invoke it, passing the argument that I handed to Foo (i.e., 10).  But object.__init__ doesn’t take any arguments. And thus we get the error message

TypeError: object() takes no parameters

What’s especially confusing, for me and many of my students, is that Python doesn’t say, “object.__init__()” takes no parameters. So they’re not sure how object figures into this, or where their mistake might be.

After reading this, though, I’m hoping that you can guess what it means: Simply put, this error message says, “You forgot to define an __init__ method on your object.”  This can be out of forgetfulness, but I’ve also seen people forget one or more of the underscores on either side of “__init__”, or even (my favorite) define a method called “__int__”, which is great for converting objects into integers, but not for initializing attributes.

So, is the error message wrong? No, it’s perfectly logical. But as with many “perfectly logical” things, it makes sense after you are steeped in the overall logic of the system, and tends to confuse those who most need the help.

foo(y=y), and similar code that confuses Python newbies

Let’s define a simple Python function:

In [1]: def foo(x):
 ...: return x * x
 ...:

In [2]: foo(5)
Out[2]: 25

In [3]: foo(10)
Out[3]: 100

As we can see, this function has a single parameter, “x”.  In Python, parameters are local variables whose values are set by whoever is calling the function. So we know that x is a local variable in the function “foo” — which means that “x” doesn’t exist outside of the function “foo”.

I can define a more complex function, which has two parameters:

In [4]: def mul(x, y):
 ...: return x * y
 ...:

In [5]: mul(5, 3)
Out[5]: 15

In [6]: mul(6, 8)
Out[6]: 48

In this example, the “mul” function must take two arguments. “x” and “y” are both local variables within “mul”, meaning that they only exist within the “mul” function.

What happens if I define a “y” variable outside of our “mul” function?  That would be a global variable, which shouldn’t be confused with a local one. Local variables exist only within a function, whereas global variables exist outside of functions. For example:

In [7]: y = 100

In [8]: mul(5,3)
Out[8]: 15

I have thus defined the global variable “y”, which has the value 100. Inside of the function, Python ignores our global “y” variable, because according to Python’s scoping (LEGB) scoping rules, local variables get priority.

So far, so good. But now let’s make things a bit more complex: Let’s change our “mul” function such that the “y” parameter takes a default value. In other words, I’ll be able to call “mul” with two arguments (as before) or with one argument (and thus use the default value for “y”):

In [9]: def mul(x, y=10):
 ...: return x * y
 ...:

In [10]: mul(5)
Out[10]: 50

In [11]: mul(7)
Out[11]: 70

In [12]: mul(5,7)
Out[12]: 35

Notice that once again, our global “y” value has no effect whatsoever on our local “y” parameter: Inside of the function, when Python looks for the value of “y”, it finds the local variable by that name, and uses the value accordingly.

We can even go so far as to give both “x” and “y” default values. Here’s how that would look:

In [13]: def mul(x=3, y=10):
 ...: return x * y
 ...:

In [14]: mul()
Out[14]: 30

In [15]: mul(5)
Out[15]: 50

In [16]: mul(7,3)
Out[16]: 21

Let’s say I want to use the default value of x, but pass a value to y.  How can I do that?  By calling the function, but explicitly naming the “y” parameter, along with a value:

In [17]: mul(y=3)
Out[17]: 9

In [18]: mul(y=5)
Out[18]: 15

What happens if I do this:

In [19]: mul(y=y)

In this case, I’m calling the function, and I’m saying that I want to set the “y” local variable to a value.  But what value am I giving it?  Well, I’m not in the function when I call the function.  And thus the only “y” value available to me is the global “y” variable that I had set earlier.

In other words: I want to call the “mul” function, setting the “y” parameter to the current value of the “y” global variable.

Why would we do such a thing? Relative newcomers to Python find this hard to read, and wonder why (or “y”) we use the same variable name on both the left and right sides.  And the answer is… it’s often easier and more convenient.  The example I’ve provided here is contrived, but there are cases in which you might want to define a function called “key” that sorts your list in a particular way.  With that function defined, you can then say

mylist.sort(key=key)

I personally prefer to define my sorting functions using a different convention, starting with the word “by”, so I can say something like

mylist.sort(key=by_last_name)

At the end of the day, this “y=y” code makes sense if you understand Python’s scoping rules, as well as how function defaults are defined and assigned. Want to know more? I’m giving a live, online course about Python functions (including exactly these topics) on Sunday, August 13th. More details are here, and early-bird tickets are still available!

Where can you practice (and improve) your Python skills?

The most common question I get from students in my Python classes is: How can we practice and improve our skills after the course is over?

These students realize that no matter how good a course might be, they won’t retain very much if they don’t use and practice their Python on a regular basis.

I’ve thus created PracticeYourPython.com, a site listing all of the resources I know about that are designed to improve your Python skills.  Some of these are free, and others are paid.  (And yes, I’ve included resources that I’ve created, as well, such as Practice Makes Python and Weekly Python Exercise.)

So if you want to improve your Python skills, head over to PracticeYourPython.com!  And if you know of resources I’ve missed, please drop me a line at reuven@lerner.co.il; I’ll be sure to add it.

Globbing and Python’s “subprocess” module

Python’s “subprocess” module makes it really easy to invoke an external program and grab its output. For example, you can say

import subprocess
print(subprocess.check_output('ls'))

and the output is then

$ ./blog.py
b'blog.py\nblog.py~\ndictslice.py\ndictslice.py~\nhexnums.txt\nnums.txt\npeanut-butter.jpg\nregexp\nshowfile.py\nsieve.py\ntest.py\ntestintern.py\n'

subprocess.check_output returns a bytestring with the filenames on my desktop. To deal with them in a more serious way, and to have the ASCII 10 characters actually function as newlines, I need to invoke the “decode” method, which results in a string:

output = subprocess.check_output('ls').decode('utf-8')
print(output)

This is great, until I want to pass one or more arguments to my “ls” command.  My first attempt might look like this:

output = subprocess.check_output('ls -l').decode('utf-8')
print(output)

But I get the following output:

$ ./blog.py
Traceback (most recent call last):
 File "./blog.py", line 5, in <module>
 output = subprocess.check_output('ls -l').decode('utf-8')
 File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 336, in check_output
 **kwargs).stdout
 File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 403, in run
 with Popen(*popenargs, **kwargs) as process:
 File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 707, in __init__
 restore_signals, start_new_session)
 File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 1333, in _execute_child
 raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: 'ls -l'

The most important part of this error message is the final line, in which the system complains that I cannot find the program “ls -l”. That’s right — it thought that the command + option was a single program name, and failed to find that program.

Now, before you go and complain that this doesn’t make any sense, remember that filenames may contain space characters. And that there’s no difference between a “command” and any other file, except for the way that it’s interpreted by the operating system. It might be a bit weird to have a command whose name contains a space, but that’s a matter of convention, not technology.

Remember, though, that when a Python program is invoked, we can look at sys.argv, a list of the user’s arguments. Always, sys.argv[0] is the program’s name itself. We can thus see an analog here, in that when we invoke another program, we also need to pass that program’s name as the first element of a list, and the arguments as subsequent list elements.

In other words, we can do this:

output = subprocess.check_output(['ls', '-l']).decode('utf-8')
print(output)

and indeed, we get the following:

$ ./blog.py
total 88
-rwxr-xr-x 1 reuven 501 126 Jul 20 21:43 blog.py
-rwxr-xr-x 1 reuven 501 24 Jul 20 21:31 blog.py~
-rwxr-xr-x 1 reuven 501 401 Jul 17 13:43 dictslice.py
-rwxr-xr-x 1 reuven 501 397 Jun 8 14:47 dictslice.py~
-rw-r--r-- 1 reuven 501 54 Jul 16 11:11 hexnums.txt
-rw-r--r-- 1 reuven 501 20 Jun 25 22:24 nums.txt
-rw-rw-rw- 1 reuven 501 51011 Jul 3 13:51 peanut-butter.jpg
drwxr-xr-x 6 reuven 501 204 Oct 31 2016 regexp
-rwxr-xr-x 1 reuven 501 1669 May 28 03:03 showfile.py
-rwxr-xr-x 1 reuven 501 143 May 19 02:37 sieve.py
-rw-r--r-- 1 reuven 501 0 May 28 09:15 test.py
-rwxr-xr-x 1 reuven 501 72 May 18 22:18 testintern.py

So far, so good.  Notice that check_output can thus get either a string or a list as its first argument.  If we pass a list, we can pass additional arguments, as well:

output = subprocess.check_output(['ls', '-l', '-F']).decode('utf-8')
print(output)

As a result of adding the “-F’ flag, we now get a file-type indicator at the end of every filename:

$ ls -l -F
total 80
-rwxr-xr-x 1 reuven 501 137 Jul 20 21:44 blog.py*
-rwxr-xr-x 1 reuven 501 401 Jul 17 13:43 dictslice.py*
-rw-r--r-- 1 reuven 501 54 Jul 16 11:11 hexnums.txt
-rw-r--r-- 1 reuven 501 20 Jun 25 22:24 nums.txt
-rw-rw-rw- 1 reuven 501 51011 Jul 3 13:51 peanut-butter.jpg
drwxr-xr-x 6 reuven 501 204 Oct 31 2016 regexp/
-rwxr-xr-x 1 reuven 501 1669 May 28 03:03 showfile.py*
-rwxr-xr-x 1 reuven 501 143 May 19 02:37 sieve.py*
-rw-r--r-- 1 reuven 501 0 May 28 09:15 test.py
-rwxr-xr-x 1 reuven 501 72 May 18 22:18 testintern.py*

It’s at this point that we might naturally ask: What if I want to get a file listing of one of my Python programs? I can pass a filename as an argument, right?  Of course:

output = subprocess.check_output(['ls', '-l', '-F', 'sieve.py']).decode('utf-8')
print(output)

And the output is:

-rwxr-xr-x 1 reuven 501 143 May 19 02:37 sieve.py*

Perfect!

Now, what if I want to list all of the Python programs in this directory?  Given that this is a natural and everyday thing we do on the command line, I give it a shot:

output = subprocess.check_output(['ls', '-l', '-F', '*.py']).decode('utf-8')
print(output)

And the output is:

$ ./blog.py
ls: cannot access '*.py': No such file or directory
Traceback (most recent call last):
 File "./blog.py", line 5, in <module>
 output = subprocess.check_output(['ls', '-l', '-F', '*.py']).decode('utf-8')
 File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 336, in check_output
 **kwargs).stdout
 File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 418, in run
 output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ls', '-l', '-F', '*.py']' returned non-zero exit status 2.

Oh, no!  Python thought that I was trying to find the literal file named “*.py”, which clearly doesn’t exist.

It’s here that we discover that when Python connects to external programs, it does so on its own, without making use of the Unix shell’s expansion capabilities. Such expansion, which is often known as “globbing,” is available via the Python “glob” module in the standard library.  We could use that to get a list of files, but it seems weird that when I invoke a command-line program, I can’t rely on it to expand the argument.

But wait: Maybe there is a way to do this!  Many functions in the “subprocess” module, including check_output, have a “shell” parameter whose default value is “False”. But if I set it to “True”, then a Unix shell is invoked between Python and the command we’re running. The shell will surely expand our star, and let us list all of the Python programs in the current directory, right?

Let’s see:

output = subprocess.check_output(['ls', '-l', '-F', '*.py'], shell=True).decode('utf-8')
print(output)

And the results:

$ ./blog.py
blog.py
blog.py~
dictslice.py
dictslice.py~
hexnums.txt
nums.txt
peanut-butter.jpg
regexp
showfile.py
sieve.py
test.py
testintern.py

Hmm. We didn’t get an error.  But we also didn’t get what we wanted.  This is mighty strange.

The solution, it turns out, is to pass everything — command and arguments, including the *.py — as a single string, and not as a list. When you’re invoking commands with shell=True, you’re basically telling Python that the shell should break apart your arguments and expand them.  If you pass a list to the shell, then the parsing is done the wrong number of times, and in the wrong places, and you get the sort of mess I showed above.  And indeed, with shell=True and a string as the first argument, subprocess.check_output does the right thing:

output = subprocess.check_output('ls -l -F *.py', shell=True).decode('utf-8')
print(output)

And the output from our program is:

$ ./blog.py
-rwxr-xr-x 1 reuven 501 141 Jul 20 22:03 blog.py*
-rwxr-xr-x 1 reuven 501 401 Jul 17 13:43 dictslice.py*
-rwxr-xr-x 1 reuven 501 1669 May 28 03:03 showfile.py*
-rwxr-xr-x 1 reuven 501 143 May 19 02:37 sieve.py*
-rw-r--r-- 1 reuven 501 0 May 28 09:15 test.py
-rwxr-xr-x 1 reuven 501 72 May 18 22:18 testintern.py*

The bottom line is that you can get globbing to work when invoking commands via subprocess.check_output. But you need to know what’s going on behind the scenes, and what shell=True does (and doesn’t) do, to make it work.

Five Python function parameters you should know and use

One of Python’s mantras is “batteries included.” which means that even with a bare-bones installation, you can do quite a bit. You can (and should) install packages from PyPI, but many day-to-day tasks can be accomplished with just the built-in data structures, functions, and methods.

What I’ve discovered over the years is that some of the functions and methods have useful parameters that can make our code shorter and more elegant. Here are some of the most elegant ones that I’ve found and use in my work:

1. str.split  (part 1)

One of the methods I use most often in my work is str.split. This method always returns a list, breaking the string into different elements. For example:

In [1]: s = 'abc,def,ghi'

In [2]: s.split(',')
Out[2]: ['abc', 'def', 'ghi']

In [3]: s = 'abc::def::ghi'

In [4]: s.split('::')
Out[4]: ['abc', 'def', 'ghi']

In [5]: s = 'this is a bunch of words'

In [6]: s.split(' ')
Out[6]: ['this', 'is', 'a', 'bunch', 'of', 'words']

All of this is great, and works just fine. But what if I do the following:

In [7]: s = 'a   b   c   d'  # Note: Three spaces between each letter

In [8]: s.split(' ')
Out[8]: ['abc', '', '', 'def', '', '', 'ghi', '', '', 'jkl']

Yuck!  Of course, this is one of those times that the computer does what we tell it, not what we want: We said that every time it encounters a space character, it should give us a new element in the output list. Sure enough, by having multiple space characters between the letters, we end up having lots of empty strings in our resulting list.

What’s worse is that I often use str.split to take input from users, or from files, and break it into individual elements. I’d love to break on one or more whitespace characters, ideally without reverting to the “re” module’s re.split(‘\s’).

Solution: Don’t pass any argument. The first parameter, named “sep”, has a default value of None. And when it has a value of None, str.split does indeed use one or more whitespace characters.  That’s right — str.split, when called with zero arguments, actually does more (and is often more useful) then when called with an argument:

In [10]: s = 'abc \n\n def \n\t ghi jkl\n\n'

In [11]: s.split()
Out[11]: ['abc', 'def', 'ghi', 'jkl']

2. str.split (part 2)

Let’s say you ask a user to enter their name, which you want to split into first and last names. For example:

In [13]: person = raw_input("Enter your name: ") # "input" in Python 3
Enter your name: Reuven Lerner

In [14]: first_name, last_name = person.split()

In [15]: print("First name is '{}', last name is '{}'".format(first_name, 
                                                              last_name))
First name is 'Reuven', last name is 'Lerner'

Line 14 uses Python’s unpacking; since we know that the list produced by person.split() will contain two elements, I can safely assign those two elements into two variables (first_name and last_name).

But what if the person also enters a third name?

In [16]: person = raw_input("Enter your name: ")
Enter your name: Reuven Moshe Lerner

In [17]: first_name, last_name = person.split()
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-17-cf257c8e7997> in <module>()
----> 1 first_name, last_name = person.split()

ValueError: too many values to unpack

Yikes! str.split() returned a list of three elements. And you cannot assign three elements into two variables. (Fine, Python 3 does allow for this, but we won’t discuss this here.)

We can, however, tell str.split how many times it should split. This is done by passing the second, optional argument. If you want to split on all whitespace, as we saw above, then you must explicitly pass None as the first argument:

In [18]: first_name, last_name = person.split(None, 1)

In [19]: print("First name is '{}', last name is '{}'".format(first_name,
 ...: last_name))
First name is 'Reuven', last name is 'Moshe Lerner'

Remember that the second parameter is called “maxsplits”, meaning the number of times str.split should do its thing. This means that the number you give will be the index of the final element in the returned list. In other words: I call person.split(None, 1), which means that I’ll get back a list with two elements — the latter of which has an index of 1.

What if I want to split things in the other direction, such that my first and middle names are in the first variable, and just my last name in the second variable? We can use a variant of str.split called str.rsplit (“right-side split”):

In [20]: first_name, last_name = person.rsplit(None, 1)

In [21]: print("First name is '{}', last name is '{}'".format(first_name,
 ...: last_name))
First name is 'Reuven Moshe', last name is 'Lerner'

3. enumerate

“enumerate” is a built-in function that exists to let us number things as we iterate over them. For example, let’s assume that I have a string, and want to print the letters of the string:

In [22]: s = 'abc'

In [23]: for one_letter in s:
 ...: print(one_letter)
 ...:
a
b
c

What if I want to get the index of each letter, too? I can do this manually:

In [24]: s = 'abc'

In [25]: index = 0

In [26]: for one_letter in s:
 ...: print("{}: {}".format(index, one_letter))
 ...: index += 1
 ...:
0: a
1: b
2: c

Because this is such a common thing that people want to do, we can instead use enumerate, which returns an iterator that produces tuples. Each tuple contains two elements, the first of which is the index and the second of which is the element from the enumerated sequence. Because we know that each tuple will contain two elements, we can grab them with unpacking:

In [27]: for index, one_letter in enumerate(s):
…: print(“{}: {}”.format(index, one_letter))
…:
0: a
1: b
2: c

But wait, what if you are presenting this information to a non-programmer, for whom it seems weird to start numbering with zero? One solution is to send them to a programming course, but if there’s no time, then you can pass “enumerate” a second argument, the number with which numbering should start:

In [28]: for index, one_letter in enumerate(s, 1):
 ...: print("{}: {}".format(index, one_letter))
 ...:
 ...:
1: a
2: b
3: c

Of course, we can start with any number we want:

In [29]: for index, one_letter in enumerate(s, 72):
 ...: print("{}: {}".format(index, one_letter))
 ...:
 ...:
 ...:
72: a
73: b
74: c

4. int()

“int” is the integer type, widely used in Python to represent whole numbers. Many Python developers know that we can turn strings into integers by invoking the “int” function.

Of course, there’s not really an “int function.”  Instead, we’re using “int” as a class to create a new instance of int. So when I say

int('5')

I get a new instance of “int” back, with the value 5.  And when I say

int('12345')

I get back a different instance of “int”, with the value 12345.

But “int” takes a second argument, which lets us tell Python the base of the source data. For example, if I say

int('12345', 16)

then we get back 74565, because we asked Python to give us the value of 0x12345.

You can actually use any number base you want, from 1 through 36 — which is particularly useful for those of us with 36 fingers. But it’s not uncommon for my clients to be reading from files containing hexadecimal numbers. For example, let’s assume that we want to sum the hex numbers on each line of the following file:

10 20 30 4a 5b 6c
ff ef fa 00 20 3b
ab cd ef af be cd

We can do something like this:

In [35]: for one_line in open('hexnums.txt'):
 ...: print(sum([int(one_number, 16)
 ...: for one_number in one_line.split()]))

In other words:

  • We open the file, and read it line by line
  • We split each line on whitespace, resulting in a list
  • We interpret each number in hex
  • The resulting list of integers can then be passed to the “sum” function
  • We print the sum for each line

If you work with files that contain binary, octal, or hex numbers, this can really be handy. To be honest, I don’t do this very much — but I work with a number of companies that do, and for whom this is a real lifesaver.

5. dict.get

Dictionaries are everywhere in Python. They’re easy to define, and easy to work with. For example:

In [36]: d = {'a':1, 'b':2, 'c':3}

In [37]: d['a']
Out[37]: 1

In [38]: d['z']
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
<ipython-input-38-da9eddaf4274> in <module>()
----> 1 d['z']

KeyError: 'z'

Oh, right — but if you request a key that doesn’t exist, you’re going to get a KeyError exception.

Let’s write a little program that lets the user repeatedly query a dictionary. If the user gives us an empty string, then we’ll exit from the loop, but otherwise we’ll either print the value associated with the key, or give an error:

In [42]: while True:
 ...:         k = raw_input("Enter key: ")
 ...:         if not k:
 ...:             break
 ...:         elif k in d:
 ...:             print("d[{}] is {}".format(k, d[k]))
 ...:         else:
 ...:             print("{} isn't a key in d".format(k))
 ...:
Enter key: a
d[a] is 1
Enter key: b
d[b] is 2
Enter key: c
d[c] is 3
Enter key: d
d isn't a key in d
Enter key: <enter>

This works fine, and is a pretty standard way that I’ve seen people check for keys in order to avoid exceptions. But often, the dict.get method will work even better.  Basically, dict.get does the same thing as square brackets ([ ]), except that if the key doesn’t exist, it returns None. For example:

In [43]: while True:
 ...:         k = raw_input("Enter key: ")
 ...:         if not k:
 ...:             break
 ...:         else:
 ...:             print("value of d[{}] is {}".format(k, d.get(k)))
 ...:
 ...:
Enter key: a
value of d[a] is 1
Enter key: b
value of d[b] is 2
Enter key: c
value of d[c] is 3
Enter key: d
value of d[d] is None
Enter key:

Now, you might not want to display None to your users. But you can always trap for None in your code, and then tell the user that the key doesn’t exist.

But dict.get takes a second, optional parameter. If you pass a second argument, you can change the default value from None to something else. For example:

In [44]: p = {'first':'Reuven', 'last':'Lerner'}

In [45]: p.get('first')
Out[45]: 'Reuven'

In [46]: p.get('last')
Out[46]: 'Lerner'

In [47]: p.get('middle', '')
Out[47]: ''

Now I can query the dict for the “middle” key, getting an empty string if the key doesn’t exist.  Another example:

In [48]: countries = {'New York':'USA', 'London':'England', 'Moscow':'Russia'}

In [51]: countries.get('Amsterdam')

In [52]: countries.get('Amsterdam', "I don't know")
Out[52]: "I don't know"

In [53]: countries.get('New York', "I don't know")
Out[53]: 'USA'

In [54]: countries.get('Moscow', "I don't know")
Out[54]: 'Russia'

Notice that when the key does exist, there’s no difference between d[k] and d.get(k).  The question is how you want to deal with a key that doesn’t exist, and dict.get often makes life much easier in these cases.

What optional parameters do you find most useful in Python?

Also: I’m teaching three live Python courses (on functional programming, advanced objects, and decorators) in the next few weeks; early-bird tickets are still available for a limited time. Grab them now, and improve your Python fluency from your own computer.

One-day birthday sale — on July 14th, get 47% off my courses, books, and subscriptions

Today is my birthday!

To celebrate, I’m offering a one-day 47% sale on many of my products:

Just enter the “birthday” coupon code when buying any of these, and you’ll get 47% off. These discounts are good for one day only — Friday, July 14th.

Announcing: Three live Python courses

If you’re like many of the Python developers I know, the basics are easy for you: Strings, lists, tuples, dictionaries, functions, and even objects roll off of your fingers and onto your keyboard. Your day-to-day tasks have become significantly easier as a result of Python, and you’re comfortable using it for tasks at work and home.

But some parts of Python remain difficult, mysterious, and outside of your comfort zone:

  • When you want to use a list comprehension, you have to go to Stack Overflow to remember how they work — to say nothing of set and dict comprehensions.
  • You know that there is a difference between functions and methods, but you can’t quite put your foot on what that difference is, or how Python rewrites “self” to be the first argument to every method.
  • You keep hearing about “decorators,” and how they allow you to do all sorts of magical things to functions and classes — but every time you start reading about them, you get confused or distracted.

Sound familiar? If so, then I want to help.

As you probably know, I spend just about every day at one of the world’s best companies — Apple, Cisco, IBM, PayPal, VMWare, and Western Digital, among others — teaching their engineers how to use Python.

The engineers who learn these techniques benefit by having more “tools in their toolbox,” as I like to put it; when a problem presents itself, they have more options at their disposal. I help them to solve new types of problems, or to solve existing problems more quickly. These engineers become more valuable to their employers, and more valuable on the larger job market.

I’m announcing three courses that you can take, from the comfort of your home or office, using the content I’ve presented to these companies:

  • Tuesday, July 25: Functional programming in Python
    • comprehensions
    • custom sorting
    • passing functions as arguments
    • lambda expressions
    • map, filter, and reduce
  • Wednesday, August 2: Advanced Python objects
    • attributes
    • methods vs. functions
    • class attributes
    • inheritance
    • methods vs. functions
    • descriptors
    • dunder methods
  • Thursday, August 3: Python decorators
    • properties and other built-in decorators
    • writing decorators
    • decorating functions, objects, and methods

Each of these classes will run live, for five hours (with two 15-minute breaks):

  • New York: 7 a.m. – 2 p.m.
  • London: 12 noon – 5 p.m.
  • Israel: 2 p.m. – 7 p.m.
  • Mumbai: 5:30 p.m. – 10:30 p.m.

Each will be packed with lectures, accompanied by tons of live-coding examples, many exercises that you’ll be expected to solve (and which we’ll review together when you’re done), and plenty of time for interactions and questions.  Indeed, please come with lots of questions, to make the class more interesting and relevant.

Each course costs $350, and will give you:

  • Access to the live audio/video/chat feed,
  • PDFs of my slides,
  • the Jupyter notebook I use during my live-coding demos,
  • and solutions to all of the exercises

I’m offering discounts to people who buy more than one course:

  • Buy two courses, and save $100, for a total of $600.  Just use the “2sessions” coupon code when purchasing each one.
  • Buy all three courses, and save $250, for a total of $800.  Just use the “3sessions” coupon code when purchasing each one.

As always, I’m also offering a discount to students; e-mail me, and I’ll send you the appropriate discount code.

Convinced?  I hope so!  View the full course descriptions here, and then register for them:

But wait!  If you register before Monday, July 18th, then you can save 15% more, by purchasing an early-bird ticket.

I’m very excited to be offering these courses.  They won’t be my last ones — but I’ll next be teaching other topics, so if these subjects interest you, you should definitely attend.

I hope that you can join me for these live, online courses.