Registration closes soon for Weekly Python Exercise

Weekly Python Exercise logoWeekly Python Exercise” starts on January 2nd, and is designed to help intermediate Python developers level up their coding skills.  But registration ends on December 20th!  What does the course include, how does it work, and what do you get for $180?  I explain it all in this video…

More than 20 years of programming in Python, and more than 20 years of teaching Python (and other languages) at companies around the world, have gone into this course.  If you want to become a more fluent Python programmer, I’m positive that this course will help.

Not sure?  Have questions? Just e-mail me.  But don’t wait too long…

Focus on the process: Your Python questions, answered

Weekly Python Exercise logoA few weeks ago, I asked subscribers to my free, weekly “Better developers” list to send me their Python problems.  I got about 20 responses from around the world, some more complex than others.  I promised to answer some of them in video.

Why? Because becoming an expert Python developer means understanding, in a deep way, how Python works. The stronger your mental model of Python’s innards, the better you can use the language to solve problems.  And watching someone solve problems, or work their way through problems in real life, helps to develop and improve those mental models.

A large proportion of my teaching takes place via exercises. (And in the case of Weekly Python Exercise, it’s the overwhelming majority of the teaching, as the name implies.) But as important as it is for my students to work through the exercises, it’s also important that I walk them through the process I use when solving the exercise. Learning the correct process is more important than getting the answer right to a specific problem, because once you start thinking in the right way, with an improved mental model, you’ll be able to solve new and different problems.

I’m trying something similar here: People ask questions, and I try to answer them. Sometimes, I’ll hit a brick wall, or an unexpected detour, or I’ll just be surprised. Guess what? This happens to all of us, and it’s part of the process of solving problems. But it’s also part of the fun and excitement of development.

With that in mind, I present the first two problems/questions that my readers submitted:

If you like these questions and walkthroughs, then you’ll love Weekly Python Exercise, starting on January 2nd, a year-long course for intermediate Python developers.

And if you have Python questions you’d like me to answer, join my list and ask away!  If I choose your question, I’ll give you a coupon for 30% off any of my books or courses.

Registration is open for the January 2018 cohort of “Weekly Python Exercise”

Weekly Python Exercise logoWhen I started to program in Python more than 20 years ago, there weren’t a lot of resources out there. Sure, there were a handful of books, and a few Web sites, and (of course) forums and mailing lists. But that was about it.

Nowadays, Python is white-hot, with companies, universities, and individuals learning the language. Whether you’re a data scientist, Web developer, system administrator, test author, or hobbyist, you’re probably learning Python. Not surprisingly, there are oodles of books, courses, and resources for people learning the language.

And then? Well, then you’re on your own.

The problem is that there aren’t a lot of resources for intermediate-level Python developers to improve their fluency with the language. Sure, you can read through Stack Overflow and Reddit, as well as read blogs and watch conference videos. But given how much is out there for beginners, you would think that there would be something to help people who have already taken a Python course, and who are ready for the next stage.

The good news: I’m here to help.

More specifically: My course, Weekly Python Exercise, is just what the doctor ordered. WPE makes you a more fluent, more knowledgeable Python developer by forcing you to learn and practice your skills every week.

Every Tuesday, you’ll receive a new Python programming challenge. These challenges are designed to be relatively short and small, taking less than an hour of your time. The following Monday, you’ll receive my suggested answer, along with a description of why I chose to solve the problem in the way I did.

Along the way, you’ll learn about — or improve your understanding of — such topics as:

  • nested functions
  • iterators
  • generators
  • decorators
  • object inheritance
  • properties
  • threads and processes
  • the “logging” module
  • the “pickle” module

Most importantly, you’ll be solving these problems along with other developers from around the world. You’ll be able to participate in a forum, in which you can discuss possible solutions. Such interactions are the key to effective, deep learning, and I’m going to do my best to encourage active participation and discussion in our forum.

Moreover, I’ll offer live, online office hours every few weeks. This will give you a chance to ask me about the exercise questions, as well as the solutions. It’s your chance to discuss the exercises in greater depth.

I’m also going to include several multi-week projects, in which successive exercises build on previous ones. This will allow us to build larger projects, while remaining within the constraints of simple, weekly exercises.

This is the second time I’m offering Weekly Python Exercise. I’ve incorporated many suggestions I received from the first set of participants, making this the best way I can think of to improve your Python skills. I’ve improved the exercises, improved the social interactions, and made myself more available to discuss the questions (and answers) with all of the participants.

If you have a good knowledge of Python but want to make it even better, then Weekly Python Exercise is for you. But — and here’s the most important thing — the next cohort will be starting on January 2nd. After that, registration for WPE will be closed.

If you want to level up your Python, now is the time to sign up. I’m not sure when I’m next going to offer WPE, which means that once the course starts, you won’t be able to join.

If you have read a Python book, or taken a Python class, and want to know how to really “get” Python, look no further: Weekly Python Exercise is coming.

Click here to register for the January 2018 cohort of Weekly Python Exercise.

Wondering if WPE is for you? Contact me at reuven@lerner.co.il., and I’ll personally read (and respond to) your question.

If you’re on the fence, and want to sample WPE, you can just enter your e-mail address below, and receive (free of charge) two exercises from the most recent cohort:


A very sad day — the end of Linux Journal

In 1995, just before moving to Israel, I was working for Time Warner in New York City.  One day, I received email from a company called SSC, publishers of numerous “cheat sheets” for Linux users and programmers. They were about to publish a quick-reference guide for GNU Emacs, and since I was the maintainer of the Emacs FAQ, they wanted my input and thoughts.

I gave them my feedback, and the people at SSC were so thankful for my help that they offered to give me, free of charge, any 10 items from their online catalog. In this way, I got a bunch of quick-reference cards, as well as several issues of the then-new publication, Linux Journal.

I had been using Unix since 1988, and had just started to use this new, open-source version.  (I ordered a distribution Linux from a small company in Connecticut known as “Red Hat,” which made for a fairly straightforward installation on a PC. And by “fairly straightforward,” I mean that it only took a few days of configuration to get X Windows to work on my hardware.)  So I was excited to read through Linux Journal.  I loved what I read, and was in awe of the knowledgeable columnists they had.

In one of those issues was an ad, saying that the magazine would soon be starting a spinoff known as “Websmith,” all about a new phenomenon known as the “World Wide Web.”

Now, I loved to write: I had edited the student newspaper while studying at MIT, and I had always thought that it would be fun to write a column. So I e-mailed the editor of Websmith, asking if they would be interested in having me write a column about CGI programming — which was then the latest and greatest way to create what we now call “dynamic Web content.”  After all, I had been doing Web development since early 1993, back when I set up one of the first 100 sites in the world; I figured that my technical and writing experience could be of help.

The editor said “yes,” and my column — which he called “At the Forge,” fitting in with Websmith’s blacksmith theme — was off to the races.

Websmith didn’t survive for very long; it seems that the World Wide Web wasn’t interesting or big enough to support a business.  (Who knows?  Maybe that’ll change some day.)  But some of us who were writing for Websmith discovered that our columns and articles were simply incorporated into Linux Journal’s next issue, continuing from there.

In other words: I was suddenly and unexpectedly a Linux Journal columnist — a position that I have held, continuously and proudly, since 1996.

It’s thus extremely sad for me to know that Linux Journal is no more.  The magazine’s publisher, Carlie Fairchild, announced the magazine’s closure to the staff and writers earlier this week, and is telling subscribers today.

Just to put things in perspective: Before I met my wife, before my children were born, before I started my 11-year-long PhD program, I was writing for Linux Journal.

For as long as we have been married, my wife has heard the following, typically several times each month, each time with increased urgency: “I have to work on the column, which was due a few days ago. My editor is going to kill me.”

Ditto for my children, whose definition of “deadline” has definitely been affected — for the worse — by my monthly worrying, complaining, and late-night writing.

But I’m not complaining. Writing for Linux Journal for so long has been a defining part of me, and my career, for a very, very long time.

Thanks to LJ, I have had a chance to write, which I so love to do. I have learned a huge amount through my writing, because each column required that I spend time learning before I could teach.

Thanks to LJ, I’ve even gotten some clients, who contacted me as a result of my articles.

Thanks to LJ, my wife and I were invited to visit Alaska and the Caribbean, when I was a featured speaker with “Geek Cruises.”

Thanks to LJ, I got many free books and other resources, was invited to speak at conferences, and was (much to my pleasant surprise) recognized at technical conferences, even when I didn’t speak there.

And of course, thanks to LJ, I’ve gotten to work with some amazing people.  I’ve worked with a variety of editors, the most recent being Jill Franklin, all of whom were talented, helpful, and tolerant of my flexible interpretation of deadlines. They all gave me total freedom to write about whatever topic I wanted, whenever I wanted.  I got to explore all sorts of great topics that were of interest to me, and to my clients, and (I believe) to my readers.

I received e-mail from people all around the world who read my columns, which was deeply satisfying and gratifying.  At conferences, people would see my name badge and tell me that they had been reading my columns for many years.  Even now, as I publish a weekly newsletter for programmers, I often get messages from people saying that they read my column in Linux Journal, and are happy to reconnect with me.

The publishing industry is changing, and that reality is hitting publishers big and small. Indeed, earlier this week, Time Inc. (yes, where I worked when SSC first contacted me) sold itself to another company. The combination of publishing economics, along with the plethora of free, online content for open-source enthusiasts, makes it hard to produce a profitable magazine with top-notch technical content.

Consider the world in 1996: Linux was an oddball operating system, supported by no major manufacturers and seen as a hacker’s plaything. Perl and Python were used by lots of individuals, but not for too many serious applications. MySQL was free, but wasn’t open source, and there weren’t any open-source options for people who wanted to use a database.  And of course, Web applications were in their infancy; my “form-mail” program, which was used by countless sites to send e-mail (before it was taken and modified for the worse by Matt’s Script Archive) remained cutting edge for quite some time.

Things have changed a great deal since then. My phone uses Android, a form of Linux. When the flight attendant reboots the in-flight entertainment system, I see that it’s running Linux, as well. My 14-year-old daughter is learning Python. Companies from Apple to IBM, Cisco to PayPal, Ericsson to VMWare, in the US, Europe, Israel, and China, ask me to teach Python and Git to their employees. Who knew that the technologies I learned, and learned to love, so many years ago, would be so dominant in the world today?

And thus, while Linux Journal might have failed as a business, it succeeded in its mission: To spread the word of Linux and open-source software, to help people to understand, implement, and use open-source technology, and to bring these technologies into the mainstream as a legitimate alternative to the then-dominant commercial offerings.

To my many readers: Thanks for your support over the years. (And you can keep getting weekly writing from me via my “Better developers” newsletter.)

To the amazing staff at Linux Journal, over many years and in many iterations: Thanks for putting up with my delays, and for doing such a great job with my text.  I can’t imagine how they edited a magazine with such in-depth technical content, but they did, and did it well.

To my family: Thanks for putting up with my loud, monthly stresses, including the many times in which I wondered, out loud, how they could possibly think to have a columnist who cannot get his software to work.

Oh, and to the three (!) journalists who e-mailed me in the last 10 days, asking to whom they could pitch stories for Linux Journal: Sorry, folks.  You’re a bit too late.

RIP, Linux Journal.  It has been a fun, wild ride.

Black Friday sale: 40% off my Python and Git books and courses

Happy Black Friday!  OK, I know: It’s not really a holiday.  And I don’t even live in the US.  But somehow, Black Friday has become a world-wide shopping and discount phenomenon.

Like many other independent authors and trainers, I’m offering big discounts today — 40% off all of my books and courses.

The discount is nice, of course. But even nicer is having a more satisfying and lucrative career, which happens when you’re able to solve more and bigger problems than before.

Here’s a list of the products I’m offering:

  • Weekly Python Exercise, my year-long course that improves your Python via weekly exercises, group discussion, and live office hours, has a new cohort starting on January 2nd.  You can only join WPE as part of a cohort, part of my overall goal of getting people to learn from one another.  Which means that if you wait too long, you’ll miss the chance to level up your Python skills in the best way I know.
  • My ebooks, Practice Makes Python and Practice Makes Regexp, are available, in book and complete (book + videos) form.  Better yet: You can now preview a number of the videos!
  • My recorded courses on Understanding Git, functional Python, object-oriented Python, and decorators in Python are for sale, at 40% off.  These courses are currently recordings from live versions I gave earlier this year, and have helped many others to improve their skills.  However, I’m revamping these courses to be more watch-on-your-own-time friendly, and you’ll get a free upgrade to the newer version, when it comes out, as part of my “forever free” course purchase policy.

All of these are available at http://store.lerner.co.il/.  And as always, I offer a 100% money-back guarantee if you’re dissatisfied.

Questions? Just e-mail me at reuven@lerner.co.il!

Aha! Preview this week’s live, online Python courses

In just 48 hours, I’ll be starting my latest round of live, online courses. Wondering what it’s like to take an online course from me? Or perhaps you’re wondering what sorts of topics I’ll discuss in my “Python dictionaries” and “Python functions” courses? Well, wonder no more; here’s a short preview of my teaching style, and the sorts of things I intend to demonstrate in my courses:

Preview: Reuven’s October 2017 live, online courses about Python and Git from Reuven Lerner on Vimeo.

If you are a beginning or intermediate Python developer, then you’ll become a more effective and fluent developer — good not only for your current employer, but for your career — thanks to my courses. And we’ll have lots of fun along the way. There will be plenty of time for exercises, questions, and comments, to ensure that you understand these technologies well. You’ll return to work the next day able to do more, and more quickly, than before.

Any questions? Just send me e-mail , and I’ll be happy to answer.

I look forward to seeing you this week (for my Python courses) and next week (for my two-day “Understanding Git” course)!

Reuven

Announcing: Three new live courses, to level up your Python and Git skills

  • Confused by Python dicts, or wondering how you can take advantage of them in your programs?
  • Do you wonder how Python functions work, and how you can make them more “Pythonic,” and easier to maintain?
  • Do you wonder why everyone raves about Git, when it seems impossibly hard to understand?  Cloning, pulling, and pushing mostly work… but when they don’t, Git seems like magic, and not the good kind.

If any (or all) of the above is true, then you’ll likely be interested in one or more of the live, online classes I’m teaching later this month:

  1. Python dictionaries, on Wednesday, October 25
  2. Python functions, on Thursday, October 26
  3. Understanding Git, on Tuesday, October 31 and Wednesday, November 1

Each of these classes is live, with tons of live-coding demos, exercises, and time for Q&A. My goal is for you to understand these technologies, how they work, and (most importantly) how you can use them effectively in your work.

Previous classes have been small and highly interactive. These are the same classes I give to some of the best-known companies in the world, such as Apple, Cisco, IBM, PayPal, VMWare, and Western Digital; I’m sure that you’ll enjoy yourself, and come out a better engineer.

Better yet: Buy a ticket by this Friday, and you’ll get a substantial (20%) discount on the ticket price.

This is not a recorded class (although recordings will be available later on).  I’ll be speaking and interacting the entire time, giving you a chance to get your questions answered.  I want to make sure you really understand what’s going on, and will answer any questions you have!

Speaking of which: If you have questions, just e-mail me at reuven@lerner.co.il, and I’ll do my best to answer.

And if you’re a student, ask me for a coupon code that will give you a substantial discount off of the ticket price.

I hope that you can join me for one or more of these classes!

Level up your Python, with three new courses

Back in July, I gave three live, online courses: Object-oriented Python, functional Python, and Python decorators.  I have long found that all three subjects are misunderstood by many Python developers, and I wanted to help people to understand how and when to use each one.

I’m pleased to announce that recordings from all three courses are now available for sale.

These are the same courses that I give all over the world to engineers at such companies as Apple, Cisco, Ericsson, IBM, VMWare, and Western Digital.

If you have ever wanted to level up your understanding of these topics, I think that my courses will really help you out.  Not only do I explain what’s going on, but I also ask you to do exercises to help cement these ideas in your mind. Of course, I go over every exercise as well, and provide the Jupyter notebooks and files that I created when doing so.

Want to buy learn more? Just click on the appropriate link for each one:

If you’re a student, then e-mail me for a discount code good on any (or all!) of these courses.

If you have any questions about the courses, just reply to this e-mail, or contact me at reuven@lerner.co.il.

I’ll be offering some more live courses later this month (October); I’ll be posting dates and topics on Monday of next week, so stay tuned for even more Python goodness!

My favorite terrible Python error message

Students in my Python classes occasionally get the following error message:

TypeError: object() takes no parameters

This error message is technically true, as I’ll explain in a moment. But it’s surprising and confusing for people who are new to Python, because it doesn’t point to the source of the actual problem.

Here’s the basic idea: Python methods are attributes, which means that when we invoke methods, Python needs to search for the attribute we’ve named. In other words, if I invoke:

o.m()

then Python will first look for the “m” attribute on the “o” object. If “o” has an attribute named “m” (i.e., if hasattr(o, ‘m’) returns True) then it retrieves the attribute’s value, and tries to call it.

However, Python methods aren’t defined on individual objects. They’re defined on classes. Which means that in almost all cases, if “m” is an actual method that can be invoked on “o”, there won’t be any “m” attribute on “o”.  Instead, we’ll need to look at type(o), the class to which “o” belongs, and look there.

And indeed, that’s how attributes work in Python: First search on the named object. If the attribute isn’t there, then look at the object’s class.  So we look for “m” on o’s class.  If the attribute is there, then it is invoked.  That’s what happens in normal method calls.

But say that the attribute isn’t on the class, either. What then? Python continues its search, looking next at the class from which type(o) inherits — which is located on the attribute type(o).__bases__.  This is a tuple, because Python classes can inherit from more than one parent; let’s ignore that for now.

Most classes inherit from the base object in the Python universe, known as “object”.  In Python 3, if you don’t specify “object” as the base from which you inherit, then it’s done for you automatically. In Python 2, failing to specify that a class inherits from “object” means that you have an “old-style class,” which will operate differently. I continue to specify “object” in my Python 3 classes, partly out of habit, partly because I think it looks nicer, and partly because I want my code to be compatible across versions as much as possible.

What happens if the attribute doesn’t exist on “object”?  Then we get an “attribute error,” with Python telling us that the attribute doesn’t exist.

However, this isn’t what happens in the case of the error message I showed:

TypeError: object() takes no parameters

This error message happens when you try to create a new instance of a class. For example:

class Foo(object):
    pass

If I say

f = Foo()

then I don’t get any error message. But if I say

f = Foo(10)

then I get the TypeError.  Why?

Because Python objects are created in two stages: First, the object is created in the __new__ method. This is a method that we almost never want to write; let Python take care of the allocation and creation of new objects.

However, __new__ doesn’t immediately return the object that it has created. Rather, it first searches for an __init__ method, whose job is to add new attributes to the newly created object. How does it look for (and then invoke) __init__?  It turns to the new object, which I’ll call “o” here, and invokes

o.__init__()

So, what happens now? Python looks for “__init__” on “o”, but doesn’t find it.  It looks for “__init__” on type(o), aka the “Foo” class, and doesn’t find it.  So it keeps searching, and looks on “object” for an “__init__” attribute.

Good news: object.__init__ exists!  Moreover, it’s a method!  So Python tries to invoke it, passing the argument that I handed to Foo (i.e., 10).  But object.__init__ doesn’t take any arguments. And thus we get the error message

TypeError: object() takes no parameters

What’s especially confusing, for me and many of my students, is that Python doesn’t say, “object.__init__()” takes no parameters. So they’re not sure how object figures into this, or where their mistake might be.

After reading this, though, I’m hoping that you can guess what it means: Simply put, this error message says, “You forgot to define an __init__ method on your object.”  This can be out of forgetfulness, but I’ve also seen people forget one or more of the underscores on either side of “__init__”, or even (my favorite) define a method called “__int__”, which is great for converting objects into integers, but not for initializing attributes.

So, is the error message wrong? No, it’s perfectly logical. But as with many “perfectly logical” things, it makes sense after you are steeped in the overall logic of the system, and tends to confuse those who most need the help.

foo(y=y), and similar code that confuses Python newbies

Let’s define a simple Python function:

In [1]: def foo(x):
 ...: return x * x
 ...:

In [2]: foo(5)
Out[2]: 25

In [3]: foo(10)
Out[3]: 100

As we can see, this function has a single parameter, “x”.  In Python, parameters are local variables whose values are set by whoever is calling the function. So we know that x is a local variable in the function “foo” — which means that “x” doesn’t exist outside of the function “foo”.

I can define a more complex function, which has two parameters:

In [4]: def mul(x, y):
 ...: return x * y
 ...:

In [5]: mul(5, 3)
Out[5]: 15

In [6]: mul(6, 8)
Out[6]: 48

In this example, the “mul” function must take two arguments. “x” and “y” are both local variables within “mul”, meaning that they only exist within the “mul” function.

What happens if I define a “y” variable outside of our “mul” function?  That would be a global variable, which shouldn’t be confused with a local one. Local variables exist only within a function, whereas global variables exist outside of functions. For example:

In [7]: y = 100

In [8]: mul(5,3)
Out[8]: 15

I have thus defined the global variable “y”, which has the value 100. Inside of the function, Python ignores our global “y” variable, because according to Python’s scoping (LEGB) scoping rules, local variables get priority.

So far, so good. But now let’s make things a bit more complex: Let’s change our “mul” function such that the “y” parameter takes a default value. In other words, I’ll be able to call “mul” with two arguments (as before) or with one argument (and thus use the default value for “y”):

In [9]: def mul(x, y=10):
 ...: return x * y
 ...:

In [10]: mul(5)
Out[10]: 50

In [11]: mul(7)
Out[11]: 70

In [12]: mul(5,7)
Out[12]: 35

Notice that once again, our global “y” value has no effect whatsoever on our local “y” parameter: Inside of the function, when Python looks for the value of “y”, it finds the local variable by that name, and uses the value accordingly.

We can even go so far as to give both “x” and “y” default values. Here’s how that would look:

In [13]: def mul(x=3, y=10):
 ...: return x * y
 ...:

In [14]: mul()
Out[14]: 30

In [15]: mul(5)
Out[15]: 50

In [16]: mul(7,3)
Out[16]: 21

Let’s say I want to use the default value of x, but pass a value to y.  How can I do that?  By calling the function, but explicitly naming the “y” parameter, along with a value:

In [17]: mul(y=3)
Out[17]: 9

In [18]: mul(y=5)
Out[18]: 15

What happens if I do this:

In [19]: mul(y=y)

In this case, I’m calling the function, and I’m saying that I want to set the “y” local variable to a value.  But what value am I giving it?  Well, I’m not in the function when I call the function.  And thus the only “y” value available to me is the global “y” variable that I had set earlier.

In other words: I want to call the “mul” function, setting the “y” parameter to the current value of the “y” global variable.

Why would we do such a thing? Relative newcomers to Python find this hard to read, and wonder why (or “y”) we use the same variable name on both the left and right sides.  And the answer is… it’s often easier and more convenient.  The example I’ve provided here is contrived, but there are cases in which you might want to define a function called “key” that sorts your list in a particular way.  With that function defined, you can then say

mylist.sort(key=key)

I personally prefer to define my sorting functions using a different convention, starting with the word “by”, so I can say something like

mylist.sort(key=by_last_name)

At the end of the day, this “y=y” code makes sense if you understand Python’s scoping rules, as well as how function defaults are defined and assigned. Want to know more? I’m giving a live, online course about Python functions (including exactly these topics) on Sunday, August 13th. More details are here, and early-bird tickets are still available!