Idioms and Anti-Idioms in Python

Idioms and Anti-Idioms, AKA Do and Don't

• Welcome
• Gimmick -- Charmed quotes

Python

• Few gotchas...
• ...but not zero
• Most are easy to avoid...
• ...if you know about them.

Exceptions

• Primary method of dealing with errors
• Flexible
• Good opportunity to shoot yourself in foot...
• ...without knowing about it (bug only shows up rarely.)

Exceptions -- Catching Too Much

• Classical case: 'except:'
• Will catch anything
• Including most bugs...
  • NameError, AttributeError...

Exceptions -- Catching Too Much -- Example

try:
   f = opne("file")
except:
   sys.exit("no such file")

Exceptions -- Catching Too Soon

• The slogan:
  • Don't catch errors you can do nothing about
• Catching exceptions should by the 'user'
• The point where the value is *used*
  • Rather than passed on

Exceptions -- Catching Too Soon -- Example

def readlinesfromfile(file):
    try:
        return open(file).readlines()
    except IOError:
        pass # do what?

• What can we do?
  • Return empty list? bad
  • Print warning? what if it's one of several possibilities
  • Exit? NO!
  • Raise our own exception? Losing information

Catching multiple exceptions

• Spot bug here:

try:
   fp = open("file")
except IOError, OSError:
   print "could not open file"

Catching multiple exceptions (cont'd)

• Bug:
• Only IOError gets caught...
• and exception value is put in OSError
• But most exceptions are IOError :(
• Likely to not discover this bug

Catching multiple exceptions (cont'd 2)

• Correct way

try:
   fp = open("file")
except (IOError, OSError):
   print "could not open file"

Catching NameError

try:
   import foo
except ImportError:
   pass

try:
   foo.Function()
except NameError:
   pass # some replacement

• Bad idea!

Catching NameError (cont'd)

try:
   import foo
except ImportError:
   foo = None

if foo is not None:
   foo.Function()
else:
   pass # some replacement

• If foo.Function() sometimes has a NameError, we won't mask it...
• ...or if we misspell 'foo'

Importing Modules -- A Review

• 'import module'
• 'from module import name1, name2'
• Only imports once (or does it?)

Importing __main__

• __main__ is where the 'script' is executed
• Avoid the temptation to import __main__
• Put common function in a named module
• Then your code will be more useful

Importing a File Twice

• But it can't be, can it?
• Importing a script into itself

# file: hello.py
import hello
class Foo: pass

• Two 'Foo's, same definition, different class!
• If your sys.path includes packages...
• ...you can import a module once from a package and once plain

Importing *

• Don't do it
  • Don't do it
• Classic mistake:

from os import *

fp = open("file") # works
fp.readline() # fails with a weird error...?

• os.open returns a file descriptor (number)

Importing * Inside Functions

• Just invalid Python...
• ...but happens to work in 1.5.2...
• ...and sometimes in 2.1...
• ...never in 2.2.
• Just Say No

Importing Names

• from foo import name1, name2
• Not a bad idea always
• But be careful of repercussions:
• modules sometimes change things inside
• You won't see those changes
• Opportunity for inconsistency!

Reloading

• reload(module) -- reread module from file
• Useful in long running processes?
• Doesn't play nice with 'from import name'
• Beware of exceptions: the new classes are different from old classes

exec, execfile and eval

• Execute arbitrary Python code
• No-cost scripting language for applications
• But easy to shoot one's self in the foot

exec, execfile and eval -- Modify namespaces

• They modify the namespace they're in
  • ...but not always.
• Depends on global vs. inside functions
• Use with care -- or with explicit dictionaries

exec, execfile and eval -- Inside functions

• Unadorned exec is invalid inside functions
• execfile and eval play badly with local var. optimisation
• Always use with explicit dictionary

Conclusion: recommended usage

• d={};exec "code" in d
• d={};execfile("file", d)
• d={};eval("expression", d)
• Sometimes useful to pre-populate dictionary

exec, execfile and eval -- Restricted Execution (Don't)

• rexec never was audited
• History of holes
• Dangerous to allow arbitrary code
• DoS attacks not defended against at all
• Recursion

Syntax

• Python syntax regular and nice...
• ...but not perfect.
• Some care needed.

Syntax -- Tabs and Spaces

• Use Tabs
• Or use spaces
• But don't mix them...
• ...ever!
• Invites bugs

Syntax -- Backslash Continuations

# Extra newline
r = 1 \

+2

# Missing backslash in long series
r = 1 \
+2 \
+3 \
+4 
+5 \
+6

• Both *silently* do the wrong things

Syntax -- Backslash Continuations (cont'd)

• Better

# Extra newline
r = (1

+2)

# Long series
r = (1
+2
+3
+4
+5
+6)

Hand Hacking Batteries

• Don't write os.path functions yourself
  • os.path.join especially
• min, max
• urlparse
• Skim through modules list. A lot.

Further Reading

• http://aspn.activestate.com/ASPN/Python/Reference/Products/ActivePython/howtos/doanddont/doanddont.html
• http://www.amk.ca/python/writing/warts.html
• http://www.python.org/doc/essays/styleguide.html

Questions?

Bonus Slides

Packages and __init__.py

• Packages are determined by __init__.py files
• Temptation to put code in __init__.py
• But two namespaces mix: __init__'s and filesystem's
• Put comments, docstring and __all__

Type Checking

• Python's typing is highly dynamic
• Capability-based, not class-based
• Explicit type checks hurt code usefulness
• (common use -- proxies, for testing)

Type Checking -- Example

• Here's what not to do:

class Foo:
    def __init__(self, i):
        if type(i) is types.StringType:
            self.content = open(i).readlines()
        elif type(i) is types.ListType:
            self.content = i

• (inspired from a question on #python)
• More badness than you can shake a stick at.

Type Checking -- Example -- Fixed

class Foo:
    pass

class FooFromFile(Foo):

    def __init__(self, filename):
         self.content = open(filename).readlines()

class FooFromList(Foo):

    def __init__(self, list):
         self.content = list

Private __Attributes

• Useful in deep hierarchies to keep attributes separate
• Mangle only class name -- *not* module name
• Makes it harder to test
• Makes it harder to hand-hack for debugging

Using Mutable Default Arguments

def foo(l=[]):
    l.append(5);return l

• Will modify the same list.
• If you want that -- use object, class, not that hack.

def foo(l=None):
    if l is None: l=[]
    l.append(5);return l