Review

  • Last week read from and wrote to files.
  • Reading and writing files is one of the most common things for a program to do

Functions

  • Functions are a way to group instructions together
  • The grouped instructions have a name
  • A function is called using its name
  • The def keyword is short for define
    • def is used when you want to define a function by name.

Here's an example of a Python function:

def do_something(): 
    print ('Okay, how about print "do something."')    
    print ("Now I'll print something else")    
  • What happens if you put this code in a file and execute it?
mikematera:~/workspace/Week6 $ python3.6 ./examples.py
mikematera:~/workspace/Week6 $ 

The code does nothing!

  • The function contains the two print instructions
  • You can tell that the instructions are inside of the function because they're indented to the right
  • The code does nothing because functions have to be called
  • Adding a function call fixes the problem:
# This is the functions definition
def do_something(): 
    print ('Okay, how about print "do something."')    
    print ("Now I'll print something else")    
 
# This is where the function is called.
do_something()

Now the program operates as expected:

mikematera:~/workspace/Week6 $ python3.6 ./do_something.py
Okay, how about print "do something."
Now I'll print something else
mikematera:~/workspace/Week6 $ 
  • Most of the work of your programs so far has been done using functions.
  • You're already familiar with the functions:
    • print()
    • input()
    • write()
    • readline()
    • read()
  • Each of these functions has a def somewhere.
  • The great thing about functions is that you can call them as often as you like
    • They help you reuse code so you don't have to write the same things over and over

If I want to do_something() more than once I could do this:

# This is the functions definition
def do_something(): 
    print ('Okay, how about print "do something."')    
    print ("Now I'll print something else")    
 
# This is where the function is called.
do_something()
do_something()
do_something()
do_something() # That's a lot of something!
  • Each time I call do_something() the function is executed.

Function Parameters

  • Functions can take parameters also called arguments
  • A parameter is a variable that holds the value that is passed to the function
  • Parameters make functions very flexible
    • The print() function takes stuff to print as parameters
    • The input() function takes a prompt for the user as a parameter

Here's an example of using parameters with a function:

print ('Hello World')  # 'Hello World' is a parameter
print ('Hello', adjective, 'world') # There are three parameters 
  • So, you're already familiar with giving arguments to functions.
  • Functions receive arguments as a part of their definition
  • Function definitions can declare a parameter list

The simplest way to have parameters is to name them:

def print_nametag(name): 
    print ('Hello, my name is', name)
 
# Prints a nametag with my name.
print_nametag('Mike')
  • As you know, functions can take variables as parameters
def print_nametag(name): 
    print ('Hello, my name is', name)
 
# This is the same as the program above.
my_name = 'Mike'
print_nametag(my_name)
  • You can do math and other operations in function arguments
def print_nametag(name): 
    print ('Hello, my name is', name)
 
my_name = 'Mike'
print_nametag(my_name + " Matera")
  • Functions can have multiple arguments
  • You must name each argument in the function definition
# This function takes two arguments.
def print_nametag(first, last): 
    print ('Hello, my name is', first, last)
 
# Two arguments must be supplied. 
print_nametag("Mike", "Matera")
  • Separate the named arguments with a comma.
  • Now you have to call the function with two arguments.
  • Must functions take a specific number of arguments.
  • If you do not provide a function with the right number of arguments it's an error
>>> print_nametag('Mike')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: print_nametag() missing 1 required positional argument: 'last'
>>> 
>>> print_nametag('Mike', 'Matera', 'Extra')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: print_nametag() takes 2 positional arguments but 3 were given
  • You should be familiar with these error messages, you'll get them a lot!
  • You may be wondering, “what about print() it takes any number of arguments?”
  • print() is a variadic function.
    • Variadic functions can take any number of arguments.
  • Here's an example of a variadic function:
def variadic_nametags(*names) :
    first, last = names 
    print ('Hello, my name is', first, last)
 
variadic_nametags("Mike", "Matera")
  • Notice the following:
    • The *names argument begins with a star (*) character
    • We get first and last in the same way we expand argv
  • What happens if we use the power of a variadic function to add more names?
def variadic_nametags(*names) :
    first, last = names 
    print ('Hello, my name is', first, last)
 
variadic_nametags("Mike", "Matera", "Bonnie", "Smith")

Running that code gives an error:

Traceback (most recent call last):
  File "./examples.py", line 11, in <module>
    variadic_nametags("Mike", "Matera", 'Bonnie', 'Smith')
  File "./examples.py", line 8, in variadic_nametags
    first, last = names 
ValueError: too many values to unpack (expected 2)
  • In order to fully appreciate the use of a variadic function and *args you need to understand loops.
  • Loops are a way to do the same work over and over.
  • We'll talk about loops in a few weeks

Here's what variadic_nametags() should look like:

def print_nametags(*names) :
    for name in names : 
        print ('Hello, my name is', name)
 
print_nametags("Mike", 'Bonnie', 'Jun', 'Dweezil')

Function Return Values

  • The caller of a function passes arguments to the function
  • The caller may also receive return values
  • Return values are values that are passed from the function to the caller
  • A function can return anything you like (or nothing)

Here's an example of functions with return values:

def add_two_numbers(a, b) :
    return a + b 
 
sum = add_two_numbers(10, 100)
print (sum)
  • Functions in Python may return multiple values.
    • That's a big improvement over C and C++!

Here's a function that returns two values:

def do_math(a, b) :
  sum = a + b 
  product = a * b 
  ratio = a / b 
  return sum, product, ratio 
 
ret1, ret2, ret3 = do_math(12, 45)
print ('The sum/product/ratios are', ret1, ret2, ret3)
  • Just like with passing arguments you have to accept the right number of variables in a return.
    • There's one exception: You can accept just one
>>> sum, prod, rat = do_math(1, 3) 
>>> print (sum, prod, rat) 
4 3 0.3333333333333333
>>> sum, prod = do_math(1, 3) 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: too many values to unpack (expected 2)
>>> all = do_math(1, 3)                                                                                                                
>>> all
(4, 3, 0.3333333333333333)
  • Notice something?
    • When do_math() returns into a single value the value becomes a tuple
    • We'll talk more about tuples and lists later

Designing with Functions

  • Functions give you a way to name snippets of code
  • You should chose a name that describes what the function does.
  • Your functions should do something that's useful.

Here's an example of a function that reads the first line of a file:

def read_first_line(filename) :
  file = open(filename, 'r') 
  line = file.readline()
  file.close()
  return line
<code>
 
Of course you could also write that function like this: 
 
<code python>
def read_first_line(filename) :
    with open(filename, 'r') as file : 
        return file.readline()

Isn't that nicer? You don't have to remember to close the file. It's done for you when you leave the with/as block.

  • What a function does isn't always clear from the name
  • This is especially true when you start to write more complex functions
  • Functions can have docstrings to help users understand what they do.

Here's an example of a docstring for the read_first_line() function:

def read_first_line(filename) :
    ''' Reads and returns the first line of a file.''' 
    with open(filename, 'r') as file : 
        return file.readline()
  • When it's not obvious what the function arguments are you should be more descriptive in your docstring.
  • It's also important sometimes to describe what the return value is (or might be)
def read_some_line(file, offset) :
    '''
    Reads a line in the file, without affecting the place that the program was reading
 
    Arguments:
      file -- A file object that is opened for reading 
      offset -- The place where the line starts 
 
    Returns: 
      A string containing the desired line. 
    ''' 
    saved_place = file.tell()
    file.seek(offset)
    line = file.readline()
    file.seek(saved_place)
    return line
  • There are official Python guidelines for how to use docstrings in functions
  • It will be difficult at first to know what code you should put in a function
    • Don't worry, it's hard for everyone at first!
    • Practice, practice, practice!

Function Variables and Scope

  • So far, I've tried to use different variable names inside and outside of functions.
  • That's good practice, but it's not always possible or effective to do so.
  • When you use a variable inside of a function you get a new variable.
    • This can lead to confusion until you get the hang of it.
  • All programming languages have rules about where variables are valid.
  • Validity of variables is called scope

Here's an example showing where variable scope matters:

myvar = 10 
print (myvar)
 
def myfunc(myvar) :
    myvar = myvar + 1
    print (myvar)
 
myfunc(myvar)
print (myvar)

What happens when you run this code?

$ python3.6 ./scope.py 
10
11
10

What happened to the value 11? We incremented myvar in myfunc()!

  • The program above makes sense when you understand that there are really two different variables named myvar
  • The difference is illustrated in the code snippet below.

This program is identical to the one above:

myvar = 10 
print (myvar)
 
def myfunc(myvar_inside_of_myfunc) :
    myvar_inside_of_myfunc = myvar_inside_of_myfunc + 1
    print (myvar)
 
myfunc(myvar)
print (myvar)
  • Now do you see why the code prints 10, 11, 10?
  • Even though the name myvar is used inside of myfunc it's a different variable.
  • Variables declared inside of functions are private to that function.
    • They go away when the function ends
    • Code outside of the function can't access them

Here's an example of an error:

def add_two_numbers(a, b) :
    c = a + b 
    return c 
 
sum = add_two_numbers(1, 3)
print (c) # ERROR! c is private to add_two_numbers()
  • Code outside of add_two_numbers() has no access to c
  • The reverse is not true
    • Code inside of a function can access variables in global scope.
  • Variables declared in code that's not a part of any function are in global scope.
  • When you use a global variable you must say so.
def add_two_numbers() :
    global a, b, c 
    c = a + b 
 
a = 10 
b = 20 
add_two_numbers()
print (c)
  • The global keyword tells Python that add_two_numbers() is operating on global variables
  • Without the global keyword the program will cause an error because a and b aren't defined.

Avoiding Global Variables

  • It's a good idea to avoid global variables when you can
  • Global variables make a program harder to understand
    • Python at least makes it obvious what global variables you intend to use
  • It's easy to make a mistake when you use global variables that's subtle (and hard to find).

For example, this program has an error, but it almost looks right:

def add_two_numbers() :
    global a, b 
    c = a + b 
 
a = 10 
b = 20 
c = 0
add_two_numbers()
print (c)
  • Using global variables makes your function harder to reuse (which is the whole point of having a function)
  • Use arguments and return values instead when you can

Before you use a global ask yourself, “Can I do this another way?” If answer is yes then you probably should.

The main() Function

  • In the C programming language every program has a main() function.
  • The main() function is called to start the program.
  • When the main() function exits the program is terminated.
  • Python does not have a concept of a main() function
    • The instructions at the top level are executed in your program.
    • This makes simple Python programs easy to write and read but has some drawbacks.
    • Top level variables are by definition global
    • Top level instructions are executed when Python modules are imported with import
  • Frequently you will see a Python program with a main() function.
  • The main() function is not called implicitly.
  • You must call it explicitly

Here's an example of a program with a main function:

def main(): 
    print ('Hello World')
 
if __name__ == '__main__' :
    main()

The main() function is called because of these statements:

if __name__ == '__main__' :
    main()
  • The name variable is a special variable that contains the context that's executing.
  • When you call a program like this:
    • python3.6 program.py
  • The context contains is the string main
  • You can call a program in a module context, for example with
    • import program
  • If you were to call the program that way the context would be the name of the file
    • The the above example name would contain program

Keyword Arguments

  • Python functions are extremely flexible!
    • It's part of why Python is so popular.
  • So far we've seen named arguments to functions.

What about this function call?

>>> "I love to {verb} with {noun}".format(verb='eat', noun='sporks')                                                                            
'I love to eat with sporks'
  • format() is just a function like any other.
  • Format uses keyword arguments.
  • When a function has keyword arguments you name them as you pass them into the function.

Functions that take keyword arguments look like this:

def keyword_func(**kwargs) : 
    print ('Noun:', kwargs['noun'])
    print ('Verb:', kwargs['verb'])
 
keyword_func(noun='spork', verb='eat')
  • The variable **kwargs is just a Python dictionary.
  • The ** in front of the name indicates that you want keyword arguments stored there.
  • The name kwargs is not special. It's just a tradition. You can use any other name.
  • Keyword arguments don't have to be in order.

The program above would work just as well if you call the function this way:

keyword_func(verb='eat', noun='spork')

You could even add extra keyword arguments (they will be ignored):

keyword_func(verb='eat', noun='spork', adverb='boldly')

However, you cannot take keywords away unless in your function you first check for their presence:

def keyword_check(**kwargs) :
    if 'noun' in kwargs : 
        print ('Noun:', kwargs['noun'])
    if 'verb' in kwargs : 
        print ('Verb:', kwargs['verb'])
 
keyword_check(noun='spork')
  • You can use an existing dictionary to pass keyword arguments.
  • When you do you must put the ** in front of the dictionary

Here's an example of converting a dictionary into keyword arguments:

def keyword_func(**kwargs) : 
    print ('Noun:', kwargs['noun'])
    print ('Verb:', kwargs['verb'])
 
args = {}
args['noun'] = 'spork'
args['verb'] = 'eat'
 
keyword_func(noun='spork', verb='eat') 
keyword_func(**args) # Same as above

Syntactic Sugar: The f-string

In programming, syntactic sugar is a feature of a programming language that's designed to save typing or make something more clear. Syntactic sugar does not add a new feature, it makes an existing one look better. The f-string in Python is syntactic sugar. Consider the following:

>>> verb = 'eat'
>>> noun = 'spork'
>>> f'I like to {verb} with a {noun}' 
'I like to eat with a spork'
>>> 'I like to {verb} with a {noun}'.format(**{**globals(), **locals()})
'I like to eat with a spork'

The f-string is a lot prettier than the equivalent format() function!

Functions are Variables Too

  • In Python everything is a variable.
  • Functions too!
  • The def keyword creates a variable of type function.
>>> def myfunc() :
...   print('This is my function')
... 
>>> type(myfunc)
<class 'function'>
>>> myfunc
<function myfunc at 0x7fc356245e18>
>>> myfunc()
This is my function
  • Because a function is an ordinary variable you can assign it
  • It's like creating an alias for your function
>>> alias = myfunc 
>>> alias()
This is my function
  • Notice that myfunc is used as a variable
    • It's not called that would look like myfunc()
  • Once alias is assigned it can be called just like a function

Advanced: Function Decorators

  • You can create a function in a function using def
  • The function you create is private to your function, just like any other variable
  • If you want the caller to see your function you have to return it.

Here's an example of creating a function in a function:

def simple_wrapper(): 
    print ('This is simple_wrapper()')
    def inner_func() : 
        print ('This is inner_func().')
    return inner_func
 
alias = simple_wrapper()
print ('alias is type', type(alias))
alias()

Executing the code produces the following:

$ python3.6 ./decorator.py 
This is simple_wrapper()
alias is type <class 'function'>
This is inner_func().

Notice that inner_func() is not called until we execute the alias.

  • In programming the inner_func() is known as a closure
  • Closures can use variables from the function they were defined in
  • They can also take their own arguments.

Here's an update to the example above that shows the two ways that inner_func() can be used:

def simple_wrapper(wrapper): 
    print ('This is simple_wrapper()')
    def inner_func(inner) : 
        print ('inner_func(): wrapper:', wrapper, 'inner:', inner)
    return inner_func
 
alias1 = simple_wrapper('mike')
alias1('foo') # prints: inner_func(): wrapper: mike inner: foo
 
alias2 = simple_wrapper('sally')
alias2('bar') # prints: inner_func(): wrapper: sally inner: bar
  • Notice how the alias functions take the value given to simple_wrapper().
    • They are defined with those values baked in
    • The baked in value doesn't ever change
    • But new aliases can be made with different baked in values.
  • In your Flask applications you use a decorator to route URLs to your functions.
  • Decorators are functions that return functions that call a function.

Here's an example of how to create a decorator.

def my_decorator(wrapped_func) : 
    def inner_func(name) : 
        print ('About to call the wrapped function.')
        wrapped_func(name)
        print ('Done with the wrapped function.')
    return inner_func 
 
def print_nametag(name) :
    print ('Hello, my name is', name)
 
wrapped_nametag = my_decorator(print_nametag)
wrapped_nametag('Mike')

This is a little tricky, so follow me:

  • The my_decorator() function
    • Creates a function called inner_func() which takes one argument.
    • The inner_func() function calls a function
      • The function called by inner_func() is passed in as the argument to my_decorator just like wrapper was before
      • inner_func() is made with wrapped_func baked in
    • The inner_func() function is returned to the caller.
  • Calling the inner_func() function
    • Prints the “about to” message
    • Calls the wrapped function
    • Prints the “done with” message
  • What's happened here is that we've made a function “wrapper”
  • The wrapper controls how the function is called.
  • Decorators are syntactic sugar
  • We can simplify the code above using a decorator
def my_decorator(wrapped_func) : 
    def inner_func(name) : 
        print ('About to call the wrapped function.')
        wrapped_func(name)
        print ('Done with the wrapped function.')
    return inner_func 
 
@my_decorator
def print_nametag(name) :
    print ('Hello, my name is', name)
 
print_nametag('Mike')

Tools of the Trade: Virtualenv

  • Developers need complete control of the version of Python they use
  • They also need to control the version of important libraries
    • For example: Flask
  • Complicated projects may depend on hundreds of libraries
  • Keeping up to date versions can be complex
  • So far you've used the system versions of Python
  • You can see where Python is installed with the which UNIX command
$ which python 
/usr/bin/python
$ which python3.6 
/usr/local/bin/python3.6
  • The system versions can only be installed by administrators
    • You are an administrator on your workspace
    • You used the sudo command to install Python 3.6
  • It's important to be able to develop on a system where you are not an administrator
  • For that there's virtualenv
  • The virtualenv program makes a custom Python environment

Why I Failed in Class

Before you use virtualenv be sure that you run the following command to update to the latest version:

sudo pip install --upgrade virtualenv

Here's how to make a custom Python 3.6 environment:

mikematera:~/workspace $ virtualenv -p /usr/bin/python env2.7 
Already using interpreter /usr/bin/python
New python executable in env2.7/bin/python
Installing setuptools, pip...done.
  • The virtualenv command took the arguments:
    • -p /usr/bin/python – The Python interpreter you wish to use in your custom environment
    • env2.7 – A directory where the virutal environment will be homed
  • You can create a virtual environment in any directory you like but it virtualenv will fail if it already exists.
  • Once created your virtual environment must be activated
  • Activating an environment sets your UNIX environment to use your virtual env and not the system versions

Here's how you activate the environment you created:

mikematera:~/workspace $ source ./env2.7/bin/activate
(env2.7)mikematera:~/workspace $ 

Notice how the prompt changed! This is an important reminder. Now determine which Python you're using:

(env2.7)mikematera:~/workspace $ which python 
/home/ubuntu/workspace/env2.7/bin/python

You're using the version that's in your virtual environment. When you use the pip tool with your environment activated you install libraries into your virtual environment.

(env2.7)mikematera:~/workspace $ pip install Flask 
... don't worry about the error message you see!... 

Now you can use Flask as a part of your programs:

(env2.7)mikematera:~/workspace $ python 
Python 2.7.6 (default, Oct 26 2016, 20:30:19) 
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 
>>> from flask import Flask 
>>> app = Flask(__name__) 
  • Virtual environments are designed to be local to the developer's computer.
  • You should never copy the contents of an env directory to another computer.
  • Instead you should give instructions for someone on how to build their own.
  • pip can do this for you.

Have pip list installed packages:

argparse (1.2.1)
click (6.7)
Flask (0.12.2)
itsdangerous (0.24)
Jinja2 (2.9.6)
MarkupSafe (1.0)
pip (1.5.4)
setuptools (2.2)
Werkzeug (0.12.2)
wsgiref (0.1.2)

It's better to have pip list packages in such a way as to make it possible for pip to install them with one command.

(env2.7)mikematera:~/workspace $ pip freeze
Flask==0.12.2
Jinja2==2.9.6
MarkupSafe==1.0
Werkzeug==0.12.2
argparse==1.2.1
click==6.7
itsdangerous==0.24
wsgiref==0.1.2

You can save the output of pip freeze like this:

(env2.7)mikematera:~/workspace $ pip freeze > requirements.txt 

When someone wants to create a virtual environment with the exact same packages they can use pip like this:

<code> (env2.7)mikematera:~/workspace $ pip install -r requirements.txt <code>

That will fetch the exact versions that are needed.