Review

  • Last week read from and wrote to files.
  • Reading and writing files is one of the most common things for a program to do

Functions

  • Functions are a way to group instructions together
  • The grouped instructions have a name
  • A function is called using its name
  • The def keyword is short for define
    • def is used when you want to define a function by name.

Here's an example of a Python function:

def do_something(): 
    print ('Okay, how about print "do something."')    
    print ("Now I'll print something else")    
  • What happens if you put this code in a file and execute it?
mikematera:~/workspace/Week6 $ python3.6 ./examples.py
mikematera:~/workspace/Week6 $ 

The code does nothing!

  • The function contains the two print instructions
  • You can tell that the instructions are inside of the function because they're indented to the right
  • The code does nothing because functions have to be called
  • Adding a function call fixes the problem:
# This is the functions definition
def do_something(): 
    print ('Okay, how about print "do something."')    
    print ("Now I'll print something else")    
 
# This is where the function is called.
do_something()

Now the program operates as expected:

mikematera:~/workspace/Week6 $ python3.6 ./do_something.py
Okay, how about print "do something."
Now I'll print something else
mikematera:~/workspace/Week6 $ 
  • Most of the work of your programs so far has been done using functions.
  • You're already familiar with the functions:
    • print()
    • input()
    • write()
    • readline()
    • read()
  • Each of these functions has a def somewhere.
  • The great thing about functions is that you can call them as often as you like
    • They help you reuse code so you don't have to write the same things over and over

If I want to do_something() more than once I could do this:

# This is the functions definition
def do_something(): 
    print ('Okay, how about print "do something."')    
    print ("Now I'll print something else")    
 
# This is where the function is called.
do_something()
do_something()
do_something()
do_something() # That's a lot of something!
  • Each time I call do_something() the function is executed.

Function Parameters

  • Functions can take parameters also called arguments
  • A parameter is a variable that holds the value that is passed to the function
  • Parameters make functions very flexible
    • The print() function takes stuff to print as parameters
    • The input() function takes a prompt for the user as a parameter

Here's an example of using parameters with a function:

print ('Hello World')  # 'Hello World' is a parameter
print ('Hello', adjective, 'world') # There are three parameters 
  • So, you're already familiar with giving arguments to functions.
  • Functions receive arguments as a part of their definition
  • Function definitions can declare a parameter list

The simplest way to have parameters is to name them:

def print_nametag(name): 
    print ('Hello, my name is', name)
 
# Prints a nametag with my name.
print_nametag('Mike')
  • As you know, functions can take variables as parameters
def print_nametag(name): 
    print ('Hello, my name is', name)
 
# This is the same as the program above.
my_name = 'Mike'
print_nametag(my_name)
  • You can do math and other operations in function arguments
def print_nametag(name): 
    print ('Hello, my name is', name)
 
my_name = 'Mike'
print_nametag(my_name + " Matera")
  • Functions can have multiple arguments
  • You must name each argument in the function definition
# This function takes two arguments.
def print_nametag(first, last): 
    print ('Hello, my name is', first, last)
 
# Two arguments must be supplied. 
print_nametag("Mike", "Matera")
  • Separate the named arguments with a comma.
  • Now you have to call the function with two arguments.
  • Must functions take a specific number of arguments.
  • If you do not provide a function with the right number of arguments it's an error
>>> print_nametag('Mike')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: print_nametag() missing 1 required positional argument: 'last'
>>> 
>>> print_nametag('Mike', 'Matera', 'Extra')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: print_nametag() takes 2 positional arguments but 3 were given
  • You should be familiar with these error messages, you'll get them a lot!
  • You may be wondering, “what about print() it takes any number of arguments?”
  • print() is a variadic function.
    • Variadic functions can take any number of arguments.
  • Here's an example of a variadic function:
def variadic_nametags(*names) :
    first, last = names 
    print ('Hello, my name is', first, last)
 
variadic_nametags("Mike", "Matera")
  • Notice the following:
    • The *names argument begins with a star (*) character
    • We get first and last in the same way we expand argv
  • What happens if we use the power of a variadic function to add more names?
def variadic_nametags(*names) :
    first, last = names 
    print ('Hello, my name is', first, last)
 
variadic_nametags("Mike", "Matera", "Bonnie", "Smith")

Running that code gives an error:

Traceback (most recent call last):
  File "./examples.py", line 11, in <module>
    variadic_nametags("Mike", "Matera", 'Bonnie', 'Smith')
  File "./examples.py", line 8, in variadic_nametags
    first, last = names 
ValueError: too many values to unpack (expected 2)
  • In order to fully appreciate the use of a variadic function and *args you need to understand loops.
  • Loops are a way to do the same work over and over.
  • We'll talk about loops in a few weeks

Here's what variadic_nametags() should look like:

def print_nametags(*names) :
    for name in names : 
        print ('Hello, my name is', name)
 
print_nametags("Mike", 'Bonnie', 'Jun', 'Dweezil')

Function Return Values

  • The caller of a function passes arguments to the function
  • The caller may also receive return values
  • Return values are values that are passed from the function to the caller
  • A function can return anything you like (or nothing)

Here's an example of functions with return values:

def add_two_numbers(a, b) :
    return a + b 
 
sum = add_two_numbers(10, 100)
print (sum)
  • Functions in Python may return multiple values.
    • That's a big improvement over C and C++!

Here's a function that returns two values:

def do_math(a, b) :
  sum = a + b 
  product = a * b 
  ratio = a / b 
  return sum, product, ratio 
 
ret1, ret2, ret3 = do_math(12, 45)
print ('The sum/product/ratios are', ret1, ret2, ret3)
  • Just like with passing arguments you have to accept the right number of variables in a return.
    • There's one exception: You can accept just one
>>> sum, prod, rat = do_math(1, 3) 
>>> print (sum, prod, rat) 
4 3 0.3333333333333333
>>> sum, prod = do_math(1, 3) 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: too many values to unpack (expected 2)
>>> all = do_math(1, 3)                                                                                                                
>>> all
(4, 3, 0.3333333333333333)
  • Notice something?
    • When do_math() returns into a single value the value becomes a tuple
    • We'll talk more about tuples and lists later

Designing with Functions

  • Functions give you a way to name snippets of code
  • You should chose a name that describes what the function does.
  • Your functions should do something that's useful.

Here's an example of a function that reads the first line of a file:

def read_first_line(filename) :
  file = open(filename, 'r') 
  line = file.readline()
  file.close()
  return line
<code>
 
Of course you could also write that function like this: 
 
<code python>
def read_first_line(filename) :
    with open(filename, 'r') as file : 
        return file.readline()

Isn't that nicer? You don't have to remember to close the file. It's done for you when you leave the with/as block.

  • What a function does isn't always clear from the name
  • This is especially true when you start to write more complex functions
  • Functions can have docstrings to help users understand what they do.

Here's an example of a docstring for the read_first_line() function:

def read_first_line(filename) :
    ''' Reads and returns the first line of a file.''' 
    with open(filename, 'r') as file : 
        return file.readline()
  • When it's not obvious what the function arguments are you should be more descriptive in your docstring.
  • It's also important sometimes to describe what the return value is (or might be)
def read_some_line(file, offset) :
    '''
    Reads a line in the file, without affecting the place that the program was reading
 
    Arguments:
      file -- A file object that is opened for reading 
      offset -- The place where the line starts 
 
    Returns: 
      A string containing the desired line. 
    ''' 
    saved_place = file.tell()
    file.seek(offset)
    line = file.readline()
    file.seek(saved_place)
    return line
  • There are official Python guidelines for how to use docstrings in functions
  • It will be difficult at first to know what code you should put in a function
    • Don't worry, it's hard for everyone at first!
    • Practice, practice, practice!

Function Variables and Scope

  • So far, I've tried to use different variable names inside and outside of functions.
  • That's good practice, but it's not always possible or effective to do so.
  • When you use a variable inside of a function you get a new variable.
    • This can lead to confusion until you get the hang of it.
  • All programming languages have rules about where variables are valid.
  • Validity of variables is called scope

Here's an example showing where variable scope matters:

myvar = 10 
print (myvar)
 
def myfunc(myvar) :
    myvar = myvar + 1
    print (myvar)
 
myfunc(myvar)
print (myvar)

What happens when you run this code?

$ python3.6 ./scope.py 
10
11
10

What happened to the value 11? We incremented myvar in myfunc()!

  • The program above makes sense when you understand that there are really two different variables named myvar
  • The difference is illustrated in the code snippet below.

This program is identical to the one above:

myvar = 10 
print (myvar)
 
def myfunc(myvar_inside_of_myfunc) :
    myvar_inside_of_myfunc = myvar_inside_of_myfunc + 1
    print (myvar)
 
myfunc(myvar)
print (myvar)
  • Now do you see why the code prints 10, 11, 10?
  • Even though the name myvar is used inside of myfunc it's a different variable.
  • Variables declared inside of functions are private to that function.
    • They go away when the function ends
    • Code outside of the function can't access them

Here's an example of an error:

def add_two_numbers(a, b) :
    c = a + b 
    return c 
 
sum = add_two_numbers(1, 3)
print (c) # ERROR! c is private to add_two_numbers()
  • Code outside of add_two_numbers() has no access to c
  • The reverse is not true
    • Code inside of a function can access variables in global scope.
  • Variables declared in code that's not a part of any function are in global scope.
  • When you use a global variable you must say so.
def add_two_numbers() :
    global a, b, c 
    c = a + b 
 
a = 10 
b = 20 
add_two_numbers()
print (c)
  • The global keyword tells Python that add_two_numbers() is operating on global variables
  • Without the global keyword the program will cause an error because a and b aren't defined.

Avoiding Global Variables

  • It's a good idea to avoid global variables when you can
  • Global variables make a program harder to understand
    • Python at least makes it obvious what global variables you intend to use
  • It's easy to make a mistake when you use global variables that's subtle (and hard to find).

For example, this program has an error, but it almost looks right:

def add_two_numbers() :
    global a, b 
    c = a + b 
 
a = 10 
b = 20 
c = 0
add_two_numbers()
print (c)
  • Using global variables makes your function harder to reuse (which is the whole point of having a function)
  • Use arguments and return values instead when you can

Before you use a global ask yourself, “Can I do this another way?” If answer is yes then you probably should.

The main() Function

  • In the C programming language every program has a main() function.
  • The main() function is called to start the program.
  • When the main() function exits the program is terminated.
  • Python does not have a concept of a main() function
    • The instructions at the top level are executed in your program.
    • This makes simple Python programs easy to write and read but has some drawbacks.
    • Top level variables are by definition global
    • Top level instructions are executed when Python modules are imported with import
  • Frequently you will see a Python program with a main() function.
  • The main() function is not called implicitly.
  • You must call it explicitly

Here's an example of a program with a main function:

def main(): 
    print ('Hello World')
 
if __name__ == '__main__' :
    main()

The main() function is called because of these statements:

if __name__ == '__main__' :
    main()
  • The name variable is a special variable that contains the context that's executing.
  • When you call a program like this:
    • python3.6 program.py
  • The context contains is the string main
  • You can call a program in a module context, for example with
    • import program
  • If you were to call the program that way the context would be the name of the file
    • The the above example name would contain program

Keyword Arguments

  • Python functions are extremely flexible!
    • It's part of why Python is so popular.
  • So far we've seen named arguments to functions.

What about this function call?

>>> "I love to {verb} with {noun}".format(verb='eat', noun='sporks')                                                                            
'I love to eat with sporks'
  • format() is just a function like any other.
  • Format uses keyword arguments.
  • When a function has keyword arguments you name them as you pass them into the function.

Functions that take keyword arguments look like this:

def keyword_func(**kwargs) : 
    print ('Noun:', kwargs['noun'])
    print ('Verb:', kwargs['verb'])
 
keyword_func(noun='spork', verb='eat')
  • The variable **kwargs is just a Python dictionary.
  • The ** in front of the name indicates that you want keyword arguments stored there.
  • The name kwargs is not special. It's just a tradition. You can use any other name.
  • Keyword arguments don't have to be in order.

The program above would work just as well if you call the function this way:

keyword_func(verb='eat', noun='spork')

You could even add extra keyword arguments (they will be ignored):

keyword_func(verb='eat', noun='spork', adverb='boldly')

However, you cannot take keywords away unless in your function you first check for their presence:

def keyword_check(**kwargs) :
    if 'noun' in kwargs : 
        print ('Noun:', kwargs['noun'])
    if 'verb' in kwargs : 
        print ('Verb:', kwargs['verb'])
 
keyword_check(noun='spork')
  • You can use an existing dictionary to pass keyword arguments.
  • When you do you must put the ** in front of the dictionary

Here's an example of converting a dictionary into keyword arguments:

def keyword_func(**kwargs) : 
    print ('Noun:', kwargs['noun'])
    print ('Verb:', kwargs['verb'])
 
args = {}
args['noun'] = 'spork'
args['verb'] = 'eat'
 
keyword_func(noun='spork', verb='eat') 
keyword_func(**args) # Same as above

Syntactic Sugar: The f-string

In programming, syntactic sugar is a feature of a programming language that's designed to save typing or make something more clear. Syntactic sugar does not add a new feature, it makes an existing one look better. The f-string in Python is syntactic sugar. Consider the following:

>>> verb = 'eat'
>>> noun = 'spork'
>>> f'I like to {verb} with a {noun}' 
'I like to eat with a spork'
>>> 'I like to {verb} with a {noun}'.format(**{**globals(), **locals()})
'I like to eat with a spork'

The f-string is a lot prettier than the equivalent format() function!

Functions are Variables Too

  • In Python everything is a variable.
  • Functions too!
  • The def keyword creates a variable of type function.
>>> def myfunc() :
...   print('This is my function')
... 
>>> type(myfunc)
<class 'function'>
>>> myfunc
<function myfunc at 0x7fc356245e18>
>>> myfunc()
This is my function
  • Because a function is an ordinary variable you can assign it
  • It's like creating an alias for your function
>>> alias = myfunc 
>>> alias()
This is my function
  • Notice that myfunc is used as a variable
    • It's not called that would look like myfunc()
  • Once alias is assigned it can be called just like a function

Advanced: Function Decorators

  • You can create a function in a function using def
  • The function you create is private to your function, just like any other variable
  • If you want the caller to see your function you have to return it.

Here's an example of creating a function in a function:

def simple_wrapper(): 
    print ('This is simple_wrapper()')
    def inner_func() : 
        print ('This is inner_func().')
    return inner_func
 
alias = simple_wrapper()
print ('alias is type', type(alias))
alias()

Executing the code produces the following:

$ python3.6 ./decorator.py 
This is simple_wrapper()
alias is type <class 'function'>
This is inner_func().

Notice that inner_func() is not called until we execute the alias.

  • In programming the inner_func() is known as a closure
  • Closures can use variables from the function they were defined in
  • They can also take their own arguments.

Here's an update to the example above that shows the two ways that inner_func() can be used:

def simple_wrapper(wrapper): 
    print ('This is simple_wrapper()')
    def inner_func(inner) : 
        print ('inner_func(): wrapper:', wrapper, 'inner:', inner)
    return inner_func
 
alias1 = simple_wrapper('mike')
alias1('foo') # prints: inner_func(): wrapper: mike inner: foo
 
alias2 = simple_wrapper('sally')
alias2('bar') # prints: inner_func(): wrapper: sally inner: bar
  • Notice how the alias functions take the value given to simple_wrapper().
    • They are defined with those values baked in
    • The baked in value doesn't ever change
    • But new aliases can be made with different baked in values.
  • In your Flask applications you use a decorator to route URLs to your functions.
  • Decorators are functions that return functions that call a function.

Here's an example of how to create a decorator.

def my_decorator(wrapped_func) : 
    def inner_func(name) : 
        print ('About to call the wrapped function.')
        wrapped_func(name)
        print ('Done with the wrapped function.')
    return inner_func 
 
def print_nametag(name) :
    print ('Hello, my name is', name)
 
wrapped_nametag = my_decorator(print_nametag)
wrapped_nametag('Mike')

This is a little tricky, so follow me:

  • The my_decorator() function
    • Creates a function called inner_func() which takes one argument.
    • The inner_func() function calls a function
      • The function called by inner_func() is passed in as the argument to my_decorator just like wrapper was before
      • inner_func() is made with wrapped_func baked in
    • The inner_func() function is returned to the caller.
  • Calling the inner_func() function
    • Prints the “about to” message
    • Calls the wrapped function
    • Prints the “done with” message
  • What's happened here is that we've made a function “wrapper”
  • The wrapper controls how the function is called.
  • Decorators are syntactic sugar
  • We can simplify the code above using a decorator
def my_decorator(wrapped_func) : 
    def inner_func(name) : 
        print ('About to call the wrapped function.')
        wrapped_func(name)
        print ('Done with the wrapped function.')
    return inner_func 
 
@my_decorator
def print_nametag(name) :
    print ('Hello, my name is', name)
 
print_nametag('Mike')

Tools of the Trade: Virtualenv

  • Developers need complete control of the version of Python they use
  • They also need to control the version of important libraries
    • For example: Flask
  • Complicated projects may depend on hundreds of libraries
  • Keeping up to date versions can be complex
  • So far you've used the system versions of Python
  • You can see where Python is installed with the which UNIX command
$ which python 
/usr/bin/python
$ which python3.6 
/usr/local/bin/python3.6
  • The system versions can only be installed by administrators
    • You are an administrator on your workspace
    • You used the sudo command to install Python 3.6
  • It's important to be able to develop on a system where you are not an administrator
  • For that there's virtualenv
  • The virtualenv program makes a custom Python environment

Why I Failed in Class

The Cloud9 environment sets an alias for the python command that interferes with the way virtualenv works. You can uset the alias in the current terminal window with this command:

unalias python 

To permanently disable the alias for all future terminal windows use this command to remove the alias from your BASH startup script:

grep -v alias ~/.bashrc > /tmp/bashrc.new && mv /tmp/bashrc.new ~/.bashrc

After that virtualenv works.

Here's how to make a custom Python 3.6 environment:

mikematera:~/workspace $ virtualenv -p /usr/bin/python env2.7 
Already using interpreter /usr/bin/python
New python executable in env2.7/bin/python
Installing setuptools, pip...done.
  • The virtualenv command took the arguments:
    • -p /usr/bin/python – The Python interpreter you wish to use in your custom environment
    • env2.7 – A directory where the virutal environment will be homed
  • You can create a virtual environment in any directory you like but it virtualenv will fail if it already exists.
  • Once created your virtual environment must be activated
  • Activating an environment sets your UNIX environment to use your virtual env and not the system versions

Here's how you activate the environment you created:

mikematera:~/workspace $ source ./env2.7/bin/activate
(env2.7)mikematera:~/workspace $ 

Notice how the prompt changed! This is an important reminder. Now determine which Python you're using:

(env2.7)mikematera:~/workspace $ which python 
/home/ubuntu/workspace/env2.7/bin/python

You're using the version that's in your virtual environment. When you use the pip tool with your environment activated you install libraries into your virtual environment.

(env2.7)mikematera:~/workspace $ pip install Flask 
... don't worry about the error message you see!... 

Now you can use Flask as a part of your programs:

(env2.7)mikematera:~/workspace $ python 
Python 2.7.6 (default, Oct 26 2016, 20:30:19) 
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 
>>> from flask import Flask 
>>> app = Flask(__name__) 
  • Virtual environments are designed to be local to the developer's computer.
  • You should never copy the contents of an env directory to another computer.
  • Instead you should give instructions for someone on how to build their own.
  • pip can do this for you.

Have pip list installed packages:

argparse (1.2.1)
click (6.7)
Flask (0.12.2)
itsdangerous (0.24)
Jinja2 (2.9.6)
MarkupSafe (1.0)
pip (1.5.4)
setuptools (2.2)
Werkzeug (0.12.2)
wsgiref (0.1.2)

It's better to have pip list packages in such a way as to make it possible for pip to install them with one command.

(env2.7)mikematera:~/workspace $ pip freeze
Flask==0.12.2
Jinja2==2.9.6
MarkupSafe==1.0
Werkzeug==0.12.2
argparse==1.2.1
click==6.7
itsdangerous==0.24
wsgiref==0.1.2

You can save the output of pip freeze like this:

(env2.7)mikematera:~/workspace $ pip freeze > requirements.txt 

When someone wants to create a virtual environment with the exact same packages they can use pip like this:

<code> (env2.7)mikematera:~/workspace $ pip install -r requirements.txt <code>

That will fetch the exact versions that are needed.