This week we'll learn how to use one of the most fundamental computer data structures. The file.


  • Last week we took input from the command line
    • We'll use this again this week.
  • We also looked at how to take input from an environment variable

Here's an example of taking input on the command line:

import sys 
prog, myvar, yourvar = sys.argv 
print (f'prog = {prog} myvar = {myvar} yourvar = {yourvar}')

You should know what this program does when you call it this way:

$ python3.6 program fun profit 

What would happen if you call it like this?

$ python3.6 program fun 
  • Environment variables are essential for controlling how programs work.
  • The FLASK_DEBUG variable is great for debugging your Flask appications.

Here's an example of accessing environment variables.

import os 
print ('You are logged in as the user {}'.format(os.environ['USER']))

Try that code in your c9 project. What user are you?

Reading Files

  • Variables are the most fundamental type of storage on a computer
    • But they are gone when the power is off
  • Files are typically stored on non-volatile media.
    • Non-volatile means that it retains its data without the need for electrical power
    • Hard disks (spinning and SSD) are non-volatile
    • Flash keys are too.
  • Files are logically a bunch of bytes in a particular order
    • A byte is 8 bits
    • A bit is one binary digit (a single one or zero)
  • In order to access a file you must first open it

The examples below use this file:

This is file.txt
This is the second line
This is the third line

This code opens the file file.txt and makes it accessible through the variable f.

>>> f = open('file.txt') 
>>> f 
<_io.TextIOWrapper name='file.txt' mode='r' encoding='UTF-8'>
>>> type(f)
<class '_io.TextIOWrapper'>
  • The contents of a file can be slurped up one line at a time
  • They can also be read in bulk (the whole file) or in specific increments.

Here's code that reads the file one line at a time:

>>> f = open('file.txt') 
>>> f.readline()
'This is file.txt\n'
>>> f.readline()
'This is the second line \n'
>>> f.readline()
'This is the third line \n'
>>> f.readline()
>>> f.readline()
>>> f.readline()
  • Notice that each readline() advances our place in the file.
  • The file works like a book as you read you move forward.
  • When readline() hits the end of the file it returns the empty string
  • The read() function reads the whole file into a string.

Here's how the read() function works:

>>> f = open('file.txt') 
'This is file.txt\nThis is the second line \nThis is the third line \n\n'
You may have noticed that the file contents contains something funny, the \n character. This looks like two characters but it's only one: the newline character. The newline character starts the next line. We'll learn more about fancy characters next week
  • When you're done with a file you have to close it.
  • Closing is extra important when you're writing to a file because the contents may not be on disk until you call close()
  • After calling close() you can no longer access the file.

Here's an example of how to use close()

>>> f = open('file.txt')
'This is file.txt\nThis is the second line \nThis is the third line \n\n'
>>> f.close()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: I/O operation on closed file.
  • A ValueError happens when you read a closed file.

Writing Files

  • What happens when you want to store data permanently?
  • You need to write a file.
  • When you want to write a file you have to slightly change the call to open()

Here's an example of how to write a file:

>>> f = open('output.txt', 'w') 
>>> f.write('This is my important saved data.') 
>>> f.write('And some less important data.') 
>>> f.close()
  • Notice something?
  • The write() function returns an integer
    • The integer is the number of bytes actually written to the file.
    • This is useful if you are printing formatted or f-strings.

Here's what output.txt contains after running the code above:

This is my important saved data.And some less important data.
  • The output doesn't contain newlines (\n).
  • write() isn't like print() it never automatically adds any characters.
  • If you want your files to have newlines you must write them explicitly
>>> f = open('output.txt', 'w') 
>>> f.write('This is my important saved data.\n') 
>>> f.write('And some less important data.\n') 
>>> f.close()

Note that the character counts went up! Now the file contains:

This is my important saved data.
And some less important data.

Be Careful Writing Files

  • When you open a file with the w flag one of two things will happen:
    • If the file doesn't exist it will be created.
    • If the file exists it will be opened and the entire contents will be deleted

Seeking in a File

  • The file keeps track of the place you were reading or writing.
  • You can manipulate that place using the seek() function.
  • seek() is necessary if you want to re-read something that you've already read.

Here's an example of using seek()

>>> f = open('file.txt')
>>> f.readline()
'This is file.txt\n'
>>> f.readline()
'This is file.txt\n'
>>> f.readline()
'his is file.txt\n'
  • Notice that the seek() function returns the place that was sought to.
  • Seeking to position 1 seeks back to the second byte in the file

You can seek in a file opened for writing, too, but be careful:

>>> f = open('output.txt', 'w') 
>>> f.write('This is the first thing that I wrote.\n')
>>> f.write('Another thing\n')
>>> f.close()

The file now contains:

Another thing
rst thing that I wrote.
  • Notice that the first part of the first sentence was overwritten
  • The write() function never inserts data such that old data is moved.
  • The write() function always overwrites.

About Files and Lines

  • A common mistake for beginners is to think that lines as we see them mean more than they do in a file.
  • A file is a compact data structure
  • There is no “extra space” between the end of the line and the beginning of the next one.
  • Look at the seek() example above and notice how overwriting messed up the lines

File Modes

* So far we've opened files for reading or writing. 
* It's possible to do both.
* Here's a cheat sheet for understanding the common file modes. 
Mode Meaning
r Opens the file for reading starting from byte 0
w Opens the file for writing. The file will be created if it doesn't exist.
All contents of the file will be lost
r+ Opens the file for reading and writing, starting from byte 0
The file must exist and it's contents will be preserved
w+ Opens the file for reading and writing, starting from byte 0
The file will be created if it doesn't exist.
All contents of the file will be lost
a Open the file for writing starting at the end of the file.
Contents are safe
  • Appending a file can be very useful.
  • If you want to log what's happened opening a file to append is great.

Here's an example of a program that remembers what time it was run:

<file python> from datetime import datetime

log = open('runlog.txt', 'a') log.write(str( log.write('\n') log.close() </code>

After running this program three times on my machine I got this output in the runlog.txt file:

2017-09-25 20:04:14.998552
2017-09-25 20:04:15.933838
2017-09-25 20:04:16.532732

Python 3's Context Managers

  • Python 3 introduced a feature called a context manager
  • You'll learn how to code a contact manager later, after you learn about classes.
  • In Python 3 the open() function can be a context manager.
  • When you use open this way you don't have to remember to close() the file.
  • It's always done for you automatically.

Here's a rewrite of above to use open() as a context manager:
from datetime import datetime
with open('runlog.txt', 'a') as log :
  • Notice that the variable name log is assigned by the as word
    • The with/as line ends in a colon :
    • The write() functions are inside the with/as context manager
    • You don't call close()
  • The close() function is automatically called when the with/as block exits

Understanding Web Requests

  • Programming for the web isn't like programming on a single machine.
    • There are at least two computers involved (many more in advanced setups)
    • Program flow is based on request/response cycles
  • In web programming the two parties involved are
    • The client. This is the web browser (e.g. Chrome or Firefox).
      • The client initiates all requests.
    • The server. The server is your program.
      • The server waits for client requests
      • The server validates and responds to requests
  • Requests have to be directed somewhere.
  • The location that a request is directed to is a URL
  • URL is Universal Resource Locator.
  • URLs have four parts

Here's an example URL:
URL Part Meaning
http: Specifies the Hypertext Transfer Protocol should be used to connect to the server.
There are many protocols but HTTP is the most common
// The host or the DNS name of the server to be contacted
/cis-15 The path of the resource. This is like a file path but the server is free to interpret this any way it wants to.
This part is case sensitive
?arg=value&arg2=value2 Arguments to the path. These are a convention used by GET forms. The server is free to do anything it wants with these
  • Web browsers primarily issue two kinds of requests.
    • GET requests load a URL with arguments like the ones in the URL above
      • GET requests reveal their arguments in the browser location bar
      • That makes them a poor choice for forms with sensitive (or a lot) of data
    • POST requests load a URL with embedded arguments.
      • Embedded arguments are a bit easier to retrieve in a program.

Here's an example of what the web browser sends the server in a GET request:

GET /hello.htm HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE5.01; Windows NT)
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: Keep-Alive

Here's an example of what the web browser sends the server in a POST request:

POST /cgi-bin/process.cgi HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE5.01; Windows NT)
Content-Type: application/x-www-form-urlencoded
Content-Length: length
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: Keep-Alive

  • For more information about GET and POST requests read this:
  • Flask makes it easy to select where a request goes.
  • This selection in web speak is known as routing
    • Which is not the same as the routing you learn about in cis-81

Here's a snippet from this week's sample code:

@app.route('/result', methods=['POST'])
def result():
    return render_template('result.html')    
@app.route('/words', methods=['POST'])
def do_madlib():
    return render_template('words.html')
def start_madlib():
    return render_template('madlib.html')
  • The @app.route() decorator informs Flask where to send requests.
  • Here's what they mean:
Route Meaning
@app.route('/result', methods=['POST']) POST requests that have /result in the path
GET requests are not allowed
@app.route('/words', methods=['POST']) POST requests that have /words in the path
GET requests are not allowed
@app.route('/') GET requests sent to the root path (/).
GET is the default, no other request is allowed.
  • It's possible to have a route that handles multiple request types. Those appear like this:
@app.route('/words', methods=['POST', 'GET'])
def do_madlib():
    return render_template('words.html')

HTML: The Language of the Web

  • Web pages are written in Hypertext Markup Language (HTML)
  • HTML was designed to be written by human beings
  • The heart of HTML is the tag
    • There are opening tags and closing tags.

For example to make bold-faced text you use the <b> tag:

This sentence needs some <b>bold text</b>. 

This HTML renders in the browser like this:

This sentence needs some bold text.
  • The bold text starts with the opening tag <b>
  • The bold text ends with the closing tag </b>
  • Not all tags have opening/closing versions
    • The <p> tag starts a paragraph.
<p>Mary had a little lamb.<p>Its fleece was white as snow.

The HTML above renders like this:

Mary had a little lamb.

Its fleece was white as snow.

  • The most important tag is the <a> tag.
  • The <a> tag makes a hyperlink
  • A hyperlink is the clickable word that takes your browser somewhere else.

Here's an example of a hyperlink:

Please visit my <a href="">website</a>.
Please visit my website.
  • The hyperlink has two important parts
    • The words between the start and end tag are turned into the visible part of the link
    • The URL inside the href=“” becomes the target of the link
  • When you click the words the browser loads the page at the URL
  • URLs can be absolute
  • They can also be relative.
    • Example <a href=“page”>
  • Relative URLs are added to the URL of the current page.

Web Forms

  • So far the HTML we've learned is used to send data from the server to the client.
  • Forms are way for the client to send data to the server.
  • Forms have the following parts
    • The form action is the URL that form data is sent to.
    • The form method is what HTTP method will be used to send form data.
    • One or more form inputs are used to take input from the user.

Here's an example form:

<form action="/target" method="post">
  <p>First name:
  <p><input type="text" name="firstname">
  <p>Last name:
  <p><input type="text" name="lastname">
  <p><input type="radio" name="class" value="cis-15" checked>CIS-15
  <p><input type="radio" name="class" value="cis-75">CIS-75
  <p><input type="radio" name="class" value="cis-191">CIS-191
  <p><input type="submit" value="Submit">
  <input type="hidden" name="hidden-but-not-secret" value="somevalue">  

The form renders like this:

First name:

Last name:




When the form above is submitted (by pressing the submit button) a POST request is made that looks something like this:

POST /target HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE5.01; Windows NT)
Content-Type: application/x-www-form-urlencoded
Content-Length: length
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: Keep-Alive


When that request hits your Flask app, it's going to be routed to a function with a decorator like the one below:

@app.route('/target', methods=['POST'])
def to_target():
    first = request.form['firstname']
    last = request.form['lastname']
    course = request.form['class']
    return render_template('some_template.html', name = first + last course = course)

Example Code

You can find starter code for this week's project here: