Today we are going to start working with Python outside of the Processing platform.
So far this semester, you have effectively been learning two things together: Python, the hugely popular computer programming language used across a wide range of domains; and Processing, a platform oriented toward creative, multimedia programming for artists and designers. As we've discussed, Processing provides a set of shortcuts to make certain types of graphical, multimedia, and user interactive programming easy. The Processing project provides these shortcuts for several languages now including Java (the original language of Processing), as well as Javascript and Python. We have been focussed on Processing as a way to learn both Python and programming fundamentals because Processing is a friendly way to learn coding, and it allows programmers to more easilly visualize what these fundamental building blocks of coding do.
As we shift out of Processing, many things we've learned will
carry over: variables, if statements,
while loops, lists, and functions. But many things
will not carry over, namely all the Processing shortcuts that
are focussed on visuals and interaction such
as rect()
and ellipse(), color(), millis(), setup(), draw(),
and keyPressed(). Essentially, most of the commands
described on
the Processing
documentation page are not usable outside of Processing.
But don't despair! This has not been a waste. Nearly every time you will be using a programming language (such as Python) you will be using it along with other libraries and APIs to facilitate your work. Processing is a great one to get started with, and a good one to use in the future if you want to work on an interactive multimedia project. Other alternatives to Processing when working in Python include PyGame or the Python Image Library (PIL), and if you ever work with either of these libraries, you will find that many concepts from Processing carry over, and all of your core Python knowledge will be essential.
Back in week 1 I explained how to use the PDE: the Processing Development Environment. This is what is known as an integrated development environment or (IDE). It is comprised of all the tools you'll need to do your work on this platform. In the case of the PDE, this includes: a text editor as well as a play button to run your code, a console to see error messages, and other tools that you used to create Python programs which the Processing world calls sketches.
For this next unit, our development environment will not be one IDE comprised of an integrated bundle of tools, but several separate tools that work well together. One of these is the Atom text editor. Another piece is the command line, which we'll use to run Python programs, manage input and output to and from Python programs, and to see error messages like we used to see in the console.
The command line can feel a little less user friendly at first. It is useful to use because it offers a text-based programming paradigm, rather than the graphical mode that we've been working in. Writing programs in this way allows the programmer to experiment with algorithms and data processing without having to spend time developing user-friendly user interfaces.
This mode is also how computer programs operate that run on servers in a networked context. Computer networks generally operate when programs on different computers exchange data with each other using protocols, which you'll learn about in the reading for next week, and we'll experiment with in the coming weeks. Computer network protocols are usually comprised of text-based messages. So understanding how to write a program that can send and receive text-based inputs and outputs will give you insight into how server processes work, and it will allow you to write a server-based program to create code that operates as part of a networked system, which is your assignment for the final project.
Today we will also learn a new data structure called a dictionary.
(The following links jump down to the various topics in these notes.)
The command line is just a different kind of interface for accessing files and running programs on your computer. While Finder (on Mac) and Explorer (on Windows) use a graphical user interface (GUI) to manage our files, folders, and programs, the command line is a text-based interface, in fact sometimes it is called the command line interface or (CLI). The command line is typically thought of as a kind of legacy mode of computer use: text-based command line interfaces were common before the rise to prominence of the GUI.
Why learn the command line? A command line context is often an easier way to develop computer programs because we as developers do not need to worry about designing and implementing graphical interfaces, which often require extra code. Conversely, this means that using programs in a text-based command line mode might often feel less intuitive and less user-friendly than in graphical modes. But working in a command line mode is an important skill for a software developer, as it often allows you to more rapidly run and test new code, without having to build up a more developed interface. Also, since the command line is text-based, there are some situations, like data processing, where running a program in a command line context may actually be easier. Lastly, there may be times when you will be working in a server context — for example installing code over a network into a cloud-based machine that is physically distant and inaccessible in-person — and in these cases it is often impossible to access a GUI interface, so you would need to navigate a command line interface.
Open Terminal on Mac. On Windows, you'll need to install Cygwin, or use the the Windows Subsystem for Linux.
Let's go through a few basic commands that you'll need to get started working with the command line. Note the new formatting: whenever I use a fixed-width font on a black background with rounded corners and a thin gray bar on top, I am showing you valid instructions for the command line, like this:
$ pwd /Users/emmmma/
pwd is a command line instruction. You
can type any command line instructions in at a command
prompt and press enter. The lines that come below are
examples of what you may see, but your results will be
different depending on the files and folders on your computer.
Please note: you should not enter the dollar
sign ($) when you are entering command
line instructions. I use the dollar sign to signify the command
prompt. (The command prompt may be signified by different
punctuation in your terminal, like %
and that is fine.) The command prompt is how
the command line tells you that it's ready for more input, and
that is where you will enter new commands. Always press enter to
execute the command you've typed.
pwd — stands for "print working
directory", and this command will display the directory
(equivalent to a folder) that you are currently working in.
ls — stands for "list", and this
command will list all of the files in your current
directory. This is the command line equivalent of simply opening
a folder in Finder (or Explorer) and viewing its contents.
The command line allows you to type the name of any executable program on your system, press enter, and the command line will then try to run that program.
Let's see how to run Python in a command line context. The name
of the Python command on your system may be different, depending
on whether you are running Mac, Windows, or which version of
Python is installed. I think it should most likely
be python3. The thing to make sure of
is that when you run the command, it displays output indicating
that you are running version 3.x of Python.
$ python3 Python 3.8.3 (v3.8.3:6f8c8320e9, May 13 2020, 16:29:34) [Clang 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>>
Notice that now the punctuation indicating my command prompt has
changed from a $
to , which means that we are in
the python shell. This is a place where you can
type any valid Python code, and it will be interpretted and run.
To exit out of the Python shell you can type CONTROL-D,
or exit(). If you inadvertently run
some Python commands that won't stop executing, you can type
CONTROL-C to stop them. Note that regular command line
instructions will not work inside the Python shell, so if you
want to pwd
or ls, you need to exit out of Python.
Variables. You can create variables in Python just as in Processing:
>>> x = 250 >>> length = 50 >>> print(x) 250 >>> x 250 >>> x + length
Notice that to examine the value of a variable I can
use print() to display it's value. When I'm in the
interactive Python shell, I can also simply type the name of the
variable and press ENTER to see its value. I can also do
arithmetic with variables as before.
And I can also use comparison
operators, Boolean values, and
conditionals (a.k.a. if statements) as before:
>>> x < 50
False
>>> x > 100
True
>>> if x < 300:
... print("Less than")
...
Less than
Note that the colon (:) still starts a
new block, and that a block
must always be indented. To exit out of a block, you type an
empty line, at which point that block will be evaluated. In
Processing we have been using four spaces for indentation (and I
will continue to do that in Atom) but in an interactive Python
shell I usually only type two spaces just out of convenience.
Loops. As before, I can
create loops with the while
statement, and use variables to control them in the same way:
>>> i = 0
>>> while i < 5:
... print("This is a loop")
... i = i + 1
...
This is a loop
This is a loop
This is a loop
This is a loop
This is a loop
As before, if you forget to include your increment in your loop block, you will get an infinite loop that repeats forever and never stops. In the Python shell, you can type CONTROL-c to interrupt and stop an infinite loop:
>>> i = 0
>>> while i < 10:
... print("This is a loop")
...
This is a loop
This is a loop
^CThis is a loop
This is a loop
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
KeyboardInterrupt
Sometimes infinite loops can be useful in this context, as with user input.
Instead of things
like keyPressed() and mousePressed(),
user input in a command-line context is text-based. You can
access user input with the input() command:
>>> input()
Hello
'Hello'
>>> s = input()
Hello
>>> print(s)
Hello
>>> s = input("Type something: ")
Type something: Hello
>>> print(" You typed: " + s)
You typed: Hello
Note that you can set a variable equal to whatever text the user
typed, and then use this variable
later. The input() command also can optionally take
an argument which gets displayed to the user as a prompt.
You can ask the user for input over and over with an intentional infinite loop:
>>> while True:
... s = input("Type something: ")
... print(" You typed: " + s)
...
Type something: Hello
You typed: Hello
Type something: How are you?
You typed: How are you?
Type something: Goodbye
You typed: Goodbye
Type something: Traceback (most recent call last):
File "<stdin>", line 2, in <module>
KeyboardInterrupt
Note that I typed CONTROL-c to exit out of this loop, just as above.
Working like this can be nice. It can be a quick way to experiment with different Python commands and to get instant feedback without having to run a file. But what about if you want to run some code without having to type it in every time? What if you want to start building up Python code into a program that you can run, or that a different person can run? For this, we'll start writing Python code in a file.
As with Processing, you will write Python code in a text editor, save it, and run it. But this time the text editor will be Atom (not the PDE), and we'll run code with the command line.
First make a folder structure to work in. Create a folder for this
week called week 11.
Open Atom. Close any tabs or welcome screens that are
there. Drag the week 11 folder into
Atom. Now from the menu click File > New to make a new file,
then click File > Save, name your
file main.py, all lower case, no spaces
or other punctuation, and save it into
your week 11 folder. Saving your file
with a .py file extension will tell
Atom that you are working with Python code, and it will offer
you syntax highlighting similar to the PDE.
cd — stands for "change
directory". In the command line, this is how you change the
directory that you are currently working in.
Now let's go back to Atom and add a line of code to our Python file:
print("Hello, world")
Now flip back to the command line, and run this program by
typing python3 and the filename:
$ python3 main.py Hello, world
Note the command prompt here: $. I am
in the Unix shell, not the Python shell. The program runs,
exits, and returns me to the Unix shell. So I have just run some
Python code in a file, not in an interactive Python shell. Now I
could write a more complicated program and distribute that to
others to run.
Let's make a program using the bits of Python that we've been
experimenting with above. First let's create a
simple while loop:
i = 0
while i < 5:
print("Hello, world")
i = i + 1
Run this:
$ python main.py Hello, world Hello, world Hello, world Hello, world Hello, world
Now let's make it interactive:
s = input("Type your name: ") i = 0 while i < 5: print("Hello, " + s) i = i + 1
Run it:
$ python main.py Type your name: emmmma Hello, emmmma Hello, emmmma Hello, emmmma Hello, emmmma Hello, emmmma
So far we have talked about one kind of data
structure: lists (week
7). Lists are good at storing values in a fixed, sequential
order. Remember that you create a list with empty square
brackets (x = []), then you add values to it using
the append() command which are then stored in the
order in which they were added, and you reference those values
with a numerical index inside square brackets
(x[0], x[3], etc.)
During our discussion of lists, I mentioned that Python offers many different data structures. Which data structure to use in a given situation depends on the structure of the data that you are trying to model, and the operations that you will need to do with it. Determining which data structure(s) to use to solve a given problem can often be a question of some subjective debate, and there is often not one clear right answer.
The data structure we'll learn about today is the dictionary. While lists were sequential (added to in order, and accessed by numerical index), dictionaries are unordered, and they are accessed by an index of any kind of value (number, string, etc.) A dictionary contains a collection of mappings, also referred to as key-value pairs. Think of a key-value pair as like a pairing of a word and its definition — this is where the dictionary gets its name. Dictionaries do not preserve the order in which items are added to them, and instead are accessed, we might say, randomly, by retrieving the value (the definition) for a given key (the word).
Since at this point in our class we are starting to move out of Processing and into Python itself, I want introduce dictionary syntax to you outside of Processing. But first, we need to take a detour into the command line.
Let's use the interactive Python shell to learn about how to use dictionaries. First let's review some list commands:
>>> a = [] >>> a.append(100) >>> a.append(200) >>> a.append(300) >>> print(a) [100, 200, 300] >>> a[0] 100 >>> a[1] 200 >>> len(a) 3
It is also valid Python code to initialize a list by using this notation:
>>> a = [ 100, 200, 300 ] >>> a [100, 200, 300]
Now let's learn some new syntax for dictionaries:
>>> r = {}
>>> r["name"] = "emmmma"
>>> r["species"] = "human"
>>> r["computer"] = "mac"
>>> r["favorite thing"] = "writing"
>>> r
{'computerr': 'mac', 'name': 'emmmma',
'favorite thing': 'writing', 'species': 'human'}
>>> r["name"]
'emmmma'
Notice that the order in which the key-value pairs are printed is not the same order in which I entered them. This is because dictionaries do not preserve order, as I mentioned above.
You can also use the len() command as with
lists. In this case, len() will return the number
of key-value pairs:
>>> len(r) 4
If you try to access a key that is not in your dictionary, you
will get a KeyError. For example:
>>> r["date of birth"] Traceback (most recent call last): File "<stdin>", line 1, in <module> KeyError: 'date of birth'
This very helpful error message is telling you precisely the
problem: that the key "date of birth" does not
exist in this dictionary.
There is a command that you can use to help avoid these
errors: in. This command checks if a given key
is in the dictionary, like so:
>>> "name" in r True >>> "date of birth" in r False
Since this command returns a Boolean
(True/False) value, you can use it
inside an if statement, like so:
>>> if "date of birth" in r: ... r["date of birth"] ... else: ... "Key not in this dictionary" ... 'Key not in this dictionary'
Similar to the new array initialization above, you can also initialize a dictionary with some new syntax:
>>> r = {
"name": "emmmma",
"species": "human",
"computer": "mac",
"favorite thing": "writing"
}
>>> r["name"]
'emmmma'
One last thing, you can even combine data structures in useful and important ways. So for example:
>>> r = {
"name": "emmmma",
"species": "human",
"computer": "mac",
"favorite thing": "writing"
}
>>> g = {
"name": "Gritty",
"species": "monster",
"computer": "orange",
"favorite thing": "hockey"
}
>>> a = [ r, g ]
>>> a
[{'computer': 'mac', 'name': 'emmmma',
'favorite thing': 'writing', 'species': 'human'},
{'computer': 'orange', 'name': 'Gritty',
'favorite thing': 'hockey', 'species': 'monster'}]
>>> a[0]
{'name': 'emmmma', 'species': 'human', 'computer': 'mac', 'favorite thing': 'writing'}
>>> a[1]
{'name': 'Gritty', 'species': 'monster', 'computer': 'orange', 'favorite thing': 'hockey'}
What I've done here is create two
dictionaries, r and g, and then
created a list a, which holds both of them.
The format that Python uses to display the contents of data structures in this way is called JSON, which stands for JavaScript Object Notation. This is a text-based way to format data for storage or to exchange between different computer programs. We'll talk about it next week.
OK, well that is nice and fun, but what if you want a webserver that does more than just return static HTML pages to a user. What if you want a webserver that dynamically generates content using all that you have learned about coding, and returns the output of a program to the web user?
For this, we can use a technique called CGI, which stands for common gateway interface. This is a technique where a server receives an HTTP request , but instead of simply returning an HTML file, the server executes a computer program, and the output of that program is returned as the HTTP response. It is similar to running a Python program on the command line, but instead of typing a command to execute, a user on the web requests a URL with their browser, which then invokes the program, and the browser then displays the response. In a way, it is also somewhat similar to all the work that we have been doing in Processing, except instead of interactive keyboard and mouse input, the only inputs come from the request, and instead of displaying visual graphic results, you can only return textual content in the web response.
Let's experiment with this ...
Create a new folder called week11-webserver. Inside that folder create a subfolder called cgi-bin. Within that
subfolder, create a new file called server.py, and
open that in Atom.
cd to this new folder, and make sure
that your server.py file
is executable. You can check this by
typing the following (remember, don't type the dollar signs):
$ ls -l cgi-bin
It probably is not executable by default, so to make sure that it is, type the following command:
$ chmod a+x cgi-bin/server.py
(This command is short for "change mode" and it allows you to change read, write, and execute permissions on files.)
The first line of this file should be the executable path to Python on your system. Type:
$ which python
/usr/local/bin/python
For me, that
outputs /usr/local/bin/python, but
for you it may be different. Whatever it says, copy that
output, and past it in as the first line
of server.py, preceded by a pound sign
(#), like this:
#!/usr/local/bin/python
Now you can add any arbitrary Python code into this
program. The first thing you must do to follow the HTTP
protocol is return the following text:
server.py:
print("Content-Type: text/html")
print("")
After that, try adding some other Python commands. For example for now you could simply add:
print("Hello, world!")
Now you can run the following command to start your python http server:
python -m CGIHTTPServer 8000
And now in your browser try visiting: 127.0.0.1:8000/cgi-bin/server.py
In the async work for this week, I explain how you can create an HTML page with a form into which a user can enter values, which will then be made available as a dictionary of key-value pairs to your Python CGI script. This will allow your CGI script to treat the user values from the HTML form as inputs, and work with those to generate a dynamic response.