Labs on The Art of Computing

Lab 9: Combinatorial Circuits

Enthusiastic Hugo User — Sun, 14 May 2017 00:00:00 UTC

In this lab, you will use the Logisim tool, along with pen and paper, to analyse and create simple combinatorial circuits.

Installing and Starting Logisim

Unlike Snap!, Logisim does not run in a web browser but needs to be installed. It is not installed on the CSIT lab computers, so to use it, you will have to install it on your own computer or on your student account.

Download the Logisim v. 2.7.1 JAR file and save it in a convenient location (e.g., your home directory).
You will probably be able to start the program by simply (double-)clicking on the JAR file. If that doesn’t work, on a GNU/Linux system you can start a commandline (“terminal”) and type in
```
java -jar logisim-generic-2.7.1.jar
```
(assuming you are in the directory where the file is).

Note that Logisim requires Java version 5 or later.

Exercise 9.1 (no marks)

First, do the following simple exercise to familiarise yourself a bit with the Logisim environment.

Download the file lab9ex1.circ (using “save link as”), then open the file in Logisim (select “Open…” under the “File” menu). If everything worked, you should now see a simple combinatorial circuit.
What does this circuit do? It computes some function

Output = F(A, B, C)

where all three inputs and the output are binary (0 or 1). But what is the function F? To find out, you can:

a. Figure it out by looking at the circuit.

b. Select the “poke” tool (), click on the inputs A, B and C to change their value, and observe how the output changes.

c. From the “Project” menu, select “Analyze Circuit”. This opens a window with information about the circuit. Under “Table” you can find the complete truth table for the circuit, and under “Expression” you can see the equivalent Boolean expression. (Note that Logisim does not have a symbol for logical AND in expressions: “x AND y” is written just “x y”. The symbol for logical OR is “+” and for NOT is “~“.)

Make sure you try the first two methods before the last.
Is there a simpler/smaller circuit that computes the same function? Using the truth table for the circuit, attempt to find a smaller expression that generates the same table. Build the corresponding circuit next to the one given (using the same inputs, but a new output) and test it to see that it gives the same result.

(Again, Logisim offers you a way to cheat: The “Minimized” tab of the circuit analysis window will give you a minimal expression, and can even rebuild the circuit automatically for you. Use this to check what you came up with by hand.)

Exercise 9.2 Sorting Numbers with Circuits

In this exercise, you will construct a circuit that sorts four 4-bit binary numbers. This means the circuit has has 16 inputs and 16 outputs! If you tried build it directly, the result would probably be pretty gigantic. So, instead, we will introduce some intermediate abstractions, which will help keep the size of the result manageable.

Download the file lab9ex2.circ and open it in Logisim. The project contains several defined components: a “1-bit swap”, a “4-bit swap” and a “4-bit compare & swap”.

Task 1 1-bit swap (50%)

The 1-bit swap element is not implemented. Your first task is to build it.

The 1-bit swap has three inputs, called Input 1, Input 2 and swap?, and two outputs, called Output 1 and Output 2. The element is meant to implement a controllable swap of the two inputs into the two outputs. In other words, behaviour of the circuit should be as follows:

When swap? is 0, the inputs are just copied to the outputs, that is Output 1 equals Input 1 and Output 2 equals Input 2.
When swap? is 1, the outputs are the inputs swapped, that is Output 1 equals Input 2 and Output 2 equals Input 1.

Hints

To help you understand how to make this work, you can draw up the truth table (it has only 2³ = 8 lines) and fill in the correct outputs for each case.

If you are using more than 10 gates to build the circuit, you’re on the wrong track; there is an easier solution. Look at the truth table.

All the inputs and output that your circuit should have are already in place, so you should not have to place any new “pins”.

How to build circuits in Logisim

To view and edit the circuit that implements one of the defined elements (1-bit swap, etc), double-click on it in the left-hand-side menu. You can also double-click “main” to get back to the main circuit.
To add a gate to the circuit, select the gate you want from the menu on the left. Then just click on the circuit board to add the gate.
When you have selected the edit tool () you can move components and add wires. Right-click and select “Delete” to remove a component or wire.
To connect components, add or extend wires. Components have a small marker where they can connect (you will see a small green circle when the pointer is on the mark). You can also add a branch off an existing wire, at any point.

Here is a short video showing how it is done. (Note: The circuit built in the video is not the solution to this exercise!)

Test your circuit!

After you have built a circuit, you can check that it does the right thing by either using the poke tool to vary the inputs, or using Logisim’s built-in circuit analyzer, which will show you the truth table for both outputs.

The 4-bit swap element is implemented using several 1-bit swaps. Thus, after you have built the 1-bit swap (correctly), the 4-bit swap will also work. In the main circuit, off to the right, there is a small test circuit for the 4-bit swap: it has two 4-bit inputs and a (1-bit) swap input, and two displayed outputs. Using the poke tool, you can change the inputs; when you toggle the swap bit, you should see the outputs change place. (Note that when you use the poke tool to change multi-bit inputs, you must toggle each bit separately by clicking on it.)

Task 2 The 4-number sorting circuit (50%)

The 4-bit compare & swap element has two inputs and two outputs, which are all 4-bit binary numbers. It sorts the two input numbers: that is, the first (upper) output is the smaller of the two numbers, and the second (lower) output is the bigger one. Note that it does an unsigned comparison. This means the numbers are interpreted as positive (ranging from 0 to 15), not in 2’s complement.

The 4-bit compare & swap element is implemented using the 4-bit swap and a “comparator” (a component from Logisim’s standard library). A test circuit is provided (also to the right in “main”): use the poke tool to change the inputs and observe what happens with the output. (This is also a chance to practice reading binary numbers: how do you set the inputs to, for example, 12 and 9?)

Now, you should build the 4-number sorting circuit using only 4-bit compare and swap elements (and wires). The inputs (labelled A-D) are on the left and the outputs (labelled #1-#4) are on the right. The outputs should be the inputs sorted in increasing order, that is, #1 should be the smallest of A,B,C,D, #2 the second smallest, and so on.

Hint

If you have trouble working out how to sort the four numbers, think recursively. This circuit gives you part of one possible solution for sorting three numbers:

After the two compare & swaps, the top wire has the smallest number, and the other two hold the two larger numbers (but not in sorted order). How would you complete this circuit to sort all three numbers? If you have a circuit that can sort three numbers, how could you use it to sort four numbers?

Test and save your circuit!

After you have built the circuit, check that works by varying the inputs. (The circuit analyzer tool does not work with multi-bit inputs and outputs.)

Save your circuit in a new file (using “Save As…” from the File menu). This is the file that you will need to submit for this assignment.

Submission:

You must submit one Logisim circuit file, containing your solutions to Tasks 1 and 2. Submit the files through wattle.

Lab 8: Learning to diagnose - Machine learning with decision trees

Enthusiastic Hugo User — Wed, 26 Apr 2017 00:00:00 UTC

This lab practice the ideas of recursive computation with tree structure and core concepts of machine learning, by learning a decision tree for diagnosing breast cancer.

Submission instructions:

For this lab, you will need to submit two files:

An xml file containing Snap blocks for Task 1-3. This block should contain “Choice node …” from task 1, “Build a decision tree …” block from task 2, and “Test performance …” from task 3, as well as blocks they depend on.
A written report for Task3 (written electronically in an editor of your choice, and exported to pdf)as described below.

Breaking down by tasks:

Task 1 (40%) - implement “Choice node …” block, you need to import this block to task 2 project.
Task 2 (40%) - implement “Build a decision tree …” block.
Task 3 (20%)- implement “Test performance …” block and answer the questions in 3.2 and 3.3 in a short report format, there are 10 questions in total.

Data, data everywhere

In the 21st century, data is everywhere. Being able to make sense of large data sets and use them to make predictions is becoming increasingly relevant in many domains including scientific and medical research, government, and commerce.

Machine learning is the study of computer algorithms that learn from data. Commonly, we are given some training data containing a set of sample input and output pairs and want to use this to make predictions for future inputs whose true output value is unknown. Using a set of labelled training data to learn in this manner is known as supervised learning.

Supervised learning algorithms use training data to create a model which describes a probabilistic relationship between input and output data, and are then able to use this model to make predictions. The fundamental goal in supervised learning is to create models that generalise well. This means that they must make accurate predictions when given future, unknown inputs rather than simply modelling the training data perfectly [1].

In this tutorial you will investigate a popular model called the decision tree, and see how it can be applied as a diagnostic tool in medicine.

A data set for medical diagnosis

The training data set we will use in this tutorial is a subset of the Breast Cancer Wisconsin (Original) Data Set, courtesy of the UCI Machine Learning Repository [4]. Each training example contains a set of measurements, called features, and a corresponding classification as either malignant (true) or benign (false). The 9 features were obtained from Fine-Needle Aspiration Cytology (FNAC) measurements of human tissue. Each feature has been normalised to lie in the range 1-10, with high valued features thought to represent more malignant (cancerous) cell characteristics. All feature values are integers.

  #  Feature                       Domain
  -- -----------------------------------------
  1. Clump Thickness               1 - 10
  2. Uniformity of Cell Size       1 - 10
  3. Uniformity of Cell Shape      1 - 10
  4. Marginal Adhesion             1 - 10
  5. Single Epithelial Cell Size   1 - 10
  6. Bare Nuclei                   1 - 10
  7. Bland Chromatin               1 - 10
  8. Normal Nucleoli               1 - 10
  9. Mitoses                       1 - 10

In this tutorial we will represent a single training example as a list containing 10 elements - the 9 input features and the corresponding class. For example represents a training example whose class is benign. The training data set is then a list of training examples - a list of lists.

A test example, e.g. from a new patient, however contains only the 9 input features. For example,
represents a test example whose class is unknown. The test data set is then a list of test examples - also a list of lists in Snap. Your overall goal is to learn good decision rules from a training dataset, so as to classify the test examples correctly.

Task 1: Decision trees and classification (40%)

Trees are one of the most widely used data structures in computer science. They are used for searching, sorting, representing hierarchical relationships and even finding routes for data over the Internet. Recall that a tree consists of a single root node, a number of internal nodes and leaf nodes, and that these nodes are connected with edges.

A decision tree is a special type of tree used to classify input data into one of a number of classes. Each internal node (or decision node) of a decision tree contains a decision rule, and each leaf node contains a class label. Here is a simple decision tree for our FNAC domain with 4 decision nodes and 5 leaf nodes:

This decision tree takes a set of features as input and produces a classification of either Benign or Cancer as output. To decide the class of a new input vector we start at the root node and continue down the tree, repeatedly applying decision rules. Each decision rule compares a particular input feature with a fixed threshold, and depending on the feature value we continue down either the left or right subtree. When we reach a leaf node we stop and this becomes the classification for our input vector.

The decision function at each decision node in our tree can therefore be described by two parameters: a feature number and a split point. The number in brackets next to the feature description at each decision node is the feature number. The number next to the edges leaving a decision node is the split point.

Discussion question

Under this decision tree, would the input vector be classified as Benign or Cancer?

Recursion with trees

A tree is an example of a recursive data type. The children of a node in a tree are themselves trees - large trees are created simply by joining smaller trees together. In the example above, the subtree rooted at the node labelled Clump thickness and the subtree rooted at the node labelled Bare nuclei are both perfectly valid decision trees in their own right. We were able to construct a larger decision tree by joining these two trees with a new choice node labelled Marginal adhesion.

Defining trees in this recursive manner allows us to write beautifully simple algorithms that achieve non-trivial results. For example to classify a new input vector, all a choice node needs to do is pass the input vector to its left or right child and ask it to perform the classification. In this task, you will add a classify method to both the choice nodes and leaf nodes and see how this works.

In this task the tree structure is fixed (- later, you will see how the tree structure can be learned from training data).

Instructions

The Snap! project link for this task is here.
The code around the top of the scripting area will create a fixed decision tree and show it on the stage.

Opening the block will show you its definition in Snap! that correspond to its shape drawn on stage. If this looks very busy, don’t worry, the rest of this task will break down and explain each component in turn.
We start by implementing simplest decision tree ever - the leaf node. We also use this part to look at the concept of implementing a function as part of a list (i.e. a gray ring).

The inpupt to a leaf node is a test example, the output is a decision (true or false). There are two types of leaf nodes: One that always returns (cancer) or one that always returns (benign). Therefore the expected behaviour of a true leaf node, invoked with given input data, is shown below

Therefore, the leaf node should always report a class label. This class label should also be the same as the type of leaf node (i.e. TRUE or FALSE in angle slot).

Look at the snap workspace to see how this is implemented.
You next task is to complete the choice node:

Currently, it will also report Not implemented when asked to classify an input vector. Your goal is to edit the node type blocks so the decision tree reports either or for an input vector according to the structure of the decision tree. Similar to the leaf node, replace with your code.

Two utility blocks to help you visualize the process:
- draws a decision tree to the stage.
- can be used to visualise how a given decision tree should classify a new input vector. For example, the visualization of a small decision tree shows:
Some more hints:
- Remember, you are implementing a recursive algorithm. Rather than report or directly, a decision node should delegate this decision to one of its children.
- For leaf nodes, the implementation forms the base for your recursion and should return either or .
- Because you are changing the code that creates tree nodes, your changes will not impact trees that have already been built unless you tell Snap! to build them again. Make sure Snap! is calling and every time you test your code.
The following three tests should help you trouble shoot and test your program as you write it:
- should reproduce the two-level tree drawn above, and report .
- prints a message to the bottom of the stage. For example: this prints the classification result of the original tree above, it should say “Classification: false”.
- On the other hand, a slightly different test example should say “Classification: true”.
When you think your implementation is working run the following test:
- . This will test your implementation with a range of different inputs and report Pass or Fail.

Discussion

Execute these blocks at the end of the Snap workspace . Notice that this tree here is the exact converse of the 3-level decision tree earlier – whenever the other tree says ‘cancer’ this tree will diagnose ‘benign’ … So which tree is correct? The next few tasks will explore this question.

Reading - Choosing a decision rule

Remember that our goal in decision tree learning is to build a decision tree automatically based on our training data. How can we do that?

Greed is good

In computer science, a greedy algorithm is an algorithm that finds a solution to a problem by making locally optimal decisions. Greedy algorithms make a series of choices that seem best at the time but depending on the problem may not produce a globally optimal solution. These algorithms are often useful when finding a globally optimal solution is not computationally feasible. In the next two tasks, we will use a greedy algorithm to build our decision tree, choosing decision functions at each node that appear to best split the data we have at the time.

We are going to start with all our training data, and build our tree by splitting our data set over and over until we reach some stopping criteria. The algorithm we will use to choose a feature and split point given a set of training examples is as follows:

Input: dataset with N features
Output: (feature, split point) pair

for each feature 1 to N:
    sort dataset by feature
    for each split point in sorted dataset:
        evaluate the gain of split
        if gain is the highest seen so far:
            remember gain, feature and split point
return highest gain, and the corresponding feature and split point

The gain of splitting on a given feature at a given split point is a measure of the quality of the split. At each stage, we want to choose the split with the lowest cost. To define this cost concretely, we need to understand the concept of entropy.

Discussion questions

Why do we sort the data set before looping over possible split points?
Is finding the globally optimal split possible, and computationally feasible?

Information entropy

Entropy gives us a measure of the uncertainty associated with a probability distribution. Suppose we roll a die and wish to predict which outcome in the set $C={1,2,3,4,5,6}$ will occur next. If the die is fair, each outcome in $C$ has an equal probability of occuring ($p(x) = \frac{1}{6}$). Because of this, it is difficult to predict which number will come up next - the outcome is uncertain. We say that the distribution of outcomes has high entropy. Conversely, the distribution of an unfair die (that is more likely to roll some numbers than others) has lower entropy since there is less uncertainty associated with rolling it.

Given a random variable $X$ with probability mass function $p(x)$, the entropy $\mathbb{H}(X)$ is defined as:

$$ \mathbb{H}(X) = -\sum_x p(x)\log_2 p(x) $$

taking $p(x)\log_2 p(x)=0$ whenever $p(x)=0$ [3]. The more uniform the distribution $p(x)$, the higher $\mathbb{H}$ will be. $\mathbb{H}$ is at its highest when all outcomes are equally likely.

Cost function revisited

In this exercise, we define the cost of a split in terms of the entropy of the sublists that the split creates. If we consider the proportion of training examples from each of the two classes as an approximation of a probability distribution over the classes, then a set of training examples from mostly the same class has low entropy and a set with an equal number from each class has high entropy. We therefore define our cost function as:

cost of split = (proportion of examples in left sublist) * (entropy of left sublist) + 
                (proportion of examples in right sublist) * (entropy of right sublist)

Minimising this sum will result in both sublists having low entropy and therefore gives us a split with the best possible class separation [2].

Discussion questions

Why do we weight each term by the relative size of the sublist?
What other metrics could we use to quantify the quality of a split?

The block implements the feature selection function. The block takes a list of training data as input and should report the index of the optimum feature as well as the optimum split point for that feature. To report a pair of values in Snap!, report a list with two elements: .

Notes

We encourage you to look at the implementation of the feature split block and note the following technical aspects.

Sorting is used so we can efficiently check all the possible split points for a given feature. The variable is used to keep track of the possible split points when iterating over the sorted data.
The block has been created to implement counting sort, since comparison sorting algorithms are incredibly slow in Snap!. Note that this block relies on the features being integers from 1 to 10 and will not work otherwise.
Even using counting sort this procedure can be quite slow to run, so it is useful to add status messages such as:

to view progress of the algorithm.

Discussion question

Why does this block report the cost rather than the information gain as described in class? Do they lead to the same results?

Task 2: Building a decision tree (40%)

Now that we have a block to tell us the best feature and split point given a set of data, all that is left to do is to greedily build our decision tree. In this task, we will use a recursive algorithm to do this.

The tree building algorithm we will use is as follows [3]:

Input: training dataset, maximum tree depth
Output: root node of a decision tree constructed greedily using minimum entropy

if depth = 1 or entropy of list = 0:
    # Stopping criteria reached
    decision = most common class in dataset
    report a leaf node with this decision
else:
    j, t = choose a feature and split point for this dataset
    left sublist, right sublist = split dataset according to (j, t)
    left child = call this procedure recursively on the left sublist with depth - 1
    right child = call this procedure recursively on the right sublist with depth - 1
    return a decision node with the above split and children

Notice that this algorithm is recursive - it contains calls to itself.

Instructions

Load Snap! project here.

You will also need to import the two blocks from Task 1 for this task to work. Namely, and .
Implement the algorithm above when depth = 1.
- This block should report a decision tree of depth 1 (what kind of node is this?)
Test your code by learning on the following simple dataset with one feature, and making the tree classify two new examples, as shown below.

Hints/discussions:
- What should the ‘tree’ here look like? Can you express it using the leaf node block?
- What does it do on the input with featuer “2”? (true)
- What does it do on the input with featuer “6”? (true) What should it do? (false) – the next step would fix this.
Now let’s implement the tree with depth=2.

Hints/discussions:
- What should the ‘tree’ here look like? Can you express it using the leaf node and choice node blocks?
- Does it now classify the input with feature “2” and “6” correctly?
Refining your implementation: If you tell the tree to learn from the same 3-example dataset, but with depth=3, what should happen? Do you actually get a tree with depth 3, why or why not?

Test your implementation with the blocks below, with the 3-example dataset, and a 4-example dataset of 2 features.
Complete the implementation of block . This block takes a list of training data and a maximum tree depth as input and should report the root node of a tree constructed according to the greedy algorithm above. Test it by executing this block , speed it up using “warp” after the code is correct.
When you have constructed the tree, use to visualise your tree. Does this tree look reasonable? Does it conform to the meanings of FNAC features, i.e., higher feature values represent more malignant cell characteristics?
Use to check your results. This block takes a decision tree as input and checks the classification of some input vectors against a reference implementation.

Extension questions

In this task we used two stopping criteria:
- Maximum depth
- Entropy equal to zero.
What other stopping criteria could be used? What impact do the stopping criteria have on the performance of our model on future, unknown input data?
In computer science, an optimal solution is a solution to a problem that is at least as good as any other solution.
- How would you define an optimal solution to the decision tree problem given a set of training data?
- The problem of finding an optimal partitioning of the data is NP-complete [3]. Loosely speaking, this means we do not currently know of an efficient algorithm to solve the problem for large data sets and it is quite likely that there isn’t one. Can you describe a ‘brute force’ algorithm that reports the optimal solution given a set of training data? How do you think the running time of this algorithm would compare to that of the greedy algorithm you have implemented in this task?

Task 3: Analysing performance (20%)

The decision tree you have constructed in this tutorial classifies the training data perfectly. Is this a good thing? If our training set instead consisted of 1,000,000 samples, are we better off with a deep, complex decision tree that fits our training data perfectly or a simpler tree that makes some errors on the training set?

Creating incredibly complex models that match our training data perfectly is known as overfitting, and is something that we must actively avoid in machine learning. Remember that our goal in machine learning is to achieve good generalisation behaviour - we want our to model to predict future, unknown inputs well, not just perform well on the training set (in other words, remembering what it was taught on).

We cannot judge generalisation performance using the same data set we used to train our model. For this we must use a separate set of labelled examples which we call a test set [1].

Exercises

Complete the implementation of block that takes a decision tree and a labelled data set and reports the fraction of misclassifications that the decision tree makes on the data.

For example, should return 0.5, as the first example is correctly classified, while the second is wrongly classified. Use a few more example trees and datasets to test this block.
Training-testing protocol.

Build a decision tree using the existing data and your block .

Draw the tree on stage using

Measure training error use your block to measure how many mistakes this tree has made on training data.

Download test data Use the block to download the breast-cancer-50.data data set. This data set contains 50 new labelled examples. All datasets in this lab reside in the same web directory, you can download them and assign to a new variable like this:

Measure test error Use your block to measure how many mistakes this tree has made on test data.

In your report:
- Include a picture (snap! stage) of the learned decision trees, at depth 4, 3, and 2, respectively.
- Include a table containing the training and test errors these trees made on training and test data, respectively.
- What are the training accuracies? Should it be 1.0?
- Why are the testing accuracy not 1.0? Should it be?
- What seem to be the right depth of this decision tree, why?
- Given these results, if you were to train a tree of max-depth=5 using the original training data, what will the tree look like? – Draw it in your report or describe in words.
- Discuss (and not implement) the following scenario: If you were to write a modified decision tree learning routine, in which each choice node can split one feature to up to three branches (e.g. feature 1 <=2, 2 < feature 1 <= 4, feature 1 >4), what needs to change in our program? Will you get a different decision tree? Will this generate a fundamentally better tree than a binary tree?
The effect of training data

Build two decision trees up to depth=4. One using the original training data, the other use breast-cancer-train2.data.

Test them on two datasets: breast-cancer-50.data as above, and data-50-2.data.

In your report:
- Include pictures of the two decision trees being compared here.
- Include a small table of the testing accuracy of the two trees on the two datasets.
- Which decision tree performs better on the two datasets? Can you offer some explanations as to why?

Discussion:

So how do we generally ensure that the learned model is of good quality? One commonly used technique is cross-validation. Read about this technique in Section 1.3 of Bishop’s book [1] – this section is included in the preview here.

References

Bishop, Christopher M. Pattern recognition and machine learning. Vol. 1. New York: springer, 2006.
Mohri, Mehryar, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of machine learning. MIT Press, 2012.
Murphy, Kevin P. Machine learning: a probabilistic perspective. MIT Press, 2012.
Wolberg, W., Mangasarian, O.: Breast Cancer Wisconsin (Original) Data Set (1992), https://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+%28Original%29 (accessed May 2014).

Lab 7: The Sentiment of a Superstorm

Enthusiastic Hugo User — Tue, 18 Apr 2017 00:00:00 UTC

For this tutorial, we’ll explore how we can use social data (tweets from Twitter) to enrich our understanding of one of the most destructive hurricanes in US history: “Superstorm Sandy”.

The first part of this lab sheet contains reading material for both sentiment analysis and hashing, the second part contains the exercise questions. Skip directly to the practical part of you are impatient.

Also note that there are five scoring parts in total, Exercise 6.1 - 6.4, and visualization.

Table of Contents

Part I - Preliminaries

Making Sense of Social Media
Concept 1: Sentiment Analysis
- A Simplified Approach to Determining Sentiment
- Sentiment Lexicon
- Steps for Determining Tweet Sentiment
Interlude: The Dictionary Problem
Concept 2: Hash Tables (not #HashTags)
- Hash Functions
- Buckets and Hash Collisions

Part II - Programming tasks

Practical: Sentiment Analysis of Superstorm Sandy
- Implementing a Hash Function
  - Exercise 6.1: Implement a hashing scheme (3 points)
- Implementing a Hash Set
  - Exercise 6.2: Implement the insert operation (3 points)
  - Exercise 6.3: Implement the contains operation (2 points)
- Testing
- Exercise 6.4: Computing the sentiment score of a Tweet (1 point)
- Sentiment Analysis and Visualisation of Superstorm Sandy Tweets (1 point)
- Extension Tasks
  - Extension Task 1: Implement the remove operation
  - Extension Task 2: Implement dynamic resizing (rehashing)
  - Extension Task 3: Implement the dictionary data structure (hash table)
References

The prevalence of social media is changing not only the way society communicates and interacts, but also how we think about social data. For instance, can real-time social data be used to better detect the occurrence of an earthquake?[1] Can social media aid disaster response to wildfires?[2] What other kinds of activities could be enriched by the analysis of social data?

The abundance of data presents us with the unique opportunity to better understand and utilise social behaviour and phenomena. This has lead to the rise of Computational Social Science; an interdisciplinary field which explores how computational techniques from computer science can be applied to various fields of social science.

To do this we’ll first review two concepts: sentiment analysis and hash tables.

Concept 1: Sentiment Analysis

There are many ways we could make sense of textual data from tweets. One way is to consider whether the emotional state or attitude it expresses is either positive, negative or neutral. This is called sentiment polarity.

For example this tweet would be considered positive in sentiment:

So happy exams are over. Finally done with second year! Now can look forward to my holiday :D #summmerrr

Conversely this tweet would express negative sentiment:

I hope i do good on my finals tommorow #worried

But what about this one? Is the person actually happy? Can we be sure?

i totally failed math finals so that's good

Try determining whether the following tweets about Superstorm Sandy are positive, neutral or negative:

So happy to wake up to #sandy gone!

Everyone Panic!!!1 RT @NBCNews Full moon could make #Sandy's impact worse http://t.co/hHs5l8EY

#sandy i love you. even if your last names not cheeks*

RT @NewYorkPost: Rainbow touches down in lower Manhattan #hurricane #sandy http://t.co/M6MGqGN7 via @kurtdietrich*

RT @NickpcWx: #Hurricane #Sandy cloud cover is covering nearly 1 million square miles of water/land. @reedtimmerTVN http://t.co/STGZxi2X

RT @GovChristie: I don't give a damn about Election Day after what has happened here. I am worried about the people of New Jersey. #Sandy*

Discussion Questions:

How did you determine their sentiment polarity?
Could you write out a list of steps used to determine this?
Would others following your steps produce the same results?
What makes determining sentiment difficult? Why is it relatively easy for humans to classify sentiment but not computers?

A Simplified Approach to Determining Sentiment

A simple way to determine sentiment polarity for a tweet would be to determine how many words in the tweet are positive, negative or neutral in sentiment polarity. If there are more positive words than negative words, then the tweet is considered positive. Conversely if we have more negative words than positive words, then the tweet is negative. Otherwise if they are equal, then the tweet is neutral. But how do we determine the sentiment polarity of a word?

Sentiment Lexicon

Determining the sentiment of a word can be accomplished by using a sentiment lexicon. This is a collection of words (or phrases) along with their associated sentiment polarity. For instance a lexicon of positive words could contain the words “happy” and “good” whilst a negative word lexicon could contain the words “sad” and “bad”. Thus the sentiment of word can be determined by checking if it is in either lexicon.

Such a sentiment lexicon can be described as an example of labeled data. This is because the words have been assigned a “correct” label (sentiment polarity). This is accomplished either by experts manually assigning a “correct” value or by using machine learning techniques to determine this.

Discussion Questions:

What are some of the limitations of this approach?
Is the labelled data “correct” across different social domains?
- Compare how are the words “bull” and “bear” are used general and in the context of financial markets
Is the labelled data “correct” in different contexts?
- Compare the word ‘low’ in the context of “low cost” and “low salary”
Has social media made it easier to create or access labeled data?

Steps for Determining Tweet Sentiment

Since we are interested in whether a tweet contains more (or less) positive words than negative words, we can assign positive words a value of 1 and negative words a value of -1. Thus if we sum these values for all words in a tweet, we get a score that tell us whether a tweet is positive, negative or neutral in sentiment. We can describe these steps informally as follows:

input is tweetText
set sentimentScore to 0
for each word in tweetText
    if positiveWordLexicon contains word then
        add 1 to sentimentScore
    end                  
    if negativeWordLexicon contains word then
        subtract 1 from sentimentScore
    end
end
output is sentimentScore

Thus a sentimentScore greater than zero indicates that the tweet is positive. If it is below zero, then the tweet is negative. Otherwise if the score is zero then it has a neutral in sentiment polarity. Note that we ignore neutral words since they won’t change the score.

This description of steps used to determine sentimentScore from tweetText is an example of an algorithm. The informal description is known as pseudo-code, as it cannot be read by a computer, however to be useful it should loosely relate to a language you’re familiar with. We use pseudo-code to focus on how the algorithm works rather than language (Snap!) syntax and rules.

Discussion Questions:

Can you tie back all the pseudo-code to the Snap! blocks you have seen/used?
How would you determine the words of a tweet?
- Look back at the example tweets, what would you need to exclude or consider?
What concerns would you have if the sentiment lexicon contained a very large number of words?
Suppose that you have a physical copy of all 20 volumes of the Oxford English Dictionary (2nd Edition), which comprises of over 400,000 entries and 22,000 pages.[4] How would you quickly look up a definition for a particular word?

Interlude: The Mailbox Problem

Imagine that you are post-office clerk looking for a letter in a pile that has been sorted alphabetically. You’d flick to where you’d estimate the letter to be, then flick left or right through the letters to get closer to the desired letter (or determine that it is not present). This type of searching is fast, but is there a faster way?

Instead imagine that you live in a residential hall or college and you are trying to find your mail. There are a number of letter-box, one for ‘A’, another for ‘B, another for ‘XYZ’, etc. To find mail for a person, you simply take the first letter of the person’s last name, go to the respective letter-box and search through the letters to find the person’s mail. This scenario roughly describes the idea of a hash table.

Discussion Questions:

Some college kids are pranksters. One has randomly redistributed the mail into the letter-boxs. You face the same problem of finding your letter.
- What is the maximum number of letters you would have to look through to find your letter?
- What is the least number of letters?
- On average, how many pages would you have to look through to find your letter?
- How many letters would you have to look through to organise all the mail?

Concept 2: Hash Tables (not #HashTags)

Similar to how a residential college stores a letter associated with a person, a hash table stores a value associated with a key. Hash tables store entries using a hash fuction for faster look up.

Hash Functions

A hash function is simply a function that takes an input value of arbitrary size and outputs a value of fixed sized. For example consider the following hash function which takes an integer k as input (for positive numbers mod is that same as remainder):

$$hash(k) = k \mod 3$$
$$hash(1) = 1$$
$$hash(2) = 2$$
$$hash(3) = 0$$
$$hash(23412517) = 1$$

No matter how large the input integer is, the function will always return a value between 0 and 2 inclusively. This is called a hash value and it is useful because just as a physical residential college has a finite number of mailboxes to put letters in; computers have a finite amount of memory that can be used to store data. Where we store our entries in memory is called a bucket.

Thus if we have n buckets to store our entries in, our hash function should output a value corresponding to a bucket:

$$hash(k) = k \mod n$$

Since we want to associate a key with a value, we hash the key to determine the bucket. We then store the key along with the value in the bucket. This is just like having letters with people’s names on them. Otherwise, when we flick through the letters in a mailbox, we wouldn’t be able to tell which ones were ours.

In the mailbox example, we already defined an intuitive hash function. The key/input is a arbitrary person. To find the hash value, you take the first letter of a person’s surname. This outputs a fixed character between ‘A’ and ‘Z’. Thus, we have a hash function.

Discussion Questions:

Why would it be undesirable for a hash function to give a different output value each time it is applied to the same given input?
What would happen if we have 100 words to store but only 99 buckets to put them in?
What would happen if we have two different input values that have the same hash value? How would we store them?

Buckets and Hash Collisions

A hash collision is when two different input values have the same hash value. Collisions are problematic because we need to somehow store and look up multiple values in the same bucket. One way to handle this is to use an addressing method called separate chaining.

The idea of separate chaining is to treat buckets as a list of entries, where one entry is linked to another in a chain. Thus if we want to look up an entry in a bucket, we would need to go through the chain until we find it or reach the end of the chain. If we want to insert an entry we would need to go through the chain to check if it is already there and if so, we update its value. Otherwise we add the entry to the end of the chain.

To illustrate this, simply observe the mailbox example. When there is a hash collision, like in the case of ‘Bruce Wagne’ and ‘Andrew Wrigley’, both letters are stored in the same mailbox. If Bruce wanted to find his letter he would have to look through the pile/list of letters in the box. In this case if you wanted to add a letter to a mailbox, you look through the pile to find a letter addressed to the person and put the new letter next to it, otherwise you just put it on top of the pile.

Discussion Questions:

Are hash collisions avoidable? What would be needed to avoid any collisions?
Would it be better to have more or less buckets? What would happen if we only have one bucket?
Why would it be undesirable for our hash function to put more entries into one particular bucket over others?
Why might a residential college in Australia have a ‘XYZ’ mailbox rather than a box for ‘X’,‘Y’ and ‘Z’? Would this differ in other countries?

Practical: Sentiment Analysis of Superstorm Sandy

Files Needed:

The Snap! workspace with the Hashset project file is here
The Snap! workspace Sandy Tweet Visualiser project file is here

To demonstrate both of these concepts, we’ll use Snap! to analyse the sentiment of tweets about Superstorm Sandy from the United State of America and visualise their polarity on a map.

To do this we’ll need to use a sentiment lexicon of positive and negative words and store them in a hash table for fast look up.

But since our sentiment analysis does not take into account different degrees of positive or negative polarity (i.e. +0.5 or -0.6), it isn’t necessary to store both a word and its sentiment polarity value. Instead if we store all positive words in one hash table, and all negative words in another, then we only need to store the word itself since we will know the polarity (positive or negative) of all words we look up in that particular hash table. This is the idea of a Set, which is simply a collection of distinct elements.

Here’s an example of a set of positive words and negative words:

$$S_{positive} = \left\{ \text{happy, love, relief} \right\}\\ S_{negative} = \left\{ \text{angry, damage, afraid} \right\}$$

Note that there can be no duplicate elements in a set and that the size of the two example sets is 3.This size of a set is often referred to as it’s cardinality.

Since we are using hash tables to implement a set, we call this a hash set.

Implementing a Hash Function

As you’ve read, a hash set (or hash table) depends on a hash function for all its operations. However hash collisions are bound to happen and handling these collisions adversely affects the performance of our operations. Consider the example where our hash set uses a hash function that puts all words into the same bucket. To look up or remove a word we would need to possibly go through all the words we have inserted because the bucket contains a chain that is as long as the number of words we have inserted. This is akin to you looking for your letter, when the letters have been randomly placed in boxes.

On the other extreme end, if we have at least as many buckets as we have words, then a hash function that allocates a word to its own bucket would perform extremely fast since the chain is as small as possible since there are no collisions!. Such a hash function is called a perfect hash function.

However for most practical purposes, hash collisions are unavoidable. Instead the goal is to create a hashing function that is as uniform as possible. This means that our hash function should roughly distribute the same number of words into each bucket. Thus if we have m words and n buckets, we would expect each bucket to contain m/n buckets.

Since the input of our hash function is a word, we need to devise a method (or algorithm) of turning that word into some number. Ideally this number should be determined by using all the characters in a word. We call this a hashing scheme and it is simply a function that takes a word as input and outputs a number.

Once we define a suitable hashing scheme our hash set will use it to define its hash function as shown below:

Notice that uses another function to transform the element (in our case a word) into a number so it can perform modulo on it using in order to allocate it to a bucket. The addition of 1 is because modulo can return 0 but Snap! counts positions from 1 (indexing begins at 1).

Thus the ability of a hash function to distribute elements uniformly across the buckets depends on devising a ‘good’ hashing scheme.

Exercise 6.1: Implement a hashing scheme (3 points)

To illustrate a bad hashing scheme, we’ve implemented a very simple one that simply uses the first letter of a word. This is bad because it means that all words starting with the same letter would go to the same bucket resulting in a lot of collisions. The goal for this task is to try to come up with a better hashing scheme that results in a hash function that is uniform.

Programming Instructions:
Complete this task by editing and completing the definition:
.

Your code is expected to:

Use the block to transform a letter of a word into a number so we can perform calculations on it. This table outlines the mapping between any character to numbers – it contains not only English (or called ascii) characters, but also alphabets from any written language in the world.
Using all the letters of a word would help to avoid collisions.
Using both the letter and their position in the word will help distinguish words that are anargrams of each other, e.g. dog or god.
Use an appropriate hashing scheme so that your hash set on a modest list of English words is as uniform as possible (i.e. passes the uniformity test block after finishing the hash set implementation.)

Hint: Prime numbers are useful for preventing collisions. See the 3rd answer answer on this page reports an observation of a collision test on 50,000 English words.

Test this block: After finishing your implementation, type a few different words in the input slot. Do you get a (large) integer from each that are different?

Implementing a Hash Set

A hash set is a specialised version of a hash table. A hash table stores both a key and a value using a hash function. In our mailbox analogy, a person was the key and their associated letters was the value. However for hash sets we only store the key. Thus if we don’t need to store a value associated with a key, it is more efficient to use a hash set. Whilst both use hash functions to also look up a key, the difference is that a hash table returns the value associated with a key, where as a hash set simply tells you if it contains or doesn’t contain the key.

A hash set and a hash table are both examples of what is called a data structure. A data structure is simply a particular way of storing and organising data so that we can perform certain operations on it in a efficient manner. For example, a list is also a data structure. In this case, hash functions are used to store and organise the data for fast look up.

For the purposes of this practical we only need to implement two operations: one that insert words into the hash set and another that checks if the hash set contains a particular word.

Exercise 6.2: Implement the insert operation (3 points)

The insert operation allows you add additional elements to the set. For our use case, this will be to add additional words into the set.

To insert an element into the set using separate chaining, we’ll need to first create a hash value in order to determine the bucket we will insert it into. Then we’ll need to check if the bucket is empty; if so we can simply add the element, otherwise we’ll need to add it alongside the other elements already in the bucket. But since sets do not contain duplicates, we should only add the element if the bucket doesn’t already contain that element.

Programming Instructions:

Completing the definition for by filling in the stub with working code:

Hints:

There is a special constant value called , which is set as a global variable. Use this to check if a bucket is empty.
Buckets need to be able to hold multiple elements (due to hash collisions). What Snap! block can we use to hold multiple elements?
A set cannot contain duplicates of an element. Check that an element is not already in the set before you actually insert it.
Inside a hash set there is a variable called that keeps track of the number of elements entered into the set. Be sure to increment this by one every time you insert an element.
At the end of the ‘insert’ we report the block . It is essentially because this representation of the hash set is implemented using lists. You don’t strictly need to know why this is, for your function to work. If that didn’t make sense, then don’t worry about it.

Testing this function:

This group of blocks can be used to visualise the hash set buckets. Use it to test your insert and remove functions.

Exercise 6.3: Implement the contains operation (2 points)

The contains operation is used to check if a element is a member of a set. We will use it to check if a word is in the positive lexicon or the negative lexicon.

To check if a word is in a hash set, again we must determine which bucket it belongs to using . Once we have determined which bucket to check, we need to check if our word is contained in that bucket. If we find it then we report , otherwise we report .

Snap Instructions:
Complete this task by completing the definition for by filling in the stub with working code:

Hints:

If the bucket is empty then it clearly cannot contain the element. How can we check this?
This lab included a utility block in the ‘variables’ group, called foreach

Testing this function: You can test that your contains function is working by using the on your customised hashset, like when testing insert.

Testing

We have provided a number of test blocks, sometimes called a test harness in Software Engineering, to ensure your blocks (code) works.

To use them to test your blocks, simply provide the test harness with a hash set that has a provided hash scheme as shown below:

For the uniformity test, you need to provide the block with a set of words. You can use any of the word sets provided. Notice that each word set will produce a different histogram. You can use this histogram to evaluate how uniform your hash function is. If buckets have a roughly equal number of elements, then your hashing scheme is uniform.

The hash scheme can be , or any other suitable block. Make sure that it is put in the correct place.

Run the test by clicking on the block. It will then then tell you whether it passed or why it failed.

Your implementation should pass all three tests.

Discussion Questions:

Does testing ensure that your code is completely correct?
Why do different word sets produce different histograms?
How does changing the hashing scheme affect the histogram produced?
Why might Dr Seuss, Rhyme and Alliteration cause particularly bad results for the example hashing scheme (remember how the example hashing scheme works)?
Is it possible to make a set of words which fail the uniformity test, with your hashing scheme?
- Is this true of any hashing scheme in this scenario?

Exporting blocks for Exercise 6.1-6.3:

This assignment has two parts and you must submit them separately. You should export the blocks that you defined in this lab, along with any other custom blocks that are used by those.

“word hashing scheme”,
“insert __ into __”,
“__ contains __”,

Test before you submit! Always, test your submission file – to make sure everything that should be included is indeed included!

The first test for this assignment is included in the test harness above.
The second test you can run, is via the Superstorm Sandy visualizer below.

Sentiment Analysis and Visualisation of Superstorm Sandy Tweets (1 point)

Once you’ve finish implementing your hash set and sentiment score you can now use it to analyse the sentiment of tweets about Superstorm Sandy[3] over time and visualise them on a map of the continental United States of America (excluding Alaska and Hawaii).

Exercise 6.4: Computing the sentiment score of a Tweet (1 point)

With the hashset implemented, you are ready to compute the sentiment score of a tweet!

First use your own hash set and hashing scheme implementation to construct the positive and negative word lists, by executing the following blocks. You need to import all of your blocks from the first project to the current project.

You will need to put your blocks into their respective locations.

You then modify the block that computes sentiment score to loop over words in the tweet and compute its sentiment score. A screenshot of the function stub for you to fill in the code is shown below.

After implementing it, you should run it through a few test cases. Use the few examples from the beginning of the this lab sheet to test out how well your scoring function works. For example, the block below should return 1 for recognizing the positive sentiment.

Snap Instructions:

This part assumes that you have a saved a copy of the blocks from Exercise 6.1-6.3.

Open a new version of Snap and import the Sandy Tweet visualiser project file (available here)
Import your blocks into the project.
If you’ve imported the visualiser for the first time you need to reset the visualiser:
Similar to testing, simply drag your implemented hash set along with its hashing scheme into the load lexicon blocks and your insert function, and run them:

6. Run the visualiser from your desired starting tweet, where 1 denote the first tweet in the dataset and the 582269 is the last.

Whilst these tweets cover the dates from 15 Oct 2012 to 12 Nov 2012, the bulk of the tweets come from a very specific time period:
The tweets in our dataset were tweeted from 15 Oct 2012 to 12 Nov 2012, however the majority of tweets come from a very specific time, as shown below:

Frquency of Tweets over Time (Hours)

Discussion Questions:

What do you think is the significance of the peaks?
The peak number of the tweets occurs between tweets numbered 115000 and 275000. Try visualising and comparing their sentiment.
Does the sentiment of tweets before the peak differ from the sentiment of tweests after the peak?

Enjoy!

Submitting your sentiment score block:

Export your sentiment computation block and submit it as a separate xml file

“sentiment score (tweet text, positive word set, negative word set)”

Test before you submit! Always, test your submission file – to make sure everything that should be included is indeed included!

Submitting a screenshot of the visualization:

You let the visualizer run for while, and submit a snap of the Snap! stage when you are satisfied with the result. Capturing part of the screen can be done via keyboard shortcut on a mac, or using the Snipping tool on Windows, or the Take Screenshot tool in Ubuntu.

Here is mine, what does yours look like?

Extension Tasks

For those that have completed all the previous tasks, here’s an opportunity to further your understanding, improve your programming skills as well as to earn bonus marks! These tasks have less guidance and support than the previous ones so you are expected to do your own reading or research if needed. You only need to implement one of the following:

Extension Task 1: Implement the remove operation

The remove operation is used to take out elements from the set.In sentiment analysis we can assume all words loaded into our hash sets are correct and thus we didn’t need remove. Our hash set implementation is incomplete without it though.

Snap Instructions:
Complete this task by completing the definition for by filling in the stub with working code:

Hints:

The remove operation is the opposite of the insert operation. Determine what steps you need to take to remove an element with this in mind. Writing pseudo-code might help.
You can’t remove an element that is not already part of the set.
Use the provided test harness to check that it works:

Extension Task 2: Implement dynamic resizing (rehashing)

Ideally, the more buckets we have, the less likely we’ll have collisions. If we could estimate how many elements we’re processing then we can set the number of buckets to at least this estimate. However if we estimate wrong and have too few buckets, then we’ll have unneccesary additional collisions.

Our hash set would be more robust if it could resize itself depending on the load. To do this we will rehash all elements into a larger number of buckets once the number of elements in our hash set exceed some threshod. This threshold is the load factor.

Take a look inside the and you will notice that before any elements are inserted, there is a condition being checked to determine if the hash set is exceeding the load factor:
Unfortunately the is not currently doing anything useful.

Your task is to edit so that it rehashses all the elements into a new larger number of buckets.

Hints:

What variables do you need to reset?
How many element in the hash set do you need to go through to rehash?

Extension Task 3: Implement the dictionary data structure (hash table)

As our approach to sentiment analysis is very simple (words are either positive or negative), we didn’t need to the full capabilities of a hash table to create our lexicon. But if the words in our lexicon were associated with a polarity value between -1 and +1, using hash sets would be infeasible since we would need a hash set for word with a polarity of -1, -0.9 -0.85… all the way to +1.

In this case we should actually use a hash table (dictionary) to implement our lexicon. Since hash tables store a key and an associated value, we can use it to store a word (key) along with its sentiment polarity (value). We can then look up the sentiment polarity for all words very quickly from a single data structure.

In Computer Science, this idea of storing a key associated with a value is called a dictionary data structure and it is useful for a wide variety of applications. One important area of application is indexing which is used to quickly retrieve information from a query. For example web pages from a search or visualising the usage of certain words or phrases over time using Google Ngram Viewer.

Hints: * A hash table is very similar to a hash set, the only difference is that we now need to store both a key and it’s value. As a start point make a copy of your hash set implementation and try to tweak it, to make it a hash table. * Keep the concept of a key separate from the value; we only hash the key but we need to store both the key and the value together in a bucket. * An example of how you could store the key and the value: .

References:

Map Images of United States of America:
Showeet: Hi-Def Editable PowerPoint Map of USA - Free Map (2010), http://www.showeet.com/24/03/2010/maps/hi-def-editable-powerpoint-map-of-usa-free-map/ (accessed April 2014).
Used under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.
Sentiment Lexicon:
Liu, B.: A list of positive and negative opinion words or sentiment words for English (2004), http://www.cs.uic.edu/~liub/FBS/opinion-lexicon-English.rar (accessed April 2014).
Takeshi Sakaki, Makoto Okazaki, and Yutaka Matsuo. 2010. Earthquake shakes Twitter users: real-time event detection by social sensors. In Proceedings of the 19th international conference on World wide web (WWW ‘10). (abstract)
Bertrand De Longueville, Robin S. Smith, and Gianluca Luraschi. 2009. “OMG, from here, I can see the flames!”: a use case of mining location based social networks to acquire spatio-temporal data on forest fires. In Proceedings of the 2009 International Workshop on Location Based Social Networks (LBSN ‘09). (abstract)

Lab 6: Sequence alignment (with pointers)

Enthusiastic Hugo User — Tue, 21 Mar 2017 00:00:00 UTC

This weeks lab is a practical test of algorithms based on “dynamic programming”, applied to one of the fundamental problems in bioinformatics.

Determining Similarity

One of the central problems pursued in bioinformatics is determining similarity between sequences of nucleotides or amino acids (the basic information that makes up our genetic structure). Similarity information is required to perform some typical tasks including:

Using a given sequence of amino acids to infer a protein’s shape and function, and
Finding all the genes and proteins in a given genome.

Usually, when given a particular sequence, you would search through one of the publicly availiable databases (visit, for example, the NCBI database for more information) and find multiple sequences that have varying degrees of similarity to the sequence in question. It is then up to the researcher to select the ones of most interest. In this lab we will focus on the underlying problem of determining similarity between two nucleotide sequences.

Exercise 5.1 (70%)

For this assignment you will write a program in Snap! which finds the optimal global alignment between two sequences of nucleotides.

Snap link: A project file with the framework for the assignment is available here.

The Global Alignment (Needleman-Wunsch) Algorithm

As described in the lectures, the global alignment algorithm is a dynamic programming algorithm. Given two sequences, at any point in the alignment we have three choices:

Align the two nucleotides
Introduce a gap in the first sequence
Introduce a gap in the second sequence

So as we align our sequence, at each point the optimal alignment is given by the optimal alignment up to that point, plus whichever of the above choices gives us the greatest alignment score.

To achieve this we will construct a score matrix $F$ with the number of rows equal to the length of the first sequence plus one and the number of columns equal to the length of the second sequence plus one. Each element in this matrix represents the score of the optimal alignment up to that point in each sequence. To fill in the matrix we start at the top left element (inititalised to 0) and move left-right, top-bottom. At each element $F(i,j)$ , the elements above ( $F(i-1,j)$ ), to the top-left ( $F(i-1,j-1)$ ) and to the left ( $F(i,j-1)$ ) represent the decision to introduce a gap in the second sequence, align the two nucleotides, or introduce a gap in the first sequence respectively. So to fill in in the element $F(i,j)$ we use the formula

where $d$ is the penalty for introducing a gap and $s(x,y)$ is the score for aligning the two nucleotides.

This is illustrated in the following figure

Each element $F(i,j)$ must not only store the score of the optimal alignment, but it must also store a pointer to the cell that it obtained the optimal value from, allowing you to trace the path that corresponds to the optimal alignment when the matrix is completed. Thus, if the finished score matrix is the following

the corresponding optimal alignment is

The optimal alignment is found by starting at the bottom right element in the matrix, and tracing back the steps taken to get there. This is done by following the pointers (arrows) stored at each element. Depending on the direction of the arrow do the following:

Arrow points up: Add the nucleotide for sequence 1 and a gap for sequence 2.
Arrow points diagonally: Add the nucleotide from both sequence 1 and sequence 2.
Arrow points left: Add a gap for sequence 1 and the nucleotide for sequence 2.

Try to reconstruct the aligned sequences above by following the arrows in the completed score matrix.

Implement the global alignment algorithm

The lab project provides a block which creates the score matrix and handles the visualisation. To initialise a new matrix, select (or type in) two sequences you wish to align and run the following blocks

You should see the outline of the score matrix appear on the stage. Your task is to implement the global alignment algorithm, described above. Place your solution in the block. A basic stub of the block is provided to illustrate how to use the helper functions.

The three inputs to the solution block are the values to use for $s(x,y)$ and $d$ in the algorithm:

The first input (good alignment score) is the score for aligning two equal nucleotides, i.e., $s(x,y)$ when $x = y$ .
The second input (misalignment penalty) is the penalty for aligning two unequal nucleotides, i.e., $-s(x,y)$ when $x \neq y$ .
The third input (gap penalty) is the penalty for introducing a gap in either sequence, i.e., $-d$ .

When you have completed and run your code, you can use the following blocks to display the optimal alignment corresponding to the computed score matrix:

Hints

You can use the block to set an element of the score matrix to a particular value. Note that when you do this, the pointer for that element is automatically set to null (a value indicating no arrow). It will also update the matrix shown on the stage.
To set the pointer for an element in the score matrix you can use the block. You should set the pointer to a list which contains the indices of the cell you are pointing to in the score matrix. The pointer (if set correctly) will also show up as an arrow between the entries in the matrix shown on the stage.
Remember to check the edges of the matrix: If $i = 1$ , there is no row at $i - 1$ , and likewise when $j = 1$ , there is no column at $j - 1$ .
You can use local variables, say “Max Score” and “Max Score Pointer”, to keep track of the best choice as you evaluate each one in turn. Calculate the score for each of the choices (skipping the ones that do not apply) and update both variables when you find a new best score. Don’t forget that you must initialise “Max Score” to a value that is lower than the score for any of the choices can be.

Testing

The lab project comes with some sample sequences to test your block on, four of length 7 and two shorter of length 5. The example score matrix above is for aligning two of them, GAATTCA and GGATCGA, with default scores. The optimal alignment has one mismatch (in the second position) and one gap in each sequence.

Try comparing all sequences of the same length pair-wise.

You should also try varying the scores. If you keep the gap penalty at 1 and increase the misalignment penalty to any value greater than 2, you should get alignments that have no mismatches (because a mismatch can always be avoided by introducing two gaps). Verify that your implementation does this! On the other hand, if you increase the gap penalty while keeping the misalignment penalty at 1 you should see fewer and fewer gaps.

Exercise 5.2 (30%)

The block you have implemented so far computes the best alignment between two complete sequences. Sometimes, however, the objective is to find similar subsequences in two given sequences. Fortunately, only a few small modifications are needed to compute an optimal local alignment. (This is known as the Smith-Waterman algoritm.)

For local alignment, the maximum score function is given by

In other words, an alignment between two subsequences with a negative score is never optimal, because in that case a better alignment can be made between shorter subsequences.
When the score matrix is completed and we wish to find an optimal local alignment, we need to start tracing back from the largest score in the matrix (instead of the bottom right corner) and trace back until we reach a score of 0 which is where we stop.

Your task is to implement the local alignment algorithm, in the block called . (Remember that you can use the “duplicate” function to copy your earlier implementation so that you don’t have to rebuild it from scratch.) To find and display the optimal local alignment after you have computed the matrix, use the following blocks:

Testing

You can use the same test cases as for the global alignment. However, local alignment is more sensitive to the score given to a good aligment (match): The higher it is, relative the the misalignment and gap penalties, the more willing we are to accept a worse alignment of longer subsequences. As an example, aligning the sequences GCATGCT and GGATCGA with good alignment scores of 2 and 1, keeping both penalties at 1, gives subsequences of different lengths.

Assignment submission

To submit your assignment, you must export blocks (as described on the lab 1 page. You should export the blocks that you defined in this lab (“Solution”, “Solution for Local Alignment” and “Show optimal local alignment”), along with any other custom you created that are used by those.

Submit the file with the saved blocks through wattle.

Test before you submit! Always, test your submission file – to make sure everything that should be included is indeed included!

To do so:

Start a fresh chrome/chronium session/
Load the Lab 5 Snap project.
Use the “import … ” function from Snap to load your submission file, containing your exported blocks.

Now run these blocks – does you solution still work?

Lab 5: Solving Sudoku

Enthusiastic Hugo User — Wed, 15 Mar 2017 00:00:00 UTC

In this lab, we explore how reasoning, and figuring things out by trial-and-error, can be coded as an algorithm, and use it to solve Sudoku puzzles.

Sudoku: The search for a solution

The most well known form of a sudoku puzzle is a 9-by-9 grid in which the numbers 1-9 must be placed such that each row, column and 3-by-3 box contains exactly one of each of the 9 digits. A typical puzzle will initially have enough of the numbers filled in to ensure that the puzzle has exactly one solution.

Finding a solution to the puzzle is a constraint satisfaction problem, just like the n-queens problem described in the lecture. (This differs from the constrained optimisation problems also talked about in the lectures only in that there is no objective function: Any way of satisfying the constraints is equally good.)

To find the solution we must search through the state space (all possible sets of 81 numbers that satisfy the constraints, i.e. all valid sudoku puzzles), trying different options until we find one that works. One possible (but certainly not ideal) way to approach this problem is to search through the entire state space until you find one that matches the puzzle you have been given. Considering that the total number of valid sudoku puzzles is 6,670,903,752,021,072,936,960, you would hope that there was a better way to go about it.

Indeed, as you have seen in the lectures we can use certain search algorithms to narrow the state space considerably as we work towards finding the solution. In this assignment you will implement one (or two) of these search algorithms to create your very own sudoku solver.

Exercise 7.1 (60%)

Implement the backtracking (depth-first search) algorithm with forward checking to solve 4-by-4 sudoku puzzles. (The rules for 4-by-4 sudokus are the same for 9-by-9’s as described above, but you will only use the digits 1-4 and the box constraint applies to the four 2-by-2 boxes.)

Snap link: A project file with the framework for the assignment is available here.

You must implement the block.

To run your solver, select a sudoku using the block. Then run your Solve block, sit back and watch it solve the sudoku. To check that your finished sudoku is correct (satisfies all the constraints) you can run the block.

Note that your sudoku block must return a value, to indicate whether it is successful or not in reaching a valid solution that fills the board.

A number of helpful blocks are provided:

: This block sets the element at (row) (column) of (sudoku) to the given (value). It also updates what is shown on the stage. If the cell already had a value, it will be replaced.
: This block to reports the current value of the element at (row) (column) of (sudoku). If the element is not set, the block reports the same value as the “Null” block.
: This block clears (removes) the element at (row) (column) in (sudoku), and also updates the drawing on the stage.
: This block implements forward checking. It will check if setting the element at (row) (column) ti the given value in the current partially filled-in sudoku violates any constraints.
: This block reports whether there is any unset element in the sudoku. It returns a list, whose first element is or . If the first element is true, then the next two are the row and column of one of the empty elements in the sudoku.

Extension to Exercise 7.1:

Each time that the algorithm commits to a choice (assignment of a value to a cell in the sudoku puzzle), this choice is has been been vetted by the forward checking procedure. But forward checking is not complete, and therefore sometimes gets it wrong: When this happens, the search algorithm will eventually have to undo this bad choice, and try another option. When this kind of “mistake” is discovered, we call it a backtrack.

Add to you implementation a variable that counts how many backtracks are made while solving each puzzle. (Hint: It is probably easiest to do this with a global variable. If you use a global variable, please name it backtracks, so that we know how to make your exported blocks work. Also, don’t forget to reset it to zero before you start a new search.) Can you determine which of the example puzzles are the most difficult, based on the number of backtracks needed to solve them?

Exercise 7.2 (40%)

The amount of search (backtracks) is influenced by, among other things, the choice of which empty cell is filled in next, and the choice of the order in which the possible values are tried (such as 1,2,3,4 or 4,3,2,1). The block provided simply returns the first unset element in the sudoku. Is there a smarter strategy?

A common rule-of-thumb for variable ordering is to pick the variable (cell) with the fewest options first. In the extreme case, if a cell has only value that passes the forward check, then we cannot make a “mistake” in choosing this value (if that value doesn’t lead to a solution, it’s because of some mistake we had already made earlier).

Change the “find empty element” block so that it checks all empty spaces, and returns (in the same form as before) the one that has the fewest options (values that pass the forward check). To improve the efficiency of your solver, you can make this block return the list of possible values for the chosen empty cell, in addition to the true/false flag and the coordinates of the cell. This saves your search from having to call the forward check again on each value. Remember, though, that it must still return a list containing only “false” if there is no empty cell left.

Extension to Exercise 7.2:

Assuming you did the extension to the first exercise, does the modified variable ordering change the number of backtracks needed to solve the example puzzles?

Assignment submission

Commenting: Good programming practice includes providing adequate and useful comments in your code. Please make sure that you use comments where it is motivated: that is, where you need to explain what is going on in your code.

As important as providing good comments is to not clutter the code with useless comments. In particular, we sometimes provides hints and explanations to you as comments in the project files. These comments do not help us understand (and mark) your code, so they should, in most cases, not be included in the blocks that you submit.

Submission: To submit your assignment, you must export blocks (as described on the lab 1 page). For this assignment, you need to export and submit two files: The first with the “Solve” block that you created for exercise 7.1, and the second with “Find empty element” block, and the “Solve” block if you have modified it, for exercise 7.2. (As usual, also export any other custom blocks you created that are used by those.)

Submit both files through wattle.

Always test before you submit! Make sure everything that should be included is indeed included!

Marking: There are no marks for the extension tasks.

Sources

Nine by Nine Sudoku Puzzle: Licensed under the terms of the GNU Free Documentation License version 1.2 or later.

http://en.wikipedia.org/wiki/File:Oceans_Sudoku17_Puzzle-39451.svg

Lab 4: Recursion

Enthusiastic Hugo User — Thu, 02 Mar 2017 00:00:00 UTC

This week we will practice recursion – with Karel, and with a special-purpose calculator.

This is the farmer sowing the corn, that kept the cock that
crowed in the morn, that waked the priest all shaven and
shorn, that married the man all tattered and torn, that kissed
the maiden all forlorn, that milked the cow with the crumpled
horn, that tossed the dog, that worried the cat, that killed
the rat, that ate the malt that lay in the house that Jack
built.

Children's song "The House that Jack Built", from 
http://www.ling.upenn.edu/~dringe/CorpStuff/Thesis/IntroSyntax.html#detour

Recursion is a phenomena, that exist not only in mathematics, but also in linguistics and physics, arts ….

Snap link: the URL to open Snap! with the environment for this week already loaded is here.

Marking rubric: Getting either one of the two questions right is worth 70%, getting the other one right is worth another 30% of this assignment.

Exercise 4.1 A calculator for combinations (50%)

Motivation:

There are 10 friends wanting to play basketball, how many different arrangements are there to split them into two teams of 5?
There are 8 courses to choose from in Semester 2. You want to take one of them because a good friend is taking it, and need to choose the other three. How many possible choices do you have for the remaining 3 courses?

More broadly, computing combinations is useful in other scientific problems, such as:

Sampling design in ecology, for example https://theoreticalecology.wordpress.com/2014/01/14/sampling-design-combinatorics/
Combinatorial chemistry for designing and discovering drugs, as illustrated in this video.

Calculation: Here is a standard recipe for counting combinations. One way to visualize this recipe is to pile such counts as a triangle, known as Pascal’s triangle. The $k$ -th number in the $n$ -th row is the number of combinations for choosing $(k)$ -items out of a total of $n$ items. Note that you should count both $n$ and $k$ from zero, not from $1$ .

For example, the number $1$ circled in green means there is one way to take $0$ items among $3$ choices, denoted as $C(3,0)$ ; the number $6$ in the blue circle in the figure is the number of taking $k=2$ items from a collection of $n=4$ items, , denoted as $C(4,2)$ .

This triangle has two prominent properties, which will allow us to construct the solution to $C(n,k)$ for any number of choices $n$ and size of selection $k$ .

The boundary of this triangle are all $1$ s. Or intuitively, there is only one way to take $0$ items or all $n$ items from $n$ choices.

A number in the middle of the triangle is the sum of the two number on its shoulder rows above. Intuitively, let’s say we label the first one among $n$ items as “special”, and the combination $C(n,k)$ now breaks down into two cases: (A) We include the special item in the selection, then the problem becomes taking $k-1$ items among the remaining $n-1$ choices (i.e. $C(n-1, k-1)$ ). (B) We do not include the special item in the selection, then we still need to take $k$ items among the remaining $n-1$ choices (i.e. $C(n-1, k-1)$ ). Adding the results from these two possibilities will give us the number of combinations gives us $C(n, k)$ . This insight can be summarized with the following equation, and by the animation on the right.

$$C(n,k) = C(n-1,k-1) + C(n-1,k)$$

Programming:

In the Snap workspace you will find this block , its content has been left blank.

Finish the implementation for , with the following summary of a recursive solution to Pascal’s Triangle.

if k=0 OR k=n
	report 1
otherwise
	report compute_combination(n-1, k-1) + compute_combination(n-1, k)

Note: Your code should check for invalid input. i.e. n and k should be integers, otherwise report error and stop; and k should be less or equal than n, otherwise return 0.

Testing your block: Here, you should test your block for syntax (i.e. it runs and reports back a number), but also compare its results against known results of combinations, such as on the triangle above, and other values of $n$ and $k$ from online calculator http://stattrek.com/online-calculator/combinations-permutations.aspx

Showing errors: If you need to output a warning or an error to the user, you can get Karel to say it for you!

Explore more (this part is worth 0 marks):

This exercise focuses on building a recursive solution for the combination problem. There are ready-made formulas for computing combinations, such as listed on this page http://www.mathwords.com/c/combination_formula.htm You can compare the two solutions.

First implement another block for the formula on the webpage $C(n,k)=\dfrac{n (n-1)\ldots(n-k+1)}{k!}$ using loops.
Call both blocks with some larger $n$ and $k$ , e.g., $n=10$ ( $15$ , $20$ , $25$ , etc.) and $k=5$ .

Which block is faster? Why? Which Solution do you prefer?

Exercise 4.2 Karel as a navigator: following a trace and coming back (50%)

In Exercise 2.1 you told Karel how to follow a trail of beepers. In Exercise 4.2 Karel will gain navigation skills, by tracing the same trail backwards and return to where she started, as follows.

Karel start from the lower left corner of the grid.
Karel follows a trail of beepers until the end, picks up the beepers (at most one at each location) as she goes.
Karel stacks all the collected beepers at the end of the trail.
Karel traces back to the origin on the same trail. Furthermore, she marks her path with a beeper whenever she has made a turn.

You will find a block in the motion category, put your implementation there. You should feel free to change the input of this block as you see fit, or change it to a reporter if you feel it is useful to return values. Leaving it with a default input of 0 also works.

Here are two ways that you can implement this.

Method 1 – one step at a time, using recursion

Follow trail and come back (beepers to drop = 0)

Find the next beeper in the trail
If it is straight ahead
	go forward one step
	pick up the beeper
	Follow trail and come back (beepers to drop + 1)
	go backward one step

If it is left or right
	turn and go forward one step
	pick up the beeper
	Follow trail and come back (beepers to drop)
	go backward one step
	drop the beeper (to mark the turn)
	turn back

If there is no next beeper
	put down beeper (beepers to drop) times

Method 2 – using arrays and reusing “Follow a Trail” from lab 2

Follow a Trail -- but modified to record trail directions in an array

put down beepers at the end (remember to keep enough to mark the turns!)

Follow the recorded trail backwards to the origin, dropping a beeper at each turn

You are free to use either method to implement the solution.

Note that a few utility blocks such as “Beeper in front?” and “Move Karel backward” are included. After each attempt, you can use these set of blocks to clear the field and put Karel back to the lower-left corner.

Testing your block: The project file contains three different trails for you to test your program (as shown below), lay them out using the blocks called “Lay short trail” (recommended for debugging), “Lay beeper trail” and “Lay a long trail” – as shown below.

After executing your block for the “long trail” on the right, the stage would look like this:

Submission: You need to export and submit the blocks called “compute combinations” and “Follow trail and come back”, along with any other blocks you created that are used by these blocks. Follow the submission instructions for file naming and block testing.

Image credits:

Pascal’s triangle. Adapted from Wikipedia - User:Conrad.Irwin and originally User:Drini, http://en.wikipedia.org/wiki/Pascal%27s_triangle#mediaviewer/File:Pascal%27s_triangle_5.svg, CC BY-SA 3.0
Animation of Pascal’s triangle. From wikipedia, by User:Hersfold. Public domain, http://en.wikipedia.org/wiki/Pascal%27s_triangle#mediaviewer/File:PascalTriangleAnimated2.gif

Lab 3: Make Your Computer Sing (tunes) and Talk (nonsense)

Enthusiastic Hugo User — Wed, 01 Feb 2017 00:00:00 UTC

This week, Snap will sing and talk! – all using the notion of lists and arrays learned this week.

Note:

See here for due dates and submission instructions.
The Snap! project for lab 3 can be accessed here.

Exercise 3.1 Snap can talk … nonsense (10%)

You’re now sufficiently warmed up by the singing excercises above on reading and interpreting lists/arrays. Now let’s make Snap! talk (or write), and this is achieved by assembling a list from other lists.

The goal of this task is to get Snap to “say” some literary nonsense that look grammatically correct. One well-known example for this type of literary nonsense is “the cow jumped over the moon” from the English nursery rhyme Hey Diddle Diddle, as illustrated here.

Background: You may ask: why should I care about nonsense like this? Breaking text down into its structrual components, and assembling words are the basic operations of computational linguistics or the computer science sub-field natural language processing. The well-known Turing test, once known as one of the ultimate test of a machine’s intelligence, requires the machine to generate text and carry out conversation with a human. What you are going to do below is a small building block of internet chatterbots, such as A.L.I.C.E..

Now let’s program: This task starts with a simple three-part sentence template “SUBJECT-VERB-LOCATION”. Open the editor for the block called “talking_setup”, you will see three variables SUB, V and LOC, each containing candidates for SUBJECT, VERB, and LOCATION, respectively.

Run these two blocks in workspace , and you will see the same nonsense – “the cow jumped over the moon” printed five times on the screen. Let’s try to get some new nonsense:

Open the block generate_nonsense, and you will see a program as below. It prints “the cow jumped over the moon” by taking the first element from each of the three input lists (called subject, verb and location).

In particular, notice the join block. It does the opposite of the split block introduced above, i.e. assemble a string of text by concatenating a list of words and spaces.
Your task is to make some changes in the join block, so that each time the program will randomly pick one element from each of the candidate lists to assemble a sentence. Notice that the candidate lists have different number of candidate words, make sure your program can deal with this, and when the list of words change.

You will need a new block for this task, . Try clicking it to excute (you get a number!), then execute it again (you get a different number!) … What this block does is equivalent to picking a number (from 1 to 10) from a hat – you won’t know which exact one you will get, but the chances for getting any of the possible number is (roughly^) equal.
Change the candidate words by changing the lists inside the block talking_setup, keep generating sentences until you are happy with the result. If you want, you can also change the template to generate more complex nonsense!

Some more background: You may also ask: why is this nonsense? This can be understood with two concepts. Syntax is concerned with the structure of language. Semantics is concerned with the meaning of words and sentences. In “the cow jumped over the moon”, the syntax (grammar) is correct, but its semantics (content or meaning) does not agree with commonsense – leading to the memorable, and sometimes laughable effects of the literary nonsense. More broadly, the notions of syntax and semantics not only applies to natural languages (e.g., English), they also applies to programming languages. These topics are discussed further in later-year computer science classes such as Document analysis or Principles of programming languages.

Music Preparation (0%)

We’d like you to test your computer sound setup, and briefly review a primer on ‘singing’.

Testing your setup

Plug an earphone to your computer, open Snap!.

Find this block from the left-hand-side menu, drag it to your workspace, click to execute it.

Did you hear a note? (if not, check your sound settings and computer volume)

Try changing the first and second number in the block to something else. How did the sound change?
Review: Notes

A note is the basic building block of music.

Now lets put a few notes together and hear them. Put together five note blocks as shown below, change the first number to successively increment by one.

Click to execute, and you will hear the first five notes in this picture , i.e. the standard chromatic scale starting from C, or the five shaded white and black keys on a piano keyboard

Now let’s do this: Make a program (i.e. a set of note blocks) so that Snap! sings the C Major scale, i.e. notes below, or the white keys within an octave on the piano keyboard.
Review: Rhythms

So far we’ve been changing the first number on the note block – this is the pitch, mapped to numbers in midi notation (so that 60 is C, 69 is A, and so on).

The second number represents the note value, i.e., how long the note should be sung.

Now let’s implement rhythm on an old-and-boring scale with dotted notes, a most-commonly used note modifier. Take the first five notes from your C-major scale above, and change them as follows:

Excute this code and listen. Does this sound a little less boring to you?

Question: What rythm(s) cannot be accurately represented in the Snap! block we use?

Question: What is the score of these five notes with dotted rhythm?

Exercise 3.2 Singing a tune (30%)

How can you teach Snap! to sing any tuned? Let’s use what you learned about lists and arrays to help here.

First open this link to see the project for lab3. You’re provided an first list in a variable called “score_amazing_grace” –containing the opening phase for Amazing Grace. Click on the organge “set” block to initialize this variable.

Open the function called “sing_a_tune” by right-clicking the grey function block, and selecting the last menu item “edit” as shown below.

A skeleton of this function is already provided, it will look like the one below:

Here you are provided with a variable index to loop through the score, and the variable note stores information about each note as implemented there. Note that the “split block” takes a string as input, and returns a list of substrings that were originally separated by one given character – in the “hello world” case, it’s two words separated by a space; in the score list, we split each element of a note as two numbers (pitch and value) separated by a comma. You just need to replace the purple “think” block with code that makes the correct sound given information in variable note.

When you’re done, click on the block ‘sing_a_tune(score_amazing_grace)’ to hear the result – does this sound right?

Note: You will need to remove the placeholder purple “Think” block and place your code in its place. Otherwise Snap will still sing but it will not sound quite right!

Test your block! Testing (and testing, and testing again) is the key to creating correct software. Don’t be discouraged if it doesn’t sound right (or doesn’t even make a sound) the first time, try to find where things went wrong and fix it. Feel free to ask tutors/lectures to help

Exercise 3.3 Improving your function (20%)

Now let’s work on a differnt tune – the opening phrase of Radetzky March by J Strauss. You can listen to it on YouTube by Vienna Philharmonic orchestra in 2014 New Year concert, or by a piano. This will reveal two aspects of our “sing_a_tune” function that needs improvement.

?? What is the pitch “0” in the score? This is used to denote rest, i.e. a silent note in for a particular duration. Use the rest block to make this happen.

Making a list

The first is the input: it can be a pain to write a long list every time you’d like to sing a tune!

An easier way is to represent the score in one long string – with notes separated by semi-colons, and within each note the pitch and value are separated by a comma. Can you parse this long string in to a list of notes?

This should be implemented in the block that takes the string as input, and output a list in the format as above – and you assign the output to variable _score_radtzskymarch as in the project.
Changing the tempo

Now you use the “sing_a_tune(score_radtzsky_march)” to play this tune using block … wait, Why does this march sound like everyone is falling asleep? – It’s too slow!

In music the notion of being fast or slow is captpured in a property called tempo (as explained by bbc and wikipedia). Tempo is measured in BPM, or Beats Per Minute. When you foot-tap along the music, a beat very roughly correspond to a unit of ‘tapping’. 60bpm is one beat every second. We know that the tempo for Amazing Grace is 72bpm, as seen in the sing-a-tune function, whereas the tempo for energetic music, such as marches, is about 140bpm (i.e. a little more than two beats per second).

Now make a new function “sing_a_tune2” to take into account tempo. Hint: you can duplicate-and-drag the existing structure in sing_a_tune and move them to sing_a_tune2, and take it from there!

Once you are done, play this tune using – now does it sound right?

Exercise 3.4 Shifting keys (20%)

Now you can sing-along or hum-along your Snap! program with your friends.

But wait … you may have a different vocal range than where the tune is at. In other words, the tune may be ‘too high’ or ‘too low’ for you to sing. For example, someone with baritone or bass voice may have difficulty singing the Radtzsky March in our assignment. One commonly used solution is to shift the pitches of notes, up or down.

In Snap!, shifting keys can be done with the simple arithmetic operations of addition and substraction.

You start by stacking three blocks, initially playing a note sequence E3-G3-C4. The range of these notes starts from a low-E, your bass or baritone friends may not be able to sing this one. (And if you know a bit of how chords work, this is a transposed C major chord ending in central C)

Now you make three other blocks as below, shifting the whole chord lower to G-major ending in the middle G.

Now the note sequence sounds lower overall, but the relationships between adjacent notes sounds the same. The Baritones are happy, but the first note is so low that your soporano friends may not be able to sing this one. Note how you can substract 5 from the pitch value of each note in the first picture to get the second picture. btw, this is called five semitones, i.e. there are four black and white keys, between G and high C, and hence four intervals.

Your task:

Implement a function called “shift_and_sing” to shift the key of a song up or down for a given amount, in a variable given as “shift”.
Midi notes should be a positive integer, and “0” is used to denote rest. Within your function, check that all notes are euqal or above zero – otherwise output an error message and halt the program.

Hint: you can re-use the singing function you just made.

Now sing along the Radtzky – is this more comfortable?

Exercise 3.5 The Crab Cannon (20%)

Now we are ready to play some advanced music! We explore the possibilities of singing in Snap! via the famous Crab Canon as seen in the book Gödel, Escher, Bach.

Preparation: concurrency and playing chords

Our computers are often doing more than one thing at any given time – for example, recieving an instant message while showing you a google search result. Making music also often require doing more than one thing at a time, such as, bowing more than one string at a time on a violin, hitting more than one key on a piano, or more than one person playing at any given time in an orchestra! Let’s use the computers’ ability to mutli-task (called concurrency) to play poly-phonic music.

The way Snap! achieves multi-tasking is via sending and receiving messages. Look at the code block below, excute the following steps, observe and hear the behavior.

Step 1 – click on each of the two groups of tone-playing-blocks. Do you hear “C-E-G” and “A3-C-E”?
Step 2 – now click on the left-most group with the “broadcast” block. Do you hear chords?

The “broadcast” block is acting like a commander here, it is responsible for shouting out the “play_chord” message. The two “When I receive” block act like soldiers, they are tuned in to “play_chord” message, and will start acting (i.e. excuting their respective programs) upon hearing it.

Note: Just like naming variables, you should assign sensible names to the messages.

Additional reading: You can find another explanation of concurrency in snap! here, plus another code example of broadcasting and recieving messagings in Snap! via a fun drawing program.

The Crab Canon

The enigmatic Canon 1 à 2 from J. S. Bachs Musical Offering (1747) depicts a single musical sequence that is to be played front to back and back to front.

Here is a musical video that visualizes the manuscript while it’s being played, its reverse, the two voices played together, then the two voices visualized on a mobius strip, and finally the two voices with different instruments.

Your Crab Canon

We use the “https” block to get a copy of the top voice of the crab canon from the course website (you can see it here). And then you can use the ‘sing_a_tune2’ block to play this voice, as shown below.

The way to have the second voice for the crab canon is by implementing the function that reverses the first voice in time. Put your implementation in this place-holder block .

First test your block by playing out the reversed voice. And then test your music by playing the two voices at the same time.

For fun: Sing your own tune

Pick some other tune that you are familiar with (or make one up!), and ask Snap! to sing it.

Post your song (and score) to the piazza discussion board.

Include its score as one or a few line of text (as the example above). Include a reference to its source if you have taken it from somewhere.
Record your tune using a software, e.g. the voice memo or recorder app on your phone, and share it with the class.

Submission, and testing your submission

When you are done, export your talking_setup and generate_nonsense blocks along with the singing blocks, and include them in your submission.

As always, test your submission file – to make sure everything that should be included is indeed included!

To do so:

Start a fresh chrome/chronium session
Load the project for lab3.
Use the “import … ” function from Snap to load your submission file containing the blocks.

Now run these blocks – make sure the singing and talking works as before.

Image credits:

“Chromatic scale”, wikimedia user Chrupos, CC BY-SA 3.0
“Scale in C-Major”, wikimedia user っ, CC BY-SA 3.0

Answer:

Triplets (and sixtuplets). As their fractional note values (e.g. ¹⁄₃) cannot be written out (exactly) as a real number with a finite number of digits.
(Left for tutors to discuss/draw during labs).

Footnote:

^ Further reading: here is an explanation about the history and basic technique for peudorandom numbers at the Khan Academy https://www.khanacademy.org/computing/computer-science/cryptography/crypt/v/random-vs-pseudorandom-number-generators

Lab 1: Karel the Robot

Enthusiastic Hugo User — Sun, 15 Jan 2017 00:00:00 UTC

Lab 1: Programming Karel

First, open Snap with Karel, in a new tab or window so that you can do the lab exercises and continue reading these instructions at the same time.

Slow to load?

Loading the page can take some time - anywhere from 20 seconds to 5 minutes. Exactly how long it takes depends on which browser you’re using and the computer you’re on. We recommend using an up-to-date version of the chromium (chrome) browser, but we have tested Snap! also with firefox and Internet Explorer.

If the browser shows a message like “a script on this page has stopped responding”, and asking you whether to “Stop script” or “Continue”, choose “Continue”. (This may happen several times.)

Getting Karel ready to go

Before you can start Karel, or do run any program, you have to initialise the grid and robot: Click the block. A green highlight appears around the edges of the block while it is run. (Setup can also take some time, up to a minute. If you don’t see the setup block at the top of the work area, click the Karel icon in the lower right area.) After that, you can start Karel by clicking the block, and then move him around by clicking the two movement blocks.

Exercise 1.1

In the first lecture, we showed you how to add a new “block” that made Karel turn right (clockwise). You can find this block, as well as Karel’s basic actions, in the “Motion” category of blocks.

If you want to see how “Turn Karel right” works, right-click on it and select “edit”: an editor with the definition of the block will show up.

Now, create a new block that uses Karel’s basic actions (moving forward, turning left) to make him move as if he had taken one step backwards.

To create a new block, click on “Make a block” (in the “Variables” category). A window pops up where you can name the new block (call it “Move Karel backward”) and select its type (“Command”) and what category it should be placed in (“Motion” would be the right choice).

Click “Ok”.

This opens an editor for the new block. Create it by dragging in appropriate blocks from the collections on the left (you may find the blocks you need in different categories). When done, click “Apply” or “Ok” and the block appears in the category where you placed it. (Blocks in the “Other” category, coloured grey, appear in the “Variables” tab.)

Test your new block!

Testing (and testing, and testing again) is the key to creating correct software. To test the block, drag a copy of it into the work area, and click on it: this will run the block. You can compose it with other blocks, for example to make Karel walk forwards and then backwards again.

Saving and loading blocks

After you have created some blocks that may be useful in future labs, you may want to save them. (Starting from next week, you will also need to save blocks so that you can submit your assignment for marking.) To do this, click the button, and select “Export blocks”. This opens a window where you can select which blocks to save. The default is to save all custom blocks, which is probably more than you want so you will have to deselect some. After you’ve made your selection, press “Ok” and the browser will open a new tab (or window) with a description of your chosen blocks. To save it, use your browsers “Save as” function (usually found by right-clicking, or in the “File” menu).

If you want to see an example of how it’s done, here is a video (click on the last icon at the bottom to watch at full screen, click on the link to watch the larger version on youtube website).

Exercise 1.2

Now for something slightly more challenging. Click the to change Karel’s world to one that has some “hurdles” sticking up from the bottom wall. We want to make Karel move along the bottom wall, crossing each hurdle that he encounters. To solve the problem, begin by breaking it down into small parts, such as:

Move Karel forwards until he is in front of a wall.
Make Karel follow a wall (on his right) until it ends.
Make Karel go over a hurdle, and back down to the bottom wall.

Make a block for each part. You should find that the first two are useful in constructing the third.

Finally, make Karel traverse the whole obstacle course. How do you make him stop when he has reached the bottom-right corner?

Exercise 1.3

Karel wakes up, not knowing where he is (or which way he is facing)! In one (and only one) of the corners of the grid, there is a beeper. How can he get to it?

To put Karel in this precarious situation, use the block , and , with the coordinates of one of the corners (for example, 11 and 1 is the lower right corner). You can do this on the grid with hurdles or without.

To Preserve your work for Lab 1 Make one xml file with the new blocks you made for excercises 1.1 - 1.3 (and only those blocks), save it somewhere on your computer – this way you can refer to them in future labs if you need to!

Assignment submission

There is no assignment to submit this week. The first assignment will be on lab 2, in which you will continue to teach Karel how to do more things.

Lab 2: Karel the Robot (II)

Enthusiastic Hugo User — Sun, 15 Jan 2017 00:00:00 UTC

Lab 2: Programming Karel (part II)

Again, start by opening Snap with Karel. (Note that this is not the same link as in lab 1: This version comes with a few other blocks ready-made.) Open it in a new tab or window so that you can do the lab exercises and continue reading these instructions at the same time.

Reminder: Loading the page can take a long time - how long depends on which browser you’re using and the computer you’re on. We recommend using an up-to-date version of the chromium (chrome) browser, but we have tested Snap! also with firefox and Internet Explorer. If the browser asks you to “Stop script” or “Continue” while loading, choose “Continue”.

Reminder: Before you can start Karel, or run any program, you have to initialise the grid and robot, by clicking the block.

For the first exercise, you should use a block called instead. You will find it near the end of the list in the Variables category.

Exercise 2.1

As we talked about in the lectures, a reporter block (also known as a function) is a block that returns a value, and that can be used in expressions (like +, join, etc). Reporter blocks are distinguished from command blocks by having rounded corners. Snap! also has a third shape of block, called a predicate. This is a reporter block that always returns one of the values or . Predicate blocks can be used in conditional blocks, like if and repeat until. Snap! allows you to define custom predicate blocks, just the same as custom reporters and command blocks.

Karels perception abilities are limited: he can only look for a wall right in front of him, or a beeper on the spot where he stands. In the Sensing category, you will find a predicate called “Beeper in front?” Right-click and select “edit” to open it in the block editor. This predicate is incomplete. You must fill it in so that it

reports true if the corner in front of Karel contains a beeper, and there is no wall blocking the way there;
reports false if there is a wall in front of Karel, or no beeper on the corner in front; and
after execution of the block, Karel is in the same place and facing the same direction as he was before it started.
“In front” means in front of Karel with one step away.

Testing your new block

As always, test your new block! Set up test cases by placing a beeper in front of Karel, turning him to face a wall, and so on.

After you are satisfied that it works correctly, try using it to make Karel follow a trail. You will find a block called which lays out a trail to get Karel out of the maze, and a block called “Follow a trail” (in the Motion category) that makes Karel follow this trail. “Follow a trail” uses your “Beeper in front?” predicate, so if that does not work correctly, it is likely that trail following will not either.

Exercise 2.2

In the lectures, we have shown you how to make Karel collect beepers. Now it is time to load up Karel with a stack of beepers and instruct him to lay them out in a nice pattern. Specifically, given any number of beepers, we want him to lay them out in a square field, or as close to a square as he can. (If the number is not a perfect square, the number of rows and columns should not differ by more than one, and the number of beepers in either rows or columns should be the same except for possibly the last one.) For example,

You should create a top-level block called “Make a square with (number) beepers”, where the number is the input. To make sure that Karel has enough beepers to fill the square, use the block “tell” block, like this:

Before you start on this exercise, run the block again. This will remove the maze, giving Karel space to lay out his beepers. You can use the blocks to remove all beepers from the grid and put Karel back in his starting corner; that will probably be useful as you test and debug the square layout block.

Assignment submission

To submit your assignment, you need to export blocks (as described on the lab 1 page and shown in this video). You should export the two blocks that you defined in this lab, “Beeper in front?” and “Make a square with (number) of beepers”, as well as any other custom blocks you created that are used inside those two (or used inside those used inside those two, and so on).

Submit the file with the saved blocks through wattle.

Testing your submission As always, test your submission file – to make sure everything that should be included is indeed included!

To do so:

Start a fresh chrome/chronium session
Load the Snap+Karel workspace for Lab 2
Use the “import … ” function from Snap to load your submission file containing the blocks.

Now run these blocks – is Karel doing what she’s asked to do in 2.1 and 2.2?

Labs on The Art of Computing

Lab 9: Combinatorial Circuits

Installing and Starting Logisim

Exercise 9.1 (no marks)

Exercise 9.2 Sorting Numbers with Circuits

Task 1 1-bit swap (50%)

Task 2 The 4-number sorting circuit (50%)

Lab 8: Learning to diagnose - Machine learning with decision trees

Data, data everywhere

A data set for medical diagnosis

Task 1: Decision trees and classification (40%)

Discussion question

Recursion with trees

Instructions

Discussion

Reading - Choosing a decision rule

Greed is good

Discussion questions

Information entropy

Cost function revisited

Discussion questions

Notes

Discussion question

Task 2: Building a decision tree (40%)

Instructions

Extension questions

Task 3: Analysing performance (20%)

Exercises

Discussion:

References

Lab 7: The Sentiment of a Superstorm

Making Sense of Social Media

Concept 1: Sentiment Analysis

A Simplified Approach to Determining Sentiment

Sentiment Lexicon

Steps for Determining Tweet Sentiment

Interlude: The Mailbox Problem

Concept 2: Hash Tables (not #HashTags)

Hash Functions

Buckets and Hash Collisions

Practical: Sentiment Analysis of Superstorm Sandy

Implementing a Hash Function

Exercise 6.1: Implement a hashing scheme (3 points)

Implementing a Hash Set

Exercise 6.2: Implement the insert operation (3 points)

Exercise 6.3: Implement the contains operation (2 points)

Testing

Sentiment Analysis and Visualisation of Superstorm Sandy Tweets (1 point)

Exercise 6.4: Computing the sentiment score of a Tweet (1 point)

Extension Tasks

Extension Task 1: Implement the remove operation

Extension Task 2: Implement dynamic resizing (rehashing)

Extension Task 3: Implement the dictionary data structure (hash table)

References:

Lab 6: Sequence alignment (with pointers)

Determining Similarity

Exercise 5.1 (70%)

The Global Alignment (Needleman-Wunsch) Algorithm

Implement the global alignment algorithm

Exercise 5.2 (30%)

Assignment submission

Lab 5: Solving Sudoku

Sudoku: The search for a solution

Exercise 7.1 (60%)

Extension to Exercise 7.1:

Exercise 7.2 (40%)

Extension to Exercise 7.2:

Assignment submission

Sources

Lab 4: Recursion

Exercise 4.1 A calculator for combinations (50%)

Exercise 4.2 Karel as a navigator: following a trace and coming back (50%)

Lab 3: Make Your Computer Sing (tunes) and Talk (nonsense)

Exercise 3.1 Snap can talk … nonsense (10%)

Music Preparation (0%)

Exercise 3.2 Singing a tune (30%)

Exercise 3.3 Improving your function (20%)

Exercise 3.4 Shifting keys (20%)

Exercise 3.5 The Crab Cannon (20%)

Preparation: concurrency and playing chords