SHAHSQUATCH

NOTES - Basic Graph Theory (Part 1)

2020-07-02T00:00:00+00:00

Disclaimer: This post was written as notes for the Graph Theory series by William Fiset.

Introduction

Types of Graphs

Undirected Graph (Nodes have no edges)
Directed graphs (Digraphs or DAGS). There are arrows between nodes u and v

Graphs in this course will be represented as (u, v, w), where w is the weighting. This is for directed graphs.

Trees are undirected graphs with no cycles

Rooted tree is directed, with a root node. Can be an out-tree or an in tree (aborescence vs. anti-aborescence).

DAGS are Directed Acylcic graphs. These are graphs with directed edges and no cycles.

ALL out-trees are DAGS, but NOT ALL DAGs are out-trees.

Bipartite graph is a graph whose vertices can be split into two independent groups U, V such that every edge connects between U and V. Also called “two colorable”, or there is no odd length cycle.

Finally we have complete graph, where there is a unique edge between every pair of nodes. A complete graph with N vertices is known as K_n. Complete graph is worst case, so to test for performance, complete graph is good place to start.

How do we represent graphs on the computer?

Adjacency matrix: Store all weights in an nxn matrix. Pros are it is efficient in lookup (O(1)) and is very simple. However it requires O(V²) space and iterating time complexity. Fine for dense graphs, but not for sparse graphs.
Adjacency list: It only stores the nodes to which the current node points. Pros are that it is great for sparse graphs, and iterating over edges is efficient. Main disadvantage is that it is less space efficient on dense graphs, and the the worst case edge weight lookup is O(E).
i.e.
A -> [(B,3)]
B -> [(A,6)]
Edge List: Simply represent the graph as an unordered list of edges. These can be stored as triplets (u, v, w), which reads as “The cost from node u to node v is w”. This is very simple, but lacks structure. It is great for sparse graphs and iterating over edges is easy. The downside is that it is less space efficient for dense graphs, and the worst case edge weight lookup is O(E).
i.e.
[(C, A, 4), (A, C, 1)]

Common Problems and Algorithms

Questions to ask

Is the graph directed or undirected
Are the edges of the graph weighted
Is the graph I will encounter likely to be sparse or dense with edges?
Should I use adjacency matrix, adjacency list, edge list or some other structure?

Shortest Path

One of the most common problems is the shortest path problem. Many algorithms exist to solve this problem. BFS, Dijkstra’s, Bellman-Ford, Floyd Warshall, A*, and more.

Connectivity

Does there exist a path from node A to Node B? Typical solutions here are union find or simple search algorithm like DFS.

Negative Cycles

Negative cycles are graph cycles that end up with a negative cost. There are places where negative cycles are beneficial. Algorithms to find negative cycles are Bellman-Ford and Floyd-Warshall.

Strongly connected components

Strongly Connected Component can be thought of as self-contained cycles within a directed graph, where every vertex in a given cycle can reach every other vertex in the same cycle. Algorithms to find these components are Tarjan’s and Kosaraju’s.

Traveling Salesman Problem

Give a list of cities and the distances between them, find the shortes possible route that visits each city once and returns back. It is NP-Hard, and has several important applications. Algorithms to solve this are Held-Karp, branch and bound, and many approximation algorithms.

Bridges

A bridge/cut edge is any edge in a graph whose removal increases the number of connected components. A connected component of an undirected graph is a subgraph in which any two vertices are connected to each other by paths, and which is connected to no additional vertices in the supergraph.

Articulation Points

An articulation point/cut vertex is any node in a graph whose removal increases the number of connected components. They are important in graph theory because they often hint at weak points in a graph.

Minimum Spanning Tree

A minimum spanning tree (MST) or minimum weight spanning tree is a subset of the edges of a connected, edge-weighted undirected graph that connects all the vertices together, without any cycles and with the minimum possible total edge weight. Algorithms are Kruskal’s, Prim’s and Boruvka’s.

Network Flow: Max Flow

The question with flow networks is that if we had an infinite input source, how much “flow” can we push through the network? Edge weights represent some sense of capacity, i.e. water through a pipe, or cars through a road. Algorithms are Ford-Fulkerson, Edmonds-Karp & Dinic’s.

Depth First Search (DFS)

DFS is a core algorithm allowing you to traverse a graph. It is easy to code, and has time complexity O(V+E). By itself it isn’t that useful, but can be very useful for other tasks.

Note that in DFS, you can’t revisit the same node. So if you either visit a leafnode or get to a visited node, you have to backtrack. You can have multiple ways to do DFS.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# Global or class scope variables
n = number of nodes in graph
g = adjacency list representing graph
visited = [false, ..., false] # size n

function dfs(at):
    if visited[at]: return
    visited[at] = true

    neighbours = g[at]
    for next in neighbours:
        dfs(next)

# Start DFS at node zero
start_node = 0
dfs(start_node)

Finding Connected Components

Can we label each node in a component with the same id value? We can use DFS to identify components.

First makes sure all the nodes are labeled from [0, n) where n is the number of nodes.
Start at every node, unless that node has already been visited, marking all nodes in the same search with the same value.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# Global or class scope variables
n = number of nodes in the graph
g = adjacency list representing graph
count = 0
components = empty integer array # size n
visited = [false, ..., false] # size n

function findComponents():
    for (i=0; i<n; i++):
        if !visited[i]:
            count++
            dfs(i)
        return (count, components)

function dfs(at):
    if visited[at] return
    visited[at] = true

    for (node : g[at]):
        if !visited[node]:
            dfs(node)

What else can DFS do?

Comput a graphs MST
Detect and find cycles in a graph
Check if a graph is bipartite
Find strongly connected components
Topologically sort the nodes of a graph
Find bridges and articulations points
Find augmenting paths in a flow network
Generate mazes

Breadth First Search (BFS)

Another fundamental search algorithm, runs with a time complexity of O(V + E). It is particularly useful for finding the shortest path on unweighted graphs.

Unlike DFS, we will visit all of the neighbours first instead of drilling down to the leaf nodes. It explores in a layered fashion by maintaining a queue.

BFS uses a queue to track where to go next. Upon reaching a new node, it adds it to the queue so we can visit it later. We will need some sort of data structure that shows which nodes we have visited so we don’t revisit.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
# Function that calculates shortest path
# Global/class scope variables
n = number of nodes in the graph
g = adjacency list representing unweighted graph

# s = start node, e = end node, and 0 <= e, s < n
def bfs(s, e):

    # Do a BFS starting at node s
    prev = solve(s)

    # return reconstructPath(s, e, prev)

def solve(s):
    q = queue data structure
    q.enqueue(s)

    visited = [false, ..., false] # size n
    visited[s] = true

    prev = [null, ..., null] # size n
    while !q.isEmpty():
        node = q.dequeue()
        neighbours = g.get(node)

        for (next : neighbours):
            if !visited[next]:
                q.enqueue(next)
                visited[next] = true
                prev[next] = node
    return prev

def reconstructPath(s, e, prev):

    # Reconstruct the path going backward from e
    path = []
    for (at = e; at != null; at = prev[at]):
        path.add(at)

    path.reverse # path[::-1]

    # If s and e are connected return the path
    if path[0] == s:
        return path
    return []

Breadth First Search (BFS) Grid Shortest Path

Motivation: A suprising number of problems can be represented using a grid. Grids are implicit graph, because we can determine neighbours based on the grid.

Common approach is to convert to adjacency list/matrix. Cells are connected left right, up and down.

Label all cells in the grin with numbers [0, n)
Construct an adjacency list and matrix based on the grid
Then we can run whatever graph algorithm we want to run.

However we can usually avoid transformations due to the structure of the graph itself.

1
2
3
4
5
6
7
8
9
10
11
12
13
# Define the direction vectors for north, south, east, and west
dr = [-1, +1, 0, 0]
dc = [0, 0, +1, -1]

for (i = 0; i < 4; i++):
    rr = r + dr[i]
    cc = c + dc[i ]

    # Skip invalid cells. Assume R 
    # and C for the number of rows and columns
    if rr < 0 or cc < 0: continue
    if rr >= R or cc >= C: continue
    # (rr, cc) is neighbour of cell (r, c)

Real Example

You are trapped in a dungeon and need to find the quickest way out. The dungeon is composed of unit cubes which may or may not be filled with rock. It takes one minute to move one unit north, south, east, and west. You cannot move diagonally and the maze is surrounded by solid rock on all sides.

Is an escape possible? If yes, how long will it take?

Alternative State Representation

Right now we are storing the values as either an (x, y) pair or an object representation.

An alternative approach is to use one queue for each dimension. When we do this however, we need to make sure that we enqueue and dequeue elements all at the same time.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
# Global/class scope variables
R, C = ... # R = # rows, C = # columns
m = ... # The input character grid of size R x C
sr, sc = ... # Starting point coordinates
rq, cq = ... # Empty Row Queue and Column Queue

# Variables used to track the number of steps taken
move_count = 0
nodes_left_in_layer = 1
nodes_in_next_layer = 0

# Variable used to track whether the 'E' character
# ever gets reached during BFS
reached_end = false

# R x C matrix of false values used to track whether
# the node at position (i, j) has been visited
visited = ...

dr = [-1, +1, 0, 0]
dc = [0, 0, +1, -1]

def solve():
    rq.enqueue(sr)
    cq.enqueue(sc)
    visited[sr][sc] = true
    while rq.size() > 0:
        r = rq.dequeue()
        c = cq.dequeue()
        if m[r][c] == 'E':
            reached_end = true
            break
        explore_neighbours(r, c)
        node_left_in_layer--
        if nodes_left_in_layer == 0:
            nodes_left_in_layer = nodes_in_next_layer
            nodes_in_next_layer = 0
            move_count++
    if reached_end:
        return move_count
    return - 1

def explore_neighbours(r, c):
    for (i = 0; i < 4; i++):
        rr = r + dr[i]
        cc = c + dc[i ]

        # Skip invalid cells. Assume R 
        # and C for the number of rows and columns
        if rr < 0 or cc < 0: continue
        if rr >= R or cc >= C: continue
        # (rr, cc) is neighbour of cell (r, c)

        if visited[rr][cc]: continue
        if m[rr][cc] == #: continue

        rq.enqueue(rr)
        cq.enqueue(cc)
        visited[rr][cc] = true
        nodes_in_next_layer++

Introduction to Tree Algorithms

What is a tree? A tree is an undirected graph with no cycles.

There is an even easier way to check if a graph is a tree. Each tree has exactly n nodes, and n-1 edges

Trees in the wild

File system
Social heirarchies
Prefix Notation
Webpages (DOM)
Game Theory to model decisions and courses of action, i.e. prisoner’s dilemma
Probability trees
Taxonomy
etc…

Storing undirected trees

You can store trees by doing the following

Label from [0, n)
Setup storage representation
1. Edge list
  1. This is super fast
  2. Easy to iterate over
  3. Lacks structure to query neighbours
2. Adjacency List
  1. Mapping from node to labels
  2. Easy to find neighbours
3. Adjacency Matrix
  1. Easy
  2. Huge, huge waste of space (n²)

Rooted Trees are directed trees, you can have an out-tree (arborescence) or an in-tree (anti-arborescence).

Related to root trees are binary trees which can have at most two child nodes.

Binary search trees are binary trees which satisfy the binary search tree invariant which is:

x.left.value <= x.value <= x.right.value

Note how the last tree above is not a binary search tree because it breaks the invariant.

We can also make the invariant strict, to avoid duplicate values.

Storing rooted trees

In practice you always maintain a pointer to the root node so you can access the tree and all of its content.

Each node also has a list of all its children, known as child nodes. Leaf nodes do not have any children.

Sometimes it is also useful to have a pointer to a node’s parent node so that you can traverse up the tree. This is not usually necessary, because you can access the node on the recursive callback.

If tree is an n-ary tree, we can also store it as a flattened array.

Here, each node has an assigned index position based on where it is in the tree. Here, even if there are missing nodes, they have an index in the array.

In this representation, we can calculate the indices of nodes as follows

left node: 2*i + 1
right node: 2*i + 2

Reciprocally, the parent of node i is:

floor((i-1)/2)

Beginner Tree Algorithms

Problem 1

Find the sum of all the leaf node values in a tree.

We can do a searching algorithm algorithm like BFS or DFS.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# Sums up leaf node values in a tree.
# Call function like: leafSum(root)
def leafSum(node):
    # Handle empty tree case
    if node == null:
        return 0
    
    if isLeaf(node):
        return node.getValue()
    
    total = 0
    for child in node.getChildNodes():
        total += leafSum(child)
    return total

def isLeaf(node):
    return node.getChildNodes().size() == 0

Problem 2

Find the height of a binary tree. The height of a tree is the number of edges from the root to the lowest leaf.

Leaf nodes have a height of 0

We can set up a height function h(x)

We can set up a recurrence relation as follows.

h(x) = max(h(x.left), h(x.right)) + 1

Note that we are recursively computing the height of the subtrees. Here is the recursive pseudocode

1
2
3
4
5
6
7
8
9
10
11
12
# The height of a tree is the number of
# edges from the root to the lowest leaf.
def treeHeight(node):
    # Handle empty tree case
    if node == null:
        return -1

    # Identify leaf nodes and return zero
    if node.left == null and node.right == null:
        return 0

    return max(treeHeight(node.left), treeHeight(node.right)) + 1

Note, that we could do the following, and the algorithm will still work correctly

1
2
3
4
5
6
7
8
# The height of a tree is the number of
# edges from the root to the lowest leaf.
function treeHeight(node):
    # Handle empty tree case
    if node == null:
        return -1

    return max(treeHeight(node.left), treeHeight(node.right)) + 1

This works because we are effectively adding extra “null nodes” to the tree and correcting for the heigh by returning -1.

Keep in mind, that we can also dynamically keep track of the height of a tree when we are constructing it, although that may not always be possible.

Rooting a Tree

The motivation for rooting a tree is that often if can help to add structure and simplify the problem you are trying to solve.

Conceptually, rooting a tree is like picking up the tree by a specific node, and having all the edges point downwards.

Note that you can root a tree using any nodes, but not every selection will be well balanced. In some situations, it is useful to keep a reference to the parent node, in order to walk up the tree.

One way to root a tree, is to use depth first search.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
# TreeNode object structure
class TreeNode:
    # Unique integer id to identify this node
    int id;

    # Pointer to parent TreeNode reference. Only the 
    # root node has a null parent TreeNode reference.
    TreeNode parent;

    # List of pointers to child TreeNodes.
    TreeNode[] children;

# g is the graph/tree represented as an adjacency
# list with undirected edges. If there's an edge between
# (u, v) there's also an edge between (v, u).
# rootId is the id of the node to root the tree from.
def rootTree(g, rootId = 0):
    root = TreeNode(rootId, null, [])
    return buildTree(g, root, null)

# Build tree recursively using depth first
def buildTree(g, node, parent):
    for childId in g[node.id]:
        # Avoid adding an edge pointing back to the parent
        if parent != null and childId == parent.id:
            continue
        child = TreeNode(childId, node, [])
        node.children.add(child)
        buildTree(g, child, node)
    return node

Tree center(s)

Finding a center of a tree is a useful way to select a node to root the tree. Note here that there may be more than one tree center, but there can’t be more than two.

The center of a tree is always the middle vertex or middle two vertices in every longest path along the tree.

Another approach is to peel away the outer layers of the tree, until we reach the center.

Steps

Compute the degree of each node (the number of nodes a specific node is connected to)
Prune edges, and update degree values
Keep doing 1. and 2. until we reach the center/centers

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# g = tree represented as an undirected graph
def treeCenters(g):
    n = g.numberOfNodes()
    degree = [0, 0, ..., 0] # size n
    leaves = []
    for (i = 0; i < n; i++):
        degree[i] = g[i].size()
        if degree[i] == 0 or degree[i] == 1:
            leaves.add(i)
            degree[i] = 0
    count = leaves.size()
    while count < n:
        new_leaves = []
        for (node : leaves):
            for (neighbour : g[node]):
                degree[neighbour] = degree[neighbour] - 1
                if degree[neighbour] == 1:
                    new_leabes.add(neighbour)
            degree[node] = 0
        count += new_leaves.size()
        leaves = new_leaves
    return leaves # center(s)

Basic Graph Theory For CS - Introduction (Part 1)

2020-07-02T00:00:00+00:00

Introduction

This series of blogs aims to cover most of the foundational material covered in the graph theory portion of an undergraduate level CS course in data structures.

The goal is to provide a more in-depth understanding of the concepts, alongside real-world implementations of the theory.

Prequisites for this course are a basic understanding of simple data structures (Arrays, Linked-Lists, Queues, Stacks), and a working proficiency in Python.

What is a graph?

Definition

To understand what graphs are, let’s look at the mathematical definition, and disassemble the meaning. From Wikipedia:

Graphs are mathematical structures used to model pairwise relations between objects.

The key words here are pairwise relationships, and objects. A graph is made up of nodes (the circles), and edges (the lines). We can see this on an example graph below made up of 3 edges and 3 nodes.

The nodes may be alternatively known as vertices or points, and the edges may also be known as links, arcs or lines.

In the figure above, the nodes represent whatever object we may be trying to model. For example, this could be a group of friends. The nodes represent each person, and the edges represent that they have a relationship. This is the basis behind creating more complex graphs such as those used to model relationships and how many degrees of separation you may be from another persion.

Not that a line will only be draw between one node and another. This defines pairwise relations. A graph will only show relationships between a pair of nodes in the graph.

Weighted Edges

Occasionally, a graphs will have weighted edges. They indicate a specific attribute of the connection between two nodes. Taking our previous example, we could create a graph as follows

Note now we have named our nodes as well. Alice, Bob and Charlie. The weighting on each edge represents the strength of the relationship. We can see that Alice has very strong relationships with both Bob and Charlie, but Bob and Charlie don’t really like each other.

Another use for weighted edges could also be to represent distance between locations. Using the graph above, we could see that location A is far away from both location B and location C, but B and C are very close to each other.

Take some time to think of how applying graphs to model your daily life. This will be helpful when we do modelling in the future.

Types of Graphs

Directed and Undirected graphs

The graphs we have discussed so far are good at modelling undirected relationships betwee two nodes. The relationship works both ways. This is why such graphs are called undirected graphs What if we wanted to model a one way relationship? For example, what if had a situation where Alice thought she was good friends with Bob and didn’t care for Charlie. Bob didn’t care for Alice, but thought he was good friends with Charlie. Charlie meanwhile liked Alice, but was also good friends with Bob. This sort of one way relationship between nodes may be shown using a directed graph. Please note the double edges on the relationship between B and C. This can also be represented as two distinct directed arrows, one from B to C and one from C to B.

Directed graphs, can also be known as digraphs. Another good example for digraphs would be an instagram celebrity. They will have thousands of followers but they may only be following a handful of other people.

Traversing a Graph

There are a variety of terms we need to understand as it pertains to traversing a graph. In this section, we will always refer to the graph below during our traversals.

Walk: A walk through a graph is taking a sequence of nodes and edges through the graph. The nodes and edges we take may be repeated. So for example in the graph above, a traversal like [A, B, C, A, C, D, E, A] would be classified as a walk. Walks themselves have two different definitions
- Open Walk: An open walk is a walk where the starting and ending vertices are different. For example, a traversal like [A, B] would be classified as an open walk.
- Closed Walk: A closed walk is a walk where the starting and ending vertices are the same. For example a traversal like [A, B, A] would be classified as a closed walk.
Trail: A trail is an walk in which no edge may be repeated. So the traversal [A, B, C, A, E] would be classified as a trail, even though we have repeated node A. Generally we will refer to trails as open trails, i.e. a trail that is also an open walk, but you can also have closed trails.
Circuit: A circuit is a closed trail. Nodes may be repeated in a circuit, but edges may not. An example of a closed trail would be [A, B, C, A, D, E, A].
Path: A path is a trail that does not repeat any nodes. Remember that a trail does not repeat any edges so a path does not repeat any nodes OR edges. When we say path, we are generally referring to an open path.
Cycle: A cycle is a closed path. It starts from the same node that it ends at, while not repeating any nodes or edges. [A, B, C, D, E, A] would be a valid cycle.

These concepts readily transfer over to directed graphs as well, with the restriction being that we can only travel in the direction of the edge.

DAGS

A directed graph without cycles is known as a Directed Acyclic Graph (DAG)

Trees

General Trees

We will define a tree in graph theory to mean an undirected graph wherein there are no cycles. Formally it can be defined as a connected acyclic undirected graph. We are used to seeing trees with one node at the top, and each node below it having a specific number of child nodes. Trees do not necessarily need to have such a structure. For example, the following is a valid tree.

Rooted Tree

A rooted tree, is a DAG in which we single out a specific node to classify as the root node. An example of a rooted tree is shown below

Polytrees

If we have a graph that is a DAG, and whose underlying graph is a tree (meaning no disjoint vertices), we can refer to it as a polytree polytrees (also directed tree, oriented tree and singly connected network). An example of a polytree is show below.

Arborescence

If we have a rooted polytree, we name it a directed rooted tree. These represent trees we generally encounter in CS, for example binary trees. When we draw binary trees, we don’t generally draw the directed edges to signify direction, but they are there implicity.

The direction that we assign the edges will alter how we define the tree. When all the edges are pointing away from the root node the directed rooted tree is known as an arborescence or out-tree. When they are all pointing towards the root node, the directed rooted tree is known as an anti-arborescence of in-tree.

Keep in mind that in a directed rooted tree, all edges have to be pointing away from or towards the root node simultaneously. You cannot have any edges pointing against the “natural direction”.

Also note from this definition, that every arborescence is a DAG, but not every DAG is an arborescence.

Bipartite Graphs

Suppose you are a data scientist with the NFL, and you are mapping which players have played with which team. You set about showing all of these relationships on a graph. For simplicity, I have given an example with 5 players (blue) and 5 teams (red). Note that players may have played for multiple teams.

The interesting thing to see here is that the vertices corresponding to players and teams can be very distinctly split, forming two disjoint and independent sets. Also see that every edge in the graph connects a vertex in the Players set to a vertex in the Teams set. This is in fact the definition of a bipartite graph (also known as two-colorable graph). From Wikipedia:

A bipartite graph or bigraph is a graph whose vertices can be divided into two disjoint and independent sets $U$ and $V$ such that every edge connects a vertex in $U$ to a vertex in $V$.

A necessary and sufficient condition for bipartite graphs is that there are no odd length cycles. In other words, a graph is bipartite if and only if there are no odd cycles.

Complete Graphs

If ALL the vertices are pairwise connected by an edge, the resulting graph is known as a complete graph. A complete graph with $n$ vertices, is denoted by $K_n$. Additionally, a complete graph with 0 vertices is known as a null graph, and a complete graph with 1 vertex is known as a singleton graph.

Both are considered to be trivial graphs (a finite graph that contains only one vertex and no edge). A simple graph is a graph that does not contain more than one edge between a pair of vertices, and is what we will be exploring further.

A graph in this context is made up of vertices (also called nodes or points) which are connected by edges (also called links or lines