Lecture 6: Sorting

• Sorting algorithms in \( \bigO(n^2) \) are known as quadratic time algorithms.
• We'll cover three quadratic sorting algorithms:
  • Bubble sort, Insertion sort, and Selection sort

• Bubble sort compares adjacent items in the array and swaps them if a[i] > a[i+1].
  • This guarantees that the largest (or smallest depending on the direction the bubble is moving the item) will be in the proper place.
    • This needs to be done for each element in the collection.
    • \( \bigO(n^2) \) complexity.
    • A good illustration of how Bubble sort works is shown here.
• Can be slightly optimized
  • If a swap doesn't occur in an iteration, we know the array is already sorted and no more comparisons need to be made.

// for std::swap
#include <utility>
#include <vector>
#include <iostream>

using namespace std;

// We use a vector reference because we are mutating a. Vector is
// better than an array, because it carries size. A more generic
// implementation would use iterators, like how `std::sort` works.
template<typename T>
void bubbleSort(vector<T> & a) {
  // The core idea, go through the array and "bubble up" the largest element to
  // the end of the array. Repeat this for the remaining part of the array.
  bool swapped;

  // O(n) iterations of this loop in the worst case
  // 
  // note that we use size_t for indices
  for (size_t i = a.size() - 1; i > 0; i--) {
    swapped = false;
    // O(n) iterations of this loop
    for (size_t j = 0; j < i; j++) {
      if (a[j] > a[j + 1]) {
        // use std::swap rather than manually assigning to a
        // temporary
        std::swap(a[j], a[j+1]);
        swapped = true;
      }
    }
    if (!swapped) {
      // in order
      cout << "i = " << i << endl;
      cout << "already in order... returning" << endl;
      return;
    }
  }
}

// We don't modify a, so we take a const reference to avoid copying
// it.
void printArray(const vector<int> & a) {
  for (size_t i = 0; i < a.size(); i++) {
    cout << "[" << i << "] = " << a[i] << endl;
  }
}

int main() {
  vector<int> a{1,2,3,4,5,6,7,8,9,10};
  vector<int> b{1,10,2,9,3,8,4,7,5,6};
  vector<int> c{2,9,4,7,6,5,8,3,10,1};
  vector<int> d{10,9,8,7,6,5,4,3,2,1};
  vector<int> e{1};

  bubbleSort(a);
  printArray(a);
  cout << "---" << endl;
  bubbleSort(b);
  printArray(b);
  cout << "---" << endl;
  bubbleSort(c);
  printArray(c);
  cout << "---" << endl;
  bubbleSort(d);
  printArray(d);
  cout << "---" << endl;
  bubbleSort(e);
  printArray(e);
}

• From top-down (or bottom-up), look through the array and find the largest (or smallest) element.
• Swap a[i] with a[index_of_largest_value]
• \( \bigO(n^2) \) complexity.
• A good illustration of how Selection Sort works is shown here.

template<typename T>
void selectionSort(vector<T> & a) {
  // The core idea to improve on bubble sort: find the last element and do a
  // single swap to put it to the end.
  size_t largestIndex;

  // Going from right-to left because we are finding the largest
  // index first
  for (size_t i = a.size() - 1; i >= 0; i--) {
    // unroll the first iteration
    largestIndex = 0;
    for (size_t j = 1; j <= i; j++) {
      if (a[j] > largest) {
        largest = a[j];
        largestIndex = j;
      }
    }
    std::swap(a[i], a[largestIndex]);
  }
}

int main() {
  vector<int> a{1,2,3,4,5,6,7,8,9,10};
  vector<int> b{1,10,2,9,3,8,4,7,5,6};
  vector<int> c{2,9,4,7,6,5,8,3,10,1};
  vector<int> d{10,9,8,7,6,5,4,3,2,1};
  vector<int> e{1};

  selectionSort(a);
  printArray(a);
  cout << "---" << endl;
  selectionSort(b);
  printArray(b);
  cout << "---" << endl;
  selectionSort(c);
  printArray(c);
  cout << "---" << endl;
  selectionSort(d);
  printArray(d);
  cout << "---" << endl;
  selectionSort(e);
  printArray(e);
}

• Similar to sorting cards in a poker hand
• For each item in the array, find where it should "belong" and insert the item in its proper place.
  • Before insertion happens, shift all elements in order to to make an empty slot where the item should belong.
  • Do this for each element, and by the end of the algorithm, all elements will be in their proper place.
  • \( \bigO(n^2) \) complexity.
  • \(\bigO(n) \) complexity for elements that are mostly-sorted.
    • Need to do the \( \bigO(n) \) insertion only a few times.
  • Works really well for small arrays.
    • Most versions of C++ standard library's std::sort uses insertion sort when the array being sorted is small (depends on the standard library implementation, this happens when there are less than ~30 elements in the array).

template<typename T>
void insertionSort(vector<T> & a) {
  // core idea: keep all the elements before the current element to be
  // sorted. Then, do a linear search to find where to "insert" the current
  // element in the prefix. Insert the current element to its position and move
  // all the larger elements to the right.
  size_t shiftIndex;
  for (size_t i = 1; i < a.size(); i++) {
    T item = a[i];
    shiftIndex = i - 1;

    // In the loop we move the larger elements to the right while looking for
    // the correct place. This does not change the number of basic operations
    // (compares and swaps) but it is faster because we do a single sweep rather
    // than two (one for finding, one for moving), so we fetch the data from the
    // memory only once.
    while (shiftIndex >= 0 && a[shiftIndex] > item) {
      a[shiftIndex + 1] = a[shiftIndex];
      shiftIndex -= 1;
    }
    a[shiftIndex + 1] = item;
  }   
}

int main() {
  vector<int> a{1,2,3,4,5,6,7,8,9,10};
  vector<int> b{1,10,2,9,3,8,4,7,5,6};
  vector<int> c{2,9,4,7,6,5,8,3,10,1};
  vector<int> d{10,9,8,7,6,5,4,3,2,1};
  vector<int> e{1};

  insertionSort(a);
  printArray(a);
  cout << "---" << endl;
  insertionSort(b);
  printArray(b);
  cout << "---" << endl;
  insertionSort(c);
  printArray(c);
  cout << "---" << endl;
  insertionSort(d);
  printArray(d);
  cout << "---" << endl;
  insertionSort(e);
  printArray(e);
}

All of the algorithms presented here have the same asymptotic worst-case complexity. However, their run time differs by a constant factor. This constant factor may be important when we have multiple algorithms to choose with the same asymptotic complexity. See the code from the live coding session for how we measured this constant factor.

Moreover, other effects like cache locality affect how fast our programs will run on modern hardware. So, it is important to profile and benchmark your code on realistic workloads when trying to optimize an algorithm.

The lecture video on GauchoCast contains some discussion about these topics as well. We also discussed best-case and average-case complexity.

Although these algorithms have the same worst-case complexity and average-case complexity, bubble sort and insertion sort have only a linear ( \( \bigO(n) \) ) best-case complexity. Can you figure out what the best case is? Analyzing different cases for complexity will be covered later in CS 130A/B.