\chapter{Course Introduction and OpenMP}

\section{Flynn's taxonomy}
$\rightarrow$ SIMD mentioned on slide 6

\noindent Single instruction, multiple data (SIMD) is part of Flynn's taxonomy, a classification of parallel computer architecture coined by Michael J. Flynn\footnote{\url{https://en.wikipedia.org/wiki/Michael_J._Flynn} a professor at Stanford's University} in 1966.
Flynn's taxonomy differentiates four architecture classifications based on the number of instruction and data streams they can use at a time.

\begin{definition}[Instruction Stream]
    This refers to the sequence of individual instructions that a processor executes. Usually represented by machine code which gets produced by assemblers from assembly. Base level examples would be \verb|ADD| and \verb|MOV| instructions on registers.
    
    \begin{enumerate}
        \item Single Instruction: One instruction at a time, sequentially. The program will only move to the next instruction after completing the current one. \hfill $\rightarrow$ SISD, SIMD
        \item Multiple Instructions: This means that the architecture utilises multiple processors (or cores) to execute different instructions simultaneously. \hfill $\rightarrow$ MISD, MIMD
    \end{enumerate}
\end{definition}

\begin{definition}[Data Stream]
    This refers to the data elements within registers or memory that get changed by the processor. 
    
    \begin{enumerate}
        \item Single data: Instructions will only ever change one singular data element at the same time, i.e. adding two variables and storing the results in another variable. \hfill $\rightarrow$ SISD, MISD
        \item Multiple data: Instructions can be used on multiple data elements at once. For example when adding two vectors and storing the result in a third vector, multiple data architectures can process the instruction(s) for all elements of the vector simultaneously ($\rightarrow$ vector/array processors). \hfill $\rightarrow$ SIMD, MIMD
    \end{enumerate}
\end{definition}

\pagebreak
\section{How programs are run on a system}
$\rightarrow$ Discuss two things you find particularly interesting.
\begin{itemize}
    \item \underline{Theoretical vs. Practical Performance} Back from algorithms and data structures we got used to the Big O notation to describe the asymptotic complexity of an algorithm regarding on how the performance scales with varying input sizes. However those analysations ignore the constant factors and lower-order terms of the runtime equation which can actually have a great impact in practise. A good such example is the Fibonacci Heap, an implementation of a priority queue, that has a theoretical worst case runtime of O(log n) (for {\scshape Delete-Min}) but is a lot less efficient than similiar priority queue implementations in practise due to requiring way more overhead to implement (requiring way more pointers per node leading to high memory consumption and high constant factors) \footnote{Good brief explanation with references found on \url{https://en.wikipedia.org/wiki/Fibonacci_heap} in the Performance chapter}. It's good to keep in mind that for the actual performance of an algorithm, not only it's Big O notation runtime counts but also factors like how well it utilises the hardware (like CPU cache, memory bandwidth and parallelism) and what are the actual typical input sizes in reality.
    \item \underline{Virtual memory} While we know how the physical memory hierarchy is looking like it's interesting to consider the abstracted virtual memory that a program is working within. Apparently there is a whole Memory Management Unit responsible to map virtual to physical memory addresses. This technique gets used to simplify the memory management within a program itself by making it believe it has access to the entire stack all by itself. 
\end{itemize}
