Updates

1 files changed, 136 insertions, 0 deletions

diff --git a/chapters/beyond-dsp.tex b/chapters/beyond-dsp.tex
index 222dd14..7a0df37 100644
--- a/chapters/beyond-dsp.tex
+++ b/chapters/beyond-dsp.tex
@@ -1289,6 +1289,142 @@ get better asymptotic performance.
 \subsection{Concurrency Control}
 \label{ssec:dyn-concurrency}
 
+The decomposition-based dynamization scheme we are considering in this
+work lends itself to a very straightfoward concurrency control scheme,
+because it is founded upon static data structures. As a result, concurrent
+writes only need to be managed within the mutable buffer. Beyond this,
+reconstructions within the levels of the structure only reorganize
+the existing data. So, a simple multi-versioning scheme to ensure that
+queries only ever see one version of the structure and buffer is entirely
+sufficient to allow concurrent reads and writes. In this section, we'll
+discuss our basic concurrency implementation.
+
+\subsubsection{High Level Architecture}
+
+\begin{figure}
+\includegraphics[width=\textwidth]{diag/concurrency.pdf}
+\caption{\textbf{Framework Concurrency Architecture.} A high level
+view of the concurrency architecture of the framework, detailing what
+operations are run on what threads, and how they interact with the
+major components. }
+\label{fig:dyn-concurrency}
+\end{figure}
+
+The general architecture of our framework is shown in
+Figure~\ref{fig:dyn-concurrency}. The framework contains a scheduler and a
+thread pool, which it uses to schedule and execute concurrent operations.
+The scheduler itself is actually a user configurable component, but we
+provide two default options: a serial scheduler that runs everything on
+the client thread to emulate single-threaded behavior, and a standard
+FIFO queue-based scheduler. The thread-pool contains a user-configurable
+number of threads, which are shared between reconstructions and queries.
+Additionally, the scheduler itself has a single thread that is used to
+assign jobs to the thread pool.
+
+\subsubsection{The Mutable Buffer}
+
+Our mutable buffer is an unsorted array to which new records are appended.
+This makes concurrent writes very straightfoward to support using a simple
+fetch-and-add instruction on the tail pointer of the buffer. When a write
+is issued, a fetch-and-add is executed against the tail pointer. This
+effectively reserves a slot at the end of the array for the new record
+to be written into, as each thread will recieve a unique index from this
+operation. Then, the record can be directly assigned to that index. If
+the buffer is full, then a reconstruction is scheduled (if one isn't
+already running) and a failure is immediately returned to the user.
+
+In order to take advantage of concurrent reconstructions, the buffer
+allocation is controlled by two numbers: a high and low watermark. The
+buffer is physically allocated based upon the high watermark, but
+reconstructions are triggered based on the low watermark. This system
+allows the user to trigger reconstructions \emph{before} the buffer
+fills. These reconstructions run in the background, allowing the system
+to hide some of the reconstruction latency.\footnote{
+    This system helps a little, but not very much. The amount of time it
+    takes to fill the buffer is vastly smaller than the time that some of
+    the more expensive reconstructions take, and so the overall effect
+    on performance is minute. In Chapter~\ref{chap:tail-latency}, we
+    discuss a more sophisticated background reconstruction scheme that is
+    much more effective at latency hiding.
+} Once the buffer has filled
+to the high watermark, insertions will block.
+
+Internal threads do not access the buffer directly, but rather retrieve
+a read-only \emph{buffer view}, consisting of a head and tail pointer
+indicating the set of records within the buffer that the thread can
+``see''.  This view defines the records removed from the buffer by a
+flush, or that are seen by a query. These views are reference counted,
+to ensure that records are not removed from the buffer until all threads
+that can see them have finished. Because of this, queries holding a
+buffer view can stall flushes. To reduce the effect of this problem,
+the buffer is actually allocated with twice the physical space required
+to support the specified high water mark, allowing multiple versions of
+the buffer to coexist at once.
+
+\subsubsection{Structure Version Management}
+
+\begin{figure}
+\includegraphics[width=\textwidth]{diag/version-transition.pdf}
+\caption{\textbf{Structure Version Transition.} The transition of
+active structure version proceeds in three phases. In the first
+phase (A), the system waits for all references to the old version (V1)
+to be released. Once a version is marked as old, it will no longer
+accumulate references, so this is guaranteed to happen eventually. Once
+all references have been released, the system moves to the second phase (B),
+in which the active version (V2) is copied over the old version. During this
+phase, both the old version and active version reference the same version
+object (V2). Finally, in the third phase, the in-flight version (V3) is
+made active. At this point, the reconstruction is complete, and another
+reconstruction can immediately be scheduled, potentially result in a new
+in-flight version (V4).}
+\label{fig:dyn-conc-version}
+\end{figure}
+
+The dynamized index itself is represented as a structure containing
+levels which contain pointers to shards. Because the shards are static,
+the entire structure can be cheaply shallow-copied to create a version.
+The framework supports up to three versions at once: the ``old'' version,
+the active version, and an ``in-flight'' version. Version transitions
+occur at the conclusion of a reconstruction operation.
+
+Only one reconstruction can be active in the system at a time. When a
+reconstruction begins, the active version of the structure is copied
+to create the in-flight version, and a buffer view is claimed by the
+reconstruction thread. The reconstruction is then performed on this copy,
+building and removing shards according to the layout policy, and a new
+shard is built from the buffer view.
+
+Once the reconstruction is complete, the framework attempts to convert
+the in-flight version into the active one. This process first waits for
+all threads currently accessing the old version to complete. Then, the
+old version is replaced by the active version. During this replacement
+process, the active version remains active, and so any threads entering
+the system during the transition will be assigned to it. Once the active
+version has been made old, the in-flight version is made active. The
+versions are reference counted, so any threads using the active version
+will not be affected by the deactivation of it. This process is summarized
+in Figure~\ref{fig:dyn-conc-version}
+
+
+\subsubsection{Concurrent Queries}
+
+Queries are answered asynchronously. When the user calls the query
+routine, the query is given to the scheduler and an \texttt{std::future}
+is returned to the user to access the result. Once the scheduler assigns
+the query to a thread, it will retrieve a buffer view, and a reference
+to the currently active version of the structure. The query will then
+be processed against these structures.
+
+Because the query keeps a reference to the version and buffer view it is
+using, it will be unaffected by the completion of a reconstruction. When
+the active version becomes old, the reference remains valid, and the
+query can continue to run. However, this reference will prevent the
+old version from being retired, and so queries can block the completion
+of reconstructions. Typically the expected runtime of a query is much
+less than that of a reconstruction, and so this is not considered to
+be a serious problem, but it is addressed in our more sophisticated
+concurrency scheme discussed in Chapter~\ref{chap:tail-latency}.
+
 \section{Evaluation}
 
 Having described the framework in detail, we'll now turn to demonstrating