From e0039390b09e802a66c2be4842dd95fee695b433 Mon Sep 17 00:00:00 2001 From: Douglas Rumbaugh Date: Mon, 2 Jun 2025 20:31:18 -0400 Subject: Updates --- chapters/beyond-dsp.tex | 136 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 136 insertions(+) (limited to 'chapters/beyond-dsp.tex') diff --git a/chapters/beyond-dsp.tex b/chapters/beyond-dsp.tex index 222dd14..7a0df37 100644 --- a/chapters/beyond-dsp.tex +++ b/chapters/beyond-dsp.tex @@ -1289,6 +1289,142 @@ get better asymptotic performance. \subsection{Concurrency Control} \label{ssec:dyn-concurrency} +The decomposition-based dynamization scheme we are considering in this +work lends itself to a very straightfoward concurrency control scheme, +because it is founded upon static data structures. As a result, concurrent +writes only need to be managed within the mutable buffer. Beyond this, +reconstructions within the levels of the structure only reorganize +the existing data. So, a simple multi-versioning scheme to ensure that +queries only ever see one version of the structure and buffer is entirely +sufficient to allow concurrent reads and writes. In this section, we'll +discuss our basic concurrency implementation. + +\subsubsection{High Level Architecture} + +\begin{figure} +\includegraphics[width=\textwidth]{diag/concurrency.pdf} +\caption{\textbf{Framework Concurrency Architecture.} A high level +view of the concurrency architecture of the framework, detailing what +operations are run on what threads, and how they interact with the +major components. } +\label{fig:dyn-concurrency} +\end{figure} + +The general architecture of our framework is shown in +Figure~\ref{fig:dyn-concurrency}. The framework contains a scheduler and a +thread pool, which it uses to schedule and execute concurrent operations. +The scheduler itself is actually a user configurable component, but we +provide two default options: a serial scheduler that runs everything on +the client thread to emulate single-threaded behavior, and a standard +FIFO queue-based scheduler. The thread-pool contains a user-configurable +number of threads, which are shared between reconstructions and queries. +Additionally, the scheduler itself has a single thread that is used to +assign jobs to the thread pool. + +\subsubsection{The Mutable Buffer} + +Our mutable buffer is an unsorted array to which new records are appended. +This makes concurrent writes very straightfoward to support using a simple +fetch-and-add instruction on the tail pointer of the buffer. When a write +is issued, a fetch-and-add is executed against the tail pointer. This +effectively reserves a slot at the end of the array for the new record +to be written into, as each thread will recieve a unique index from this +operation. Then, the record can be directly assigned to that index. If +the buffer is full, then a reconstruction is scheduled (if one isn't +already running) and a failure is immediately returned to the user. + +In order to take advantage of concurrent reconstructions, the buffer +allocation is controlled by two numbers: a high and low watermark. The +buffer is physically allocated based upon the high watermark, but +reconstructions are triggered based on the low watermark. This system +allows the user to trigger reconstructions \emph{before} the buffer +fills. These reconstructions run in the background, allowing the system +to hide some of the reconstruction latency.\footnote{ + This system helps a little, but not very much. The amount of time it + takes to fill the buffer is vastly smaller than the time that some of + the more expensive reconstructions take, and so the overall effect + on performance is minute. In Chapter~\ref{chap:tail-latency}, we + discuss a more sophisticated background reconstruction scheme that is + much more effective at latency hiding. +} Once the buffer has filled +to the high watermark, insertions will block. + +Internal threads do not access the buffer directly, but rather retrieve +a read-only \emph{buffer view}, consisting of a head and tail pointer +indicating the set of records within the buffer that the thread can +``see''. This view defines the records removed from the buffer by a +flush, or that are seen by a query. These views are reference counted, +to ensure that records are not removed from the buffer until all threads +that can see them have finished. Because of this, queries holding a +buffer view can stall flushes. To reduce the effect of this problem, +the buffer is actually allocated with twice the physical space required +to support the specified high water mark, allowing multiple versions of +the buffer to coexist at once. + +\subsubsection{Structure Version Management} + +\begin{figure} +\includegraphics[width=\textwidth]{diag/version-transition.pdf} +\caption{\textbf{Structure Version Transition.} The transition of +active structure version proceeds in three phases. In the first +phase (A), the system waits for all references to the old version (V1) +to be released. Once a version is marked as old, it will no longer +accumulate references, so this is guaranteed to happen eventually. Once +all references have been released, the system moves to the second phase (B), +in which the active version (V2) is copied over the old version. During this +phase, both the old version and active version reference the same version +object (V2). Finally, in the third phase, the in-flight version (V3) is +made active. At this point, the reconstruction is complete, and another +reconstruction can immediately be scheduled, potentially result in a new +in-flight version (V4).} +\label{fig:dyn-conc-version} +\end{figure} + +The dynamized index itself is represented as a structure containing +levels which contain pointers to shards. Because the shards are static, +the entire structure can be cheaply shallow-copied to create a version. +The framework supports up to three versions at once: the ``old'' version, +the active version, and an ``in-flight'' version. Version transitions +occur at the conclusion of a reconstruction operation. + +Only one reconstruction can be active in the system at a time. When a +reconstruction begins, the active version of the structure is copied +to create the in-flight version, and a buffer view is claimed by the +reconstruction thread. The reconstruction is then performed on this copy, +building and removing shards according to the layout policy, and a new +shard is built from the buffer view. + +Once the reconstruction is complete, the framework attempts to convert +the in-flight version into the active one. This process first waits for +all threads currently accessing the old version to complete. Then, the +old version is replaced by the active version. During this replacement +process, the active version remains active, and so any threads entering +the system during the transition will be assigned to it. Once the active +version has been made old, the in-flight version is made active. The +versions are reference counted, so any threads using the active version +will not be affected by the deactivation of it. This process is summarized +in Figure~\ref{fig:dyn-conc-version} + + +\subsubsection{Concurrent Queries} + +Queries are answered asynchronously. When the user calls the query +routine, the query is given to the scheduler and an \texttt{std::future} +is returned to the user to access the result. Once the scheduler assigns +the query to a thread, it will retrieve a buffer view, and a reference +to the currently active version of the structure. The query will then +be processed against these structures. + +Because the query keeps a reference to the version and buffer view it is +using, it will be unaffected by the completion of a reconstruction. When +the active version becomes old, the reference remains valid, and the +query can continue to run. However, this reference will prevent the +old version from being retired, and so queries can block the completion +of reconstructions. Typically the expected runtime of a query is much +less than that of a reconstruction, and so this is not considered to +be a serious problem, but it is addressed in our more sophisticated +concurrency scheme discussed in Chapter~\ref{chap:tail-latency}. + \section{Evaluation} Having described the framework in detail, we'll now turn to demonstrating -- cgit v1.2.3