Interface update (#5)

* Query Interface Adjustments/Refactoring

Began the process of adjusting the query interface (and also the shard
interface, to a lesser degree) to better accommodate the user. In
particular the following changes have been made,

    1. The number of necessary template arguments for the query type
       has been drastically reduced, while also removing the void pointers
       and manual delete functions from the interface.

       This was accomplished by requiring many of the sub-types associated
       with a query (parameters, etc.) to be nested inside the main query
       class, and by forcing the SHARD type to expose its associated
       record type.

    2. User-defined query return types are now supported.

       Queries no longer are required to return strictly sets of records.
       Instead, the query now has LocalResultType and ResultType
       template parameters (which can be defaulted using a typedef in
       the Query type itself), allowing much more flexibility.

       Note that, at least for the short term, the LocalResultType must
       still expose the same is_deleted/is_tombstone interface as a
       Wrapped<R> used to, as this is currently needed for delete
       filtering. A better approach to this is, hopefully, forthcoming.

    3. Updated the ISAMTree.h shard and rangequery.h query to use the
       new interfaces, and adjusted the associated unit tests as well.

    4. Dropped the unnecessary "get_data()" function from the ShardInterface
       concept.

    5. Dropped the need to specify a record type in the ShardInterface
       concept. This is now handled using a required Shard::RECORD
       member of the Shard class itself, which should expose the name
       of the record type.

* Updates to framework to support new Query/Shard interfaces

Pretty extensive adjustments to the framework, particularly to the
templates themselves, along with some type-renaming work, to support
the new query and shard interfaces.

Adjusted the external query interface to take an rvalue reference, rather
than a pointer, to the query parameters.

* Removed framework-level delete filtering

This was causing some issues with the new query interface, and should
probably be reworked anyway, so I'm temporarily (TM) removing the
feature.

* Updated benchmarks + remaining code for new interface

1 files changed, 120 insertions, 16 deletions

diff --git a/include/framework/interface/Query.h b/include/framework/interface/Query.h
index 577d6cd..1b64646 100644
--- a/include/framework/interface/Query.h
+++ b/include/framework/interface/Query.h
@@ -1,7 +1,7 @@
 /*
  * include/framework/interface/Query.h
  *
- * Copyright (C) 2023 Douglas B. Rumbaugh <drumbaugh@psu.edu> 
+ * Copyright (C) 2023-2024 Douglas B. Rumbaugh <drumbaugh@psu.edu>
  *
  * Distributed under the Modified BSD License.
  *
@@ -10,23 +10,127 @@
 
 #include "framework/QueryRequirements.h"
 
-namespace de{
+namespace de {
 
-template <typename Q, typename R, typename S>
-concept QueryInterface = requires(void *p, S *sh, std::vector<void*> &s, std::vector<std::vector<Wrapped<R>>> &rv, BufferView<R> *bv, std::vector<R> &resv) {
-    {Q::get_query_state(sh, p)} -> std::convertible_to<void*>;
-    {Q::get_buffer_query_state(bv, p)} -> std::convertible_to<void *>;
-    {Q::process_query_states(p, s, p)};
-    {Q::query(sh, p, p)} -> std::convertible_to<std::vector<Wrapped<R>>>;
-    {Q::buffer_query(p, p)} -> std::convertible_to<std::vector<Wrapped<R>>>;
-    {Q::merge(rv, p, resv)};
+/*
+ * FIXME: It would probably be best to absorb the std::vector into
+ *        this type too; this would allow user-defined collections for
+ *        intermediate results, which could allow for more merging
+ *        optimizations. However, this would require an alternative
+ *        approach to doing delete checks, so we'll leave it for now.
+ */
+template <typename R>
+concept LocalResultInterface = requires(R res) {
+  { res.is_deleted() } -> std::convertible_to<bool>;
+  { res.is_tombstone() } -> std::convertible_to<bool>;
+};
+
+/*
+ *
+ *
+ */
+template <typename QUERY, typename SHARD,
+          typename RESULT = typename QUERY::ResultType,
+          typename LOCAL_RESULT = typename QUERY::LocalResultType,
+          typename PARAMETERS = typename QUERY::Parameters,
+          typename LOCAL = typename QUERY::LocalQuery,
+          typename LOCAL_BUFFER = typename QUERY::LocalQueryBuffer>
+concept QueryInterface = LocalResultInterface<LOCAL_RESULT> &&
+    requires(PARAMETERS *parameters, LOCAL *local, LOCAL_BUFFER *buffer_query,
+             SHARD *shard, std::vector<LOCAL *> &local_queries,
+             std::vector<std::vector<LOCAL_RESULT>> &local_results,
+             std::vector<RESULT> &result,
+             BufferView<typename SHARD::RECORD> *bv) {
+
+  /*
+   * Given a set of query parameters and a shard, return a local query
+   * object for that shard.
+   */
+  { QUERY::local_preproc(shard, parameters) } -> std::convertible_to<LOCAL *>;
+
+  /*
+   * Given a set of query parameters and a buffer view, return a local
+   * query object for the buffer.
+   * NOTE: for interface reasons, the pointer to the buffer view MUST be
+   *       stored inside of the local query object. The future buffer
+   *       query routine will access the buffer by way of this pointer.
+   */
+  {
+    QUERY::local_preproc_buffer(bv, parameters)
+    } -> std::convertible_to<LOCAL_BUFFER *>;
+
+  /*
+   * Given a full set of local queries, and the buffer query, make any
+   * necessary adjustments to the local queries in-place, to account for
+   * global information. If no additional processing is required, this
+   * function can be left empty.
+   */
+  {QUERY::distribute_query(parameters, local_queries, buffer_query)};
+
+  /*
+   * Answer the local query, defined by `local` against `shard` and return
+   * a vector of LOCAL_RESULT objects defining the query result.
+   */
+  {
+    QUERY::local_query(shard, local)
+    } -> std::convertible_to<std::vector<LOCAL_RESULT>>;
+
+  /*
+   * Answer the local query defined by `local` against the buffer (which
+   * should be accessed by a pointer inside of `local`) and return a vector
+   * of LOCAL_RESULT objects defining the query result.
+   */
+  {
+    QUERY::local_query_buffer(buffer_query)
+    } -> std::convertible_to<std::vector<LOCAL_RESULT>>;
+
+  /*
+   * Process the local results from the buffer and all of the shards,
+   * stored in `local_results`, and insert the associated ResultType
+   * objects into the `result` vector, which represents the final result
+   * of the query. Updates to this vector are done in-place.
+   */
+  {QUERY::combine(local_results, parameters, result)};
 
-    {Q::delete_query_state(p)} -> std::same_as<void>;
-    {Q::delete_buffer_query_state(p)} -> std::same_as<void>;
+  /*
+   * Process the post-combine `result` vector of ResultType objects,
+   * in the context of the global and local query parameters, to determine
+   * if the query should be repeated. If so, make any necessary adjustments
+   * to the local query objects and return True. Otherwise, return False.
+   *
+   * If no repetition is needed for a given problem type, simply return
+   * False immediately and the query will end.
+   */
+  {
+    QUERY::repeat(parameters, result, local_queries, buffer_query)
+    } -> std::same_as<bool>;
 
-    {Q::repeat(p, resv, s, p)} -> std::same_as<bool>;
+  /*
+   * If this flag is True, then the query will immediately stop and return
+   * a result as soon as the first non-deleted LocalRecordType is found.
+   * Otherwise, every Shard and the buffer will be queried and the results
+   * merged, like normal.
+   *
+   * This is largely an optimization flag for use with point-lookup, or
+   * other single-record result queries
+   */
+  { QUERY::EARLY_ABORT } -> std::convertible_to<bool>;
 
-    {Q::EARLY_ABORT} -> std::convertible_to<bool>;
-    {Q::SKIP_DELETE_FILTER} -> std::convertible_to<bool>;
+  /*
+   * If false, the built-in delete filtering that the framework can
+   * apply to the local results, prior to calling combine, will be skipped.
+   * This general filtering can be inefficient, particularly for tombstone
+   * -based deletes, and so if a more efficient manual filtering can be
+   * performed, it is worth setting this to True and doing that filtering
+   * in the combine step.
+   *
+   * If deletes are not a consideration for your problem, it's also best
+   * to turn this off, as it'll avoid the framework making an extra pass
+   * over the local results prior to combining them.
+   *
+   * TODO: Temporarily disabling this, as we've dropped framework-level
+   *       delete filtering for the time being.
+   */
+   /* { QUERY::SKIP_DELETE_FILTER } -> std::convertible_to<bool>; */
 };
-}
+} // namespace de