summaryrefslogtreecommitdiffstats
path: root/include/framework/structure
Commit message (Collapse)AuthorAgeFilesLines
* Comment updates/fixesDouglas Rumbaugh2024-02-093-29/+42
|
* Updated VPTree to new shard/query interfacesDouglas Rumbaugh2024-02-081-3/+0
|
* Fully realized shard concept interfaceDouglas Rumbaugh2024-02-072-3/+3
|
* Fully implemented Query concept and adjusted queries to use itDouglas Rumbaugh2024-02-072-3/+3
|
* BufferView: Adjusted BV to avoid repeated modulus operationsDouglas Rumbaugh2024-02-052-15/+48
|
* Insert throughput benchmarkDouglas Rumbaugh2024-01-311-1/+1
|
* Multiple concurrency bug fixesDouglas Rumbaugh2024-01-173-36/+50
| | | | | | | | A poorly organized commit with fixes for a variety of bugs that were causing missing records. The core problems all appear to be fixed, though there is an outstanding problem with tombstones not being completely canceled. A very small number are appearing in the wrong order during the static structure test.
* Concurrency testing and bug fixesDouglas B. Rumbaugh2024-01-151-1/+9
|
* Use 16-byte CAS to control buffer headDouglas B. Rumbaugh2024-01-151-61/+59
|
* Various single-threaded bug fixesDouglas B. Rumbaugh2024-01-152-8/+36
|
* Initial integration of new buffering scheme into frameworkDouglas Rumbaugh2024-01-122-29/+16
| | | | | It isn't working right now (lotsa test failures), but we're to the debugging phase now.
* BufferView.h: Hopefully the last necessary tweak to the move semantics stuffDouglas Rumbaugh2024-01-121-24/+2
| | | | | | | | You can't move assign an std::Bind, but you can move construct it. So I had to disable the move assignment operator. This means that when you change the BufferView ownership over to, say, a QueryBufferState object, you need to do it by passing std::move(buffview) into a constructor call only--you cannot assign it.
* InternalLevel update and testsDouglas Rumbaugh2024-01-112-16/+39
| | | | | Plus some assorted fixes for move semantics stuff in BufferView that accompanied these changes.
* BufferView: enforce move semanticsDouglas Rumbaugh2024-01-111-2/+19
| | | | | | Because a BufferView's lifetime is so tightly linked to the lifetime of regions of the buffer, it can't be copied without potentially breaking things.
* Fixed some potential buffer-related concurrency bugsDouglas Rumbaugh2024-01-112-57/+94
|
* MutableBuffer: multithreaded insert test + bugfixesDouglas Rumbaugh2024-01-101-4/+9
|
* MutableBuffer+View: Implementation with unit testsDouglas Rumbaugh2024-01-102-49/+116
|
* Initial update of buffer to new specificationsDouglas B. Rumbaugh2024-01-092-152/+62
| | | | | | | | | | | | | | | | | | | There are a few minor issues that this introduces, however. Global tracking of a lot of secondary information, such as weights for WIRS/WSS, or the exact number of tombstones will need to be approached differently than they have been historically with this new approach. I've also removed most of the tombstone capacity related code. We had decided not to bother enforcing this at the buffer level anyway, and it would greatly increase the complexity of the problem of predicting when the next compaction will be. On the whole this new approach seems like it'll simplify a lot. This commit actually removes significantly more code than it adds. One minor issue: the currently implementation will have problems in the circular array indexes once more than 2^64 records have been inserted. This doesn't seem like a realistic problem at the moment.
* InternalLevel: appending an empty level is a no-opDouglas Rumbaugh2023-12-211-0/+7
| | | | | | | | The existing reconstruction logic will occasionally attempt to append an empty level to another empty level, for some reason. While the underlying cause of this needs to be looked into, this special case should prevent shard constructors being called with a shard count of 0 under tiering, reducing the error handling overhead of shard code.
* ExtensionStructure: adjusted leveling logic to avoid unneeded copiesDouglas Rumbaugh2023-12-211-5/+14
| | | | | | | | This also reduces the special-case overhead on shards. As it was, shards would need to handle a special case when constructing from other shards where the first of the two provided shards was a nullptr, which caused a number of subtle issues (or outright crashes in some cases) with existing shard implementations.
* Refactoring: corrected a number of names and added more commentsDouglas Rumbaugh2023-12-133-98/+120
|
* Tombstone Compaction: re-enabled tombstone compactionDouglas Rumbaugh2023-11-132-1/+52
| | | | | | | Currently, proactive buffer tombstone compaction is disabled by forcing the buffer tombstone capacity to match its record capacity. It isn't clear how to best handle proactive buffer compactions in an environment where new buffers are spawned anyway.
* Fixed merge logic bug in tieringDouglas Rumbaugh2023-11-131-0/+1
| | | | | | | | | | | | In InternalLevel::clone(), the m_shard_cnt variable was not being set appropriately in the clone, resulting in the record counts reported for a multi-shard level to be reported incorrectly. In DynamicExtension::merge(), the merges were being performed in the wrong order, resulting in multi-level merges deleting records. The leveling tests all passed even with this bug for some reason, but it caused tiering tests to fail. It isn't clear _why_ leveling appeared to work, but the bug is now fixed, so that's largely irrelevant I suppose.
* Comment and License updatesDouglas Rumbaugh2023-11-074-4/+4
|
* InternalLevel: switched to std::sharedptr for shard memory managementDouglas Rumbaugh2023-11-061-52/+29
|
* MutableBuffer: added explicit tail variableDouglas Rumbaugh2023-11-061-5/+12
| | | | | | | Use an explicit m_tail variable for insertion, rather than using m_reccnt. This ensures that the record count doesn't increase despite new records being inserted, and allows for the m_tail variable to be decremented on failure without causing the record count to momentarily change.
* Fixes for various bugs under SerialSchedulerDouglas B. Rumbaugh2023-10-311-1/+4
|
* MutableBuffer: removed most concurrency control stuffDouglas B. Rumbaugh2023-10-312-52/+13
| | | | | | The buffer isn't responsible for a lot of CC anymore (just the append operation), so this code was no longer necessary. Also removed the only calls to some of these CC operations within the rest of the framework.
* ExtensionStructure: fixed incorrect constructor args in clone()Douglas B. Rumbaugh2023-10-311-2/+2
|
* General Comment + Consistency updatesDouglas Rumbaugh2023-10-304-7/+7
|
* DynamicExtension: adjusted a few operations to ensure conistencyDouglas Rumbaugh2023-10-301-0/+4
| | | | | | | | | | | | get_memory_usage, get_aux_memory_usage, get_record_count, get_tombstone_count, and create_static_structure have been adjusted to ensure that they pull from a consistent epoch, even if a change-over occurs midway through the function. These functions also now register with the epoch as a job, to ensure that the epoch they are operating own isn't retired midway through the function. Probably not a big issue for the accessors, but I could see it being very important for create_static_structure.
* Concurrency updates + fixes for compile errorsDouglas Rumbaugh2023-10-302-12/+13
|
* Began moving to an explicit epoch-based systemDouglas Rumbaugh2023-10-233-0/+154
| | | | | | | I started moving over to an explicit Epoch based system, which has necessitated a ton of changes throughout the code base. This will ultimately allow for a much cleaner set of abstractions for managing concurrency.
* BugfixesDouglas Rumbaugh2023-10-231-0/+1
|
* Initial pass w/ new scheduler setupDouglas Rumbaugh2023-10-202-12/+20
| | | | currently there's a race condition of some type to sort out.
* Re-structuring Project + scheduling updatesDouglas Rumbaugh2023-09-253-0/+928
This is a big one--probably should have split it apart, but I'm feeling lazy this morning. * Organized the mess of header files in include/framework by splitting them out into their own subdirectories, and renaming a few files to remove redundancies introduced by the directory structure. * Introduced a new framework/ShardRequirements.h header file for simpler shard development. This header simply contains the necessary includes from framework/* for creating shard files. This should help to remove structural dependencies from the framework file structure and shards, as well as centralizing the necessary framework files to make shard development easier. * Created a (currently dummy) SchedulerInterface, and make the scheduler implementation a template parameter of the dynamic extension for easier testing of various scheduling policies. There's still more work to be done to fully integrate the scheduler (queries, multiple buffers), but some more of the necessary framework code for this has been added as well. * Adjusted the Task interface setup for the scheduler. The task structures have been removed from ExtensionStructure and placed in their own header file. Additionally, I started experimenting with using std::variant, as opposed to inheritence, to implement subtype polymorphism on the Merge and Query tasks. The scheduler now has a general task queue that contains both, and std::variant, std::visit, and std::get are used to manipulate them without virtual functions. * Removed Alex.h, as it can't build anyway. There's a branch out there containing the Alex implementation stripped of the C++20 stuff. So there's no need to keep it here.