summaryrefslogtreecommitdiffstats
path: root/include
Commit message (Collapse)AuthorAgeFilesLines
...
* DynamicExtension: added a way to block on merge completionDouglas B. Rumbaugh2023-10-311-3/+31
| | | | | This is mostly just for testing purposes at the moment, though I'd imagine it may be useful for other reasons too.
* General Comment + Consistency updatesDouglas Rumbaugh2023-10-3024-282/+167
|
* DynamicExtension: comment cleanup/adjustmentsDouglas Rumbaugh2023-10-301-11/+14
|
* Epoch/DynamicExtension: added cv to epoch retirement checkDouglas Rumbaugh2023-10-302-4/+36
| | | | | | Instead of busy waiting on the active job count, a condition variable is now used to wait for all active jobs to finish before freeing an epoch's resources.
* FIFOScheduler: correctly protect m_cv with a lockDouglas Rumbaugh2023-10-301-0/+5
|
* DynamicExtension: adjusted a few operations to ensure conistencyDouglas Rumbaugh2023-10-302-15/+37
| | | | | | | | | | | | get_memory_usage, get_aux_memory_usage, get_record_count, get_tombstone_count, and create_static_structure have been adjusted to ensure that they pull from a consistent epoch, even if a change-over occurs midway through the function. These functions also now register with the epoch as a job, to ensure that the epoch they are operating own isn't retired midway through the function. Probably not a big issue for the accessors, but I could see it being very important for create_static_structure.
* Concurrency updates + fixes for compile errorsDouglas Rumbaugh2023-10-306-99/+106
|
* Began moving to an explicit epoch-based systemDouglas Rumbaugh2023-10-239-103/+481
| | | | | | | I started moving over to an explicit Epoch based system, which has necessitated a ton of changes throughout the code base. This will ultimately allow for a much cleaner set of abstractions for managing concurrency.
* BugfixesDouglas Rumbaugh2023-10-233-1/+7
|
* Initial pass w/ new scheduler setupDouglas Rumbaugh2023-10-207-341/+196
| | | | currently there's a race condition of some type to sort out.
* Checkpointing workDouglas Rumbaugh2023-10-202-33/+84
| | | | I'll probably throw all this out, but I want to stash it just in case.
* Re-structuring Project + scheduling updatesDouglas Rumbaugh2023-09-2521-427/+377
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a big one--probably should have split it apart, but I'm feeling lazy this morning. * Organized the mess of header files in include/framework by splitting them out into their own subdirectories, and renaming a few files to remove redundancies introduced by the directory structure. * Introduced a new framework/ShardRequirements.h header file for simpler shard development. This header simply contains the necessary includes from framework/* for creating shard files. This should help to remove structural dependencies from the framework file structure and shards, as well as centralizing the necessary framework files to make shard development easier. * Created a (currently dummy) SchedulerInterface, and make the scheduler implementation a template parameter of the dynamic extension for easier testing of various scheduling policies. There's still more work to be done to fully integrate the scheduler (queries, multiple buffers), but some more of the necessary framework code for this has been added as well. * Adjusted the Task interface setup for the scheduler. The task structures have been removed from ExtensionStructure and placed in their own header file. Additionally, I started experimenting with using std::variant, as opposed to inheritence, to implement subtype polymorphism on the Merge and Query tasks. The scheduler now has a general task queue that contains both, and std::variant, std::visit, and std::get are used to manipulate them without virtual functions. * Removed Alex.h, as it can't build anyway. There's a branch out there containing the Alex implementation stripped of the C++20 stuff. So there's no need to keep it here.
* Bugfixes for tieringDouglas Rumbaugh2023-09-203-4/+9
| | | | | | | | | | | | Fixed a few issues that manifested during the tiering tests, 1) When a version is copied, it now contains copies of the levels, not just pointers (the levels themselves still hold pointers to the shards, though). 2) Ensure that tasks are scheduled with the correct timestamp, they were originally being scheduled backwards. The get_merge_tasks() method already returns them in the correct order, so reversing them again put it in the wrong order.
* The scheduler now spawns a seperate merge threadDouglas Rumbaugh2023-09-184-58/+221
| | | | | | | | | | | Merges are now executed from a seperate thread within the scheduler that wakes up via condition variables when new merge tasks are scheduled. In addition, tombstone limits are now enforced by the scheduler, with new merges being scheduled as needed. There are still a few tests failing, notably the zero tombstones in the last run invarient is not holding under tiering with tombstones. Need to look into that yet.
* Moved individual merge task execution into the schedulerDouglas Rumbaugh2023-09-183-35/+98
| | | | | | | | | | | | | | | | | | | | This change is made in anticipation of scheduling each task using a specific thread, and required some modification to the interface of ExtensionStructure. Namely, 1. ExtensionStructure now supports a get_merge_tasks() interface, which returns a list of the individual level merges that would need to be performed to complete a buffer flush of specified size. 2. merge_levels and merge_buffer have been promoted to the public interface, to allow their use within the scheduler. 3. merge_buffer has been modified to assume that the structure already can support a direct flush of the buffer into L0, it is now the responsibility of the caller to ensure that the necessary merges have already been completed prior to calling this method. Currently, preemptive tombstone compactions are non-functional, so some unit tests are failing. This will be fixed when the thread scheduling system is set up.
* General bugfixesDouglas Rumbaugh2023-09-182-1/+5
|
* Began re-architecting the project for concurrency supportDouglas Rumbaugh2023-09-137-308/+645
| | | | | The project is now in a state where it builds, but it probably has a lot of bugs still.
* Migrated over to using psudb-common utilities/headersDouglas Rumbaugh2023-08-2421-4512/+77
|
* Removed unused pagedfile headerDouglas Rumbaugh2023-08-241-434/+0
|
* Change PGM to range count.Dong Xie2023-07-282-32/+58
|
* Expanded query interfaceDouglas Rumbaugh2023-07-279-50/+269
| | | | | Query interface now enables skipping of delete processing and stopping query processing when first match is found.
* Alex shard progressDouglas Rumbaugh2023-07-262-0/+361
|
* Exposed epsilon configuration parameter for PGMDouglas Rumbaugh2023-07-261-4/+5
|
* Adjusted calculation for index sizeDouglas Rumbaugh2023-07-266-46/+53
|
* VPTree: account for size of pointer array in memory usageDouglas Rumbaugh2023-07-251-2/+1
|
* VPTree: Added template configuration to use/not use a hash tableDouglas Rumbaugh2023-07-251-6/+32
|
* VPTree: bugfixDouglas Rumbaugh2023-07-241-1/+1
|
* VPTree: added a level of indirection to avoid repeated point copiesDouglas Rumbaugh2023-07-241-23/+52
|
* VPTree: fixed knn queryDouglas Rumbaugh2023-07-243-53/+99
|
* Cosine Similarity TypeDouglas Rumbaugh2023-07-241-4/+48
|
* VPTree: KNN query initial implementationDouglas Rumbaugh2023-07-241-4/+126
|
* ISAM IRS query bugfixesDouglas Rumbaugh2023-07-241-9/+28
|
* BTree-based baselines for IRS and Range queriesDouglas Rumbaugh2023-07-231-0/+3924
|
* Triespline RQ fixesDouglas Rumbaugh2023-07-231-7/+26
|
* Benchmarking updatesDouglas Rumbaugh2023-07-231-2/+124
|
* Added the parameter argument to query merge routineDouglas Rumbaugh2023-07-176-6/+6
| | | | | This is necessary for KNN, but it unused for all currently implemented query types.
* VPTree: added fixme for tombstone handling in the hashtableDouglas Rumbaugh2023-07-171-2/+7
|
* PriorityQueue: generalized priority queue comparison operationDouglas Rumbaugh2023-07-172-4/+41
| | | | | Generalized the comparison used for the priority queue to enable its use within the KNN query code.
* VPTree: changed Point format to a D-dimensional point.Douglas Rumbaugh2023-07-171-6/+24
|
* VPTree: use a secondary hash-table for point lookupsDouglas Rumbaugh2023-07-172-8/+24
|
* Initial commit of VPTree-related codeDouglas Rumbaugh2023-07-132-11/+350
| | | | | | Point lookups are currently broken; I suspect that there is something wrong with tree construction, although the quickselect implementation seems to be fine.
* Slightly optimized merge functionDouglas Rumbaugh2023-07-031-3/+11
|
* BugfixesDouglas Rumbaugh2023-07-031-1/+1
|
* Fixed query errorsDouglas Rumbaugh2023-07-032-8/+16
|
* PGM Shard bugfixesDouglas Rumbaugh2023-06-261-2/+2
|
* TrieSpline: fixed error in min/max key determinationDouglas Rumbaugh2023-06-131-5/+10
| | | | | | An elif was used when the conditions were _not_ supposed to be mutually exclusive, resulting in the wrong minimum key value being used in some cases.
* MemISAM: Removed friend relationship with InternalLevelDouglas Rumbaugh2023-06-131-1/+0
|
* TrieSpline: parameterized TrieSpline template with max errorDouglas Rumbaugh2023-06-131-5/+3
|
* Updated bloom filters to use whole recordDouglas B. Rumbaugh2023-06-095-30/+30
|
* Fixed bug(s) in tagging-based deletesDouglas B. Rumbaugh2023-06-097-7/+27
|