-
Notifications
You must be signed in to change notification settings - Fork 10
Home
Welcome to the LunarBase Engin wiki!
LunarBase engine targets to a real time analysis and free-style database engine, managing 2 billions records in one table, where each record has a size limitation up to 32k bytes. Therefore 64 TB data can be stored and queried via one table of LunarBase.
The key challenge for a server end software is the memory fragments generated after a period of time running. Fragments slow down the system, unless user restart the OS, the resource will be eaten up soon or later. LunarBase includes a MMU(Memory Management Unit) to solve this problem by none-pause garbage collection technique. Then we are able to manage hundreds of GB memory in practice.
Administrators are enabled to configure a big direct memory to cache hot data for quick access. Never need to deploy another third party cache solution. It is not only cost effective, but also much more important for data consistency and validation, if applications have no choice but independent outer cache solutions. Consult why internal big cache for an elaboration on this subject.
Besides the basic CRUD(Create, Read, Update, Delete) operations that a database shall provide, LunarBase achieves ACID by L-transaction interface that gets eventually consistency data. Operations like JOIN in traditional RDBMSs, are supported via materialized view by LunarMView. Materialized view separates read and write, then greatly accelerates query speed, and gets eventually consistency for any data updates. For very busy sites with data modeled by tons of 1-to-1, 1-to-many, many-to-many relations, LunarMView is the suitable interface managing this.
For records stored, user queries any column by specifying its value you seek. Queries are "payment=300", "payment=300 AND 10<= age <=25", or something like these. These functionalities are supported by LunarMax real time engine. Consult chapter "real time computation" for its mechanism and usage.
Vertical scalability, real time computation, big memory and none-pause garbage collection, join relational tables, transactional log, these features together make LunarBase your best choice for your forever-online business.
Knowledge base can be found here: https://github.com/LunarBaseEngin/Application/blob/master/LunarBase%20--%20A%20database%20engin%20for%20managing%20very%20large%20amounts%20of%20data%20--%20EN%20--V0.8.pdf
1 Home
1.1 summary
1.2 System Preparation
1.3 When LunarBase is your best choice
1.4 Benchmark
1.5 Power consumption
2 Data Model And Process
2.1 Why internal big cache
2.2 Memory Management: LunarMMU
2.3 Garbage Collection
2.4 Transaction Log
2.5 JOIN via materialized view
3 Real Time Computation: LunarMax
3.1 In-Memory File System: Memory Estimation
3.2 Configuration
3.3 Use SSD as a cheaper memory
3.4 Data Safety
3.5 HE Server VS. Cluster
3.6 High Availability
4 Create a database
4.1 Three modes
4.2 creation.conf settings
4.3 Table space
4.4 Multiple Instance
4.5 Database Status
4.6 Remove and Restore a table
5 Insertion
5.1 Insert as normal record
5.2 Insert to search engine
6 Query
6.1 Point Query
6.2 Result Handler: register your own event handler
6.3 Interpreter Pattern: complex query conditions
6.4 Range Query
6.5 Full-text Search
6.6 Algebraic Logical Query
8 Deletion
9 Materialized view
9.1 Eventual consistency
9.2 Update
9.3 MVCC in LunarBase
9.4 Easy JOIN via denormalization
9.5 CRUD in view
10 Distributed integration with
10.1 Kafka
10.2 Storm
10.3 Spark
11 Storage: Lunar Virtual File System
13 Roadmap of LunarBase future
15 FAQ