Key-Value Engine

Overview

The Key-Value Engine in ShibuDB provides fast, reliable storage for simple key-value pairs. It's built on top of a B-tree index for efficient lookups and includes Write-Ahead Logging (WAL) for data durability.

Key Features

High Performance: B-tree indexing for fast key lookups
Data Durability: Write-Ahead Logging ensures data persistence
Atomic Operations: Each PUT/GET/DELETE operation is atomic
Space Isolation: Data is organized in separate spaces
Role-Based Access: Fine-grained permissions per space

Architecture

Key-Value Engine Architecture

┌─────────────────────────────────────┐
│           Key-Value Space           │
├─────────────────────────────────────┤
│  B-Tree Index (for fast lookups)    │
├─────────────────────────────────────┤
│  In-Memory Buffer (for performance) │
├─────────────────────────────────────┤
│  Write-Ahead Log (for durability)   │
├─────────────────────────────────────┤
│  Data Files (persistent storage)    │
└─────────────────────────────────────┘

Space Management

Key-value data is organized in spaces, which provide isolation and access control.

Creating a Key-Value Space

Create Space

# Create a basic key-value space
CREATE-SPACE users --engine key-value

# Create with custom name
CREATE-SPACE product_catalog --engine key-value

Note: Only admin users can create spaces.

Listing Spaces

List Spaces

# List all available spaces
LIST-SPACES

Response:

Response

{
  "status": "OK",
  "spaces": ["users", "product_catalog", "session_data"]
}

Using a Space

Switch Space

# Switch to a specific space
USE users

# Verify current space (prompt will show current space)
[users]>

Deleting a Space

Delete Space

# Delete a space (admin only)
DELETE-SPACE users

Warning: This permanently removes all data in the space.

Basic Operations

Core key-value operations for storing, retrieving, and managing data.

PUT - Store Data

Store Data

# Store a simple value
PUT user:1 "John Doe"

# Store with complex key
PUT user:profile:123 "{\"name\":\"John\",\"age\":30}"

# Store multiple values
PUT user:2 "Jane Smith"
PUT user:3 "Bob Johnson"

GET - Retrieve Data

Retrieve Data

# Get a single value
GET user:1

# Get multiple values
GET user:1 user:2 user:3

# Get with complex key
GET user:profile:123

DELETE - Remove Data

Delete Data

# Delete a single key
DELETE user:1

# Delete multiple keys
DELETE user:2 user:3

# Delete with complex key
DELETE user:profile:123

EXISTS - Check Key Existence

Check Existence

# Check if key exists
EXISTS user:1

# Check multiple keys
EXISTS user:1 user:2 user:3

Advanced Operations

Advanced features for efficient data management and querying.

Batch Operations

# Batch PUT operations
PUT user:1 "John Doe" user:2 "Jane Smith" user:3 "Bob Johnson"

# Batch GET operations
GET user:1 user:2 user:3 user:4 user:5

# Batch DELETE operations
DELETE user:1 user:2 user:3

Key Patterns and Wildcards

Pattern Matching

# Get all user keys
GET user:*

# Get all profile keys
GET user:profile:*

# Get keys with specific pattern
GET user:profile:1*

Data Types and Formats

Understanding the data types and formats supported by the key-value engine.

Supported Data Types

Strings: Simple text values
JSON: Structured data in JSON format
Numbers: Numeric values (stored as strings)
Binary: Binary data (base64 encoded)

Data Format Examples

Data Examples

# String values
PUT name "John Doe"
PUT email "john@example.com"

# JSON data
PUT user:profile "{\"name\":\"John\",\"age\":30,\"city\":\"New York\"}"

# Numeric values (as strings)
PUT user:age "30"
PUT user:score "95.5"

# Complex nested data
PUT user:preferences "{\"theme\":\"dark\",\"notifications\":true,\"language\":\"en\"}"

Performance Considerations

Optimizing performance for key-value operations.

Performance Tips

Use Batch Operations: Group multiple operations together
Optimize Key Design: Use meaningful, hierarchical keys
Limit Data Size: Keep individual values reasonably sized
Use Appropriate Spaces: Separate different data types into different spaces

Memory Usage

The key-value engine uses memory efficiently with:

B-tree index for fast lookups
In-memory buffer for recent operations
Write-ahead logging for durability
Automatic memory management

Best Practices

Recommended practices for using the key-value engine effectively.

Key Design

Use hierarchical keys: user:123:profile
Keep keys consistent and predictable
Avoid very long keys
Use meaningful separators (colons, dashes)

Data Organization

Group related data in the same space
Use different spaces for different data types
Consider access patterns when designing keys
Plan for future scalability

Error Handling

Always check operation results
Handle missing keys gracefully
Validate data before storing
Use appropriate error handling in applications

Examples and Use Cases

Common use cases and practical examples.

User Session Management

Session Management

# Create sessions space
CREATE-SPACE sessions --engine key-value
USE sessions

# Store session data
PUT session:abc123 "{\"user_id\":123,\"login_time\":\"2024-01-01T10:00:00Z\",\"expires\":\"2024-01-01T18:00:00Z\"}"

# Retrieve session
GET session:abc123

# Delete expired session
DELETE session:abc123

Configuration Management

Configuration

# Create config space
CREATE-SPACE config --engine key-value
USE config

# Store application settings
PUT app:theme "dark"
PUT app:language "en"
PUT app:notifications "true"
PUT app:timezone "UTC"

# Get all config
GET app:*

Feature Flags

# Create features space
CREATE-SPACE features --engine key-value
USE features

# Store feature flags
PUT feature:new_ui "true"
PUT feature:beta_mode "false"
PUT feature:analytics "true"

# Check feature status
GET feature:new_ui

Table of Contents