Overview

File-based event store persisting events as JSON on disk. Provides durability without requiring a database. Suitable for single-node deployments with moderate event volumes.

Events stored in append-only events.log file. Multi-process support via file locking and fsnotify change detection.

Quick Start

import Hindsight.Store.Filesystem

main :: IO ()
main = do
  -- Create store with default config
  config <- mkDefaultConfig "./events"
  store <- newFilesystemStore config

  -- Insert events (see Hindsight.Store for details)
  streamId <- StreamId <$> UUID.nextRandom
  let event = mkEvent MyEvent myData
  result <- insertEvents store Nothing $ singleEvent streamId NoStream event

  -- Subscribe to events
  handle <- subscribe store matcher (EventSelector AllStreams FromBeginning)
  -- ... process events ...

  -- Cleanup when done
  cleanupFilesystemStore store

Configuration

FilesystemStoreConfig has three parameters:

• storePath - Directory for event log and lock file
• syncInterval - Disk sync frequency (microseconds, 0 = sync every write)
• lockTimeout - Max time to wait for file lock (microseconds)

Use mkDefaultConfig for sensible defaults or construct manually for custom settings.

Use Cases

When to use Filesystem store:

• Single-node applications requiring durability
• Development/staging environments
• Embedded systems or edge deployments
• Apps that can't run PostgreSQL (resource constraints, deployment complexity)
• Multi-process applications on same host

When NOT to use Filesystem store:

• Distributed multi-node systems (use PostgreSQL)
• Very high event throughput (PostgreSQL performs better)
• Large event volumes (startup replay becomes slow)

Trade-offs

Advantages:

• Events survive process restarts (durable)
• No database dependency
• Multi-process support on same host
• Simple deployment (just a directory)
• ACID guarantees via file locking

Limitations:

• Startup time grows with event count (linear log replay)
• All indices must fit in memory
• Single-node only (no distributed support)
• Performance limited by disk I/O
• Not suitable for very large datasets

Implementation

Persistence layer over Memory store infrastructure. Events written to disk then loaded into in-memory STM structures. File locking serializes writes. fsnotify detects changes from other processes for incremental reloading.

Storage format: Append-only JSON log (events.log), one transaction per line. Stream indices rebuilt on startup by replaying log.