Apocrypha

This is a complete Haskell implementation of the Apocrypha JSON server.

You can see the original Python Implementation here, and the official API documentation.

Features

• Performance - up to 18,000 queries per second per CPU, semi linear performance scaling with additional CPUs. Benchmarking shows 8 CPUS ~= 86,000 queries/sec using a local Unix Domain Socket.

• Compatability - full compatability with the Python implementation. Scripts written using the Python API may see a 2x to 5x performance increase. The Haskell client API uses the same patterns as the Python API.

• Easy of Use - the client API integrates the Data.Aeson package for straight forward JSON object usage.

ACID

• Atomicity - all operations are atomic
• Consistency - invariants are enforced by the client application only, there are no schemas and no types
• Isolation - strong within one server, where each query is one STM transaction, and between serverless queries, which hold a lock beside the database file (<db>.lock) from read to save. None for a server or a caching client against other writers on the same file: they save their whole copy over the other's changes
• Weak Durability - the server writes to disk asynchronously; a serverless query that changes something has saved before it returns, without an fsync

Usage

• --headless - do not log to the console, may make execution up to %30 faster
• --no-cache - disable caching, decreases read performance depending on value complexity
• --stateless - do not write changes back to disk
• --no-unix - do not listen on a local Unix Domain Socket, only TCP
• --tcp-port <port> - listen for TCP on an alternate port, default 9999
• --bind <ip> - listen for TCP on this address, default 127.0.0.1. There's no authentication, so only widen this on a trusted network: :: is every interface over IPv4 and IPv6, 0.0.0.0 every interface over IPv4. A server that a --proxy points at needs it
• --database <path> - use an alternate file on disk for the database. With --bind, you can use this to serve an arbitrary JSON file to the network.
• --proxy <host[:port]> - forward every query to the server at host, rather than serving a database

Benchmarks

Performance testing done with the bench-apocrypha utility, run against the Haskell binary produced by make release and the latest Python implementation available through pip. These results were produced on Ubuntu 16.04, 32 Platinum 8168 CPU @ 2.70GHz

Workloads

Seconds to complete

	Python	Haskell	Speedup	Queries
Single reader	4.0533	2.5743	157%	100,000
Single reader cache	4.0559	2.3132	175%	100,000
Single writer	4.7642	3.0133	158%	100,000
Multi reader	53.3462	9.3792	568%	10 * 100,000
Multi reader cache	43.5055	8.9738	484%	10 * 100,000
Many reader	96.2097	19.6056	490%	20 * 100,000
Many reader cache	87.5551	19.2085	455%	20 * 100,000

Throughput

Average queries completed per second

	Python	Haskell	Difference	Queries
Single reader	24,671	38,845	+14,174	100,000
Single reader cache	24,655	43,230	+18,575	100,000
Single writer	20,989	33,186	+12,197	100,000
Multi reader	18,745	106,618	+87,873	10 * 100,000
Multi reader cache	22,985	111,435	+88,450	10 * 100,000
Many reader	20,787	102,011	+81,224	20 * 100,000
Many reader cache	22,842	104,120	+81,278	20 * 100,000

API

This library includes client bindings for applications that want to use an Apocrypha database natively.

type Query = [String]

set    :: (ToJSON a)   => Context -> Query -> a -> IO ()
get    :: (FromJSON a) => Context -> Query -> IO (Maybe a)
keys   :: Context -> Query -> IO [String]
del    :: Context -> Query -> IO ()
pop    :: Context -> Query -> IO (Maybe String)
append :: Context -> Query -> String -> IO ()

import Apocrypha.Client

main :: IO ()
main = do
    keys' [] >>= mapM_ print

    append' ["nested", "key"] "value"
    value <- pop' ["nested", "key"]

    case value of
      Nothing -> putStrLn "Nothing found"
      Just v  -> putStrLn $ "Found " <> v