Databases
Interesting
- RocksDB - Persistent Key-Value Store for Flash and RAM Storage. (Rocksplicator - RocksDB Replication)
- LevelDB - Fast key-value storage library written at Google that provides an ordered mapping from string keys to string values.
- NodeLevelDB - Fast & simple storage. A Node.js-style LevelDB wrapper for Node.js, Electron and browsers. (Awesome)
- TiKV - Distributed transactional key-value database, originally created to complement TiDB. (pd - Placement driver for TiKV) (TiKV Rust Client)
- RedixDB - Persistent real-time key-value store, with the same redis protocol with powerful features.
- EdgeDB - Open-source object-relational database built on top of PostgreSQL. (EdgeDB 1.0 Beta) (Web) (GitHub) (Talk)
- Cassandra - Highly-scalable partitioned row store. Rows are organized into tables with a required primary key.
- Scylla - Drop-in Apache Cassandra alternative big data database that powers your applications with ultra-low latency and extremely high throughput, while reducing TCO to a fraction of most NoSQL databases. Code (Scylla University)
- DuckDB - Embeddable SQL OLAP Database Management System. (HN) (Playing With DuckDB) (5 minute intro) (Web)
- Pebble - RocksDB/LevelDB inspired key-value database in Go. (Intro article) (HN) (Pebble vs RocksDB: Implementation Differences)
- SQLite - C-language library that implements a small, fast, self-contained, high-reliability, full-featured, SQL database engine.
- MongoDB - General purpose, document-based, distributed database built for modern application developers.
- Delta Lake - Storage layer that brings scalable, ACID transactions to Apache Spark and other big-data engines.
- Neo4j - High performance graph store with all the features expected of a mature and robust database, like a friendly query language and ACID transactions.
- Dgraph - Horizontally scalable and distributed graph database, providing ACID transactions, consistent replication and linearizable reads.
- CrateDB - Distributed SQL database that makes it simple to store and analyze massive amounts of machine data in real-time.
- OmniSciDB - Open source SQL-based, relational, columnar database engine that leverages the full performance and parallelism of modern hardware.
- Nebula Graph - Open-source graph database capable of hosting super large scale graphs with dozens of billions of vertices (nodes) and trillions of edges, with milliseconds of latency. (HN) (Active Fork)
- MinIO - High Performance, Kubernetes Native Object Storage. (Web) (GitHub) (MinIO: A Bare Metal Drop-In for AWS S3) (MinIO Console) (MinIO Operator)
- SwayDB - Fast embeddable persistent and in-memory key-value storage engine that provides storage as simple data structures - Map, Set & Queue.
- TerminusDB - Full featured in-memory graph database management system with a rich query language. (Code) (HN)
- WhiteDB - Lightweight database library operating fully in main memory. Disk is used only for dumping/restoring database and logging.
- TimescaleDB - Open-source database built for analyzing time-series data with the power and convenience of SQL. (timescaledb-tune) (HN) (2.0 release)
- ClickHouse - Open-source column-oriented database management system that allows generating analytical data reports in real time. (How ClickHouse Saved our Data) (HN) (Faster ClickHouse Imports) (HN: ClickHouse, Inc.) (Article) (HN)
- ArongoDB - Natively store data for graph, document and search needs. Utilize feature-rich access with one query language. (Go Driver) (Arangolite - Go Driver) (Python Driver)
- LiteStore - Lightweight, self-contained, RESTful, multi-format NoSQL document store server written in Nim and powered by a SQLite backend for storage.
- Apache Ignite - Horizontally scalable, fault-tolerant distributed in-memory computing platform for building real-time applications that can process terabytes of data with in-memory speed.
- Pravega - Open source distributed storage service implementing Streams. It offers Stream as the main primitive for the foundation of reliable storage systems.
- Datalevin - Port of Datascript in-memory Datalog database to Lightning Memory-Mapped Database (LMDB).
- Graviton Database - Simple, fast, versioned, authenticated, embeddable key-value store database in pure Go. (HN)
- Quadrable - Authenticated multi-version database: sparse binary merkle tree with compact partial-tree proofs.
- Amazon QLDB - Fully managed ledger database that provides a transparent, immutable, and cryptographically verifiable transaction log. (Awesome)
- JavaScript Database (JSDB) - Transparent, in-memory, streaming write-on-update JavaScript database for Small Web applications that persists to a JavaScript transaction log. (Intro) (Lobsters)
- Bigbucket - Serverless NoSQL database with a focus on scalability, availability and simplicity. It has a Bigtable-style data model with storage backed by a Cloud Storage Bucket.
- OpenCog AtomSpace - In-RAM knowledge representation (KR) database, an associated query engine and graph-re-writing system, and a rule-driven inferencing engine that can apply and manipulate sequences of rules to perform reasoning. (Web)
- Sybil - Append only analytics datastore with no up front table schema requirements. Just log JSON records to a table and run queries.
- Eva - Distributed database-system implementing an entity-attribute-value data-model that is time-aware, accumulative, and atomically consistent.
- Firestore - Develop rich applications using a fully managed, scalable, and serverless document database. (Intro) (Running Google Firestore locally)
- Graphik - Identity-aware, permissioned, persistant document/graph database & pubsub server written in Go.
- DatenLord - Computing Defined Storage, an application-orientated, cloud-native distributed storage system.
- Redwood - Highly-configurable, distributed, realtime database that manages a state tree shared among many peers.
- Drivine - Best and fastest graph database client (Neo4j & AgensGraph) for Node.js & TypeScript. (Code) (Starter Template)
- Escanor - High performance key value database with useful json document indexing and manipulations.
- Condensation - General-purpose distributed data system with conflict-free synchronization, and inherent end-to-end security. (GitHub)
- ZenoDB - Go-based embeddable time series database optimized for performing aggregated analytical SQL queries on dimensional data.
- SteveCare - Peer-to-peer database system that enables people to build complex databases between peers, without any intermediary platform.
- Resql - SQL database server that uses SQLite as its SQL engine and it provides replication and automatic failover capabilities.
- Kvrocks - Distributed key value NoSQL database based on RocksDB and compatible with Redis protocol.
- Greenplum Database - Advanced, fully featured, open source data warehouse, based on PostgreSQL. (Web)
- JinDB - Small relational database engine written in Rust with the standard library and no external dependencies.
- LemonGraph - Log-based transactional graph (nodes/edges/properties) database engine that is backed by a single file.
- Vertica - Big Data Analytics On-Premises, in the Cloud, or on Hadoop. (Getting Started with Vertica)
- Datomic - Transactional database with a flexible data model, elastic scaling, and rich queries. (GitHub) (Replicating with Datomic)
- EdgelessDB - Open-source MySQL-compatible database for confidential computing. Runs entirely inside a secure enclave and comes with advanced features for collaboration, recovery, and access control. (Intro)
- Zerostash - Deduplicated, encrypted data store that provides native versioning capabilities, and was designed to secure all metadata related to the files.
- Couchbase Lite for iOS and MacOS - Lightweight, embedded, syncable NoSQL database engine for iOS and MacOS apps.
- Ambry - Distributed object store that supports storage of trillion of small immutable objects (50K -100K) as well as billions of large objects.
Tools
- litecli - Command-line client for SQLite databases that has auto-completion and syntax highlighting.
- goose - Database migration tool. Manage your database schema by creating incremental SQL changes or Go functions.
- erd - Translates a plain text description of a relational database schema to a graphical entity-relationship diagram.
- replikativ - Open, scalable and distributive infrastructure for a data-driven community of applications. (Web) (Unified storage IO)
- Maxwell's Daemon - Application that reads MySQL binlogs and writes row updates as JSON to Kafka, Kinesis, or other streaming platforms. (Code)
Notes
Links
- Algebraic Query Language - Schemas as categories, DB instances as functors, provable data migration as functor composition.
- Storage Performance Development Kit - Provides a set of tools and libraries for writing high performance, scalable, user-mode storage applications.
- Readings in Databases - List of papers essential to understanding databases and building new data systems.
- Designing Data Intensive Applications Book - Deep dives into different types of data storage solutions, their history, and how they actually work. (Review) (Notes) (Literature References) (Review)
- Scalable SQL and NoSQL Data Stores - Good paper that helps differentiate similar but different datastores. Really helpful when you're trying to pick a modern data solution.
- Apache Arrow - Development platform for in-memory analytics. It contains a set of technologies that enable big data systems to process and move data fast. (Web) (HN)
- Blazer - Explore your data with SQL. Easily create charts and dashboards, and share them with your team.
- DB Weekly - Weekly round-up of database technology news and articles covering new developments, SQL, NoSQL, document databases, graph databases, and more.