The original intention has been modern web-scale databases.
The movement began early 2009 and
is growing rapidly. Often more characteristics apply such as: schema-free,
easy replication support, simple API, eventually consistent / BASE (not ACID), a
huge amount of data and more. So the misleading term "nosql" (the community now
translates it mostly with "not only sql") should be seen as an alias to something like the
[based on 7 sources, 15 constructive feedback emails (thanks!) and 1 disliking comment
. Agree / Disagree? Tell me so! By the
way: this is a strong definition and it is out there here since 2009!]
- NoSQL DEFINITION:
Next Generation Databases mostly addressing some of the points: being non-relational,
distributed, open-source and horizontally scalable.
List Of NoSQL Databases [currently >225]
Core NoSQL Systems: [Mostly originated out of a Web 2.0 need]
Wide Column Store / Column Families
API: Java / any writer, Protocol: any write call, Query Method: MapReduce
Java / any exec, Replication: HDFS Replication, Written in: Java,
Concurrency: ?, Misc:
Links: 3 Books [1,
Hadoop Distribution and professional services .
massively scalable, partitioned row store, masterless architecture, linear scale performance, no single points of failure, read/write support across multiple data centers & cloud availability zones. API / Query Method: CQL and Thrift, replication: peer-to-peer, written in: Java, Concurrency: tunable consistency, Misc: built-in data compression, MapReduce support, primary/secondary indexes, security features. Links: Documentation, PlanetC*, Company.
API: Thrift (Java, PHP, Perl, Python, Ruby, etc.), Protocol: Thrift, Query
Method: HQL, native Thrift API, Replication: HDFS Replication,
Concurrency: MVCC, Consistency Model: Fully consistent Misc: High performance
C++ implementation of Google's Bigtable. » Commercial support
Accumulo is based on BigTable
and is built on top of Hadoop,
It features improvements on the BigTable design in the form of cell-based access control,
improved compression, and a server-side programming mechanism that can modify key/value pairs
at various points in the data management process.
Misc: not open source / part of AWS, Book (will be
outperformed by DynamoDB!)
Google's Big table clone like HBase. » Article
Column Store pioneer since 2002.
from LexisNexis, info, article
(formerly known as Stratosphere)
massively parallel & flexible data analytics platform,
API: Java, Scala,
Query Method: expressive data flows (extended M/R, rich UDFs, iteration support),
Data Store: independent (e.g., HDFS, S3, MongoDB),
Written in: Java,
License: Apache License V2.0,
Misc: good integration with Hadoop stack (HDFS, YARN), source code on Github
horizontally and vertically scalable, relational, partitioned row store, document store API / Query Method: SQL (native, DRDA, JDBC, ODBC), MongoDB wire listener, mixed mode,
replication: master / slave, peer-to-peer, sharding, grid operations,
written in: C,
Concurrency: row, page, table, db locking,
Misc: ACID, built-in data compression, scheduler, automatic cyclic storage management, extensible, in memory acceleration, native ports from ARM v6 up Links: Documentation, IIUG, Company.
Splice Machine is an RDBMS built on Hadoop, HBase and Derby. Scale real-time applications using commodity hardware without application rewrites,
Features: ACID transactions, ANSI SQL support, ODBC/JDBC, distributed computing
massively scalable in-memory and persistent storage DBMS for analytics on market data
(and other time series data).
APIs: C/C++, Java Native Interface (JNI), C#/.NET), Python, SQL (native, ODBC, JDBC),
Data layout: row, columnar, hybrid,
Written in: C, Replication: master/slave, cluster, sharding,
Concurrency: Optimistic (MVCC) and pessimistic (locking)
a distributed self-tuning database with
automatic indexing, version control and ACID transactions.
Written In: Java. API/Protocol: Thrift (many languages). Concurrency: serializable
transactions with just-in-time locking.
Misc: uses a buffered storage system to commit all data to disk
immediately while perform rich indexing in the background.
open-source analytics data store for business intelligence (OLAP) queries on event data.
Low latency (real-time) data ingestion, flexible data exploration, fast data aggregation.
Scaled to trillions of events & petabytes.
Most commonly used to power user-facing analytic applications.
Written in: Java
License: Apache 2.0
Apache Kudu (incubating) completes Hadoop's storage layer to enable fast analytics on fast data.
[OpenNeptune, Qbase, KDI]
API: REST and many languages, Protocol: REST, Query Method: via JSON, Replication + Sharding: automatic and configurable, written in: Java,
Misc: schema mapping, multi tenancy with arbitrary indexes, » Company and Support, » Article
(Doc Store & GraphDB & Key-Value. More details in Category Multimodel Databases)
API: BSON, Protocol: C, Query Method: dynamic object-based language &
MapReduce, Replication: Master Slave &
Auto-Sharding, Written in: C++,Concurrency: Update in Place. Misc:
Indexing, GridFS, Freeware + Commercial License Links:
» Talk, » Notes,
API: Memcached API+protocol (binary and ASCII) , most languages, Protocol:
Memcached REST interface for cluster conf + management,
Written in: C/C++ + Erlang (clustering), Replication: Peer to Peer, fully
consistent, Misc: Transparent topology changes during operation, provides memcached-compatible
caching buckets, commercially supported version available,
Links: » Wiki,
Written in: Erlang, Concurrency: MVCC, Misc:
» 3 CouchDB books , » Couch Lounge
(partitioning / clusering), » Dr. Dobbs
API: protobuf-based, Query Method: unified chainable query language
(incl. JOINs, sub-queries, MapReduce, GroupedMapReduce); Replication:
Sync and Async Master Slave with per-table acknowledgements, Sharding:
guided range-based, Written in: C++,
Concurrency: MVCC. Misc: log-structured storage engine with concurrent incremental garbage compactor
Query Method: dynamic object-based language,
Replication: Master Slave & Auto-Sharding,
Written in: C++,
Misc: Indexing, Large Object Store, Transaction, Free + Commercial License,
.Net solution. Provides HTTP/JSON
queries & Sharding
supported. » Misc
API: JSON, XML, Java
Protocols: HTTP, REST
Query Method: Full Text Search and Structured Query, XPath, XQuery, Range, Geospatial,
Written in: C++
Concurrency: Shared-nothing cluster, MVCC
Misc: Petabyte-scalable and elastic (on premise in the cloud), ACID + XA transactions,
auto-sharding, failover, master slave replication (clusters), replication (within cluster), high
availablity, disaster recovery, full and incremental backups,
government grade security at the doc level, developer community »
(freeware+commercial) API: XML, PHP, Java, .NET Protocols: HTTP, REST, native
TCP/IP Query Method: full text search, XML, range and Xpath queries; Written in
C++ Concurrency: ACID-compliant, transactional, multi-master cluster Misc:
Petabyte-scalable document store and full text search engine. Information ranking. Replication. Cloudable.
It queries the collections with a gremlin-like DSL that uses MongoDB's API methods, but also provides joining.
The collections extend the native array objects, which gives the overall ODM a good performance. Queries 500.000 elements in less then a second.
API: Java & http,
Querying: Range queries, Predicates, Replication: Partitioned with consistent
hashing, Consistency: Per-record
strict consistency, Misc: Based on Terracotta
Architected to unify the best of search engine, NoSQL and NewSQL DB technologies. API: REST and many languages. Query method: SQL. Written in C++. Concurrency: MVCC. Misc: ACID transactions, data distribution via consistent hashing, static and dynamic schema support, in-memory processing. Freeware + Commercial License
Lightweight open source document database written in Java for high performance, runs in-memory, supports Android.
API: JSON, Java
Query Method: REST OData Style Query language, Java fluent Query API
Concurrency: Atomic document writes
Indexes: eventually consistent indexes
JSON based, Document store database with compiled .net map functions and automatic hybrid
bitmap indexing and LINQ query filters
djonDB API: BSON, Protocol: C++,
Query Method: dynamic queries and map/reduce, Drivers:
Java, C++, PHP Misc: ACID compliant, Full shell console over google v8 engine, djondb
requirements are submited by users, not market. License: GPL and commercial
Embedded JSON database engine based on tokyocabinet.
API: C/C++, C# (.Net, Mono), Lua, Ruby, Python, Node.js binding,
Written in: C,
Query language: mongodb-like dynamic queries,
Concurrency: RW locking, transactional ,
Misc: Indexing, collection level rw locking, collection level transactions, collection joins.,
DensoDB is a new NoSQL document database. Written for .Net environment in c# language.
It’s simple, fast and reliable. Source
A Document Store on top of SQL-Server.
For small online databases, PHP / JSON interface, implemented in PHP.
(please help provide more facts!) Uses Apache Thrift
to integrate multiple backend databases as BerkeleyDB, Disk, MySQL, S3.
Transactional embedded database, it can embed into mobile, desktop and web applications,
supports on-disk and in-memory storages. API: Java,C# (Android, Mono, Xamarin, Unity3D).
Query Method: SQL-like and KeyValue. Written In: Java, C#. Replication: MasterSlave, MasterMaster.
Written in: Java. Replication: Master/Slave.
License: AGLP. Historical queries. ACID. Schemaless.
Concurrency: STM and persistent data structure.
Append-only storage. Encrypted storage. Flexible durability control.
Secondary & composite indexes. Transparently serializes Java/.NET objects.
Cloud based, open-source, zero-config. Based on CouchDB and BigCouch.
Key Value / Tuple Store
Automatic ultra scalable NoSQL DB based on fast SSDs. Multiple Availability Zones. Elastic MapReduce Integration. Backup to S3 and much more...
Collections of free form entities (row key, partition key, timestamp). Blob and Queue Storage available, 3 times
redundant. Accessible via REST or ATOM.
API: tons of languaes, JSON,
Query Method: MapReduce term matching , Scaling: Multiple Masters; Written in:
Concurrency: eventually consistent (stronger then MVCC via Vector Clocks)
Tons of languages,
Written in: C,
Concurrency: in memory
and saves asynchronous disk after a defined time. Append only mode
available. Different kinds of fsync policies. Replication: Master / Slave,
Misc: also lists, sets, sorted sets, hashes, queues. Cheat-Sheet:
», great slides
» Admin UI
» From the Ground up
Fast and web-scale database. RAM or SSD. Predictable performance; achieves 2.5 M TPS (reads and
writes), 99% under 1 ms. Tunable consistency. Replicated, zero configuration, zero downtime,
auto-clustering, rolling upgrades, Cross Datacenter Replication (XDR). Written in: C. APIs: C, C#,
Erlang, Go, Java, Libevent, Node, Perl, PHP, Python, Ruby. Links:
2.5M TPS on 1 node at Intel,
1M TPS on AmazonWS,
1M TPS w/ SSDs on 1 Server,
Combating Memory Fragmentation
& Batch updates. DB from Google.
Written in C++.
hot Benchmark », Article »
(in German). Java access.
API: C++. Written in C++.
Facebook`s improvements to Google`s LevelDB to speed throughput
for datasets larger than RAM. Embedded solution.
API: Many languages, Written in: C, Replication: Master / Slave,
Berkeley DB Java
Edition: API: Java, Written in: Java,
Replication: Master / Slave,
Concurrency: serializable transaction isolation, License: Sleepycat
Oracle NoSQL Database is a distributed key-value database. It is designed to provide highly reliable, scalable and available data storage across a configurable set of systems that function as storage nodes.
NoSQL and the Enterprise Data is stored as key-value pairs, which are written to particular storage node(s), based on the hashed value of the primary key. Storage nodes are replicated to ensure high availability, rapid failover in the event of a node failure and optimal load balancing of queries. API: Java/C.
Immediate consistency sharded KV store with an eventually consistent AP
store bringing eventual consistency issues down to the
theoretical minimum. It features efficient record coalescing.
GenieDB speaks SQL and co-exists / do intertable
Protocol: Native, HTTP,
Flavor: Embedded, Network, Elastic Cache, Replication: P2P based Network Overlay,
Written in: C++, Concurrency:
Misc: robust, crash proof, Elastic, throw machines to scale linearly, Btree/Ehash
API: Java & simple RPC to vals, Protocol: internal, Query Method: M/R
inside value objects, Scaling: every node is master for its slice of namespace, Written
in: Java, Concurrency: serializable transaction isolation,
(please help provide more facts!) Written in: Erlang, Replication: Strong consistency over
replicas, Concurrency: non blocking Paxos.
Links: nice talk
Kyoto Cabinet »
API / Protocol: http
(text, html, JSON), C, C++, Python, Java, Ruby, PHP,Perl. Concurrency:
Open-Source implementation of Amazons Dynamo Key-Value Store.
implementation of Amazons Dynamo Key-Value Store. written in Erlang.
With "data partitioning, versioning, and read repair, and user-provided
storage engines provide persistence and query processing".
Open Source Amazon Dnamo implementation, Misc: slides
API: Memcache protocol
(get, set, add, replace, etc.), Written in: C, Data
Misc: Is Memcached writing to BerkleyDB.
API: C, C++, C#, Java, PHP, Perl, Written in: C,C++. Misc:
Transaction logging. Client/server. Embedded. SQL wrapper (not core). Been
around since 1979.
Key-Value database that was written as part of SQLite4, They claim it is faster then LevelDB. Instead of supporting custom comparators, they have a recommended data encoding for keys that allows various data types to be sorted.
(embedded solution) API: C, C++, .NET, Java, Erlang.
Written in C,C++. Fast key/value store with a
parameterized B+-tree. Keys are "typed" (i.e. 32bit integers, floats,
variable length or fixed length binary data). Has built-in analytical
functions like SUM, AVERAGE etc.
API: C#, Written in C#, embedded solution, generic XTable<TKey,TRecord>
ACID transactions, snapshots, table versions, shared records, vertical data compression, custom compression,
composite & custom primary
keys, available backend file system layer, works over multiple volumes, petabyte scalability, LINQ.
API: C, Perl, PHP, Python, Java and Ruby. Written in: Objective
,Protocol: asynchronous binary, memcached, text (Lua console). Data model:
collections of dimensionless tuples, indexed using primary + secondary keys.
Concurrency: lock-free in memory, consistent with disk (write ahead log).
Replication: master/slave, configurable.
Other: call Lua stored procedures.
In-memory (opt. persistence via mmap), highly concurrent, low-latency key-value store.
Written in: Java.
Protocol: in-process Java, remote via
Chronicle Engine + Wire:
binary, text, Java, C# bindnings.
Concurrency: in-memory lock striping, read-write locks.
Replication: multi-master, eventually consistent.
API: C, Query Method: MQL, native API, Replication:
DFS Replication, Consistency: strict consistency Written
very high-performance associative database. Highly scalable. API: C, C++, Java, Python and (limited) RESTful Protocol: binary Query method: key-value, iteration, Replication: Distributed, Written in: C++ 11/Assembly, Concurrency: ACID, Misc: built-in data compression, native support for FreeBSD, Linux and Windows. License: Commercial.
For geolocalized apps. Concurrency: in-memory with asynchronous disk writes.
API: HTTP/JSON. Written in: C.
A pure key value store with optimized b+tree and murmur
hashing. (In the near future it will be a JSON document database much like
mongodb and couchdb.)
peer-to-peer distributed in-memory (with persistence) datagrid that
implements and expands on the concept of the Tuple Space. Has SQL
Queries and ACID (=> NewSQL).
Key-Value concept. Variable number of keys per record. Multiple key
values, Hierarchic records. Relationships. Diff. record types in same
DB. Indexing: B*-Tree. All aspects configurable. Full scripting
language. Multi-user ACID. Web interfaces (PHP, Perl, ActionScript)
plus Windows client.
A fast key-value Database (using LSM-Tree storage engine), API: Redis protocol
(SET,MSET,GET,MGET,DEL etc.), Written in: ANSI C
Distributed searchable key-value store. Fast (latency &
throughput), scalable, consistent, fault tolerance, using hyperscpace
hashing. APIs for C, C++ and Python.
Fast, open source, shared memory (using memory mapped files e.g. in /dev/shm or on SSD), multi process, hash table, e.g. on an 8 core i7-3720QM CPU @ 2.60GHz using /dev/shm, 8 processes combined have a 12.2 million / 2.5 to 5.9 million TPS read/write using small binary keys to a hash filecontaining 50 million keys. Uses sharding internally to mitigate lock contention. Written in C.
Ultra-fast, ultra-compact key-value embedded data store developed by Symas for the OpenLDAP Project.
It uses memory-mapped files, so it has the read performance of a pure in-memory database while still offering the persistence of standard disk-based databases, and is only limited to the size of the virtual address space, (it is not limited to the size of physical RAM)
Sophia is a modern embeddable key-value database designed for a high load environment. It has unique architecture that was created as a result of research and rethinking of primary algorithmical constraints, associated with a getting popular Log-file based data structures, such as LSM-tree. Implemented as a small C-written, BSD-licensed library.
.NET Open Source Distributed Cache. Written in C#. API .NET & Java. Query Parallel SQL Query, LINQ & Tags. Misc: Linear Scalability, High Availability, WAN Replication, GUI based Administration & Monitoring Tools, Messaging, Runtime Data Sharing, Cache & Item Level Events, Continuous Query & Custom Events, DB Dependencies & Expirations
Open Source In-Memory JCache compliant Data Grid. Written in Java. API Java, JCache JSR 107 & .NET. Query SQL & DB Synchronization. Misc: Linear Scalability, High Availability, WAN Replication, GUI based Administration & Monitoring Tools, Distributed Messaging, MapReduce, Entry Processor and Aggregator
Redis inspired K/V store for Python object serialization.
(based on Tokyo Tyrant)
Hibari is a highly available, strongly consistent, durable, distributed
key-value data store
Key-value store, B+tree. Lightning fast reads+fast bulk loads.
Memory-mapped files for persistent storage with all the speed of an in-memory
database. No tuning conf required. Full ACID support.
MVCC, readers run lockless. Tiny code, written in C, compiles to under 32KB of
x86-64 object code. Modeled after the BerkeleyDB API for easy migration from
Berkeley-based code. Benchmarks against LevelDB, Kyoto Cabinet, SQLite3, and
BerkeleyDB are available, plus full paper and presentation slides.
High availability, concurrency-oriented event-based K/V database with transactions and
causal consistency. Protocol: MsgPack, API: Erlang, Elixir, Node.js.
Written in: Elixir, Github-Repo.
BinaryRage is designed to be a lightweight ultra fast key/value store for .NET with no dependencies. Tested with more than 200,000 complex objects written to disk per second on a crappy laptop :-) No configuration, no strange driver/connector, no server, no setup - simply reference the dll and start using it in less than a minute.
Github Page »
Professional, open-source, NoSql (embedded Key/Value storage), transactional, ACID-compliant, multi-threaded, object database management system for .NET 3.0> MONO. Written in C#.
API: Scala. Written in Scala.
Replication: Replicas vote on writes and reads.
Sharding: Hashes keys onto array of replica cohorts.
Concurrency: Optimistic + Multiversion Concurrency Control.
Provides multirow atomic writes. Exposes optimistic concurrency through API
to support HTTP Etags.
Key/Value DB written in Go.
Serenity database implements basic Redis commands and extends them with support of Consistent
Cursors, ACID transactions, Stored procedures, etc. The database is designed to store data bigger
then available RAM.
[Scality », KaTree » TomP2P »,
Kumofs » , TreapDB
Wallet » ,
NMDB, luxio, actord, keyspace, flare, schema-free,
API: lots of langs,
Protocol: Java embedded / REST, Query Method: SparQL, nativeJavaAPI, JRuby,
Replication: typical MySQL style
Written in: Java, Concurrency: non-block reads, writes locks involved
nodes/relationships until commit,
possible, Links: Video », Blog »
(Doc Store & GraphDB & Key-Value. More details in Category Multimodel Databases)
(by Objectivity) API: Java,
Protocol: Direct Language Binding, Query Method: Graph Navigation API,
Predicate Language Qualification, Written in: Java (Core C++),
Data Model: Labeled Directed Multi Graph, Concurrency: Update locking on subgraphs,
concurrent non-blocking ingest, Misc: Free for Qualified Startups.
API: Java, .NET, C++, Python, Objective-C, Blueprints Interface
Query Method: as above + Gremlin (via Blueprints),
Written in: C++,
Data Model: Labeled Directed Attributed Multigraph,
Misc: ACID possible, Free community edition up to 1 Mio objects,
API: Java, Blueprints, Gremlin, Python, Clojure Protocol: Thrift, RexPro(Binary), Rexster (HTTP/REST) Query Method: Gremlin, SPARQL Written In: Java Data Model: labeled Property Graph, directed, multi-graph adjacency list Concurrency: ACID Tunable C Replication: Multi-Master License: Apache 2 Pluggable backends: Cassandra, HBase, MapR M7 Tables, BDB, Persistit, Hazelcast Links: Titan User Group
API: Java, http/REST, Protocol: as API + XPRISO, OpenID, RSS, Atom, JSON, Java
embedded, Query Method: Web user interface with html, RSS, Atom, JSON output, Java
Written in: Java,
Concurrency: concurrent reads, write lock within one MeshBase, Misc: Presentation »
(and Java Langs), Written in:Java,
Query Method: Java
or P2P, Replication: P2P,
Misc: Open-Source, Especially for AI and Semantic Web.
Sub-graph-based API, query language, tools & transactions.
Embedded Java, remote-proxy Java or REST.
Distributed storage & processing. Read/write all Nodes.
Permissions & Constraints frameworks. Object storage, vertex-embedded agents.
Supports multiple graph models. Written in Java
API: C#, Protocol: C# Language Binding, Query Method:
Graph Navigation API, Replication: P2P with Master Node,
Written in: C#, Concurrency: Yes (Transactional update in online query mode,
in Batch Mode) Misc: distributed in-memory storage, parallel graph computation
platform (Microsoft Research Project)
API: Java, Python, Ruby, C#, Perl, Clojure, Lisp Protocol:
Query Method: SPARQL and Prolog, Libraries:
Social Networking Analytics & GeoSpatial,
Written in: Common Lisp, Links: Learning Center
A native, .NET, semantic web database with code first Entity Framework, LINQ
and OData support. API: C#, Protocol: SPARQL
HTTP, C#, Query Method: LINQ, SPARQL, Written in: C#
API: Java, Jini service discovery, Concurrency: very high
Written in: Java, Misc: GPL + commercial, Data: RDF data with inference, dynamic
key-range sharding of indices,
Misc: Blog » (parallel database, high-availability architecture,
immortal database with historical views)
RDF enterprise database management system. It is cross-platform and can be used with most programming languages.
Main features: high performance, guarantee database transactions with ACID, secure with ACL's, SPARQL &
SPARUL, ODBC & JDBC drivers, RDF & RDFS.
WhiteDB is a fast lightweight graph/N-tuples shared memory database library written in C with focus on speed, portability and ease of use. Both for Linux and Windows, dual licenced with GPLv3 and a free nonrestrictive royalty-free commercial licence.
Hybrid DBMS covering the following models: Relational, Document, Graph
by twitter »
scalable, fast, consistent
[Java Universal Network / Graph Framework, OpenRDF / Sesame, Filament, OWLim, NetworkX, iGraph, Jena]
List of SPARQL implementations can be found here
REST, Graph Blueprints, C#, D, Ruby, Python, Java, PHP, Go, Python, etc.
Documents, Graphs and Key/Values.
HTTP using JSON.
declarative query language (AQL), query by example.
API: REST, Binary Protocol, Java, Node.js, Tinkerpop Blueprints, Python, PHP, Go, Elixir, etc.,
Schema: Has features of an Object-Database, DocumentDB, GraphDB and Key-Value DB,
Written in: Java,
Query Method: SQL, Gremlin, SparQL,
Concurrency: MVCC, tuneable,
Indexing: Primary, Secondary, Composite indexes with support for Full-Text and Spatial,
Replication: Master-Master + sharding,
Misc: Really fast, Lightweight, ACID with recovery.
API: Many jvm languages, Protocol: Native + REST, Query Method: Datalog + custom extensions, Scaling: elastic via underlying DB (in-mem, DynamoDB, Riak, CouchBase, Infinispan, more to come), Written in: Clojure, Concurrency: ACID MISC: smart caching, unlimited read scalability, full-text search, cardinality, bi-directional refs 4 graph traversal, loves Clojure + Storm.
Bought by Apple Inc. Not available any more.
GraphDB + RDBMS + KV Store + Document Store. Alchemy Database is a
low-latency high-TPS NewSQL RDBMS embedded in the NOSQL datastore
redis. Extensive datastore-side-scripting is provided via deeply
embedded Lua. Bought and integrated with Aerospike.
CortexDB is a dynamic schema-less multi-model data base providing nearly all advantages of up to now known NoSQL data base types (key-value store, document store, graph DB, multi-value DB, column DB) with dynamic re-organization during continuous operations, managing analytical and transaction data for agile software configuration,change requests on the fly, self service and low footprint.
API:JDBC,SQL; WonderDB is fully transactional, distributed NewSQL database implemented in java based on relational architectures. So you can get best of both worlds, sql, joins and ease of use from SQL and distribution, replication and sharding from NoSQL movement. Tested performance is over 60K per node with Amazon m3.xlarge VM.
The following section containts Soft NoSQL Systems
[Mostly NOT originated out of a Web 2.0 need but worth a look for great non relational solutions]
Object Databases »
Java, C#, C++, Python. Schema: language class model (easy changable). Modes: always
consistent and eventually consistent
Replication: synchronous fault tolerant and peer to peer asynchronous. Concurrency: optimistic
and object based locks. Scaling: can add physical nodes on fly for scale out/in
and migrate objects between nodes without impact to application code. Misc: MapReduce
via parallel SQL like query across logical database groupings.
API: Java, C#, .Net Langs, Protocol: language, Query Method: QBE (by
Example), Soda, Native Queries, LINQ (.NET),
Replication: db4o2db4o & dRS to relationals, Written in: Java,
Cuncurrency: ACID serialized, Misc: embedded lib, Links:
DZone Refcard #53 »,
Languages: Java, C#, C++, Python, Smalltalk, SQL access through ODBC. Schema: native
language class model, direct support for references, interoperable across all language bindings. 64 bit unique
object ID (OID) supports multi exa-byte. Platforms: 32 and 64 bit Windows, Linux, Mac OSX,
*Unix. Modes: always consistent (ACID). Concurrency: locks at cluster of
objects (container) level. Scaling: unique distributed architecture, dynamic addition/removal
of clients & servers, cloud environment ready. Replication: synchronous with quorum fault
tolerant across peer to peer partitions.
API: Java, C, C++, Smalltalk
Schema: language class model
Platforms: Linux, AIX, Solaris, Mac OSX, Windows clients
Modes: always consistent (ACID)
Replication: shared page cache per node, hot standby failover
Concurrency: optimistic and object based locks
Scaling: arbitrarily large number of nodes
Misc: SQL via GemConnect
API: C# (.NET languages),
Schema: Native language class model, Query method: SQL, Concurrency: Fully
ACID compliant, Storage: In-memory with transactions secured on disk, Reliability:
Full checkpoint recovery, Misc: VMDBMS - Integrating the DBMS with the virtual machine
for maximal performance and ease of use.
API: Java,Java ME,C#,Mono. Query method: OO via Perst collections, QBE, Native Queries,
LINQ, native full-text search, JSQL Replication: Async+sync (master-slave) Written in:
Java, C#. Caching: Object cache (LRU, weak, strong), page pool, in-memory database Concurrency: Pessimistic+optimistic
(MVCC) + async or sync (ACID) Index types: Many tree models + Time
Series. Misc: Embedded lib., encryption, automatic recovery, native full text search, on-line or off-line backup.
Written in100% pure C#, Concurrency: ACID/transactional, pessimistic/optimistic locking, Misc: compact data, B-tree indexes,
LINQ queries, 64bit object identifiers (Oid) supporting multi millions of
databases and high performance. Deploy with a single DLL of around 400KB.
Written in: 100% C#,
The HSS DB v3.0 (HighSpeed-Solutions Database), is a client based, zero-configuration, auto schema evolution, acid/transactional, LINQ Query, DBMS for Microsoft .NET 4/4.5, Windows 8 (Windows Runtime), Windows Phone 7.5/8, Silverlight 5, MonoTouch for iPhone and Mono for Android
Method: Direct object access, zope.catalog, gocept.objectquery,
Replication: ZEO, ZEORAID, RelStorage
Written in: Python, C Concurrency: MVCC, License:
Zope Public License (OSI approved)
Misc: Used in production since 1998
Smalltalk DB, optimistic locking, Transactions, etc.
API: Python - ZODB "Storage" interface, Protocol: native, Query Method:
Written in: Python,
(internally), License: GPL "v2 or later", Misc: Load balancing, fault tolerant, hot-extensible.
An object database engine that currently runs on .NET, Mono, Silverlight,Windows
Phone 7, MonoTouch, MonoAndroid, CompactFramework; It has
implemented a Sync Framework Provider and can be synchronized with MS
is a lightweight object-oriented database for .NET with support for
Silverlight and Windows Phone 7. It features in-memory keys and
indexes, triggers, and support for compressing and encrypting the
Stores .NET classes in a datapool. Build for speed. SQL Server
integration. LINQ support.
EyeDB is an LGPL OODBMS,
provides an advanced object model (inheritance, collections, arrays,
methods, triggers, constraints, reflexivity), an object definition
language based on ODMG ODL, an object query and manipulation language
based on ODMG OQL. Programming interfaces for C++ and Java.
Object-Oriented Database designed to support the maintenance and
sharing of knowledge bases. Optimized for pointer-intensive data
structures used by semantic networks, frame systems, and many
intelligent agent applications. Written in: ANSI C.
Ninja Database Pro is a .NET ACID compliant relational object database that
supports transactions, indexes, encryption, and compression. It currently
runs on .NET Desktop Applications, Silverlight Applications, and Windows Phone 7 Applications.
API: C#, .Net, Mono, Windows Phone 7, Silverlight,
Protocol: language, Query Method: Soda, LINQ (.NET),
Written in: C#, Misc: embedded lib, indexes, triggers, handle circular ref,
LinqPad support, Northwind sample, refactoring, in-memory database,
Transactions Support (ACID) and more, Documentation: »
Language and Object Database, can be viewed as a Database Development
Framework. Schema: native language class model with relations + various indexes.
Queries: language build in + a small Prolog like DSL Pilog. Concurrency: synchronization
Replication, distribution and fault tolerance is not implemented per default but can be implemented with native
functionality. Written in C (32bit) or assembly (64bit).
Query Method: Functional programming,
Written in: Haskell,
Concurrency: ACID, GHC concurrent runtime,
Misc: In-memory with disk-based log, supports remote access
Links: Wiki »,
API: Java (JPA / JDO)
Query method: JPA JPQL, JDO JDOQL
Written in: 100% Pure Java
Caching: Object cache, Data cache, Page cache, Query Result cache, Query program cache
Concurrency: Object level locking (pessimistic + optimistic)
Index types: BTree, single, path, collection
Misc: Used in production since 2004, Embedded mode, Client Server mode, automatic recovery, on-line backup.
CoreObject: Version-controlled OODB, that supports powerful undo, semantic merging, and real-time collaborative editing.
MIT-licensed, API: ObjC, Schema: EMOF-like, Concurrency: ACID, Replication: differential sync, Misc: DVCS based on object graph diffs, selective undo, refs accross versioned docs, tagging, temporal indexing, integrity checking.
KiokuDB » (Perl solution),
Grid & Cloud Database Solutions
Oracle Coherence offers distributed, replicated, multi-datacenter, tiered (off-heap/SSD) and near (client) caching. It provides distributed processing, querying, eventing, and map/reduce, session management, and prorogation of database updates to caches. Operational support provided by a Grid Archive deployment model.
Popular SpaceBased Grid Solution.
GemFire offers in-memory globally distributed data management with dynamic scalability, very high performance and granular control supporting the most demanding applications. Well integrated with the Spring Framework, developers can quickly and easily provide sophisticated data management for applications. With simple horizontal scale-out, data latency caused by network roundtrips and disk I/O can be avoided even as applications grow.
scalable, highly available data grid platform, open source,
written in Java.
NOSQL Data Integration Environment, can integrate relational, object, BigData – NOSQL easily and without
Hazelcast is a in-memory data grid that offers distributed data in Java with dynamic scalability under the Apache 2 open source license. It provides distributed data structures in Java in a single Jar file including hashmaps, queues, locks, topics and an execution service that allows you to simply program these data structures as pure java objects, while benefitting from symmetric multiprocessing and cross-cluster shared elastic memory of very high ingest data streams and very high transactional loads.
nothing, document-oriented cluster data store. Accessed via SQL
and has builtin BLOB
support. Uses the cluster state implementation and node
discovery of Elasticsearch.
License: Apache 2.0,
Query Method: SQL,
Clients: HTTP (REST), Python, Java (JDBC or native),
Ruby, JS, Erlang,
Replication + Sharding: automatic and configurable,
written in: Java,
» Crate Data GitHub Project,
(commercial system) API: Java, XQuery,
Protocol: WebDAV, web
services, Query method: XQuery, XPath, XPointer,
primary copy replication (master/replicas), Written in: Java,
Concurrency: concurrent reads, writes with
lock; transaction isolation,
Misc: Fully transactional persistent DOM; versioning; multiple index
types; metadata and non-XML data support; unlimited horizontal scaling.
Developer Network »
API: XQuery, XML:DB API,
DOM, SAX, Protocols: HTTP/REST,
WebDAV, SOAP, XML-RPC, Atom, Query Method: XQuery, Written in: Java
reads, lock on write;
Misc: Entire web applications can be written in XQuery, using XSLT,
new full text search index based on Apache Lucene, a lightweight URL
rewriting and MVC framework, and support for XProc.
Misc: ACID transactions, security, indices, hot backup. Flexible XML
processing facilities include W3C XQuery implementation, tight
integration of XQuery with full-text search facilities and a node-level
BaseX is a fast, powerful, lightweight XML database system
XPath/XQuery processor with highly conformant support for the latest
W3C Update and Full Text Recommendations. Client/Server architecture,
ACID transaction support, user management, logging, Open Source,
BSD-license, written in Java, runs out of the box.
commercial and open source version, API: Java, Protocols:
HTTP, REST, Query
Method: XQuery, XQuery
Full-Text, XQuery Update, Written in: Java, full source can be
reads & writes, isolation, Misc: Terabyte scalable, emphasizes
API: Many languages,
Written in: C++,
Query Method: XQuery,
Replication: Master /
Slave, Concurrency: MVCC,
by Intersystems, multidimensional
API, array based APIs (Java / .NET), and a Java based document API.
System. Multidimensional array APIs, Object APIs, Relational Support
(Fully SQL capable JDBC, ODBC, etc.) and Document APIs are new
the upcoming 2012.2.x versions. Availible for
Windows, Linux and OpenVMS.
API: M, C, Python, Perl,
Protocol: native, inprocess C, Misc: Wrappers: M/DB for SimpleDB
for XML »,
for mapping to tables for SQL »,
Features: Small footprint (17MB), Terabyte Scalability, Unicode
support, Database encryption, Secure, ACID transactions (single node),
eventual consistency (replication), License: AGPL v3 on x86 GNU/Linux, Links: Slides
Array Data Model for Scientists,
» paper, » poster,
Multidimensional arrays, API: M, C, Pascal, Perl,
.NET, ActiveX, Java, WEB.
Available for Windows and Linux.
Short description: Rasdaman is a scientific database that allows to store and retrieve
multi-dimensional raster data (arrays) of unlimited size through an SQL-style query language.
API: C++/Java, Written in C++, Query method: SQL-like query language rasql,
as well as via OGC standards WCPS, WCS, WPS link2
is a new Realtime analytics database written in .NET C#.
ACID compliant. fluent .NET query API, Client / server or in-process.
In-memory and persistent mode.
EGTM: GT.M for Erlang,
"IODB: EGTM-powered ObjectDB for Erlang
(UniVerse, UniData): MultiValue
Databases, Data Structure: MultiValued, Supports nested
entities, Virtual Metadata, API: BASIC, InterCall,
Socket, .NET and Java API's, IDE: Native, Record Oriented,
table space allocation, Protocol: Client Server, SOA,
Terminal Line, X-OFF/X-ON, Written in: C, Query Method: Native
mvQuery, (Retrieve/UniQuery) and SQL,
Replication: yes, Hot
standby, Concurrency: Record
and File Locking (Fine and Coarse Granularity)
.Net, COM, Socket, ODBC, Protocol: TCP/IP, Named Pipes, Telnet,
VT100. HTTP/S Query Method: RList, SQL & XPath Written in:
Native 4GL, C, C++, Basic+, .Net,
Java Replication: Hot Standby
Concurrency: table &/or row locking, optionally transaction based & commit
& rollback Data structure: Relational &/or
MultiValue, supports nested entities Scalability: rows and tables size dynamically
(D3, mvBase, mvEnterprise) Data Structure: Dynamic multidimensional PICK data model, multi-valued, dictionary-driven, API: NET, Java, PHP, C++, Protocol: C/S, Written In: C, Query Method: AQL, SQL, ODBC, Pick/BASIC, Replication: Hot Backup, FFR, Transaction Logging + real-time replication, Concurrency: Row Level Locking, Connectivity: OSFI, ODBC, Web-Services, Web-enabled, Security: File level AES-128 encryption
The original MultiValue data set database, virtual machine, enquiry and
rapid development environment. Delivers ultra efficiency, scalability
and resilience while extended for the web and with built-in auto
sizing, failsafe and more. Interoperability includes Web Services, Java
ActiveX, Sockets, C and, for those that have to
interoperate with the SQL
and two-way transparent SQL data access.
Supports nested data. Fully automated table space allocation. Concurrency
control via task locks, file locks & shareable/exclusive record
locks. Case insensitivity option. Secondary key indices. Integrated
data replication. QMBasic programming language for rapid development.
OO programming integrated into QMBasic. QMClient connectivity from
Visual Basic, PowerBasic, Delphi, PureBasic, ASP, PHP, C and more.
Extended multivalue query language.
A high performance dbms that runs on IBM mainframes (IBM z/OS, z/VM, zVSE),
+SQL interface with nested entity support
API: native 4GL (SOUL + o-o support), SQL, Host Language (COBOL, Assembler, PL1) API, ODBC, JDBC, .net, Websphere MQ, Sockets
Scalability: automatic table space allocation, 64-bit support
Written in: IBM assembler, C
Query method: SOUL, SQL, RCL ( invocation of native language from client )
Concurrency: record and file level locking
Connectivity: TN3270, Telnet, Http
Hybrid database / search engine system with characteristics of multi-value, document, relational, XML and graph databases. Used in production since 1985 for high-performance search and retrieve solutions. Full-text search, text classification, similarity search, results ranking, real time facets, Unicode, Chinese word segmentation, and more. Platforms: Windows, Linux, AIX and Solaris. API: .NET, Java and C/C++. Query methods: native (CCL), SQL subset, XPath. Commercial.
(by Microsoft) ISAM storage technology. Access using index or
cursor navigation. Denormalized schemas, wide tables with sparse
columns, multi-valued columns, and sparse and rich indexes. C# and
Delphi drivers available. Backend for a number of MS Products as
more info »
Time Series / Streaming Databases
Distributed DB designed to store and analyze high-frequency time-series data at scale.
Includes a large set of built-in features:
Rule Engine, Visualization, Data Forecasting, Data Mining.
API: RESTful API, Network API, Data API, Meta API, SQL
API Clients: R, Java, Ruby, Python, PHP, Node.js
Replication: Master Slave
Major protocol & format support: CSV, nmon, pickle, StatsD, collectd,
tcollector, scollector, JMX, JDBC, SNMP, JSON, ICMP, OVPM, SOAP.
time-series database optimized for Big Data analytics.
other NoSQL related databases
LotusScript, C, @Formulas, Protocol: HTTP, NRPC, Replication: Master/Master, Written in: C, Concurrency:
Consistent, Scaling: Replication
Database; Written in: C;
API: C/C++, SQL, JNI,
(master-slave), Cluster; Scalability: 64-bit and MVCC
APIs: C++, Navigational C.
Embedded Solution that is ACID Compliant with Multi-Core, On-Disk
In-Memory Support. Distributed Capabilities, Hot Online Backup,
supports all Main Platforms. Supported
B Tree & Hash Indexing. Replication: Master/Slave,
NoSql, in-memory, flat-file, cloud-based. API interfaces. Small data footprint and very fast data retrieval. Stores 200 million records with 200 attributes in just 10GB. Retrieves 150 million records per second per CPU core. Often used to visualize big data on maps. Written in C.
Next-gen NoSQL encrypted document store. Multi-recipient / group encryption. Featuers: concurrency, indices, ACID transactions, replication and PKI management. Supports PHP and many others. Written in C++. Commercial but has a free version. API: JSON
Service oriented, schema-less, network data model DBMS.
Client application invokes methods of vyhodb services, which are
written in Java and deployed inside vyhodb. Vyhodb services reads
and modifies storage data. API: Java, Protocol:
RSI - Remote service invocation, Written in: Java,
ACID: fully supported, Replication:
async master slave, Misc: online backup,
Java RAM Data structure
Python wrapper over sqlite3
Scientific and Specialized DBs
BayesDB, a Bayesian database table, lets users query the probable implications of their tabular data as easily as an SQL database lets them query the data itself. Using the built-in Bayesian Query Language (BQL), users with no statistics training can solve basic data science problems, such as detecting predictive relationships between variables, inferring missing values, simulating probable observations, and identifying statistically similar database entries.
A distributed database for many core devices.
GPUdb leverages many core devices such as NVIDIA GPUs to provide
an unparallelled parallel database experience. GPUdb is a scalable,
distributed database with SQL-style query capability, capable of
storing Big Data. Developers using the GPUdb API add data, and query
the data with operations like select, group by, and join.
GPUdb includes many operations not available in other "cloud database" offerings.
unresolved and uncategorized
(by Pervasive Software) key/index/tupel DB. Using Pages.
Written in: Ruby. github: »
GNU Tool for text files containing records and fields. Manual
Mainly targeted to Silverlight/Windows Phone developers but its also
great for any .NET application where a simple local database is
required, extremely Lightweight - less than 50K, stores one table per
file, including index, compiled versions for Windows Phone 7,
Silverlight and .NET, fast, free to use in your applications
written in Python
illuminate Correlation Database
», FluidDB (Column Oriented DB)
»,Fleet DB »,
Twisted Storage »,
Java-Chronicle », Ringo, Sherpa,
tin, Dryad, SkyNet, Disco
Possibly the oldest NoSQL DB (together with MUMPS and IBMs IMS & IDMS [1968,1964]):
Adabas VSAM by IBM is also a good candidate.