Antirez mentions that a limited set of on-disk data structures could work well but that it will take one or two years to even reach the drawing board. Fair enough -- Redis is first and foremost an in-memory database.
If there was the time though, I'd love to see LevelDB[1] merge the gap between in-memory and on-disk. Inspired by Google's BigTable, all keys are kept sorted on disk (see: SSTable[2]). Keys, lists, sets, sorted sets and hashtables could be encoded in sorted key-value form and would be reasonably (though not tremendously) efficient to retrieve, especially if the query can be converted to a range query (i.e. list retrieval or set intersection). Keep hot data in memory, the rest ends up securely on disk.
Yes, Cassandra is based on the same lineage but it's not simple or clean to operate. Redis is a pain free setup, features a simple API but has no transparent way to overflow to disk for cold data. LevelDB is an optional backend for Riak but I must admit I've not explored Riak heavily... Have I missed a contender from another database crowd?
Speaking of LevelDB and Redis, did you know about Edis[1]? It is a protocol compatible implementation of Redis that uses LevelDB as its data store. I haven't had a chance (nor the need) to try it, but it sounds interesting.
Indeed I hadn't -- Edis doesn't rank highly for searches involving "LevelDB + Redis" which could be how it avoided me for so long. This sounds like what I envisioned, so I'll be looking at it with a keen interest =] The real question is how they implemented the encoding of the Redis data structures into the LevelDB SSTable format and the implications that will have on performance. If the Github issues are any indication it's an interesting proof of concept but hasn't been used or tested widely yet[1]. Along those lines, leveldb-server also looks interesting as a simple (API) and easy to install LevelDB backed DB.
If there was the time though, I'd love to see LevelDB[1] merge the gap between in-memory and on-disk. Inspired by Google's BigTable, all keys are kept sorted on disk (see: SSTable[2]). Keys, lists, sets, sorted sets and hashtables could be encoded in sorted key-value form and would be reasonably (though not tremendously) efficient to retrieve, especially if the query can be converted to a range query (i.e. list retrieval or set intersection). Keep hot data in memory, the rest ends up securely on disk.
Yes, Cassandra is based on the same lineage but it's not simple or clean to operate. Redis is a pain free setup, features a simple API but has no transparent way to overflow to disk for cold data. LevelDB is an optional backend for Riak but I must admit I've not explored Riak heavily... Have I missed a contender from another database crowd?
[1]: http://code.google.com/p/leveldb/ [2]: http://www.igvita.com/2012/02/06/sstable-and-log-structured-...