News posted by antirez

Twemproxy, a Redis proxy from Twitter

antirez 1833 days ago.
While a big number of users use large farms of Redis nodes, from the point of view of the project itself currently Redis is a mostly single-instance business.

I've big plans about going distributed with the project, to the extent that I'm no longer evaluating any threaded version of Redis: for me from the point of view of Redis a core is like a computer, so that scaling multi core or on a cluster of computers is the same conceptually. Multiple instances is a share-nothing architecture. Everything makes sense AS LONG AS we have a *credible way* to shard :-)

Redis Crashes

antirez 1833 days ago.
Premise: a small rant about software reliability.

I'm very serious about software reliability, and this is not just a good thing.
It is good in a sense, as I tend to work to ensure that the software I release is solid. At the same time I think I take this issue a bit too personally: I get upset if I receive a crash report that I can't investigate further for some reason, or that looks like almost impossible to me, or with an unhelpful stack trace.

Guess what? This is a bad attitude because to deliver bugs free software is simply impossible. We are used to think in terms of labels: "stable", "production ready", "beta quality". I think that these labels are actually pretty misleading if not put in the right perspective.

Redis children can now report amount of copy-on-write

antirez 1848 days ago.
This is one of this trivial changes in Redis that can make a big difference for users. Basically in the unstable branch I added some code that has the following effect, when running Redis on Linux systems:

[32741] 19 Nov 12:00:55.019 * Background saving started by pid 391
[391] 19 Nov 12:01:00.663 * DB saved on disk
[391] 19 Nov 12:01:00.673 * RDB: 462 MB of memory used by copy-on-write

As you can see now the amount of additional memory used by the saving child is reported (it is also reported for AOF rewrite operations).

On Twitter, at Twitter

antirez 1854 days ago.
On Twitter:

@War3zRub1 "Hahaha it's silly how people use Redis when they need a reverse proxy"
@C4ntc0de "ZOMG! Use a real message queue, Redis is not a queue!"
@L4m3tr00l "My face when Redis BLABLABLA..."

Meanwhile *at* Twitter:

OP1: "Hey guys, there is a spike in the number of lame messages today, load is increasing..."
OP2: "Yep noticed, it's the usual troll fiesta trowing shit at Redis, 59482 messages per second right now."
OP1: "Ok, no prob, let's spawn two additional Redis nodes to serve their timelines as smooth as usually".

Eventual consistency: when, and how?

antirez 1857 days ago.
This post by Peter Bailis is a must read. "Safety and Liveness: Eventual consistency is not safe" [1].

[1] http://www.bailis.org/blog/safety-and-liveness-eventual-consistency-is-not-safe/

An extreme TL;DR of this is.

1) In an eventually consistent system, when all the nodes will agree again after a partition?
2) In an eventually consistent system, HOW the nodes will agree about inconsistencies?
3) In immediately consistent systems, when I'm no longer able to write? When I'm no longer able to read?

Welcome to RethinkDB

antirez 1858 days ago.
There is a new DB option out there, I know it took a long time to be developed. While I don't know very well how it works I hope it will be an interesting player in the database landscape.

My initial feeling is that it will compete closely with Riak and MongoDB (the system seems more similar to MongoDB itself, but if it can scale well multi-nodes people that don't need high write availability may pick an immediate consistent database such as RethinkDB instead of Riak for certain applications).

Redis data model and eventual consistency

antirez 1858 days ago.
While I consider the Amazon Dynamo design, and its original paper, one of the most interesting things produced in the field of databases in recent times, in the Redis world eventual consistency was never particularly discussed.

Redis Cluster for instance is a system biased towards consistency than availability. Redis Sentinel itself is an HA solution with the dogma of consistency and master slave setups.

This bias for consistent systems over more available but eventual consistent systems has some good reasons indeed, that I can probably reduce to three main points: