Category Archives: NOSQL

redis.conf Redis配置文件详解

如果认为Redis是一个key value存储, 可以使用它来代替MySQL;如果认为它是一个可以持久化的cache, 可能只是用它保存一些频繁访问的临时数据(代替Memcached);除此之外,还可以把Redis当做一个轻量级的消息队列使用,因为它内置就支持list数据结构和PUB/SUB命令;还可以当做一个轻量级的分布式锁系统。Redis是REmote DIctionary Server的缩写,在Redis在官方网站的解释是:

Redis is an open source, advanced key-value store.   It is often referred to as a data structure server since keys   can contain strings, hashes, lists, sets and sorted sets.

本文将会详细介绍Redis的配置文件。

1. Redis默认不是以守护进程的方式运行,可以通过该配置项修改,使用yes启用守护进程

daemonize no

2. 当Redis以守护进程方式运行时,Redis默认会把pid写入/var/run/redis.pid文件,可以通过pidfile指定

pidfile /var/run/redis.pid

3. 指定Redis监听端口,默认端口为6379,作者在自己的一篇博文中解释了为什么选用6379作为默认端口,因为6379在手机按键上MERZ对应的号码,而MERZ取自意大利歌女Alessia Merz的名字

port 6379

4. 绑定的主机地址

bind 127.0.0.1

5.当 客户端闲置多长时间后关闭连接,如果指定为0,表示关闭该功能

timeout 300

6. 指定日志记录级别,Redis总共支持四个级别:debug、verbose、notice、warning,默认为verbose

loglevel verbose

7. 日志记录方式,默认为标准输出,如果配置Redis为守护进程方式运行,而这里又配置为日志记录方式为标准输出,则日志将会发送给/dev/null

logfile stdout

8. 设置数据库的数量,默认数据库为0,可以使用SELECT <dbid>命令在连接上指定数据库id

databases 16

9. 指定在多长时间内,有多少次更新操作,就将数据同步到数据文件,可以多个条件配合

save <seconds> <changes>

Redis默认配置文件中提供了三个条件:

save 900 1  save 300 10  save 60 10000

分别表示900秒(15分钟)内有1个更改,300秒(5分钟)内有10个更改以及60秒内有10000个更改。

10. 指定存储至本地数据库时是否压缩数据,默认为yes,Redis采用LZF压缩,如果为了节省CPU时间,可以关闭该选项,但会导致数据库文件变的巨大

rdbcompression yes

11. 指定本地数据库文件名,默认值为dump.rdb

dbfilename dump.rdb

12. 指定本地数据库存放目录

dir ./

13. 设置当本机为slav服务时,设置master服务的IP地址及端口,在Redis启动时,它会自动从master进行数据同步

slaveof <masterip> <masterport>

14. 当master服务设置了密码保护时,slav服务连接master的密码

masterauth <master-password>

15. 设置Redis连接密码,如果配置了连接密码,客户端在连接Redis时需要通过AUTH <password>命令提供密码,默认关闭

requirepass foobared

16. 设置同一时间最大客户端连接数,默认无限制,Redis可以同时打开的客户端连接数为Redis进程可以打开的最大文件描述符数,如果设置maxclients 0,表示不作限制。当客户端连接数到达限制时,Redis会关闭新的连接并向客户端返回max number of clients reached错误信息

maxclients 128

17. 指定Redis最大内存限制,Redis在启动时会把数据加载到内存中,达到最大内存后,Redis会先尝试清除已到期或即将到期的Key,当此方法处理后,仍然到达最大内存设置,将无法再进行写入操作,但仍然可以进行读取操作。Redis新的vm机制,会把Key存放内存,Value会存放在swap区

maxmemory <bytes>

18. 指定是否在每次更新操作后进行日志记录,Redis在默认情况下是异步的把数据写入磁盘,如果不开启,可能会在断电时导致一段时间内的数据丢失。因为redis本身同步数据文件是按上面save条件来同步的,所以有的数据会在一段时间内只存在于内存中。默认为no

appendonly no

19. 指定更新日志文件名,默认为appendonly.aof

appendfilename appendonly.aof

20. 指定更新日志条件,共有3个可选值: no:表示等操作系统进行数据缓存同步到磁盘(快)
always:表示每次更新操作后手动调用fsync()将数据写到磁盘(慢,安全) everysec:表示每秒同步一次(折衷,默认值)

appendfsync everysec

21. 指定是否启用虚拟内存机制,默认值为no,简单的介绍一下,VM机制将数据分页存放,由Redis将访问量较少的页即冷数据swap到磁盘上,访问多的页面由磁盘自动换出到内存中(在后面的文章我会仔细分析Redis的VM机制)

vm-enabled no

22. 虚拟内存文件路径,默认值为/tmp/redis.swap,不可多个Redis实例共享

vm-swap-file /tmp/redis.swap

23. 将所有大于vm-max-memory的数据存入虚拟内存,无论vm-max-memory设置多小,所有索引数据都是内存存储的(Redis的索引数据就是keys),也就是说,当vm-max-memory设置为0的时候,其实是所有value都存在于磁盘。默认值为0

vm-max-memory 0

24. Redis swap文件分成了很多的page,一个对象可以保存在多个page上面,但一个page上不能被多个对象共享,vm-page-size是要根据存储的数据大小来设定的,作者建议如果存储很多小对象,page大小最好设置为32或者64bytes;如果存储很大大对象,则可以使用更大的page,如果不确定,就使用默认值

vm-page-size 32

25. 设置swap文件中的page数量,由于页表(一种表示页面空闲或使用的bitmap)是在放在内存中的,,在磁盘上每8个pages将消耗1byte的内存。

vm-pages 134217728

26. 设置访问swap文件的线程数,最好不要超过机器的核数,如果设置为0,那么所有对swap文件的操作都是串行的,可能会造成比较长时间的延迟。默认值为4

vm-max-threads 4

27. 设置在向客户端应答时,是否把较小的包合并为一个包发送,默认为开启

glueoutputbuf yes

28. 指定在超过一定的数量或者最大的元素超过某一临界值时,采用一种特殊的哈希算法

hash-max-zipmap-entries 64  hash-max-zipmap-value 512

29. 指定是否激活重置哈希,默认为开启(后面在介绍Redis的哈希算法时具体介绍)

activerehashing yes

30. 指定包含其它的配置文件,可以在同一主机上多个Redis实例之间使用同一份配置文件,而同时各个实例又拥有自己的特定配置文件

include /path/to/local.conf

from: http://zhumeng8337797.blog.163.com/blog/static/100768914201121324915446/?fromdm&fromSearch&isFromSearchEngine=yes

redis-cli 命令总结

Redis提供了丰富的命令(command)对数据库和各种数据类型进行操作,这些command可以在Linux终端使用。在编程时,比如使用Redis 的Java语言包,这些命令都有对应的方法。下面将Redis提供的命令做一总结。

官网命令列表:http://redis.io/commands (英文)

1、连接操作相关的命令

  • quit:关闭连接(connection)
  • auth:简单密码认证

2、对value操作的命令

  • exists(key):确认一个key是否存在
  • del(key):删除一个key
  • type(key):返回值的类型
  • keys(pattern):返回满足给定pattern的所有key
  • randomkey:随机返回key空间的一个key
  • rename(oldname, newname):将key由oldname重命名为newname,若newname存在则删除newname表示的key
  • dbsize:返回当前数据库中key的数目
  • expire:设定一个key的活动时间(s)
  • ttl:获得一个key的活动时间
  • select(index):按索引查询
  • move(key, dbindex):将当前数据库中的key转移到有dbindex索引的数据库
  • flushdb:删除当前选择数据库中的所有key
  • flushall:删除所有数据库中的所有key

3、对String操作的命令

  • set(key, value):给数据库中名称为key的string赋予值value
  • get(key):返回数据库中名称为key的string的value
  • getset(key, value):给名称为key的string赋予上一次的value
  • mget(key1, key2,…, key N):返回库中多个string(它们的名称为key1,key2…)的value
  • setnx(key, value):如果不存在名称为key的string,则向库中添加string,名称为key,值为value
  • setex(key, time, value):向库中添加string(名称为key,值为value)同时,设定过期时间time
  • mset(key1, value1, key2, value2,…key N, value N):同时给多个string赋值,名称为key i的string赋值value i
  • msetnx(key1, value1, key2, value2,…key N, value N):如果所有名称为key i的string都不存在,则向库中添加string,名称key i赋值为value i
  • incr(key):名称为key的string增1操作
  • incrby(key, integer):名称为key的string增加integer
  • decr(key):名称为key的string减1操作
  • decrby(key, integer):名称为key的string减少integer
  • append(key, value):名称为key的string的值附加value
  • substr(key, start, end):返回名称为key的string的value的子串

4、对List操作的命令

  • rpush(key, value):在名称为key的list尾添加一个值为value的元素
  • lpush(key, value):在名称为key的list头添加一个值为value的 元素
  • llen(key):返回名称为key的list的长度
  • lrange(key, start, end):返回名称为key的list中start至end之间的元素(下标从0开始,下同)
  • ltrim(key, start, end):截取名称为key的list,保留start至end之间的元素
  • lindex(key, index):返回名称为key的list中index位置的元素
  • lset(key, index, value):给名称为key的list中index位置的元素赋值为value
  • lrem(key, count, value):删除count个名称为key的list中值为value的元素。count为0,删除所有值为value的元素,count>0从头至尾删除count个值为value的元素,count<0从尾到头删除|count|个值为value的元素。 lpop(key):返回并删除名称为key的list中的首元素 rpop(key):返回并删除名称为key的list中的尾元素 blpop(key1, key2,… key N, timeout):lpop命令的block版本。即当timeout为0时,若遇到名称为key i的list不存在或该list为空,则命令结束。如果timeout>0,则遇到上述情况时,等待timeout秒,如果问题没有解决,则对keyi+1开始的list执行pop操作。
  • brpop(key1, key2,… key N, timeout):rpop的block版本。参考上一命令。
  • rpoplpush(srckey, dstkey):返回并删除名称为srckey的list的尾元素,并将该元素添加到名称为dstkey的list的头部

5、对Set操作的命令

  • sadd(key, member):向名称为key的set中添加元素member
  • srem(key, member) :删除名称为key的set中的元素member
  • spop(key) :随机返回并删除名称为key的set中一个元素
  • smove(srckey, dstkey, member) :将member元素从名称为srckey的集合移到名称为dstkey的集合
  • scard(key) :返回名称为key的set的基数
  • sismember(key, member) :测试member是否是名称为key的set的元素
  • sinter(key1, key2,…key N) :求交集
  • sinterstore(dstkey, key1, key2,…key N) :求交集并将交集保存到dstkey的集合
  • sunion(key1, key2,…key N) :求并集
  • sunionstore(dstkey, key1, key2,…key N) :求并集并将并集保存到dstkey的集合
  • sdiff(key1, key2,…key N) :求差集
  • sdiffstore(dstkey, key1, key2,…key N) :求差集并将差集保存到dstkey的集合
  • smembers(key) :返回名称为key的set的所有元素
  • srandmember(key) :随机返回名称为key的set的一个元素

6、对zset(sorted set)操作的命令

  • zadd(key, score, member):向名称为key的zset中添加元素member,score用于排序。如果该元素已经存在,则根据score更新该元素的顺序。
  • zrem(key, member) :删除名称为key的zset中的元素member
  • zincrby(key, increment, member) :如果在名称为key的zset中已经存在元素member,则该元素的score增加increment;否则向集合中添加该元素,其score的值为increment
  • zrank(key, member) :返回名称为key的zset(元素已按score从小到大排序)中member元素的rank(即index,从0开始),若没有member元素,返回“nil”
  • zrevrank(key, member) :返回名称为key的zset(元素已按score从大到小排序)中member元素的rank(即index,从0开始),若没有member元素,返回“nil”
  • zrange(key, start, end):返回名称为key的zset(元素已按score从小到大排序)中的index从start到end的所有元素
  • zrevrange(key, start, end):返回名称为key的zset(元素已按score从大到小排序)中的index从start到end的所有元素
  • zrangebyscore(key, min, max):返回名称为key的zset中score >= min且score <= max的所有元素 zcard(key):返回名称为key的zset的基数 zscore(key, element):返回名称为key的zset中元素element的score zremrangebyrank(key, min, max):删除名称为key的zset中rank >= min且rank <= max的所有元素 zremrangebyscore(key, min, max) :删除名称为key的zset中score >= min且score <= max的所有元素
  • zunionstore / zinterstore(dstkeyN, key1,…,keyN, WEIGHTS w1,…wN, AGGREGATE SUM|MIN|MAX):对N个zset求并集和交集,并将最后的集合保存在dstkeyN中。对于集合中每一个元素的score,在进行AGGREGATE运算前,都要乘以对于的WEIGHT参数。如果没有提供WEIGHT,默认为1。默认的AGGREGATE是SUM,即结果集合中元素的score是所有集合对应元素进行SUM运算的值,而MIN和MAX是指,结果集合中元素的score是所有集合对应元素中最小值和最大值。

7、对Hash操作的命令

  • hset(key, field, value):向名称为key的hash中添加元素field<—>value
  • hget(key, field):返回名称为key的hash中field对应的value
  • hmget(key, field1, …,field N):返回名称为key的hash中field i对应的value
  • hmset(key, field1, value1,…,field N, value N):向名称为key的hash中添加元素field i<—>value i
  • hincrby(key, field, integer):将名称为key的hash中field的value增加integer
  • hexists(key, field):名称为key的hash中是否存在键为field的域
  • hdel(key, field):删除名称为key的hash中键为field的域
  • hlen(key):返回名称为key的hash中元素个数
  • hkeys(key):返回名称为key的hash中所有键
  • hvals(key):返回名称为key的hash中所有键对应的value
  • hgetall(key):返回名称为key的hash中所有的键(field)及其对应的value

8、持久化

  • save:将数据同步保存到磁盘
  • bgsave:将数据异步保存到磁盘
  • lastsave:返回上次成功将数据保存到磁盘的Unix时戳
  • shundown:将数据同步保存到磁盘,然后关闭服务

9、远程服务控制

  • info:提供服务器的信息和统计
  • monitor:实时转储收到的请求
  • slaveof:改变复制策略设置
  • config:在运行时配置Redis服务器

from:http://slj.me/2011/04/redis-cli-commands/

案例学习:仅使用Redis+PHP设计实现一个简单的Twitter

原文位于Redis官网http://redis.io/topics/twitter-clone

Redis是NoSQL数据库中一个知名数据库,在新浪微博中亦有部署,适合固定数据量的热数据的访问。

作为入门,这是一篇很好的教材,简单描述了如何使用KV数据库进行数据库的设计。新的项目www.xiayucha.com亦采用Redis + MySQL进行开发,考虑Redis文档比较少,故翻译了此文。

其他参考资料:

我会在此文中描述如何使用PHP以及仅使用Redis来设计实现一个简单的Twitter克隆。
很多编程社区常认为KV储存是一个特别的数据库,在web应用中不能替代关系数据库。
本文尝试证明这恰恰相反。

这个twitter克隆名为Retwis,结构简单,性能优异,能很轻易地用N个web服务器和Redis服务器以分布式架构。
在此获取源码http://code.google.com/p/redis/downloads/list
我们使用PHP作为例子是因为它能被每个人读懂,也能使用Ruby、Python、Erlang或其他语言获取同样(或者更佳)的效果。

注意:Retwis-RB是一个由Daniel Lucraft用Ruby与Sinatra写的Retwis分支!
此文全部代码在本页尾部的Git repository链接里。
此文以PHP为例,但是Ruby程序员也能检出其他源码。他们很相似。

注意Retwis-J是Retwis的一个分支,由Costin Leau以Java和Spring框架写成。
源码能在GitHub找到,并且在springsource.org有综合的文档。

Key-value 数据库基础

KV数据的精髓,是能够把value储存在key里,此后该数据仅能够通过确切的key来获取,无法搜索一个值。
确切的来讲,它更像一个大型HASH/字典,但它是持久化的,比如,当你的程序终止运行,数据不会消失。
比如我们能用SET命令以key foo 来储存值 bar
 SET foo bar
Redis会永久储存我们的数据,所以之后我们可以问Redis:“储存在key foo里的数据是什么?”,Redis会返回一个值:bar
 GET foo => bar
KV数据库提供的其他常见操作有:DEL,用于删除指定的key和关联的value;
SET-if-not-exists (在Redis上称为SETNX )仅会在key不存在的时候设置一个值;
INCR能够对指定的key里储存的数字进行自增。
 SET foo 10
 INCR foo => 11
 INCR foo => 12
 INCR foo => 13

原子操作

目前为止它是相当简单的,但是INCR有些不同。设想一下,为什么要提供这个操作?毕竟我们自己能用以下简单的命令实现这个功能:
 x = GET foo
 x = x + 1
 SET foo x
问题在于要使上面的操作正常进行,同时只能有一个客户端操作x的值。看看如果两台电脑同时操作这个值会发生什么:
 x = GET foo (返回10)
 y = GET foo (返回10)
 x = x + 1 (x现在是11)
 y = y + 1 (y现在是11)
 SET foo x (foo现在是11)
 SET foo y (foo现在是11)
问题发生了!我们增加了值两次,本应该从10变成12,现在却停留在了11。这是因为用GET和SET来实现INCR不是一个原子操作(atomic operation)。
所以Redis\memcached之类提供了一个原子的INCR命令,服务器会保护get-increment-set操作,以防止同时的操作。
让Redis与众不同的是它提供了更多类似INCR的方案,用于解决模型复杂的问题。
因此你可以不使用任何SQL数据库、仅用Redis写一个完整的web应用,而不至于抓狂。

超越Ke-Value数据库

本节我们会看到构建一个Twitter克隆所需Redis的功能。首先需要知道的是,Redis的值不仅仅可以是字符串(String)。
Redis的值可以是列表(Lists)也可以是集合(Sets),在操作更多类型的值时也是原子的,所以多方操作同一个KEY的值也是安全的。
让我们从一个Lists开始:
 LPUSH mylist a (现在mylist含有一个元素:’a’的list)
 LPUSH mylist b (现在mylist含有元素’b,a’)
 LPUSH mylist c (现在mylist含有’c,b,a’)
LPUSH的意思是Left Push, 就是把一个元素加在列表(list)的左边(或者说头上)。
在PUSH操作之前,如果mylist这个键(key)不存在,Redis会自动创建一个空的list。
就像你能想到的一样,同样有个RPUSH操作可以把元素加在列表(list)的右边(尾部)。
这对我们复制一个twitter非常有用,例如我们可以把用户的更新储存在username:updates里。
当然,我们也有相应的操作来获取数据或者信息。比如LRANGE返回列表(list)的一个范围内的元素,或者所有元素
 LRANGE mylist 0 1 => c,b
LRANGE使用从零开始的索引(zero-based indexes),第一个元素的索引是0,第二个是1,以此类推。该命令的参数是:LRANGE key first-index last-index
参数last index可以是负数,具有特殊的意义:-1是列表(list)的最后一个元素,-2是倒数第二个,以此类推。
所以,如果要获取整个list,我们能使用以下命令:
 LRANGE mylist 0 -1 => c,b,a
其他重要的操作有LLEN,返回列表(list)的长度,LTRIM类似于LRANGE,但不仅仅会返回指定范围内的元素,而且还会原子地把列表(list)的值设置这个新的值。
我们将会使用这些list操作,但是注意阅读Redis文档来浏览所有redis支持的list操作。

数据类型:集合(set)

除了列表(list),Redis还提供了集合(sets)的支持,是不排序(unsorted)的元素集合。
它能够添加、删除、检查元素是否存在,并且获取两个结合之间的交集。当然它也能请求获取集合(set)里一个或者多个元素。
几个例子可以使概念更为清晰。记住:SADD是往集合(set)里添元素;SREM是从集合(set)里删除元素;SISMEMBER是检测一个元素是否包含在集合里;SINTER用于显示两个集合的交集。
其他操作有,SCARD用于获取集合的基数(集合中元素的数量);SMEMBERS返回集合中所有的元素
 SADD myset a
 SADD myset b
 SADD myset foo
 SADD myset bar
 SCARD myset => 4
 SMEMBERS myset => bar,a,foo,b
注意SMEMBERS不会以我们添加的顺序返回元素,因为集合(Sets)是一个未排序的元素集合。如果你要储存顺序,最好使用列表(Lists)取而代之。以下是基于集合的一些操作:
 SADD mynewset b
 SADD mynewset foo
 SADD mynewset hello
 SINTER myset mynewset => foo,b
SINTER能够返回集合之间的交集,但并不仅限于两个集合(Sets),你能获取4个、5个甚至1000个集合(sets)的交集。
最后,让我们看下SISMEMBER是如何工作的:
 SISMEMBER myset foo => 1
 SISMEMBER myset notamember => 0
Okay,我觉得我们可以开始coding啦!

先决条件

如果你还没下载,请前往<http://code.google.com/p/redis/downloads/list>下载Retwis的源码。它包含几个PHP文件,是个简单的tar.gz文件。
实现的非常简单,你会在里面找到PHP客户端(redis.php),用于redis与PHP的交互。该库由Ludovico Magnocavallo(http://qix.it/ )编写,你可以在自己的项目中免费使用。
但如果要更新库的版本请下载Redis的发行版。(注意:现在有更好的PHP库了,请检查我们的客户端页面<http://redis.io/clients>)
你需要的另一个东西是正常运行的Redis服务器。仅需要获取源码、用make编译、用./redis-server就完工了,点儿也不须配置就可以在你的电脑上运行Retwis。

数据结构规划

当使用关系数据库的时候,这一步往往是在设计数据表、索引的表单里处理。我们没有表,那我们设计什么呢? 我们需要确认物体使用的key以及key采用的类型。
让我们从用户这块开始设计。当然了,首先需要展示用户的username, userid, password, followers,自己follow的用户等。第一个问题是:如何在我们的系统中标识一个用户?
username是个好主意,因为它是唯一的。不过它太大了,我们想要降低内存的使用。如果我们的数据库是关系数据库,我们能关联唯一ID到每一个用户。每一个对用户的引用都通过ID来关联。
做起来很简单,因为我们有我们的原子的INCR命令!当我们创建一个新用户,我们假设这个用户叫”antirez”:
 INCR global:nextUserId => 1000
 SET uid:1000:username antirez
 SET uid:1000:password p1pp0
我们使用global:nextUserId为键(Key)是为了给每个新用户分配一个唯一ID,然后用这个唯一ID来加入其他key,以识别保存用户的其他数据。这就是kv数据库的设计模式!请牢记于心,
除了已经定义的KEY,我们还需要更多的来完整定义一个用户,比如有时需要通过用户名来获取用户ID,所以我们也需要设置这么一个键(Key)
 SET username:antirez:uid 1000
一开始看上去这样很奇怪,但请记住我们只能通过key来获取数据!这不可能告诉Redis返回包含某值的Key,这也是我们的强处。
用关系数据库方式来讲,这个新实例强迫我们组织数据,以便于仅使用primary key访问任何数据。

关注\被关注与更新

这也是在我们系统中另一个重要需求.每个用户都有follower,也有follow的用户.对此我们有最佳的数据结构!那就是…..集合(Sets).那就让我们在结构中加入两个新字段:
 uid:1000:followers => Set of uids of all the followers users
 uid:1000:following => Set of uids of all the following users
另一个重要的事情是我们需要有个地方来放用户主页上的更新。这个要以时间顺序排序,最新的排在旧的前面。所以,最佳的类型是列表(List)。
基本上每个更新都会被LPUSH到该用户的updates key.多亏了LRANGE,我们能够实现分页等功能。请注意更新(updates)和帖子(posts)讲的是同一个东西,实际上更新(updates)是有点小的帖子(posts)。
 uid:1000:posts => a List of post ids, every new post is LPUSHed here.

验证

OK,除了验证,或多或少我们已经有了关于该用户的一切东西。我们处理验证用一个简单而健壮(鲁棒)的办法:我们不使用PHP的session或者其他类似方式。
我们的系统必须是能够在不同不同服务器上分布式部署的,所以一切状态都必须保存在Redis里。所以我们所需要的一个保存在已验证用户cookie里的随机字符串。
包含同样随机字符串的一个key告诉我们用户的ID。我们需要使用两个key来保证这个验证机制的健壮性:
 SET uid:1000:auth fea5e81ac8ca77622bed1c2132a021f9
 SET auth:fea5e81ac8ca77622bed1c2132a021f9 1000
为了验证一个用户,我们需要做一些简单的工作(login.php):
* 从登录表单获取用户的用户名和密码
* 检查是否存在一个键 username:<username>:uid
* 如果这个user id存在(假设1000)
* 检查 uid:1000:password 是否匹配,如果不匹配,显示错误信息
* 匹配则设置cookie为字符串”fea5e81ac8ca77622bed1c2132a021f9″(uid:1000:auth的值)
实例代码:

PHP代码
  1. include(“retwis.php”);   
  2.   
  3. # Form sanity checks   
  4. if (!gt(“username”) || !gt(“password”))   
  5.     goback(“You need to enter both username and password to login.”);   
  6.   
  7. # The form is OK, check if the username is available   
  8. $username = gt(“username”);   
  9. $password = gt(“password”);   
  10. $r = redisLink();   
  11. $userid = $r->get(“username:$username:id”);   
  12. if (!$userid)   
  13.     goback(“Wrong username or password”);   
  14. $realpassword = $r->get(“uid:$userid:password”);   
  15. if ($realpassword != $password)   
  16.     goback(“Wrong useranme or password”);   
  17.   
  18. # Username / password OK, set the cookie and redirect to index.php   
  19. $authsecret = $r->get(“uid:$userid:auth”);   
  20. setcookie(“auth”,$authsecret,time()+3600*24*365);   
  21. header(“Location: index.php”);   
  22.   

每次用户登录都会运行,但我们需要一个函数isLoggedIn用于检验一个用户是否已经验证。
这些是isLoggedIn的逻辑步骤
* 从用户获取cookie里auth的值。如果没有cookie,该用户未登录。我们称这个cookie为<authcookie>
* 检查auth:<authcookie>是否存在,存在则获取值(例子里是1000)
* 为了再次确认,检查uid:1000:auth是否匹配
* 用户已验证,在全局变量$User中载入一点信息
也许代码比描述更短:

PHP代码
  1. function isLoggedIn() {   
  2.     global $User, $_COOKIE;   
  3.   
  4.     if (isset($User)) return true;   
  5.   
  6.     if (isset($_COOKIE[‘auth’])) {   
  7.         $r = redisLink();   
  8.         $authcookie = $_COOKIE[‘auth’];   
  9.         if ($userid = $r->get(“auth:$authcookie”)) {   
  10.             if ($r->get(“uid:$userid:auth”) != $authcookie) return false;   
  11.             loadUserInfo($userid);   
  12.             return true;   
  13.         }   
  14.     }   
  15.     return false;   
  16. }   
  17.   
  18. function loadUserInfo($userid) {   
  19.     global $User;   
  20.   
  21.     $r = redisLink();   
  22.     $User[‘id’] = $userid;   
  23.     $User[‘username’] = $r->get(“uid:$userid:username”);   
  24.     return true;   
  25. }   
  26.   

把loadUserInfo作为一个独立函数对于我们的应用而言有点杀鸡用牛刀了,但是对于复杂的应用而言这是一个不错的模板。
作为一个完整的验证,还剩下logout还没实现。在logout的时候我们怎么做呢?
很简单,仅仅改变uid:1000:auth里的随机字符串,删除旧的auth:<oldauthstring>并增加一个新的auth:<newauthstring>
重要:logout过程解释了为什么我们不仅仅查找auth:<randomstring>而是再次检查了uid:1000:auth。真正的验证字符串是后者,auth:<randomstring>是易变的.
假设程序中有BUGs或者脚本被意外中断,那么就有可能有多个auth:<something>指向同一个用户id。
logout代码如下:(logout.php)

PHP代码
  1. include(“retwis.php”);   
  2.   
  3. if (!isLoggedIn()) {   
  4.     header(“Location: index.php”);   
  5.     exit;   
  6. }   
  7.   
  8. $r = redisLink();   
  9. $newauthsecret = getrand();   
  10. $userid = $User[‘id’];   
  11. $oldauthsecret = $r->get(“uid:$userid:auth”);   
  12.   
  13. $r->set(“uid:$userid:auth”,$newauthsecret);   
  14. $r->set(“auth:$newauthsecret”,$userid);   
  15. $r->delete(“auth:$oldauthsecret”);   
  16.   
  17. header(“Location: index.php”);   
  18.   

以上是我们所描述过的,应该比较易于理解。

更新(Updates)


更新,或者称为帖子(posts)的实现则更为简单。为了在数据库里创建一个新的帖子,我们做了以下工作:
 INCR global:nextPostId => 10343
 SET post:10343 “$owner_id|$time|I’m having fun with Retwis”
就像你看到的一样,帖子的用户id和时间直接储存在了字符串里。
在这个例子中我们不需要根据时间或者用户id来查找帖子,所以把他们紧凑地挤在一个post字符串里更佳。
在新建一个帖子之后,我们获得了帖子的id。需要LPUSH这个帖子的id到每一个follow了作者的用户里去,当然还有作者的帖子列表。
update.php这个文件展示了这个工作是如何完成的:

PHP代码
  1. include(“retwis.php”);   
  2.   
  3. if (!isLoggedIn() || !gt(“status”)) {   
  4.     header(“Location:index.php”);   
  5.     exit;   
  6. }   
  7.   
  8. $r = redisLink();   
  9. $postid = $r->incr(“global:nextPostId”);   
  10. $status = str_replace(“\n”,” “,gt(“status”));   
  11. $post = $User[‘id’].”|”.time().”|”.$status;   
  12. $r->set(“post:$postid”,$post);   
  13. $followers = $r->smembers(“uid:”.$User[‘id’].”:followers”);   
  14. if ($followers === false) $followers = Array();   
  15. $followers[] = $User[‘id’]; /* Add the post to our own posts too */  
  16.   
  17. foreach($followers as $fid) {   
  18.     $r->push(“uid:$fid:posts”,$postid,false);   
  19. }   
  20. # Push the post on the timeline, and trim the timeline to the   
  21. # newest 1000 elements.   
  22. $r->push(“global:timeline”,$postid,false);   
  23. $r->ltrim(“global:timeline”,0,1000);   
  24.   
  25. header(“Location: index.php”);   
  26.   

函数的核心是foreach。 通过SMEMBERS获取当前用户的所有follower,然后循环会把帖子(post)LPUSH到每一个用户的 uid:<userid>:posts里
注意我们同时维护了一个所有帖子的时间线。为此我们还需要LPUSH到global:timeline里。
面对这个现实,你是否开始觉得:SQL里面用ORDER BY来按时间排序有一点儿奇怪? 我确实是这么想的。

分页

现在很清楚,我们能用LRANGE来获取帖子的范围,并在屏幕上显示。代码很简单:

PHP代码
  1. function showPost($id) {   
  2.     $r = redisLink();   
  3.     $postdata = $r->get(“post:$id”);   
  4.     if (!$postdata) return false;   
  5.   
  6.     $aux = explode(“|”,$postdata);   
  7.     $id = $aux[0];   
  8.     $time = $aux[1];   
  9.     $username = $r->get(“uid:$id:username”);   
  10.     $post = join(array_splice($aux,2,count($aux)-2),”|”);   
  11.     $elapsed = strElapsed($time);   
  12.     $userlink = “<a class=\”username\” href=\”profile.php?u=”.urlencode($username).”\”>”.utf8entities($username).”</a>”;   
  13.   
  14.     echo(‘<div class=”post”>’.$userlink.’ ‘.utf8entities($post).”<br>”);   
  15.     echo(‘<i>posted ‘.$elapsed.’ ago via web</i></div>’);   
  16.     return true;   
  17. }   
  18.   
  19. function showUserPosts($userid,$start,$count) {   
  20.     $r = redisLink();   
  21.     $key = ($userid == -1) ? “global:timeline” : “uid:$userid:posts”;   
  22.     $posts = $r->lrange($key,$start,$start+$count);   
  23.     $c = 0;   
  24.     foreach($posts as $p) {   
  25.         if (showPost($p)) $c++;   
  26.         if ($c == $count) break;   
  27.     }   
  28.     return count($posts) == $count+1;   
  29. }   
  30.   

当showUserPosts获取帖子的范围并传递给showPost时,showPost会简单输出一篇帖子的HTML代码。

Following users 关注的用户

如果用户id 1000 (antirez)想要follow用户id1000的pippo,我们做到这个仅需两步SADD:
SADD uid:1000:following 1001
SADD uid:1001:followers 1000
再次注意这个相同的模式:在关系数据库里的理论里follow的用户和被follow的用户是一张包含类似following_id和follower_id的单独数据表。
用查询你能明确follow和被follow的每一个用户。在key-value数据里有一点特别,需要我们分别设置1000follow了1001并且1001被1000follow的关系。
这是需要付出的代价,但是另一方面讲,获取这些数据即简单又超快。并且这些是独立的集合,允许我们做一些有趣的事情,比如使用SINTER获取两个不同用户的集合。
这样我们也许可以在我们的twitter复制品中加入一个功能:当你访问某个人的资料页时显示”你和foobar有34个共同关注者”之类的东西。
你能够在follow.php中找到增加或者删除following/folloer关系的代码。它如你所见般平常。

使它能够水平分割

亲爱的读者,如果你看到这里,你已经是一个英雄了,谢谢你。在讲到水平分割之前,看看单台服务器的性能是个不错的主意。
Retwis让人惊讶地快,没有任何缓存。在一台非常缓慢和高负载的服务器上,以100个线程并发请求100000次进行apache基准测试,平均占用5ms。
这意味着你可以仅仅使用一台linux服务器接受每天百万用户的访问,并且慢的跟个傻猴似的,就算用更新的硬件。
虽然,就算你有一堆用户,也许也不需要超过1台服务器来跑应用,但让我们假设我们是Twitter,需要处理海量的访问量呢?该怎么做?

Hashing the key

第一件事是把KEY进行hash运算并基于hash在不同服务器上处理请求。有大量知名的hash算法,例如ruby客户端自带的consistent hashing
大致意思是你能把key转换成数字,并除以你的服务器数量
 server_id = crc32(key) % number_of_servers
这里还有大量因为添加一台服务器产生的问题,但这仅仅是大致的意思,哪怕使用一个类似consistent hashing的更好索引算法,
是不是key就可以分布式访问了呢?所有用户数据都分布在不同的服务器上,没有inter-keys使用到(比如SINTER,否则你需要注意要在同一台服务器上进行)
这是Redis不像memcached一样强制指定索引算法的原因,需要应用来指定。另外,有几个key访问的比较频繁。

特殊的Keys

比如每次发布新帖,我们都需要增加global:nextPostId。单台服务器会有大量增加的请求。如何修复这个问题呢?一个简单的办法是用一台专门的服务器来处理增加请求。
除非你有大量的请求,否则矫枉过正了。另一个小技巧是ID并不需要真正地增加,只要唯一即可。这样你可以使用长度为不太可能发生碰撞的随机字符串(除了MD5这样的大小,几乎是不可能)。
完工,我们成功消除了水平分割带来的问题。

另一个问题是global:timeline。这里有个不是解决办法的解决办法,你可以分别保存在不同服务器上,并且在需要这些数据时从不同的服务器上取出来,或者用一个key来进行排序。
如果你确实每秒有这么多帖子,你能够再次用一台独立服务器专门处理这些请求。请记住,商用硬件的Redis能够以100000/s的速度写入数据。我猜测对于twitter这足够了。
请随意在下面评论处提问以及反馈。

 

本文采用Creative Commons协议,复制本文需遵守三点:1、保留署名(链接);2、非商业性使用;3、再次创作的作品必须以相同的许可协议发布。法律顾问:庄毅雄律师

Redis作者谈Redis应用场景

毫无疑问,Redis开创了一种新的数据存储思路,使用Redis,我们不用在面对功能单调的数据库时,把精力放在如何把大象放进冰箱这样的问题上,而是利用Redis灵活多变的数据结构和数据操作,为不同的大象构建不同的冰箱。希望你喜欢这个比喻。

下面是一篇新鲜出炉的文章,其作者是Redis作者@antirez,他描述了Redis比较适合的一些应用场景,NoSQLFan简单列举在这里,供大家一览:

1.取最新N个数据的操作

比如典型的取你网站的最新文章,通过下面方式,我们可以将最新的5000条评论的ID放在Redis的List集合中,并将超出集合部分从数据库获取

  • 使用LPUSH latest.comments<ID>命令,向list集合中插入数据
  • 插入完成后再用LTRIM latest.comments 0 5000命令使其永远只保存最近5000个ID
  • 然后我们在客户端获取某一页评论时可以用下面的逻辑(伪代码)
FUNCTION get_latest_comments(start,num_items):
    id_list = redis.lrange("latest.comments",start,start+num_items-1)
    IF id_list.length < num_items
        id_list = SQL_DB("SELECT ... ORDER BY time LIMIT ...")
    END
    RETURN id_list
END

如果你还有不同的筛选维度,比如某个分类的最新N条,那么你可以再建一个按此分类的List,只存ID的话,Redis是非常高效的。

2.排行榜应用,取TOP N操作

这个需求与上面需求的不同之处在于,前面操作以时间为权重,这个是以某个条件为权重,比如按顶的次数排序,这时候就需要我们的sorted set出马了,将你要排序的值设置成sorted set的score,将具体的数据设置成相应的value,每次只需要执行一条ZADD命令即可。

3.需要精准设定过期时间的应用

比如你可以把上面说到的sorted set的score值设置成过期时间的时间戳,那么就可以简单地通过过期时间排序,定时清除过期数据了,不仅是清除Redis中的过期数据,你完全可以把Redis里这个过期时间当成是对数据库中数据的索引,用Redis来找出哪些数据需要过期删除,然后再精准地从数据库中删除相应的记录。

4.计数器应用

Redis的命令都是原子性的,你可以轻松地利用INCR,DECR命令来构建计数器系统。

5.Uniq操作,获取某段时间所有数据排重值

这个使用Redis的set数据结构最合适了,只需要不断地将数据往set中扔就行了,set意为集合,所以会自动排重。

6.实时系统,反垃圾系统

通过上面说到的set功能,你可以知道一个终端用户是否进行了某个操作,可以找到其操作的集合并进行分析统计对比等。没有做不到,只有想不到。

7.Pub/Sub构建实时消息系统

Redis的Pub/Sub系统可以构建实时的消息系统,比如很多用Pub/Sub构建的实时聊天系统的例子。

8.构建队列系统

使用list可以构建队列系统,使用sorted set甚至可以构建有优先级的队列系统。

9.缓存

这个不必说了,性能优于Memcached,数据结构更多样化。

from:http://blog.nosqlfan.com/html/2235.html?ref=rediszt

Document Databases : A look at them

DocumentDBs.zip : NOTE this code will not run unless you download and  reference the prerequisites mentioned in this article

 

Introduction

I don’t know how many of you work with SQL. Loads? Well I do, as we know it’s  a relational database which we can store um well relational SQL data types in,  such as INT/CHAR/NVARCHAR etc etc, I am sure you all know what I am talking  about and have used relational databases loads in the past.

Do you think there are other sort of databases out there? No? Well actually  there are some different types of databases other than relational databases,  such as

  • Flat file
  • Object
  • NoSQL / Document / KeyValue pair

Now I do not profess to know much about Flat file or Object databases per se,  but I have spent some time evaluating and getting to know some of the newer  Document databases. In this article I have chosen to look at 3 different  Document databases, which I have created demos for in the code attached to this  article, but before we go on to talk about each of them and how to get started  with them, let’s just spend a bit of time talking about the gaining popularity  of these document databases.

 

So I have stated that this article will talk about document databases, but  what are these document databases, and why might you want to use one.

The reason to use a document database may come from any number of  requirements, such as

  • A more accurate representation of your business model may be able to be expressed if you ditch the relational model
  • RESTFul API (though most users would try and find a native client for their language of choice)
  • Schema changes do not really matter as much as they would in a relational database, ad-hoc changes to the schema are supported

So those are some of the reasons, how about looking at some of the features  that a typical document database might minimally provide:

  • Http enabled server, which is capable of handling standard HTTP requests for data (think PUT/GET/POST/DELETE), so if there was no driver for your language of choice available you could always just use standard Http requests.
  • Documents are typically stored as some sort of serializable format (most typically this appears to be JSON or BSON)
  • The abilty to store an entire document. Yes I mean a document in comparison to a rich object, with methods inheritence heirachy etc etc. These are not present in a document database; code is not part of the database.
  • Sharding
  • Replication

One thing that is of note when working with document databases, is this idea  of “eventual consistency”, which is not what we might be used to seeing by and  large, in the relational world.

What is meant by this term, it sounds scary right. Well I guess it is a bit,  but when working with document databases, its seems a common approach that they  all follow, to allow you to push updates/inserts into the document store, but  that does not mean these will be changes will be nesecarrily seen by all readers  of the data straight away They will of course be written to to all reader  sources eventually, but not straight away. Which means we may occassionally see  inconsistencies.

Why is this, how has it come about.  It’s all about scaling and  availability really, if you only have a single source of data, a you do  typically in a relational database world, then you must lock reads while you are  writing. It’s a simple model but it remains totally consistent, but it will not  scale that well, some of sort of sharding must be used, which is not that common  in relational databases. In fact having never done that is a RDBMS I am not sure  its even possible, may be it is maybe not.

Anyway that is one area I thought I would alert you to straight away, I  found these links to be highy informative on this subject, should you want to  know more

  1. http://ayende.com/blog/4447/that-no-sql-thing (Ayende wrote RavenDB, so its worth a read)
  2. http://smoothspan.wordpress.com/2007/12/22/eventual-consistency-is-not-that-scary/

You may find that this alone is reason enough that document databases may not  be a good fit for you, but that is a decison only you can make

Anyway you do not need to be too concerned with these issues, as this article  will be more focussed on elementary usage of  document databases, such as  how to perform simple CRUD operations, I just thought it was worth mentioning  upfront, so you knew about it, so there we go you have been warned.

Now there are loads and loads of document databases out there, far too many  for me to go through, for my initial evaluations I chose to look at a few based  on what I perceived to be the best attributes, such as

  • Features
  • Ease of use
  • Reputation
  • .NET driver availability (I am a .NET developer these day, so this article is about using .NET with the relevant document database)

With those list of attributes, I ended up with quite a large list, which I  whittled down further to end up with 3 document databases, which I will talk  about in this article.

  1. Redis
  2. Raven
  3. Mongo

Do not expect to reach the end of this article and be an expert in document  databases, but I would hope by the end of reading this article you will be able  to understand how they work (kinda) and would be able to carry on using them and  finding any remaining answers you need by yourselves.

 

PreRequisites

Before we start you will need to make sure you have downloaded the relevant NoSQL server and .NET client APIs.  I would have liked to have upload them with this article but unfortunately they  are just too big for codeprojects limits, so this task must fall to you. So  shown below are the components you will need to download

Redis

For Redis you will need to download the following 3 items, and ensure the  correct .NET portions are references correctly

Server

  1. This can be downloaded from : http://redis.io/download
  2. Ensure you change the strings at the top of RedisServer.cs class within the DocumentDB.Redis project

.NET Client : Service Stack

  1. This can be downloaded from : https://github.com/ServiceStack/ServiceStack.Redis
  2. Once you have downloaded this ensure you fix all the references within the DocumentDB.Redis project

 

 

Raven

For Raven you will need to download the following 2 items, and ensure the  correct .NET portions are references correctly

Server

  1. This can be downloaded from : http://builds.hibernatingrhinos.com/builds/RavenDB
  2. Ensure you change the strings at the top of RavenServer.cs class within the DocumentDB.Raven project

.NET Client

  1. The .NET client actually comes as part of the overall download mentioned above
  2. Once you have downloaded Raven ensure you fix all the references within the DocumentDB.Raven project

 

MongoDB

For MongoDB you will need to download the following 2 items, and ensure the  correct .NET portions are references correctly

Server

  1. This can be downloaded from : http://www.mongodb.org/downloads
  2. Ensure you change the strings at the top of MongoDBServer.cs class within the DocumentDB.Mongo project

.NET Client (Mongo supported client)

  1. This can be downloaded from : https://github.com/mongodb/mongo-csharp-driver/downloads
  2. Once you have downloaded this ensure you fix all the references within the DocumentDB.Mongo project

 

     IMPORTANT NOTE : Once you have downloaded these items and put them  somewhere good, you will need to do the following

  1. Ensure each of the 3 projects within the downloadable demo code attached to this article, has its references fixed to point to where you downloaded the .NET client API Dlls to, which I am hoping you did when you following the steps above. If not make sure to do that now
  2. Within each of the 3 projects within the downloadable demo code attached to this article is a simple server wrapper which simply spawns the correct actual NoSQL server process. This is more for convience than anything else, and you will need to change the path to the actual NoSQL server location to match where you downloaded it to. This is done by changing a string value within each of the XXXXServer.cs classes in the 3 demo projects within the downloadable demo code attached to this article.

Just for the record, when I was developing the code for  this article I put the NoSQL servers/.NET Clients with the downloadable solution  something like shown below, but you put them where you want, and just make sure  to do the 2 steps above for all 3 projects in the demo code

Redis Document Database Usage

In this section I will discuss usingRedis. Redis commands are at the  core of Redis, where a full list of commands can be found athttp://redis.io/commands, here is screen  shot to show you the sort of thing I am talking about

This screen shot shows only a small portion of the available Redis commands.

Now even though this is how Redis works internally there is not really a need  for you to get to know these commands, as any Redis client will already be  calling these commands on your behalf when it talks to the server. I just  thought it might be useful to show you how redis works under the hood.

One other very import aspect of getting to know Redis is that fact that it is  designed to work with extremely dynamic datasets, as such it is really intended  that your entire dataset should fit into memory. This may sound mad, but it  really depends on the type of application you need to write.

Although Redis operates in memory it does have multiple disk persistance modes i.e Journalling and/or entire snapshots. See this link for more information on this http://redis.io/topics/persistence

If you want years and years or storage that you could bring back into memory at any time then  Redis may not be a good fit for you. On the other had is you have some very  dynamic fast moving data, that you could live with expiring after x-time, then  Redis would be a very good fit.

 

The Server

The Redis server (available for download here :http://redis.io/download) is witten in C and can be run using  the “redis-server.exe” process. In fact when you have downloaded  Redis server  you should see something like this

Where there are a number of different processes that can be used for managing  Redis server.

The .NET Client

There are quite a few Redis clients for all sorts of languages. This aricles  demo code uses .NET, so I obviously had to pick a .NET client, but which one.  Well that was down to me doing a bit of research and picking one, and it seemed  that ServiceStack seemed to be quite popular, so that is the one I chose. It can  be foud here: https://github.com/ServiceStack/ServiceStack.Redis

It would be  pretty much impossible for me to outline every feature of Reis, but I shall  outline what I think are the most important parts when getting started with  Redis.

1st Steps

You must have the Redis actual server running. I have tried to make this  easy for by creating a helper class called “RedisServer” which  you should modify to point to your Redis server downloads. Once the Redis actual  server is running we need to create a software RedisClient (which  connects to the actual Redis server instance).

Some skeleton code is shown below, that all the Redis code uses in the  attached demo code

Collapse | Copy Code
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using ServiceStack.Redis;

namespace DocumentDB.Redis
{
    public class Program
    {
        [STAThread]
        static void Main(string[] args)
        {
            Run();
        }

        public static void Run()
        {
            try
            {
                RedisClient redisClient = new RedisClient("localhost");

                if (RedisServer.Instance.Start())
                {
                   //Use the software RedisClient which talks to actual Redis server
                   //Use the software RedisClient which talks to actual Redis server
                   //Use the software RedisClient which talks to actual Redis server
                   //Use the software RedisClient which talks to actual Redis server
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine("============= OH NO : ERROR ============");
            }
            Console.ReadLine();

        }
    }
}

This software RedisClient is then used by the various classes in  the demo code, so you can expect to see the use of  a RedisClient object in use in the attached demo code

 

Basic CRUD Using Typed Objects

In order to show you how to use my chosen Redis client (Service Stack) all  you really need to know how to do is use an instance of a RedisClient which would typically be used as follows, to obtain a IRedisTypedClient

Collapse | Copy Code
private IList<Blog> Blogs()
{
    return redisClient.As<Blog>().GetAll();
}

Shown below are various listings that show how to carry out various operations using the  Service Stack RedisClient where the following document classes are used to store

Collapse | Copy Code
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace DocumentDB.Redis
{
    public class Blog
    {
        public Blog()
        {
            this.Tags = new List<string>();
            this.BlogPostIds = new List<long>();
        }

        public long Id { get; set; }
        public long UserId { get; set; }
        public string UserName { get; set; }
        public List<string> Tags { get; set; }
        public List<long> BlogPostIds { get; set; }
    }

    public class BlogPost
    {
        public BlogPost()
        {
            this.Categories = new List<string>();
            this.Tags = new List<string>();
        }

        public long Id { get; set; }
        public long BlogId { get; set; }
        public string Title { get; set; }
        public string Content { get; set; }
        public List<string> Categories { get; set; }
        public List<string> Tags { get; set; }
    }

    public class User
    {
        public User()
        {
            this.BlogIds = new List<long>();
        }

        public long Id { get; set; }
        public string Name { get; set; }
        public List<long> BlogIds { get; set; }
    }
}

Obtain Data

Collapse | Copy Code
private IList<Blog> Blogs()
{
    return redisClient.As<Blog>().GetAll();
}

private IList<User> Users()
{
    return redisClient.As<User>().GetAll();
}

Insert Data

Collapse | Copy Code
private void InsertSingleBlog(IRedisClient client)
{

    var redisUsers = client.As<User>();
    var redisBlogs = client.As<Blog>();
    var redisBlogPosts = client.As<BlogPost>();
    var ayende = new User { Id = redisUsers.GetNextSequence(), Name = "Oren Eini" };

    var ayendeBlog = new Blog
    {
        Id = redisBlogs.GetNextSequence(),
        UserId = ayende.Id,
        UserName = ayende.Name,
        Tags = new List<string> { "Architecture", ".NET", "Databases" },
    };

    var blogPosts = new List<BlogPost>
	{
		new BlogPost
		{
			Id = redisBlogPosts.GetNextSequence(),
			BlogId = ayendeBlog.Id,
			Title = "RavenDB",
			Categories = new List<string> { "NoSQL", "DocumentDB" },
			Tags = new List<string> {"Raven", "NoSQL", "JSON", ".NET"}
		}
	};

    ayende.BlogIds.Add(ayendeBlog.Id);
    ayendeBlog.BlogPostIds.AddRange(blogPosts.Where(x => x.BlogId == ayendeBlog.Id).ConvertAll(x => x.Id));

    redisUsers.Store(ayende);
    redisBlogs.StoreAll(new[] { ayendeBlog });
    redisBlogPosts.StoreAll(blogPosts);

}

It can be see that using the IRedisClient we are able to use batch operations such as StoreAll which makes things easy for us

Delete Data

Collapse | Copy Code
private void DeleteSpecificBlog(long blogId)
{
    Console.WriteLine("DELETING SINGLE Blog\r\n");
    var redisBlogs = redisClient.As<Blog>();
    redisBlogs.DeleteById(blogId);
}

An important IMPORTANT note here is that the Save() method here is not quite what it seems. This actually does a foreground/synchronous snapshot save to disk – you generally never want to do this in Production. You would be better of doing a BGSAVE (background save) for Production environments

Linq Support

As I have already stated on numerous occassions Redis works with in memory  datasets, that is the entire dataset MUST fit into memory as such it exposes  numerous collection classes for managing the in memory store. Within the Service  Stack Redis client these collections are typically managed using standard .NET  collection classes, as such any of the standard LINQ extension methods may be  applied straight to the collections. For example here is where I get a List<Blog> items, and filter it using LINQ

Collapse | Copy Code
private IList<Blog> Blogs()
{
    IList<Blog> blogs = new List<Blog>();
    var redisBlogs = redisClient.As<Blog>())
    blogs = redisBlogs.GetAll().Where(x => x.Id > 20).ToList();
    return blogs;
}

It should of course be noted that since Redis works with a in memory dataset, it is not using a true IQueryProvider, and it really just using Linq to objects, there is no lazy loading occurring in the database. So just be mindful of this fact

 

Transaction Support

Within the Service Stack Redis client, transactions are managed by obtaining  a new IRedisTransaction of the IRedisClient by using  the CreateTransaction() method, after you have a IRedisTransaction you can simply use the following methods to work with  the IRedisTransaction

  • QueueCommand(..) : Will inlist the contained command in the transaction
  • Rollback() : Will rollback the transaction
  • Commit() : Will commit the transaction

Anyway here is some demo code

Collapse | Copy Code
public void InsertInsideTransaction(bool shouldTransactionRollback)
{
    RedisClient transClient = new RedisClient("localhost");

    ClearAll();
    using (var trans = transClient.CreateTransaction())
    {
        var redisUsers = redisClient.As<User>();

        //Have to do this here (as redisUsers.GetNextSequence() is a READ, which MUST be done before
        //we write using the RedisTransaction)
        var sacha = new User { Id = redisUsers.GetNextSequence(), Name = "Sacha Barber" };

        trans.QueueCommand(r =>
            {
                using (redisUsers = r.As<User>())
                {
                    redisUsers.Store(sacha);
                }
            });

        //commit or rollback based on incoming flag
        if (shouldTransactionRollback)
            trans.Rollback();
        else
            trans.Commit();

        IList<User> users = Users();
        Console.WriteLine(string.Format("InsertInsideTransaction : There are currently {0}, Users", users.Count()));
    }
}

 

Cache Expiry

Sorry to harp on and on about this, but the point that makes Redis so fast is  that it works with the dataset entirely in memory. So what happens to data you  no need, is there a way to delete is, well yeah we could do it programatically,  but is there a way of use Redis as some sort of Most Recently Used (MRU) cache  where data can expire on its own?

Turns out this is possible and what we need to do is use one of the standard  Redis Service Stack typed client methods ExpireIn which has a  method signature which looks like this

Collapse | Copy Code
bool ExpireIn(object id, TimeSpan expiresAt);

To see this in action I have provided a bit of code in the demo which is as follows

Collapse | Copy Code
public void CacheInsertWhichDeletes()
{
    ClearAll();

    var redisUsers = redisClient.As<User>();
    var frankenFurter = new User { Id = redisUsers.GetNextSequence(), Name = "FrankenFurter" };
    redisUsers.Store(frankenFurter);
    redisUsers.ExpireIn(frankenFurter.Id, TimeSpan.FromSeconds(1));

    User fetchedUser = redisUsers.GetById(frankenFurter.Id);
    Console.WriteLine(fetchedUser != null ? "Still exists" : "Removed from cache");
    Thread.Sleep(2000);
    fetchedUser = redisUsers.GetById(frankenFurter.Id);
    Console.WriteLine(fetchedUser != null ? "Still exists" : "Removed from cache");
}

So that is how you might manage fast moving data that you only want to live for x amount of time.

 

Pooling

Service Stack (The Redis Client I chose to use) does offer connection pooling  via the use of 2 classes PooledRedisClientManager and any class of  your own that implements the Servce Stack interface IRedisClientFactory

The following code declared multiple client addresses

Collapse | Copy Code
readonly string[] testReadWriteHosts = new[] {
	"readwrite1", "readwrite2:6000", "192.168.0.1", "localhost"
};

readonly string[] testReadOnlyHosts = new[] {
	"read1", "read2:7000", "127.0.0.1"
};

Where we then proceed to create a PooledRedisClientManager like this

Collapse | Copy Code
private PooledRedisClientManager CreateAndStartManager()
{
    var manager = CreateManager();
    manager.Start();
    return manager;
}

private PooledRedisClientManager CreateManager()
{
    return CreateManager(new RedisClientFactory(), testReadWriteHosts, testReadOnlyHosts);
}

private PooledRedisClientManager CreateManager(
    IRedisClientFactory clientFactory, string[] readWriteHosts, string[] readOnlyHosts)
{
    return new PooledRedisClientManager(readWriteHosts, readOnlyHosts,
        new RedisClientManagerConfig
        {
            MaxWritePoolSize = readWriteHosts.Length,
            MaxReadPoolSize = readOnlyHosts.Length,
            AutoStart = false,
        })
    {
        RedisClientFactory = clientFactory
    };
}

Where it can be seen that we set a RedisClientFactory to an instance of IRedisClientFactory.  So what does that look like well it looks like this

Collapse | Copy Code
public class RedisClientFactory : IRedisClientFactory
{
    public RedisClient CreateRedisClient(string host, int port)
    {
        return new RedisClient(host, port);
    }
}

After following these steps all that is left to do is spin up some and use a new PooledRedisClientManager. We get access to a client via the use of the PooledRedisClientManager. In the demo code I spin up new clients in new threads to simulate concurrent access to the connection pool, here is the relevant code

Collapse | Copy Code
/// <summary>
/// Use the PooledRedisClientManager to gain access to n-many clients
/// </summary>
public void Start()
{
    Thread t = new Thread((state) =>
        {

            const int noOfConcurrentClients = 5; //WaitHandle.WaitAll limit is <= 64
            var clientUsageMap = new Dictionary<string, int>();

            var clientAsyncResults = new List<IAsyncResult>();
            using (var manager = CreateAndStartManager())
            {
                for (var i = 0; i < noOfConcurrentClients; i++)
                {
                    var clientNo = i;
                    var action = (Action)(() => UseClient(manager, clientNo, clientUsageMap));
                    clientAsyncResults.Add(action.BeginInvoke(null, null));
                }
            }

            WaitHandle.WaitAll(clientAsyncResults.ConvertAll(x => x.AsyncWaitHandle).ToArray());

            Console.WriteLine(TypeSerializer.SerializeToString(clientUsageMap));

            var hostCount = 0;
            foreach (var entry in clientUsageMap)
            {
                hostCount += entry.Value;
            }

        });
    t.SetApartmentState(ApartmentState.MTA);
    t.Start();
}

private static void UseClient(IRedisClientsManager manager, int clientNo,
    Dictionary<string, int> hostCountMap)
{
    using (IRedisClient client = manager.GetClient())
    {
        lock (hostCountMap)
        {
            int hostCount;
            if (!hostCountMap.TryGetValue(client.Host, out hostCount))
            {
                hostCount = 0;
            }
            hostCountMap[client.Host] = ++hostCount;
        }

        Console.WriteLine("Client '{0}' is using '{1}'", clientNo, client.Host);

        //YOU COULD USE THE SPECIFIC CLIENT HERE, YOU MAY HAVE TO TEST THE HOST TO SEE IF ITS THE ACTUAL ONE YOU WANT
        //YOU COULD USE THE SPECIFIC CLIENT HERE, YOU MAY HAVE TO TEST THE HOST TO SEE IF ITS THE ACTUAL ONE YOU WANT
        //YOU COULD USE THE SPECIFIC CLIENT HERE, YOU MAY HAVE TO TEST THE HOST TO SEE IF ITS THE ACTUAL ONE YOU WANT
        //YOU COULD USE THE SPECIFIC CLIENT HERE, YOU MAY HAVE TO TEST THE HOST TO SEE IF ITS THE ACTUAL ONE YOU WANT
    }
}

It is slightly backward in its usage in that you must ask the PooledRedisClientManager for a IRedisClient where it will use the IRedisClientFactory you  provided, and just give you a IRedisClient which you can use. But  which IRedisClient you get is up to the PooledRedisClientManager.  So if you are relying on it being a specific IRedisClient guess  again, you will need to check which IRedisClient has been dished  out by the PooledRedisClientManager.

 

 

Redis Admin UI

There is an Admin UI (which runs on .NET/Mono) which also includes JSON, XML, JSV & SOAP services for all Redis operations at: https://github.com/ServiceStack/ServiceStack.RedisWebServices/

 

Raven Document Database Usage

In this section I will discuss usingRaven.

There are a couple of points that are worth note before we start to look at  using Raven, so lets  give these a quick bit of discussion right now shall we:

  1. Raven is written entirely in .NET, yes even the Server is .NET. I have seen plenty of chat/internet noise about this, and people saying that it would not be fast enough for high volume dataset demands. To be frank I am not in a position to say for sure whether this is the case or not, as I was just in an evaluation mode whilst looking at various different document databases. What I can say though is that for my evaluations I found no issues at all
  2. Raven has a concept of denying unbounded results sets, so if you try and bring back to much data Raven will step in and not allow that. This is a setting that can be changed but it is not encouraged
  3. Raven borrows ideas from other well known frameworks, mainly NHibernate, so when you see a IDocumentSession this should seem pretty familiar and almost be the same to use as ISession was/is in NHibernate.
  4. The commercial version of Raven is not free, but its not that much if it fits your needs

 

The Server

The Raven server (available for download here : http://ravendb.net/download) is witten in C# and can be run using  the “Raven.Server.exe” process. In fact when you have downloaded  Raven server  you should see something like this

Where there are a number of different processes that can be used for managing  Raven server.

The .NET Client

There is only one .NET client for Raven which is the one available on the  download page: http://ravendb.net/download

It would be  pretty much impossible for me to outline every feature of Raven, but I shall  outline what I think are the most important parts when getting started with  Raven.

1st Steps

You must have the Raven actual server running. I have tried to make this  easy for by creating a helper class called “RavenServer” which  you should modify to point to your Raven server downloads. Once the Raven actual  server is running we need to create a software DocumentStore (which  connects to the actual Raven server instance).

Some skeleton code is shown below, that all the Raven code uses in the  attached demo code

Collapse | Copy Code
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Raven.Client.Document;
using Raven.Client;
using System.IO;

namespace DocumentDB.Raven
{
    class Program
    {
        public static string documentStoreLocation = @"http://localhost:8080";

        [STAThread]
        static void Main(string[] args)
        {
            Run();
        }

        public static void Run()
        {
            try
            {
                if (RavenServer.Instance.Start())
                {
                    using (var documentStore = new DocumentStore { Url = documentStoreLocation, DefaultDatabase = "ravenTest" })
                    {
			//Use DocumentStore here
			//Use DocumentStore here
			//Use DocumentStore here
			//Use DocumentStore here
			//Use DocumentStore here
			//Use DocumentStore here
                    }
                }

            }
            catch (Exception ex)
            {
                Console.WriteLine("============= OH NO : ERROR ============");
            }
            Console.ReadLine();
        }
    }
}

This software DocumentStore is then used by the various classes in  the demo code, so you can expect to see the use of  a DocumentStore object in use in the attached demo code

 

Basic CRUD Using Typed Objects

In order to show you how to use my chosen Raven client all  you really need to know how to do is use an instance of a IDocumentSession which would typically be used as follows, where a Query is run using a  generic type of the Document that you would like to obtain data for

Collapse | Copy Code
private IList<Blog> Blogs()
{
    IList<Blog> blogs = new List<Blog>();
    using (IDocumentSession session = documentStore.OpenSession())
    {
        blogs = session.Query<Blog>().ToList();
    }
    return blogs;
}

Shown below are various listings that show how to carry out various operations using the  Service Stack RedisClient where the following document classes are used to store

Collapse | Copy Code
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace DocumentDB.Raven
{
    public class Blog
    {
        public Blog()
        {
            this.Tags = new List<string>();
            this.BlogPostIds = new List<long>();
        }

        public long Id { get; set; }
        public long UserId { get; set; }
        public string UserName { get; set; }
        public List<string> Tags { get; set; }
        public List<long> BlogPostIds { get; set; }
    }

    public class BlogPost
    {
        public BlogPost()
        {
            this.Categories = new List<string>();
            this.Tags = new List<string>();
        }

        public long Id { get; set; }
        public long BlogId { get; set; }
        public string Title { get; set; }
        public string Content { get; set; }
        public List<string> Categories { get; set; }a
        public List<string> Tags { get; set; }
    }

    public class User
    {
        public User()
        {
            this.BlogIds = new List<long>();
        }

        public long Id { get; set; }
        public string Name { get; set; }
        public List<long> BlogIds { get; set; }
    }

}

Obtain Data

Collapse | Copy Code
private IList<Blog> Blogs()
{
    IList<Blog> blogs = new List<Blog>();
    using (IDocumentSession session = documentStore.OpenSession())
    {
        blogs = session.Query<Blog>().ToList();
    }
    return blogs;
}

private IList<User> Users()
{
    IList<User> users = new List<User>();
    using (IDocumentSession session = documentStore.OpenSession())
    {
        users = session.Query<User>().ToList();
    }

    return users;
}

Insert Data

Collapse | Copy Code
public void InsertTestData()
{

    using (IDocumentSession session = documentStore.OpenSession())
    {
        var ayende = new User { Name = "Oren Eini" };
        var mythz = new User { Name = "Demis Bellot" };

        var ayendeBlog = new Blog
        {
            UserId = ayende.Id,
            UserName = ayende.Name,
            Tags = new List<string> { "Architecture", ".NET", "Databases" },
        };

        var mythzBlog = new Blog
        {
            UserId = mythz.Id,
            UserName = mythz.Name,
            Tags = new List<string> { "Architecture", ".NET", "Databases" },
        };

        session.Store(ayende);
        session.Store(mythz);
        session.Store(ayendeBlog);
        session.Store(mythzBlog);
        session.SaveChanges();

        var mythzBlogPosts = new List<BlogPost>
		{
			new BlogPost
			{
				BlogId = mythzBlog.Id,
				Title = "Redis",
				Categories = new List<string> { "NoSQL", "Cache" },
				Tags = new List<string> {"Redis", "NoSQL", "Scalability", "Performance"},
			},
			new BlogPost
			{
				BlogId = mythzBlog.Id,
				Title = "Couch Db",
				Categories = new List<string> { "NoSQL", "DocumentDB" },
				Tags = new List<string> {"CouchDb", "NoSQL", "JSON"},
			}
        };

        var ayendeBlogPosts = new List<BlogPost>
		{
			new BlogPost
			{
				BlogId = ayendeBlog.Id,
				Title = "RavenDB",
				Categories = new List<string> { "NoSQL", "DocumentDB" },
				Tags = new List<string> {"Raven", "NoSQL", "JSON", ".NET"} ,
			},
			new BlogPost
			{
				BlogId = ayendeBlog.Id,
				Title = "Cassandra",
				Categories = new List<string> { "NoSQL", "Cluster" },
				Tags = new List<string> {"Cassandra", "NoSQL", "Scalability", "Hashing"},
			}
        };

        foreach (BlogPost blogPost in ayendeBlogPosts.Union(mythzBlogPosts))
        {
            session.Store(blogPost);
        }

        session.SaveChanges();

        ayende.BlogIds.Add(ayendeBlog.Id);
        ayendeBlog.BlogPostIds.AddRange(ayendeBlogPosts.Select(x => x.Id));
        mythz.BlogIds.Add(mythzBlog.Id);
        mythzBlog.BlogPostIds.AddRange(mythzBlogPosts.Select(x => x.Id));

        session.Store(ayende);
        session.Store(mythz);
        session.Store(ayendeBlog);
        session.Store(mythzBlog);
        session.SaveChanges();

    }
}

 

Delete Data

Collapse | Copy Code
private void DeleteSpecificBlog(long blogId)
{
    Console.WriteLine("DELETING SINGLE Blog\r\n");
    using (IDocumentSession session = documentStore.OpenSession())
    {
        session.Delete<Blog>(session.Query<Blog>().Where(x => x.Id == blogId).Single());
        session.SaveChanges();
    }
}

 

 

LINQ Support

As Raven is built entirely in .NET it would be pretty strange if it did not  support LINQ, and low and behold it does. For example here is where I get a List<Blog> items, and filter it using LINQ

Collapse | Copy Code
private IList<Blog> Blogs()
{
    IList<Blog> blogs = new List<Blog>();
    using (IDocumentSession session = documentStore.OpenSession())
    {
        blogs = session.Query<Blog>().ToList();
    }

    return blogs;
}

 

Transaction Support

Raven supports transactions whole heartedly, and as Raven is all written in  .NET you even get to use familiar transaction classes such as TransactionScope, which I have to say does make life easier. Here is an  example of how to use Transactions with Raven.

Collapse | Copy Code
public void InsertInsideTransaction(bool shouldTransactionRollback)
{

    var users = new List<User>();

    using (IDocumentSession session = documentStore.OpenSession())
    {
        try
        {
            users = session.Query<User>().ToList();
            Console.WriteLine(string.Format("Before Transaction : There are currently {0}, Users", users.Count()));

            using (var transaction = new TransactionScope())
            {
                var sacha = new User { Name = "Sacha Barber" };
                session.Store(sacha);
                session.SaveChanges();

                if (shouldTransactionRollback)
                    throw new InvalidOperationException("testing transactions");

                transaction.Complete();

            }
            users = session.Query<User>().ToList();
            Console.WriteLine(string.Format("After Transaction : There are currently {0}, Users", users.Count()));

        }
        catch
        {
            users = session.Query<User>().ToList();
            Console.WriteLine(string.Format("On Transaction Error : There are currently {0}, Users", users.Count()));
        }
    }
}

 

 

Direct Database Operations

Sometimes you just need direct access to the underlying Raven database  commands. In Raven this is done using the DocumentStore.DatabaseCommands  property which will give you an instance of a IDatabaseCommands,  which allows you to carry out the various tasks. Shown below is the IDatabaseCommandsinterface definition straight from Raven, which shows you what sort of things you can do with aIDatabaseCommands instance

Collapse | Copy Code
using Raven.Abstractions.Commands;
using Raven.Abstractions.Data;
using Raven.Abstractions.Indexing;
using Raven.Client.Connection.Profiling;
using Raven.Client.Indexes;
using Raven.Json.Linq;
using System;
using System.Collections.Generic;
using System.Collections.Specialized;
using System.IO;
using System.Net;

namespace Raven.Client.Connection
{
    // Summary:
    //     Expose the set of operations by the RavenDB server
    public interface IDatabaseCommands : IHoldProfilingInformation
    {
        // Summary:
        //     Gets or sets the operations headers
        NameValueCollection OperationsHeaders { get; set; }
        //
        // Summary:
        //     Gets a value indicating whether [supports promotable transactions].
        bool SupportsPromotableTransactions { get; }

        // Summary:
        //     Executed the specified commands as a single batch
        //
        // Parameters:
        //   commandDatas:
        //     The command data.
        BatchResult[] Batch(IEnumerable<ICommandData> commandDatas);
        //
        // Summary:
        //     Commits the specified tx id
        //
        // Parameters:
        //   txId:
        //     The tx id.
        void Commit(Guid txId);
        //
        // Summary:
        //     Deletes the document with the specified key
        //
        // Parameters:
        //   key:
        //     The key.
        //
        //   etag:
        //     The etag.
        void Delete(string key, Guid? etag);
        //
        // Summary:
        //     Deletes the attachment with the specified key
        //
        // Parameters:
        //   key:
        //     The key.
        //
        //   etag:
        //     The etag.
        void DeleteAttachment(string key, Guid? etag);
        //
        // Summary:
        //     Perform a set based deletes using the specified index, not allowing the operation
        //     if the index is stale
        //
        // Parameters:
        //   indexName:
        //     Name of the index.
        //
        //   queryToDelete:
        //     The query to delete.
        void DeleteByIndex(string indexName, IndexQuery queryToDelete);
        //
        // Summary:
        //     Perform a set based deletes using the specified index
        //
        // Parameters:
        //   indexName:
        //     Name of the index.
        //
        //   queryToDelete:
        //     The query to delete.
        //
        //   allowStale:
        //     if set to true [allow stale].
        void DeleteByIndex(string indexName, IndexQuery queryToDelete, bool allowStale);
        //
        // Summary:
        //     Deletes the specified index
        //
        // Parameters:
        //   name:
        //     The name.
        void DeleteIndex(string name);
        //
        // Summary:
        //     Disable all caching within the given scope
        IDisposable DisableAllCaching();
        //
        // Summary:
        //     Create a new instance of Raven.Client.Connection.IDatabaseCommands that will
        //     interacts with the specified database
        IDatabaseCommands ForDatabase(string database);
        //
        // Summary:
        //     Create a new instance of Raven.Client.Connection.IDatabaseCommands that will
        //     interacts with the default database
        IDatabaseCommands ForDefaultDatabase();
        //
        // Summary:
        //     Retrieves the document for the specified key
        //
        // Parameters:
        //   key:
        //     The key.
        JsonDocument Get(string key);
        //
        // Summary:
        //     Retrieves documents with the specified ids, optionally specifying includes
        //     to fetch along
        //
        // Parameters:
        //   ids:
        //     The ids.
        //
        //   includes:
        //     The includes.
        MultiLoadResult Get(string[] ids, string[] includes);
        //
        // Summary:
        //     Retrieves the attachment with the specified key
        //
        // Parameters:
        //   key:
        //     The key.
        Attachment GetAttachment(string key);
        //
        // Summary:
        //     Returns the names of all tenant databases on the RavenDB server
        //
        // Returns:
        //     List of tenant database names
        string[] GetDatabaseNames();
        //
        // Summary:
        //     Using the given Index, calculate the facets as per the specified doc
        IDictionary<string, IEnumerable<FacetValue>> GetFacets(string index, IndexQuery query, string facetSetupDoc);
        //
        // Summary:
        //     Gets the index definition for the specified name
        //
        // Parameters:
        //   name:
        //     The name.
        IndexDefinition GetIndex(string name);
        //
        // Summary:
        //     Returns the names of all indexes that exist on the server
        //
        // Parameters:
        //   start:
        //     Paging start
        //
        //   pageSize:
        //     Size of the page.
        string[] GetIndexNames(int start, int pageSize);
        //
        // Summary:
        //     Create a new instance of Raven.Client.Connection.IDatabaseCommands that will
        //     interact with the root database. Useful if the database has works against
        //     a tenant database
        IDatabaseCommands GetRootDatabase();
        //
        // Summary:
        //     Retrieve the statistics for the database
        DatabaseStatistics GetStatistics();
        //
        // Summary:
        //     Get the all terms stored in the index for the specified field You can page
        //     through the results by use fromValue parameter as the starting point for
        //     the next query
        IEnumerable<string> GetTerms(string index, string field, string fromValue, int pageSize);
        //
        // Summary:
        //     Retrieves the document metadata for the specified document key.
        //
        // Parameters:
        //   key:
        //     The key.
        //
        // Returns:
        //     The document metadata for the specifed document, or null if the document
        //     does not exist
        JsonDocumentMetadata Head(string key);
        //
        // Summary:
        //     Perform a single POST requst containing multiple nested GET requests
        GetResponse[] MultiGet(GetRequest[] requests);
        //
        // Summary:
        //     Sends a patch request for a specific document, ignoring the document's Etag
        //
        // Parameters:
        //   key:
        //     Id of the document to patch
        //
        //   patches:
        //     Array of patch requests
        void Patch(string key, PatchRequest[] patches);
        //
        // Summary:
        //     Sends a patch request for a specific document
        //
        // Parameters:
        //   key:
        //     Id of the document to patch
        //
        //   patches:
        //     Array of patch requests
        //
        //   etag:
        //     Require specific Etag [null to ignore]
        void Patch(string key, PatchRequest[] patches, Guid? etag);
        //
        // Summary:
        //     Promotes the transaction
        //
        // Parameters:
        //   fromTxId:
        //     From tx id.
        byte[] PromoteTransaction(Guid fromTxId);
        //
        // Summary:
        //     Puts the document in the database with the specified key
        //
        // Parameters:
        //   key:
        //     The key.
        //
        //   etag:
        //     The etag.
        //
        //   document:
        //     The document.
        //
        //   metadata:
        //     The metadata.
        PutResult Put(string key, Guid? etag, RavenJObject document, RavenJObject metadata);
        //
        // Summary:
        //     Puts a byte array as attachment with the specified key
        //
        // Parameters:
        //   key:
        //     The key.
        //
        //   etag:
        //     The etag.
        //
        //   data:
        //     The data.
        //
        //   metadata:
        //     The metadata.
        void PutAttachment(string key, Guid? etag, Stream data, RavenJObject metadata);
        //
        // Summary:
        //     Creates an index with the specified name, based on an index definition
        //
        // Parameters:
        //   name:
        //     The name.
        //
        //   indexDef:
        //     The index def.
        string PutIndex(string name, IndexDefinition indexDef);
        //
        // Summary:
        //     Creates an index with the specified name, based on an index definition that
        //     is created by the supplied IndexDefinitionBuilder
        //
        // Parameters:
        //   name:
        //     The name.
        //
        //   indexDef:
        //     The index def.
        //
        // Type parameters:
        //   TDocument:
        //     The type of the document.
        //
        //   TReduceResult:
        //     The type of the reduce result.
        string PutIndex<TDocument, TReduceResult>(string name, IndexDefinitionBuilder<TDocument, TReduceResult> indexDef);
        //
        // Summary:
        //     Creates an index with the specified name, based on an index definition
        //
        // Parameters:
        //   name:
        //     The name.
        //
        //   indexDef:
        //     The index def.
        //
        //   overwrite:
        //     if set to true [overwrite].
        string PutIndex(string name, IndexDefinition indexDef, bool overwrite);
        //
        // Summary:
        //     Creates an index with the specified name, based on an index definition that
        //     is created by the supplied IndexDefinitionBuilder
        //
        // Parameters:
        //   name:
        //     The name.
        //
        //   indexDef:
        //     The index def.
        //
        //   overwrite:
        //     if set to true [overwrite].
        //
        // Type parameters:
        //   TDocument:
        //     The type of the document.
        //
        //   TReduceResult:
        //     The type of the reduce result.
        string PutIndex<TDocument, TReduceResult>(string name, IndexDefinitionBuilder<TDocument, TReduceResult> indexDef, bool overwrite);
        //
        // Summary:
        //     Queries the specified index in the Raven flavoured Lucene query syntax
        //
        // Parameters:
        //   index:
        //     The index.
        //
        //   query:
        //     The query.
        //
        //   includes:
        //     The includes.
        QueryResult Query(string index, IndexQuery query, string[] includes);
        //
        // Summary:
        //     Resets the specified index
        //
        // Parameters:
        //   name:
        //     The name.
        void ResetIndex(string name);
        //
        // Summary:
        //     Rollbacks the specified tx id
        //
        // Parameters:
        //   txId:
        //     The tx id.
        void Rollback(Guid txId);
        //
        // Summary:
        //     Retrieves documents for the specified key prefix
        JsonDocument[] StartsWith(string keyPrefix, int start, int pageSize);
        //
        // Summary:
        //     Stores the recovery information
        //
        // Parameters:
        //   resourceManagerId:
        //     The resource manager Id for this transaction
        //
        //   txId:
        //     The tx id.
        //
        //   recoveryInformation:
        //     The recovery information.
        void StoreRecoveryInformation(Guid resourceManagerId, Guid txId, byte[] recoveryInformation);
        //
        // Summary:
        //     Returns a list of suggestions based on the specified suggestion query
        //
        // Parameters:
        //   index:
        //     The index to query for suggestions
        //
        //   suggestionQuery:
        //     The suggestion query.
        SuggestionQueryResult Suggest(string index, SuggestionQuery suggestionQuery);
        //
        // Summary:
        //     Perform a set based update using the specified index, not allowing the operation
        //     if the index is stale
        //
        // Parameters:
        //   indexName:
        //     Name of the index.
        //
        //   queryToUpdate:
        //     The query to update.
        //
        //   patchRequests:
        //     The patch requests.
        void UpdateByIndex(string indexName, IndexQuery queryToUpdate, PatchRequest[] patchRequests);
        //
        // Summary:
        //     Perform a set based update using the specified index
        //
        // Parameters:
        //   indexName:
        //     Name of the index.
        //
        //   queryToUpdate:
        //     The query to update.
        //
        //   patchRequests:
        //     The patch requests.
        //
        //   allowStale:
        //     if set to true [allow stale].
        void UpdateByIndex(string indexName, IndexQuery queryToUpdate, PatchRequest[] patchRequests, bool allowStale);
        //
        // Summary:
        //     Returns a new Raven.Client.Connection.IDatabaseCommands using the specified
        //     credentials
        //
        // Parameters:
        //   credentialsForSession:
        //     The credentials for session.
        IDatabaseCommands With(ICredentials credentialsForSession);
    }
}

Here is a small example of using it to add an index

Collapse | Copy Code
this.documentStore.DatabaseCommands.PutIndex("Users/ByName",
                            new IndexDefinitionBuilder<User>
                            {
                                Map = users => from user in users
                                                select new { user.Name }
                            });

Full Text Search

One of the more interesting things that Raven allows you to do is to do full  text searches. To do this we would typically create an Index where we supply a  Map LINQ Exrpression to build the Index. Here is an example

Collapse | Copy Code
public class User_ByName_FullTextSearch : AbstractIndexCreationTask<User>
{
    public User_ByName_FullTextSearch()
    {
        Map = users => from user in users
                        select new { user.Name };
        Index(x => x.Name, FieldIndexing.Analyzed);

    }

    public override string IndexName
    {
        get
        {
            return @"Users\ByNameIndex";
        }
    }
}

We would could then use this Index in a simple or complex manner as follows

Simple Case

Collapse | Copy Code
Console.WriteLine(string.Format("Looking for users with Name starting with '{0}'\r\n", searchName));

Console.WriteLine("Simple starts with example:");
foreach (var person in Queryable.Where(session.Query<User, User_ByName_FullTextSearch>(), x => x.Name.StartsWith(searchName)))
{
    Console.WriteLine(person.Name);
}

Complex Case

Collapse | Copy Code
Console.WriteLine(string.Format("Looking for users with Name starting with '{0}'\r\n", searchName));

Console.WriteLine("Simple starts with example:");
IQueryable<User> query = session.Query<User, User_ByName_FullTextSearch>();
query = searchName.Split().Aggregate(query, (current, part) => current.Where(x => x.Name.StartsWith(part)));

foreach (var person in query)
{
    Console.WriteLine(person.Name);
}

Where we would see the following results

 

 

Batch Operations

This section will outline how to carry out batch operations using Raven.

Bulk Updates

Sometimes you may need to update/delete a whole batch worth of data. Raven  supports these type of operations by using what it called “set based  operations”. To do “set based operations” in Raven you have to use its “patching  api”. We will now see some examples of how to use Ravens “patching api”.

As the heart of Ravens patching API are 2 objects that you will need to get  familiar with, which are

  • PatchCommandData : A single batch operation for Document patch
  • PatchRequest : A patch request for a specified Document

Here is an example of how to do a bulk update

Collapse | Copy Code
public void DoUpdate(int numUsers)
{

    this.documentStore.DatabaseCommands.Batch(

        //patch 1st 1/2 of users
        Enumerable.Range(0, numUsers / 2).Select(i => new PatchCommandData
            {
                Key = "users/" + i,
                Patches = new[]
                {
                new PatchRequest
                {
                    Name = "Name",
                    Value = "Users-" + i
                },
            }
            }
    ).ToArray());

    this.documentStore.DatabaseCommands.Batch(

        //patch 1st 1/2 of users
        Enumerable.Range(numUsers / 2, numUsers).Select(i => new PatchCommandData
        {
            Key = "users/" + i,
            Patches = new[]
                {
                new PatchRequest
                {
                    Name = "Name",
                    Value = "Users-" + i
                },
            }
        }
    ).ToArray());

    using (IDocumentSession session = documentStore.OpenSession())
    {
        var users = session.Query<User>()
                        .Customize(x => x.WaitForNonStaleResults(TimeSpan.FromSeconds(10)))
                        .ToList();

        for (int i = 0; i < numUsers; i++)
        {
            Console.WriteLine("Expecting UserName of '{0}', actual UserName now is '{1}'", "Users-"
                + (i + 1), users[i].Name);
        }
    }

}

This code relies on there being an index which specifies an Index (which Raven builds using LINQ queries) to put a “UsersByName” index in place that thePatchRequest can use when its doing its bulk update to obtain the correct document.

Collapse | Copy Code
this.documentStore.DatabaseCommands.PutIndex("Users/ByName",
                            new IndexDefinitionBuilder<User>
                            {
                                Map = users => from user in users
                                                select new { user.Name }
                            });

When run this code produces uutput something like this

 

Bulk Delete

To do a bulk delete we simply need an index in place, such the one we used  previously for the bulk update

Collapse | Copy Code
this.documentStore.DatabaseCommands.PutIndex("Users/ByName",
                            new IndexDefinitionBuilder<User>
                            {
                                Map = users => from user in users
                                                select new { user.Name }
                            });

Then we can simply issue a DatabaseCommand to DeleteByIndex where we specify our Index string, and we also  supply an IndexQuery to pick out the documents we want to match,  which will then be deleted.

Collapse | Copy Code
this.documentStore.DatabaseCommands.DeleteByIndex("Users/ByName",
    new IndexQuery
    {
        Query = "Name:*" // where entity.Name contains anything
    }, allowStale: false);

 

 

The Studio

Raven comes with a Sliverlight web client application called “The Studio”  which is a graphical front end tool that can be used to crry out various tasks  such as

  • Manage collections/Indexes/Documents
  • Edit Document
  • Create Document
  • Import Database (which uses the Smuggler.exe tool)
  • Export Database (which uses the Smuggler.exe tool)

Here is a screen shot of a working Raven Studio instance reflecting some of  the demo code (click it for a bigger image)

 

 

 

MongoDB Document Database Usage

In this section I will discuss usingMongoDB

The Server

The MongoDB server (available for download here : http://www.mongodb.org/downloads) is witten in C++ and can be run using  the “mongod.exe” process. In fact when you have downloaded MongoDB  you should see something like this

Where there are a number of different processes that can be used for managing  MongoDB.

The .NET Client

The .NET client that this article uses is the official (supported) .NET  client, which can be downloaded from https://github.com/mongodb/mongo-csharp-driver/downloads. It would be  pretty much impossible for me to outline every feature of MongoDB, but I shall  outline what I think are the most important parts when getting started with  MongoDB.

1st Steps

You must have the MongoDB actual server running. I have tried to make this  easy for by creating a helper class called “MongoDBServer” which  you should modify to point to your MongoDB downloads. Once the MongoDB actual  server is running we need to create a software MongoServer (which  connects to the actual MongoDB server instance).

Some skeleton code is shown below, that all the MongoDB code uses in the  attached demo code

Collapse | Copy Code
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using MongoDB.Bson;
using MongoDB.Driver;
using MongoDB.Driver.Linq;
using MongoDB.Bson.Serialization.Attributes;

namespace DocumentDB.Mongo
{
    class Program
    {
        public static void Run()
        {
            try
            {
                if (MongoDBServer.Instance.Start())
                {

                    MongoServer server = MongoServer.Create(); // connect to localhost
		    //Use the software server
		    //Use the software server
		    //Use the software server
		    //Use the software server
                 }
            }
            catch (Exception ex)
            {
                Console.WriteLine("============= OH NO : ERROR ============");
            }
            Console.ReadLine();
        }

        static void Main(string[] args)
        {
            Run();
        }
    }
}

This software MongoServer is then used by the various classes in  the demo code, so you can expect to see the use of  a MongoServerobject in use in the attached demo code

Raw Document Storage

So now that we have seen that we need to start the actual MongoDB server and  also a software MongoServer instance, let see how to store some  documents.

Here is some code that will store a BsonDocument which is what  MongoDB uses.

Collapse | Copy Code
MongoDatabase rawDocumentDemoDB = server.GetDatabase("rawDocumentDemoDB");
rawDocumentDemoDB.Drop();

MongoCollection<BsonDocument> users = rawDocumentDemoDB.GetCollection("users");

// Create BsonDocument object for new user
var user = new BsonDocument();
user["firstname"] = "Goat";
user["lastname"] = "Head";
user["age"] = 12;
user["createdate"] = DateTime.Now;

// Insert new user object to collection
users.Insert(user);

And here is some code to update a BsonDocument

Collapse | Copy Code
MongoDatabase rawDocumentDemoDB = server.GetDatabase("rawDocumentDemoDB");
rawDocumentDemoDB.Drop();

MongoCollection<BsonDocument> users = rawDocumentDemoDB.GetCollection("users");
var savedUsers = users.FindAll();
if (savedUsers.Any())
{
    BsonDocument userToModify = savedUsers.First();
    if (userToModify != null)
    {
        Console.WriteLine(string.Format("Decrementing User Id : {0}, Age By -1", userToModify["_id"]));
        userToModify["age"] = userToModify["age"].AsInt32 - 1;
        users.Save(userToModify);
    }
}

It can be seen that when working with MongoDB we are working closely withMongoCollection<T> which we can use to store BsonDocument objects in. But these BsonDocuments don’t seem  that handy, we don’t have any typing at all there, they are nothing more than a  dictionary of key/values, mmm not that useful. Surely there is a way we can  store our own object structures.

Well yes as it turns out there is, lets see that.

Serializing Your Own Types

Ok so we have seen hw to deal with BsonDocument objects and found them lacking, and we now want to serialize  our own objects, so how do we do that. Well its simply a question of doing the following:

Collapse | Copy Code
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using MongoDB.Bson.Serialization.Attributes;
using MongoDB.Bson;

namespace DocumentDB.Mongo
{
    public class Blog
    {
        public Blog()
        {
            this.Tags = new List<string>();
            this.BlogPostIds = new List<ObjectId>();
        }

        public ObjectId Id { get; set; }
        public ObjectId UserId { get; set; }
        public string UserName { get; set; }
        public List<string> Tags { get; set; }
        public List<ObjectId> BlogPostIds { get; set; }
    }
}

We can also customise the serialization process by using special Mongo attributes as shown below, where I have attributed the same Blog class:

Collapse | Copy Code
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using MongoDB.Bson.Serialization.Attributes;
using MongoDB.Bson;

namespace DocumentDB.Mongo
{
    public class Blog
    {
        public Blog()
        {
            this.Tags = new List<string>();
            this.BlogPostIds = new List<ObjectId>();
        }

        [BsonElementAttribute("id")]
        public ObjectId Id { get; set; }

        [BsonElementAttribute("userid")]
        public ObjectId UserId { get; set; }

        [BsonElementAttribute("username")]
        public string UserName { get; set; }

        [BsonElementAttribute("tags")]
        public List<string> Tags { get; set; }

        [BsonElementAttribute("blogpostids")]
        public List<ObjectId> BlogPostIds { get; set; }
    }
}

 

There are a couple of points here

  1. That we use the BsonElementAttribute to mark our members as being serializable
  2. That the Ids are using ObjectId, which is a MongoDB type. You can use long/int but if you use ObjectId you will get this Id automatically filled in by MongoDB
  3. I have designed my models is to use links to Ids for foreign keys, this is  for 2 reasons
    1. I don’t want a massive graph brought back when I get a whole document stored, I want to decide when to fetch this linked data. So I just store the Ids of other data I may be interested in, if I find I need it, I’ll look it up later
    2. BSON/JSON doesn’t support circular references, so its just easier to store Ids, and have no cycles

Having this arrangement will allow these type of objects to be serialized to  Bson (Binary JSON)

I have to say I am not sure I like this too much, as its putting persistence  concerns in my model, I guess WCF does that with its DataContract/DataMember  attributes so its not so different, but WCF objects are more DTOs to me. But hey  you can decide if you like it or not.

Basic CRUD Using Typed Objects

I think the best way to show you some examples of basic CRUD operations using  strongly typed models is to literally show you a complete listing of the demo  code. So here goes

Collapse | Copy Code
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
using MongoDB.Driver;
using MongoDB.Driver.Linq;
using MongoDB.Bson;
using MongoDB.Driver.Builders;

namespace DocumentDB.Mongo
{
    public class MongoDemo
    {
        private MongoServer server;
        private MongoDatabase mongoDemoDB;

        public MongoDemo(MongoServer server)
        {
            this.server = server;
            mongoDemoDB = server.GetDatabase("rawDocumentDemoDB");
            mongoDemoDB.Drop();
        }

        public void ClearAll()
        {
            mongoDemoDB.Drop();
        }

        public void InsertFollowedByDelete()
        {
            StartRequest();
            ClearAll();
            InsertSingleBlog();
            IList<Blog> blogs = Blogs();
            Console.WriteLine(string.Format("There are currently {0}, Blogs", blogs.Count()));
            DeleteSpecificBlog(blogs.First().Id);
            blogs = Blogs();
            Console.WriteLine(string.Format("There are currently {0}, Blogs", blogs.Count()));
            StopRequest();
        }

        public void ShowListOfBlogs()
        {
            StartRequest();
            Console.WriteLine("RECENT BLOGS:\r\n");
            foreach (Blog blog in Blogs())
            {
                Console.WriteLine(blog.ToString());
            }
            StopRequest();
        }

        public int UserCount()
        {
            int users = 0;
            StartRequest();
            users = Users().Count();
            StopRequest();
            return users;
        }

        public void ShowListOfRecentPosts()
        {
            StartRequest();
            Console.WriteLine("RECENT BLOG POSTS:\r\n");
            foreach (BlogPost blogPost in BlogPosts())
            {
                Console.WriteLine(blogPost.ToString());
            }
            StopRequest();
        }

        public void InsertTestData()
        {
            StartRequest();
            ClearAll();
            var blogs = mongoDemoDB.GetCollection<Blog>("blogs");
            var users = mongoDemoDB.GetCollection<User>("users");
            var blogposts = mongoDemoDB.GetCollection<User>("blogposts");

            List<User> newUsers = new List<User>();
            User orenUser = new User { Name = "Oren Eini" };
            User demisUser = new User { Name = "Demis Bellot" };
            newUsers.Add(orenUser);
            newUsers.Add(demisUser);
            users.InsertBatch(newUsers);

            List<Blog> newBlogs = new List<Blog>();
            Blog orenBlog = new Blog
                {
                    UserId = orenUser.Id,
                    UserName = orenUser.Name,
                    Tags = new List<string> { "Architecture", ".NET", "Databases" },
                };
            Blog demisBlog = new Blog
                {
                    UserId = demisUser.Id,
                    UserName = demisUser.Name,
                    Tags = new List<string> { "Architecture", ".NET", "Databases" },
                };

            newBlogs.Add(orenBlog);
            newBlogs.Add(demisBlog);
            blogs.InsertBatch(newBlogs);

            List<BlogPost> newBlogPosts = new List<BlogPost>
            {
                new BlogPost
                {
                    BlogId = newBlogs.First().Id,
                    Title = "RavenDB",
                    Categories = new List<string> { "NoSQL", "DocumentDB" },
                    Tags = new List<string> {"Raven", "NoSQL", "JSON", ".NET"} ,
                },
                new BlogPost
                {
                    BlogId = newBlogs.First().Id,
                    Title = "Redis",
                    Categories = new List<string> { "NoSQL", "Cache" },
                    Tags = new List<string> {"Redis", "NoSQL", "Scalability", "Performance"},
                },
                new BlogPost
                {
                    BlogId = newBlogs.First().Id,
                    Title = "Cassandra",
                    Categories = new List<string> { "NoSQL", "Cluster" },
                    Tags = new List<string> {"Cassandra", "NoSQL", "Scalability", "Hashing"},
                },
                new BlogPost
                {
                    BlogId = newBlogs.First().Id,
                    Title = "Couch Db",
                    Categories = new List<string> { "NoSQL", "DocumentDB" },
                    Tags = new List<string> {"CouchDb", "NoSQL", "JSON"},
                }
            };

            blogposts.InsertBatch(newBlogPosts);
            orenUser.BlogIds.Add(orenBlog.Id);
            demisUser.BlogIds.Add(demisBlog.Id);
            users.Save(orenUser);
            users.Save(demisUser);

            orenBlog.BlogPostIds.Add(newBlogPosts.First().Id);
            demisBlog.BlogPostIds.Add(newBlogPosts.Last().Id);
            blogs.Save(orenBlog);
            blogs.Save(demisBlog);

            foreach (BlogPost newBlogPost in newBlogPosts)
            {
                newBlogPost.BlogId = orenBlog.Id;
                blogposts.Save(newBlogPost);
            }
        }

        private void InsertSingleBlog()
        {

            var blogs = mongoDemoDB.GetCollection<Blog>("blogs");
            var users = mongoDemoDB.GetCollection<User>("users");
            var blogposts = mongoDemoDB.GetCollection<User>("blogposts");

            User jayUser = new User { Name = "Jay Rays" };
            users.Insert(jayUser);

            Blog jayBlog = new Blog
            {
                UserId = jayUser.Id,
                UserName = jayUser.Name,
                Tags = new List<string> { "Architecture", ".NET", "Databases" },
            };
            blogs.Insert(jayBlog);

            BlogPost jayBlogPost = new BlogPost
            {
                Title = "RavenDB",
                Categories = new List<string> { "NoSQL", "DocumentDB" },
                Tags = new List<string> { "Raven", "NoSQL", "JSON", ".NET" }
            };
            blogposts.Insert(jayBlogPost);

            jayUser.BlogIds.Add(jayBlog.Id);
            jayBlog.BlogPostIds.Add(jayBlogPost.Id);
            jayBlogPost.BlogId = jayBlog.Id;

            users.Save(jayUser);
            blogs.Save(jayBlog);
            blogposts.Save(jayBlogPost);
        }

        private IList<Blog> Blogs()
        {
            return mongoDemoDB.GetCollection<Blog>("blogs").AsQueryable<Blog>().Where(x => true).ToList();
        }

        private IList<User> Users()
        {
            return mongoDemoDB.GetCollection<User>("users").AsQueryable<User>().Where(x => true).ToList();
        }

        private IList<BlogPost> BlogPosts()
        {
            return mongoDemoDB.GetCollection<BlogPost>("blogposts").AsQueryable<BlogPost>().Where(x => true).ToList();
        }

        private void DeleteSpecificBlog(ObjectId blogId)
        {
            Console.WriteLine("DELETING SINGLE Blog\r\n");
            mongoDemoDB.GetCollection<Blog>("blogs").Remove(Query.EQ("_id", blogId));
        }

        private void StartRequest()
        {
            server.RequestStart(mongoDemoDB);
        }

        private void StopRequest()
        {
            server.RequestDone();
        }
    }
}

I hope that makes sense to pick through, I feel it’s pretty obvious

SQL Like Syntax Expressions

One thing that MongoDB users seem to like is that it offers a SQL like API  via the use of Query classes. Here is a very small example that deletes a Blog that has a certain id

Collapse | Copy Code
private void DeleteSpecificBlog(ObjectId blogId)
{
    Console.WriteLine("DELETING SINGLE Blog\r\n");
    mongoDemoDB.GetCollection<Blog>("blogs").Remove(Query.EQ("_id", blogId));
}

Here are some more example of Query methods that you might use

 

Linq Support

MongoDB does have support for LINQ where you simply use its collections and use the AsQueryable methods and you can also then do the typical LINQ like things such as the following:

Collapse | Copy Code
return mongoDemoDB.GetCollection<Blog>("blogs").AsQueryable<Blog>().ToList();

Connection Sharing

One neat feature that MongoDB has is that is allows you to share a connection  to the database such that a group of operations can all use the same connection.

This is easily achieved using the following 2 methods

Collapse | Copy Code
MongoDatabase rawDocumentDemoDB = server.GetDatabase("rawDocumentDemoDB");
server.RequestStart(rawDocumentDemoDB);
//DO STUFF HERE
//DO STUFF HERE
//DO STUFF HERE
//DO STUFF HERE
server.RequestDone();

Bulk Operations

MongoDB does allow bulk operations by the use of xxxBatch() methods of its collections, so an example might be InsertBatch(..), where a working example is as shown below

Collapse | Copy Code
MongoDatabase rawDocumentDemoDB = server.GetDatabase("rawDocumentDemoDB");
rawDocumentDemoDB.Drop();

MongoCollection<BsonDocument> users = rawDocumentDemoDB.GetCollection("users");
List<User> newUsers = new List<User>();
User orenUser = new User { Name = "Oren Eini" };
User demisUser = new User { Name = "Demis Bellot" };
newUsers.Add(orenUser);
newUsers.Add(demisUser);
users.InsertBatch(newUsers);

Transaction Support

This is not something that MongoDB offers.

 

 

Comparisons

Whist evaluating the 3 document databases that I chose, these were my take  away points from them, as I say these are my own opionions and only I can be blamed  for them. I am also talking more from a beginners point of view as well, the  links I include below this table may help you if you are looking for deeper  answers, such as what sort of replication mechanisms are supported, but from a  purely zero to simple CRUD usability stance this is what I thought.

  Redis   Raven   Mongo
  • As Redis uses an approach where the entire dataset must fit into memory, it is fast. Lightning fast in fact.
  • Supports transactions
  • Supports LINQ
  • Good support for different types of collections
    • Sets/Lists etc etc
  • Values can expire
  • Replication
  • Very feature rich
  • Web based tool for inspecting/managing database store
  • Supports transactions
  • Supports LINQ
  • Map/Reduce
  • Full text search
  • Bulk operations/Patching API
  • Supports Sharding
  • Embedded mode
  • InMemory mod (for testing)
  • No where near as fast as Redis (due to Redis everything in memory approach)
  • Replication
  • Mature well used API
  • NO support for transactions
  • Supports LINQ (though there is some strangeness with LINQ API)
  • Connection sharing
  • Map/Reduce
  • Bulk operations
  • No where near as fast as Redis (due to Redis everything in memory approach)
  • You need to polute your models with document storage attributes, poor seperation of concerns
  • Replication

I also found these to be very good for making informed comparisons

  1. http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis
  2. http://stackoverflow.com/questions/4720508/redis-couchdb-or-cassandra
  3. http://www.servicestack.net/mythz_blog/?p=474

In terms of which document database I would choose to use it would reall  depend on what I was trying to do, for example if I had a very quick changing  data such as FX Rate ticks/tweets which would quicky expire I would use Redis (I  also hear very good things about Cassandra for this sort of scenario, but I did  not look at that particular document database).

If I was dealing with more standard storage requirements, which is what  really drew us to look into a document database that was scalability, I would use  RavenDB as I found it to be much richer and better thought out than Mongo. Mongo  seemed like it had been written a while ago, and was showing signs of needing a  breath of fresh air, in fact I read somewhere that the guys behind Mongo are  starting ove again on a new project which I imagine will be great. However at  the time of writing this article, out of the document databases I looked at for  this scenario (ie not very fast moving data requirments) I would choose RavenDB.

 

That’s It

Thats all I wanted to say in this article, hope you have got something out of it, and enjoyed my rantings on this subject,  and can see that there may be times when a No SQL based solution may be exactly what you need