Couchdb Performance on a MacPro 

Couchdb 0.11.0
2x3 QUAD-Core Intel Xeon
16 GB 667Mhz DDR2
OS X 10.6.4

Inserting

NUMBLOCKtimeBytesMb/srecords/s
110.003010.000330
1010.02511000.004398
100013.241510000000.308308
10000134.62631000000002.888289
1000103.4122106100003.1092931
1001001.5112106010007.0156617
1010001.9464106001005.4465138
250002.3308106000204.5484290
1100002.0068106000105.2824983
101000015.71761060001006.7446362

Average Top:

33423  beam.smp     48.9      05:42.89 13    0    62   151-  39M-   264K  
33421  CouchDBX     12.9      00:52.85 6/1   3    124- 322   77M-   29M

Comments

Post a Comment

Popular posts from this blog

Serializion Performance

Last week  I stuck my head out  in a meeting and declared that XML is verbose and slow to parse and that we should move to something like Google's protocols buffers,  or something readable such as json or YAML, which are  easier to parse etc etc etc! Well is this really true ? The statement seems logical considering how verbose XML can be. Still, after the meeting, some questions stayed in my mind. So I thought I would do some tests. I used  a FIX Globex (CME) swap trade confirmation message to test my theory. Size from Python to Python json cjson 2332 0.222238063812 0.0943419933319 pickle cPickle 1778 0.233518123627 0.128826141357 XML cElementTree 2083 0.407706975937 2.77832698822 json simplejson 2332 3.37723612785 5.11316084862 So this simple test shows that using XML with cElementTree parser  is not so slow, cjson wins in speed and the conclusion must be: Your performance will ultimately depend on your data and the quality of the l...
Read more

JSON Performance

I have some code that processes tweets, about 5 million a day, in realtime. They are currently stored in mongodb and also posted on various celery/rabbitmq work queues. The average message size is 5524, so encoding and decoding these messages is an issue. Using the following test code below. Standard Tweet message Encode/Decode with python built-in json package. message size Test Msg Size De-serialize Obj Size Serialize cjosn, bson, ujson Storage Storoge Cost Empty object 41 14.790, 37.565, 0.970 54 6.341, 41.856, 1.249 2050Mb 0.21 {} Empty list 41 15.069, 38.021, 1.005 54 6.675, 41.475, 1.400 2050Mb 0.21 [] Object of objects 843 107.750, 145.440, 25.525 3226 63.555, 828.235, 28.051 42150Mb 4.21 List of lists 563 58.805, 81.950, 16.960 104 43.426, 815.965, 18.311 28150Mb 2.81 Object with only tweet id 93 25.030, 53.360, 2.280 422 23.570, 83.445, 3.295 4650Mb 0.47 Full tweet message 4386 697.221, 867.780, 188.560 12606 360.290, 5847.335, 201.610 219300Mb 21...
Read more

My Second Super Computer

Cluster GPU Quadruple Extra Large 22 GB memory: 22 GB EC2 Compute Units: 33.5 , GPU: 2 x NVIDIA Tesla “Fermi” M2050 GPUs, 1690 GB of local instance storage, 64-bit platform, 10 Gigabit Ethernet cores each: 448 os: CENTOS 64bit Monte Carlo on One Telsa Device Options : 256 Simulation paths CPU GPU Time (ms.) options/sec. Time (ms.) options/sec. 262144 6000 42 3.586 71388 Monte Carlo on Two Telsa Devices Options : 256 split across two Tesla boards Simulation paths CPU GPU Time (ms.) options/sec. Time (ms.) options/sec. 262144 6000 42 3.405 151999 TOTAL Cost: $0.04 including building the environment and sample code from scratch. CUDA Device Query (Runtime API) version (CUDART static linking) There are 2 devices supporting CUDA Device 0: "Tesla M2050" CUDA Driver Version: 3.20 CUDA Runtime Version: 3.10 CUDA Capability Major/Minor version number: 2.0 Total amount of...
Read more