ZODB vs Durus

Soon after, I posted my last article about ZODB performance against SQLite3, I got a polite comment from Michael Watkins reminding me of Durus. Durus is a simpler object database inpired by ZODB. Despite not having many of the features of ZODB, such as multi-threaded storage access, multiple storage backends, asynchronous IO, versions, undo and conflict resolution (according to Durus own FAQ), It is a very capable database. So I decided to adapt my benchmark script and pitch Durus against ZODB. Please note that my benchmark code is very simple and does not explore well the differences between Durus and ZODB. A better comparison is left as an exercise to the reader. ;-)

Despite the simplicity of my test code, There was one suprinsing result of my test. Both databases used files as storages, but the file size for Durus was 3.7MB for a million records, while ZODB file size was 23.7MB !!!

Both database systems offer the option of packing their stores, to reduce size, but this feature was not used. Besides, to pack a ZODB storage file, the same ammount of free disk space is required, wich only makes matters worse for ZODB. Please, also check Michael's Blog for a very interesting benchmark of Durus vs cPickle.

Here is the code:

import time, os, glob
import ZODB
from   ZODB import FileStorage, DB
import pylab as P

from durus.file_storage import FileStorage as FS
from durus.connection import Connection


def zInserts(n):
    print "Inserting %s records into ZODB"%n
    for i in xrange(n):
        dbroot[i] = {'name':'John Doe','sex':1,'age':35}
    connection.transaction_manager.commit()

def DurusInserts(n):
    print "Inserting %s records into Durus"%n
    for i in xrange(n):
        Droot[i] = {'name':'John Doe','sex':1,'age':35}
    conndurus.commit()

recsize = [1000,5000,10000,50000,100000,200000,400000,600000,800000,1000000]
zperf = []
durusperf =[]
for n in recsize:
    # remove old databases
    if os.path.exists('testdb.fs'):
        [os.remove(i) for i in glob.glob('testdb.fs*')]
    if os.path.exists('test.durus'):
        os.remove('test.durus')
    # setup ZODB storage
    dbpath = 'testdb.fs'
    storage     = FileStorage.FileStorage(dbpath)
    db          = DB(storage)
    connection  = db.open()
    dbroot      = connection.root()
    #Setting up durus database
    conndurus = Connection(FS("test.durus"))
    Droot = conndurus.get_root()
    #begin tests
    t0 = time.clock()
    zInserts(n)
    t1 = time.clock()
    # closing and reopening ZODB' database to make sure
    # we are reading from file and not from some memory cache
    connection.close()
    db.close()
    storage  = FileStorage.FileStorage(dbpath)
    db  = DB(storage)
    connection  = db.open()
    dbroot = connection.root()
    t2 = time.clock()
    print "Number of records read from ZODB: %s"%len(dbroot.items())
    t3 = time.clock()
    ztime = (t1-t0)+(t3-t2)
    zperf.append(ztime)
    print 'Time for ZODB: %s seconds\n'%ztime
    t4 = time.clock()
    DurusInserts(n)
    t5 = time.clock()
    conndurus = Connection(FS("test.durus"))
    Droot = conndurus.get_root()
    t6 = time.clock()
    print "Number of records read from Durus: %s"%len(Droot.items())
    t7 = time.clock()
    Dtime = (t5-t4)+(t7-t6)
    durusperf.append(Dtime)
    print 'Time for Durus with db on Disk: %s seconds\n'%Dtime
P.plot(recsize,zperf,'-v',recsize,durusperf,'-^')
P.legend(['ZODB','Durus'])
P.xlabel('inserts')
P.ylabel('time(s)')
P.show()