leveldown: LevelDB read performance degradation

i’m not sure if my issue is related to leveldown or not, but right now i don’t have any ideas. i’m experiencing heavy performance degradation after a huge amount of writes/deletes. The db seems to be huge:

$ du -sh db/
3.3G    db/
$ ls -l db | wc -l
1633

But the db is totally empty. Also it takes 30s to finish scanning.

var db = require('levelup')('./db');
var s = Date.now();
var keys = 0;
db.createKeyStream().on('data', function (key) { keys++; }).on('error', function (e) { console.log(e); }).on('end', function () { console.log(keys, Date.now() - s); })
// 0 30970

I’ve tried to run leveldown.repair, but the numbers are roughly the same:

$ du -sh db/
2.6G    db/
$ ls -l cache | wc -l
1331

I’ve tried to inspect the db:

> console.log(db.getProperty('leveldb.stats'))
                               Compactions
Level  Files Size(MB) Time(sec) Read(MB) Write(MB)
--------------------------------------------------
  0        3        6         0        0         0
  1        5        9         0        0         0
  2       48       99         0        0         0
  3      485      998         0        0         0
  4      603     1219         0        0         0

Is this behavior something i should expect with leveldb?

About this issue

  • Original URL
  • State: closed
  • Created 8 years ago
  • Comments: 15 (5 by maintainers)

Most upvoted comments

@madbence you might consider manually triggering compaction when you delete a significant number of entries and see if performance improves at all. this was added in the last release:

https://github.com/Level/leveldown#leveldown_compactRange

the keys begin with numbers, so i guess this should cover the whole domain of my keys:

> db.db.approximateSize('', 'z', function () { console.log(arguments); })
{ '0': null, '1': 2445270120 }