leveldown: LevelDB read performance degradation
i’m not sure if my issue is related to leveldown
or not, but right now i don’t have any ideas. i’m experiencing heavy performance degradation after a huge amount of writes/deletes. The db seems to be huge:
$ du -sh db/
3.3G db/
$ ls -l db | wc -l
1633
But the db is totally empty. Also it takes 30s to finish scanning.
var db = require('levelup')('./db');
var s = Date.now();
var keys = 0;
db.createKeyStream().on('data', function (key) { keys++; }).on('error', function (e) { console.log(e); }).on('end', function () { console.log(keys, Date.now() - s); })
// 0 30970
I’ve tried to run leveldown.repair
, but the numbers are roughly the same:
$ du -sh db/
2.6G db/
$ ls -l cache | wc -l
1331
I’ve tried to inspect the db:
> console.log(db.getProperty('leveldb.stats'))
Compactions
Level Files Size(MB) Time(sec) Read(MB) Write(MB)
--------------------------------------------------
0 3 6 0 0 0
1 5 9 0 0 0
2 48 99 0 0 0
3 485 998 0 0 0
4 603 1219 0 0 0
Is this behavior something i should expect with leveldb?
About this issue
- Original URL
- State: closed
- Created 8 years ago
- Comments: 15 (5 by maintainers)
@dominictarr and @maxogden
@madbence you might consider manually triggering compaction when you delete a significant number of entries and see if performance improves at all. this was added in the last release:
https://github.com/Level/leveldown#leveldown_compactRange
the keys begin with numbers, so i guess this should cover the whole domain of my keys: