realm-swift: Slow performance with multiple realms / Caching RLMRealm objects

I have data structure like this:

Trick : RLMObject {
    trickID, // primary key
    name,
    ...
    @readonly progress {
        return [TrickProgress objectForPrimaryKey:self.trickID];
    }
}

TrickProgress : RLMObject {
    trickID, // primary key
    progress
}

I have table view with data source like this:

- cellForRowAtIndexPath {

    Trick *trick = tricks[indexPath.row];
    something = trick.progress; // Some computes with Trick `progress` property
    ...
}

The question is: If I store Trick and TrickProgress entities in the same realm the performance is really great, but if i store them in the different realms (what I want to do) the performance is so bad (even on iPhone 6s) so I can’t scroll table view without visible lagging. Yeah, maybe this is not so great idea to fetch entity from database in the cellForRow, but the question is why this is such a big difference in performance?