oj: Memory leak in Oj.dump

Ruby version: ruby 2.4.4p296 (2018-03-28 revision 63013) [x86_64-darwin17] Oj version: oj (3.6.8, 2.18.5)

👋 I’ve observed some leaky behaviour in my application when using Oj to dump JSON. It’s more visible when the object(hash/array) being dumped contains symbols. Oj keeps references to all symbols that are dumped so they can’t GC’d

Here’s an example:

require 'objspace'
require 'oj'
require 'json'
require 'get_process_mem'

def record_allocation
  GC.start
  GC.start

  mem = GetProcessMem.new
  puts "Before - Process Memory: #{mem.mb} mb"
  puts "Before - Objects count: #{ObjectSpace.each_object.count}"
  puts "Before - Symbols count: #{Symbol.all_symbols.size}"

  yield

  GC.start
  GC.start
  GC.start

  puts "After - Process Memory: #{mem.mb} mb"
  puts "After - Objects count: #{ObjectSpace.each_object.count}"
  puts "After - Symbols count: #{Symbol.all_symbols.size}"
end

record_allocation do
  data = 1_000_000.times.map { |i| "string_number_#{i}".to_sym } # array of symbols
  Oj.dump(data)
end

output

$ RBENV_VERSION=2.4.4 time ruby -v -W0 test.rb
ruby 2.4.4p296 (2018-03-28 revision 63013) [x86_64-darwin17]
Before - Process Memory: 12.703125 mb
Before - Objects count: 16070
Before - Symbols count: 3496
After - Process Memory: 308.328125 mb
After - Objects count: 2016040
After - Symbols count: 1003496
        3.97 real         3.58 user         0.32 sys

We can see that a alot of symbols are retained after the block.

This is the same test but using ruby JSON and we can see that no symbols or extra objects are retained.

record_allocation do
  data = 1_000_000.times.map { |i| "string_number_#{i}".to_sym } # array of symbols
  JSON.dump(data)
end

output

$ RBENV_VERSION=2.4.4 time ruby -v -W0 test.rb
ruby 2.4.4p296 (2018-03-28 revision 63013) [x86_64-darwin17]
Before - Process Memory: 13.1171875 mb
Before - Objects count: 16070
Before - Symbols count: 3496
After - Process Memory: 339.734375 mb
After - Objects count: 16042
After - Symbols count: 3496
        5.30 real         4.88 user         0.37 sys

Let me know if you need help reproducing this. Thanks