newrelic-ruby-agent: Upgrading from V8 to V9 causes transactions with custom names to disappear in UI and errors in the logs

I’m in the process of upgrading a large Rails 6.2 app running APM 8.16 to version 9.0.

In the transactions interface on some web transactions (seems to be the ones with custom transaction names) I see now that most time is attributed to a segment by the name #. What is this and why?

Skærmbillede 2023-03-06 kl  15 32 57

In the newrelic.log I can see the following:

[2023-03-06 15:13:18 +0100 web-01 (1107039)] ERROR : Exception during Transaction#finish
[2023-03-06 15:13:18 +0100 web-01 (1107039)] ERROR : NoMethodError: undefined method `name' for nil:NilClass
[2023-03-06 15:13:18 +0100 web-01 (1107039)] DEBUG : Debugging backtrace:
/home/app/a/shared/bundle/ruby/2.7.0/gems/newrelic_rpm-9.0.0/lib/new_relic/agent/transaction/abstract_segment.rb:267:in `force_finish'
/home/app/a/shared/bundle/ruby/2.7.0/gems/newrelic_rpm-9.0.0/lib/new_relic/agent/transaction/abstract_segment.rb:91:in `finalize'
/home/app/a/shared/bundle/ruby/2.7.0/gems/newrelic_rpm-9.0.0/lib/new_relic/agent/transaction/tracing.rb:52:in `block in finalize_segments'
/home/app/a/shared/bundle/ruby/2.7.0/gems/newrelic_rpm-9.0.0/lib/new_relic/agent/transaction/tracing.rb:52:in `each'
/home/app/a/shared/bundle/ruby/2.7.0/gems/newrelic_rpm-9.0.0/lib/new_relic/agent/transaction/tracing.rb:52:in `finalize_segments'
/home/app/a/shared/bundle/ruby/2.7.0/gems/newrelic_rpm-9.0.0/lib/new_relic/agent/transaction.rb:555:in `commit!'
/home/app/a/shared/bundle/ruby/2.7.0/gems/newrelic_rpm-9.0.0/lib/new_relic/agent/transaction.rb:534:in `finish'
/home/app/a/shared/bundle/ruby/2.7.0/gems/newrelic_rpm-9.0.0/lib/new_relic/agent/instrumentation/middleware_tracing.rb:111:in `call'
/home/app/a/shared/bundle/ruby/2.7.0/gems/unicorn-5.8.0/lib/unicorn/http_server.rb:634:in `process_client'
/home/app/a/shared/bundle/ruby/2.7.0/gems/unicorn-5.8.0/lib/unicorn/http_server.rb:732:in `worker_loop'
/home/app/a/shared/bundle/ruby/2.7.0/gems/unicorn-5.8.0/lib/unicorn/http_server.rb:548:in `spawn_missing_workers'
/home/app/a/shared/bundle/ruby/2.7.0/gems/unicorn-5.8.0/lib/unicorn/http_server.rb:144:in `start'
/home/app/a/shared/bundle/ruby/2.7.0/gems/unicorn-5.8.0/bin/unicorn:128:in `<top (required)>'
/home/app/a/shared/bundle/ruby/2.7.0/bin/unicorn:23:in `load'
/home/app/a/shared/bundle/ruby/2.7.0/bin/unicorn:23:in `<top (required)>'
/home/app/.rbenv/versions/2.7.2/lib/ruby/site_ruby/2.7.0/bundler/cli/exec.rb:63:in `load'
/home/app/.rbenv/versions/2.7.2/lib/ruby/site_ruby/2.7.0/bundler/cli/exec.rb:63:in `kernel_load'
/home/app/.rbenv/versions/2.7.2/lib/ruby/site_ruby/2.7.0/bundler/cli/exec.rb:28:in `run'
/home/app/.rbenv/versions/2.7.2/lib/ruby/site_ruby/2.7.0/bundler/cli.rb:481:in `exec'
/home/app/.rbenv/versions/2.7.2/lib/ruby/site_ruby/2.7.0/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/home/app/.rbenv/versions/2.7.2/lib/ruby/site_ruby/2.7.0/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/home/app/.rbenv/versions/2.7.2/lib/ruby/site_ruby/2.7.0/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
/home/app/.rbenv/versions/2.7.2/lib/ruby/site_ruby/2.7.0/bundler/cli.rb:31:in `dispatch'
/home/app/.rbenv/versions/2.7.2/lib/ruby/site_ruby/2.7.0/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
/home/app/.rbenv/versions/2.7.2/lib/ruby/site_ruby/2.7.0/bundler/cli.rb:25:in `start'
/home/app/.rbenv/versions/2.7.2/lib/ruby/gems/2.7.0/gems/bundler-2.2.25/exe/bundle:49:in `block in <top (required)>'
/home/app/.rbenv/versions/2.7.2/lib/ruby/site_ruby/2.7.0/bundler/friendly_errors.rb:128:in `with_friendly_errors'
/home/app/.rbenv/versions/2.7.2/lib/ruby/gems/2.7.0/gems/bundler-2.2.25/exe/bundle:37:in `<top (required)>'
/home/app/.rbenv/versions/2.7.2/bin/bundle:23:in `load'
/home/app/.rbenv/versions/2.7.2/bin/bundle:23:in `<main>'

The custom naming in rails controller looks like this:

NewRelic::Agent.set_transaction_name("#{NewRelic::Agent.get_transaction_name} - search_backend: #{resolver.backend}")

I can see here that it seems to be parent which is nil https://github.com/newrelic/newrelic-ruby-agent/blob/68a34050cff2034559ea1acc59132d3ee245ce42/lib/new_relic/agent/transaction/abstract_segment.rb#L267 - How can I further debug this and what might be causing it and could it be related?

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Reactions: 1
  • Comments: 18 (11 by maintainers)

Commits related to this issue

Most upvoted comments

@kaylareopelle, setting instrumentation.thread.tracing: false appears to have stopped the problem for us as well.

Hi @NielsKSchjoedt, @boomer196 - I have a bug fix branch I’d like you both to test out. It resolves the raised error, but I’m not sure if it’ll solve the segments named #.

To install it, please replace your current installation of newrelic_rpm in your Gemfile with:

gem 'newrelic_rpm', git: 'https://github.com/newrelic/newrelic-ruby-agent.git', branch: 'bugfix/transaction_finish_nil_parent'

We’d also like you to turn thread instrumentation back on while testing. To do this, you can either change the value of instrumentation.thread.tracing to true or remove the instrumentation.thread.tracing line from your config.

Thank you, @boomer196. This helps us narrow down which PRs may have introduced the bug! I’m going to do a bit more tinkering and will reach out with a bug fix branch when ready.

Thanks for the follow-up, @boomer196! I’ll keep searching for a solution.

It sounds like you’re using a Rails app. Which version are you on? Is your Ruby version 3.2? (it seems like this from the logs in the Explorer’s Hub post)

Yes @kaylareopelle, we are running rails@7.0.4.2 with ruby@3.2.1