rails: Rails standardized error reporting interface

Context

A very common pattern when dealing with exceptions is to report them asynchronously to the developer and continue with some fallback. E.g.

begin
  do_something
rescue SomethingIsBroken => error
  MyErrorReportingService.notify(error)
  some_fallback_code_or_value
end

An example in Rails is in ActiveSupport::Cache::RedisCacheStore we simply log handled errors.

That is the kind of errors you’d want reported to your error monitoring service so that even though the error is not directly breaking the user experience, you are alerted early that some part of your infrastructure is no longer working, and might take action, or that some edge case is causing a degraded experience.

Another example is the recent ActiveSupport::Notifications handling of exceptions raised by finish. In such case the most appropriate behaviour could be to swallow the exception, as to not interrupt the notification chain, but to report it to the application developer.

In application code it’s easy because you know where to send the exception report, but in libraries and frameworks, usually the best you can do is log the error. But people usually don’t look at logs unless they know something is happening.

Introducing a standard interface for error reporting services

Since the vast majority of Rails application out there do setup an error reporting service (Sentry, Airbrake, Errbit, HoneyBadger, and plenty others), I believe there’s an opportunity for Rails to provide a generic interface to report errors, with an adapter API to allow using the service of your choice. In other term, introducing an Active Job or Active Support Cache for error reporting.

There would be several upsides to this.

Reporting the errors handled by Rails

When Rails handles an error like in the examples listed above, it could report them properly to the developer, instead of logging them where no-one will see.

Reporting warnings

Similarly there are places in Rails were we emit warnings, but they aren’t easy to notice. Many error reporting service have some kind of severity attribute, so Rails could report these as exceptions with severity: warning.

Simplified collection of unhandled errors

The hooking mechanism would be simplified. The various error reporting gems out there are doing a lot of work to hook into Rack / Sidekiq / Resque etc. It’s brittle and hard to maintain and setup. It’s also often not covering all entry points such as rails runner being invoked from a crontab, etc.

If Rails was in charge of collecting unhandled errors, it could do it more easily, likely as part of ActiveSupport::Executor, and it would automatically cover all Rails entry points.

Simplified errors notification service setup

The setup of the various error reporting service could be heavily simplified. They could ship a Railtie in their gem and automatically register themselves to receive the exceptions. No need to add a middleware in Rack, or require some code to hook into Sidekiq and such.

Interface Requirements

I believe that along the error itself with its backtrace, there are two essential attributes that need to be communicated:

• handled: true/false: Wether the error was properly handled by the application, or if it was “final” (e.g. displayed a 500)
• severity: error/warning: Even when it’s handled by the application, an error can either be “nice to know about”, or important to fix quickly. As such I think it’s important to be able to communicate this.

Of course all reporting services don’t just log the error, they collect various contextual data alongside it. This often include request or job parameters, sometimes app specific data such as an user_id, etc.

So the interface would need to be extensible.

We can probably draw some inspiration from the recent ActiveRecord::QueryLogs, the need is similar, as we enter some paths we do want to append some data to the error reporter context. e.g. Job class and parameters when performing a job.

Or even refactor ActiveRecord::QueryLogs.set_context, to move it to a more generic namespace, and share it with the error reporter, as many of the data will be the same.

And it is also important to be able to pass extra data when reporting an error, e.g.:

Rails.error.report(some_error, context: { foo: "bar" })

The signature could be something like:

def report(error, handled: true, severity: :error, context: {})
end

There is likely some other interface requirements I’m not thinking about, so I’d like to involve as many of these gems maintainers as possible to collect some feedback.

Feedback

At this early stage I would love to hear some feedback from the maintainers of the various error monitoring gems out there:

@imjoehaines (Bugsnag) @st0012 (Sentry) @joshuap, @shalvah (HoneyBadger) @kyrylo (Airbrake) @waltjones (Rollbar) @samuel-holt, @mogest (Raygun) @composit, @dlanderson, @okapusta (Scout) @angelatan2, @amhuntsman, @tannalynn, @jasonrclark (NewRelic)

Sorry if I forgot other popular services. And apologies if you didn’t wish to be pinged.

If not, I’d have a few questions:

• Would you integrate with such interface?
• What kind of metadata do you need provided by this interface?

cc @rafaelfranca @eileencodes @jhawthorn