BGPalerter: Possible memory leak when monitoring large number of prefixes

Hello,

Before anything, thanks for this project. I’ve been watching it for a while, and now finally deployed it… I hope I’ll be able to contribute to at some point, for now, at least, with some reports.

Describe the bug

Well, it is not exactly a bug (or maybe it is), but I’ve noticed the memory is steadily increasing. In 24 hours after deploying the app, the memory footprint grew from 150MB to over 800MB, and it looks like it’s only going to be getting worse - see screenshot below:

BGPalerter Mem Usage Grafana

The CPU usage seems pretty stable however.

I’m yet to play with Node Js, but in general a pattern like that means there’s a possibility of a memory leak somewhere.

Provide an example Provide an example in terms of prefixes and BGP messages. Possibly provide a snippet of config.yml and prefixes.yml.

Nothing really specific, pretty much all the prefixes from https://bgp.he.net/AS14061#_prefixes and https://bgp.he.net/AS14061#_prefixes6, e.g.,

"5.101.96.0/21":
  description: "Some description"
  asn: 14061
  ignoreMorespecifics: false
  ignore: false

3581 prefixes in total.

Expected behavior

Given the high number of prefixes, I’d expect to see a pretty important memory footprint, but I find an always growing figure more worrying because it’s not easily predictable how much resources you’re going to need on the long term.

Are you using the binary or the source code?

A custom Docker image based on nttgin/BGPalerter, so source code.

Your information Provide your name and your AS/company (see Bert Hubert’s post https://berthub.eu/articles/posts/anonymous-help/).

AS14061, DigitalOcean.

Cheers!

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 19 (19 by maintainers)

Most upvoted comments

I just want to say you guys rock 😃

On Tue, Jan 21, 2020 at 23:03 Massimo Candela notifications@github.com wrote:

Hi @mirceaulinic https://github.com/mirceaulinic,

This is interesting! I need to replicate it. Could you please provide me with the rule in prefixes.yml covering this ip: 195.66.227.90

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/nttgin/BGPalerter/issues/114?email_source=notifications&email_token=AABFRWECS7BFSLPCO3VOAS3Q636BVA5CNFSM4J56R3OKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJP2ZMQ#issuecomment-576695474, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABFRWCYHZCULWNXIIAMN5DQ636BVANCNFSM4J56R3OA .

To be clear, the frequent re-announcements are real independently from the duplication. So there there are 2 phenomena:

  • re-announcements or anyway frequent updates
  • duplication of them (RIS bug)

The RIS team is working on the second one!

Sorry, I provided you with the peer instead of the prefix. I meant 206.189.244.0/22. Anyway, I think I already have all the information I need. I will let you know asap.

Ok!! We are there! Please, run the latest code in dev for some hours and let me know if you see the memory pattern!

Yes, I will write you a detailed answer soon. Unfortunately I believe the patch I released in dev for this issue introduced another bug I’m trying to address. So don’t invest time in testing it yet.