google-cloud-node: pubsub: app with default settings eventually runs out of memory (but does not crash)

Environment details

  • OS: alpine 3.6
  • Node.js version: 9.8.0
  • npm version: 5.6.0
  • google-cloud-node version: @google-cloud/pubsub@0.16.5

Steps to reproduce

Run the following on GKE, pump 30 million messages into pubsub:

const PubSub = require('@google-cloud/pubsub')
const express = require('express')

console.log('Node starting up')

PROJECT_ID = 'some-project'
TOPIC_ID = 'some-topic'
SUBSCRIPTION_ID = 'some-node-subscription'

const pubsub = new PubSub({
  projectId: PROJECT_ID,
})

const topic = pubsub.topic(TOPIC_ID)
topic.createSubscription(SUBSCRIPTION_ID, function (err, subscription) {
  if (err) {
    console.log('Node subscription already created, moving on')
  }
})
const subscription = topic.subscription(SUBSCRIPTION_ID)

subscription.on('error', function (err) {
  console.error('Node error!', err)
})

function onMessage(message) {
  console.log(`Node received ${message.id} ${message.data}`)
  setTimeout(function () {
    console.log('Node acking')
    message.ack()
  }, 5000)
}

console.log('Node receiving..')

subscription.on('message', onMessage)

// because I have no idea how to wait forever..
const app = express()
app.listen(3000, () => console.log('Example app listening on port 3000!'))

// Remove the listener from receiving `message` events.
// subscription.removeListener('message', onMessage)

Once the app runs out of memory (1.5GB), reception slows from ~20k/min to something like 200/min:

screen shot 2018-03-17 at 5 58 42 pm

(removing the timeout, it still runs out of memory)

Let me know if I’m missing something silly here.

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 22 (22 by maintainers)

Most upvoted comments

@jadekler hooray! Thank you for all your help, I definitely would not have resolved this issue as quickly without it.

@callmehiphop Happy to report that I saw no problems over the weekend, and that the node pubsub library chugged through 67M messages at a rate of 61,000 acks / minute.