google-cloud-python: pubsub - modify_ack_deadline does not work, message gets redelivered after 10 minutes always

  1. Specify the API at the beginning of the title (for example, “BigQuery: …”) General, Core, and Other are also allowed as types google-cloud-pubsub - 0.29.2
  2. OS type and version Linux 4.4.0-104-generic #127-Ubuntu SMP Mon Dec 11 12:16:42 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
  3. Python version and virtual environment information python --version Python 2.7.12
  4. google-cloud-python version pip show google-cloud, pip show google-<service> or pip freeze google-cloud: 0.30.0 google-cloud-pubsub - 0.29.2
  5. Stacktrace if available
  6. Steps to reproduce I need to have very long ack deadlines since our consumer process could be running for long time per message. No matter what I set as ack_deadline_seconds in subscription definition, or call subscriber.modify_ack_deadline every 1 minute, the message gets redelivered in exact ten minutes.
  7. Code example

I have a test case with a simple subscriber::

subscriber.create_subscription(sname, tname, ack_deadline_seconds=120)
subscription = subscriber.subscribe(
        sname #, flow_control=flow_control
 )
future = subscription.open(callback)

def callback(pubsub_message):
    logging.warn("Received message: %s %s at %s", pubsub_message.message_id, pubsub_message.data, time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(time.time())))
    i = 0
    while i < 480:
        i = i + 1
        time.sleep(60)
        subscriber.modify_ack_deadline(sname, [pubsub_message._ack_id], 120)
        logging.warn("Renewed: %s %s %s at %s", i, pubsub_message.message_id, pubsub_message.data, time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(time.time())))

    logging.warn("Ack: %s %s at %s", pubsub_message.message_id, pubsub_message.data, time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(time.time())))
    pubsub_message.ack()

enabling DEBUG logging, I see ack messages every few seconds DEBUG:google.cloud.pubsub_v1.subscriber.policy.base:The current p99 value is 10 seconds. DEBUG:google.cloud.pubsub_v1.subscriber.policy.base:Renewing lease for 5 ack IDs. DEBUG:google.cloud.pubsub_v1.subscriber.policy.base:Snoozing lease management for 7.362112 seconds. DEBUG:google.cloud.pubsub_v1.subscriber._consumer:Sending request:

but message gets redelivered in 10 minutes.

Using GitHub flavored markdown can help make your request clearer. See: https://guides.github.com/features/mastering-markdown/

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Comments: 27 (10 by maintainers)

Most upvoted comments

@explicitcall No concrete plans, but I have ideas in my head once we can split google-cloud-python into N repositories (one for each subpackage). It’s a difficult task right now that would require a HUGE investment of time making modifications to the RTD Sphinx theme.

Thankyou @dhermes. Do you think a potential fix would be available anytime soon? I am trying to figure out if I should build application level logic to handle this or wait for a fix.