boto3: SQS Queue.receive_messages returns one message when set MaxNumberOfMessages to 10.

Don’t count on the order of receiving the messages - put a timestamp in the message.

[Update] I take it back - SQS does support FIFO - more here: http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/FIFO-queues.html

There are only two hard problems in distributed systems:

2 . Exactly-once delivery 1 . Guaranteed order of messages 2 . Exactly-once delivery…

Yeah I’m still having this issue now. I have a queue with 2 messages in it. When I call receive_message() through boto3 I only get 1 message back. When I use the Poll for Messages button through the AWS console I get back two messages (which is correct). This is incredibly discouraging and doesn’t build much trust in boto3.

This is still an issue, I have a queue with 4 messages on it and only get 1 back for each request to receive-message but polling in the console returns all 4. Why is this closed, it’s not desirable behaviour and makes me think I should not use SQS.

Even repeat calls with an extended wait time I only get 1 message at a time back and it does not always return unique messages each time a repeat the call. So i could write some code which loops through the queue doing repeat receive message requests and never get a response with the message I want. This is just not usable or reliable.

I take it, this is still an “issue” (not w/ the lib, more w/ SQS).

Context

I have a tiny queue (we’ll only ever have ~8 messages in the queue at a time) that I’d like to expose over a Slack “slash” command. For example, typing /queue, would hit an API Gateway that would call a Lambda that would pull and display all messages in the queue (in order).

Temp Solution

I have some code that hacks around SQS’s limitations (based on the feedback in this thread):

def lambda_handler(event, context):
    """
    Get messages from an SQS message queue and return them to the caller.
    """
    sqs = boto3.resource('sqs')
    queue = sqs.get_queue_by_name(QueueName='test-messages')

    messages = set()
    for i in range(0, 6):
        msg_list = queue.receive_messages(VisibilityTimeout=1, MaxNumberOfMessages=10, WaitTimeSeconds=5)
        for msg in msg_list:

            messages.add(msg.body)

    return list(messages)

Question

While this obviously isn’t an optimal approach, it does work. That said, I was wondering if was possible to order these messages by the timestamp w/ which they were submitted to the queue?

Potential Approach

Set the timestamp as an attr on the message during message creation and order on that attr once I get my set…

I am still having the same issue/bug. Please resolve it or provide a workaround that does not include adding new messages and works with a small no of messages.

Adding a while True loop in your code will solve it Ex - while True: for message in queue.receive_messages(): print(message.body)

This prints all the messages in the queue.

I’ve been investigating this issue, and it seems to be SQS’s specified behavior. You are not guaranteed to receive 10 messages if there are 10 or more messages (especially with a small number of messages). All that long polling (enabled by setting WaitTimeSeconds) guarantees is that all the machines will be searched, and that it won’t return empty until the time is up. When you have a large number of messages in the queue you are much more likely to receive 10 messages in a call.

Let me know if you have any more questions.