copilot-cli: Worker Service getting permission errors when reading from its own queue
Hi! I used copilot to add a scheduled job and a worker service. The scheduled job manifest looks something like this:
name: scheduled-job
type: Scheduled Job
on:
schedule: "@every 1m"
image:
build: scheduler-service/Dockerfile
cpu: 256
memory: 512
platform: linux/x86_64
publish:
topics:
- name: jobs
The worker service manifest looks something like this:
name: worker-service
type: Worker Service
image:
build: worker-service/Dockerfile
count: 1
exec: true
# storage:
# readonly_fs: true
# The events can be received from an SQS queue via the env var $COPILOT_QUEUE_URI.
subscribe:
topics:
- name: jobs
service: scheduled-job
In my worker-service/Dockerfile I run my node script. It uses the AWS SDK to interact with the SQS queue created for the worker service. Here is a simplified version of the server:
import { prisma } from "./prisma/prisma-client";
import AWS from "aws-sdk";
const queueUrl = process.env.COPILOT_QUEUE_URI;
const sqs = new AWS.SQS();
const receiveAndProcessMessage = async () => {
const params = {
MaxNumberOfMessages: 1,
QueueUrl: queueUrl,
VisibilityTimeout: 20,
WaitTimeSeconds: 0,
};
try {
const data = await sqs.receiveMessage(params).promise();
if (data.Messages) {
const message = data.Messages[0];
await processMessage(message);
await deleteMessage(message);
} else {
console.log("No messages in queue.");
}
} catch (err) {
console.error(err);
}
};
async function main() {
await receiveAndProcessMessage();
}
main()
I am seeing a permission error saying:
AccessDenied: Access to the resource https://sqs.us-east-2.amazonaws.com/ is denied.
I checked and it appears that the SQS queue’s permissions (the default one that got created for this worker service) has the correct IAM role (the same one that the created ECS has)
I haven’t messed with any permissions outside of copilot. Is my setup incorrect, am I missing something? Seems the service should have access to this queue without any additional trouble.
Thanks!
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 34 (17 by maintainers)
Hi! Thanks so much for your help!!
^ You were totally right about this, that is exactly what was going on!
Everything is working as expected now 🎉 Haha finally 😅
Thanks again. 🎉 It feels almost sad to close this issue, been going on for so long! I’ll miss you 😛 !
Yeah AWS support should be able to help! Probably not related but just in case 💭 In
const sqs = new SQSClient({ region: "use-east-2" });
, “us-east-2” was mistyped!Oh… I think I see what’s going. When I created the deploy the “worker-service” queue got briefly created but then it got deleted because the deploy was rolled back. The other two queues I think just belong to a separate worker service (
queue-worker
) that someone else created.So what I’m going to try to do is deploy without referencing the SQS queue. Then hopefully the deploy will succeed and I can copy the ARN of the created queue and add it to my addon. Then deploy again with the code accessing the queue.
🧐🧐🧐
Yes! You’d use an addon resource. You may have to hardcode the queue ARN in this policy template, but you should be able to look it up either in the console or via
copilot svc show --resources
.Then you’d add the following template into
copilot/svcname/addons/sqs-iam.yml
.Hey @marytal, you’re right that this shouldn’t be an issue. Let me look into this and see if there’s a quick fix.