firebase-tools: Cloud Tasks is not supported by the Firebase Emulator Suite

Any news on the state of the emulator ?

The way my team handled this was to create an internal mock for the CloudTaskClient that is used when our local flag is set. It just sends the request straight to the local emulated function instead of interacting with the Cloud Tasks client at all, which allows the function to be ran locally. It bypasses the queue functionality entirely, but that has been fine for us during local development since the rate of requests are so low and we usually just need the functions to run. Here is my implementation of the mock file if it is useful to you all.

const fetch = require("node-fetch");

class CloudTasksMock {
  constructor() {}

  queuePath(project, location, queue) {
    return `projects/${project}/locations/${location}/queues/${queue}`;
  }

  /**
   *
   * @param { import ("@google-cloud/tasks").protos.google.cloud.tasks.v2.ICreateTaskRequest } request
   */
  async createTask(request) {
    const { parent } = request;
    const httpRequest = request.task.httpRequest;

    console.log(
      `Received local queue request for parent ${parent}. Sending straight to function...`
    );

    if (httpRequest) {
      const { httpMethod, url, body } = httpRequest;

      const res = await fetch(url, {
        method: httpMethod,
        headers: {
          "content-type": "application/json",
        },
        //need to make sure to undo the base64 encoding before dispatching
        body: Buffer.from(body, "base64").toString("ascii"),
      });
      return [res];
    } else {
      throw new Error("cloudTasksMock - httpRequest is undefined");
    }
  }
}

module.exports = {
  CloudTasksMock,
};

Note that our Cloud Task processor functions are http.onRequest functions, and we have only been able to get these working properly with Cloud Task queues when the body is encoded in base64. Looking at my code it doesn’t make much sense that I convert it back to ascii before sending to the local function and then perform another conversion in my local cloud function, but it works fine from what I can tell. Here is a small snippet of my Cloud Task processor function for context:

exports.cloudTaskProcessor = functions
  .runWith({
    memory: "512MB",
    timeoutSeconds: 540,
  })
  .https.onRequest(async (req, res) => {
    const reqBody = JSON.parse(
      Buffer.from(req.rawBody, "base64").toString("ascii")
    );
    ...
});

If nothing else, better explanation of the limitation of queueing tasks in the emulator environment should be included in the documentation. Would have saved a lot of time before finding this thread.

I agree that this is disappointing but since you found this thread you may workaround using some of the solutions provided. I feel your disappointment here. But let’s assume that the people that provided this useful toolset may just be pretty occupied and not simply mean.

I love the emulator but I agree that it would also be lovely to have the task queue there as well so a +1 from me 😃

+1

Any plan to support it? It’s very useful.

@NuclearGandhi Thanks for writing up a detailed issue.

Sadly, Emulator doesn’t support Cloud Tasks functions today - there isn’t a “Google Cloud Task Emulator” that’s hooked up to the Firebase Emulator Suite.

We also think it would be a great addition to the Emulator Suite, but we are focusing on other areas of improvement at the moment. I’m going to convert this issue into a feature request and will try to remember to post update here.

@NuclearGandhi Thanks for writing up a detailed issue.

Sadly, Emulator doesn’t support Cloud Tasks functions today - there isn’t a “Google Cloud Task Emulator” that’s hooked up to the Firebase Emulator Suite.

We also think it would be a great addition to the Emulator Suite, but we are focusing on other areas of improvement at the moment. I’m going to convert this issue into a feature request and will try to remember to post update here.

It’s frankly ridiculous that this still isn’t supported after almost a year, given that the official code example enqueues a task directly from a cloud function.

There’s no way to test locally a cloud task being called from a cloud function, despite it being the main use case. Do I have that right?

+1 for making this smoothly in the local emulator. Yes, there are workarounds which can get us there, but it is a friction point for working with this easily. Surely a useful addition which I hope will be shipped at some point by the team.

This was my solution in Typescript. I had to separate the task function from the actual task creation.

This is an example code for the emulator enqueuing the selectBiddingWinner task.

import {TaskQueue as FTaskQueue} from "firebase-admin/functions";

declare interface TaskQueue {
    functionName: string;
    client: unknown;
    extensionId?: string;
    enqueue(data: Record<string, string>, opts?: TaskOptions): Promise<void>
}

if (process.env.FUNCTIONS_EMULATOR) {
    FTaskQueue.prototype.enqueue = function (this: TaskQueue, data: Record<string, string>, opts?: TaskOptions) {
        logger.debug(this.functionName, data);
        return new Promise(() => {
            if (this.functionName == "locations/southamerica-east1/functions/selectBiddingWinnerTask") {
                return selectBiddingWinner(data.orderId);
            } else {
                return;
            }
        });
    };
}

This is an example of the task definition

exports.selectBiddingWinnerTask = tasks.taskQueue({
    retryConfig: {
        maxAttempts: 5,
        minBackoffSeconds: 3,
    },
    rateLimits: {
        maxConcurrentDispatches: 10,
    },
}).onDispatch(async (data: Record<string, string>) => {
    const orderId = data.orderId;
    return selectBiddingWinner(orderId)
        .then((resp) => logger.info(resp))
        .catch((e) => logger.error(`selectWinner failed for order:${orderId}`, e));
});

selectBiddingWinner is the actual function executed.

I’ll be glad to receive comments/corrections that would help me generalize the code 😃

+1

Like @Bastianowicz, this is my workaround. Put the code below just after your initializeApp(), it will display the message send to your queue.

Console

i  functions: Loaded environment variables from .env, .env.meeting-work-01, .env.local.
i  functions: Beginning execution of "meeting.task.sendCreatedAndFinishedEvents"
>  tasks:  { id: '4vu1i5g4cn4v93t2dvmfnt9ar6@google.com', status: 'started' } { scheduleTime: 2023-03-07T08:00:00.000Z }
>  tasks:  { id: '4vu1i5g4cn4v93t2dvmfnt9ar6@google.com', status: 'finished' } { scheduleTime: 2023-03-07T09:00:00.000Z }
i  functions: Finished "meeting.task.sendCreatedAndFinishedEvents" in 5.472542ms

Solution

import { initializeApp } from 'firebase-admin/app'
import { getFunctions, TaskQueue } from 'firebase-admin/functions'

const app = initializeApp()
if (process.env.FUNCTIONS_EMULATOR) {
    Object.assign(TaskQueue.prototype, {
      enqueue: (data: any, params: any) =>
        console.debug('tasks: ', data, params),
  })
}
const functions = getFunctions(app)