nest: It is not possible to use RegEx for event matching on the @EventPattern() adn @MessagePattern()

Bug Report

Current behavior

It is not possible to use a RegEx for event and message patterns. This is possible when using KafkaJS library but is not possible in NestJS

Input Code

Having a controller emitting a message 'pre-notify-post'

@Post('notify-with-regex')
  async sendNotificationForRegExMatching(): Promise<any> {
    return this.client.emit('pre-notify-post', { notify: true });
  }

Also having a controller with the event handler and expecting messages that match the regular expression of /.*notify.*/ :

 @EventPattern(/.*notify.*/)
  secondEventHandler(data: any) {
    KafkaController.IS_NOTIFIED_WITH_REGEX = data.value.notify;
  }

Currently, the event handler never gets called, because the handler is not matched using regular expressions.

Expected behavior

It is expected to listen to events and messages by using RegEx patterns. This is possible in KafkaJS library and it is useful when having a set of a topic that the consumer wants to listen to.

Alternatively, you can subscribe to multiple topics at once using a RegExp:

await consumer.connect() await consumer.subscribe({ topic: /topic-(eu|us)-.*/i })

https://kafka.js.org/docs/consuming

Possible Solution

This could be fixed by getting the matching routes.

Environment

Nest version: 6.7

About this issue

Original URL
State: open
Created 5 years ago
Reactions: 6
Comments: 31 (1 by maintainers)

Commits related to this issue

feat(microservices) add capability to use RegExp in Kafka Events closes #3083 — committed to alfredoperez/nest by alfredoperez 5 years ago
feat(microservices): Allow RegExps in Patterns for Kafka closes #3083 — committed to edeesis/nest by edeesis 2 months ago
feat(microservices): Allow RegExps in Patterns for Kafka closes #3083 — committed to edeesis/nest by edeesis 2 months ago
feat(microservices): Allow RegExps in Patterns for Kafka closes #3083 — committed to edeesis/nest by edeesis 2 months ago

Most upvoted comments

Hi, any update on this?

Idan747 on Oct 13, 2021

NestJS feels like a collection of leaky abstractions with missing features and edge cases creeping there to surprise you every now and then and the official docs don’t go into the details deeply enough to prepare for those things so you discover them only as you go 😦

PiotrOwsiak on Jun 7, 2023

Updates?

buccfer-knauf on Oct 5, 2023

Any updates on this? Without this feature kafka is pretty much useless for us.

Vetm on May 10, 2021

I also need this feature. Some microservices need to handle different kafka topics in the same way. We have to write a separate listener for each topic.

inmativ on Jul 15, 2021

Hello! Any updates? We also stuck with this.

vviital on Jul 29, 2021

Thank you @fjodor-rybakov

We have a topic for each customer such as <<customerId>>.traffic. It is impossible to list them all in @EventPattern. Also when new customers are onboarded, new topics will be added.

We try to do with @EventPattern('/*.traffic/'), but it does not work.

Or, do I misunderstand your suggestion?

ww917352 on Aug 4, 2022

Hi everybody,

Thanks for the discussion and posting workarounds.

My question is: do these workarounds also support topics that get created in future? If my app is deployed, and then a topic whose name match the regex gets created, will the app get subscribed to this new topic?

Thank you!

Cheers, Wei

No, subscribe on topic only happens when consumer starts

You can pass a list of matching topics at once

@EventPattern(['topic.one', 'topic.two'])

I think this will solve your problem

fjodor-rybakov on Aug 4, 2022