resilience4j: Need help understanding why rate limiter doesn't behave like I think it should?
Hi,
I’m using resilience4j and the rate-limiting module together with Spring project reactor. The code I have looks essentially like this:
public class App {
private final WebClient webClient;
private final InMemoryRateLimiterRegistry rateLimiterRegistry;
public App(int port, int rps) {
webClient = WebClient.builder()
.baseUrl("http://localhost:" + port)
.build();
rateLimiterRegistry = new InMemoryRateLimiterRegistry(RateLimiterConfig.ofDefaults());
rateLimiterRegistry.rateLimiter("test", RateLimiterConfig.custom()
.limitRefreshPeriod(Duration.ofSeconds(1))
.limitForPeriod(rps)
.timeoutDuration(Duration.ofHours(1))
.build());
}
public Mono<String> makeRequest() {
RateLimiter rateLimiter = rateLimiterRegistry.rateLimiter("test");
Mono<String> result = webClient.post()
.uri("/testing")
.accept(ALL)
.contentType(TEXT_PLAIN)
.syncBody("hello world")
.retrieve().bodyToMono(String.class);
return result.transform(RateLimiterOperator.of(rateLimiter));
}
}
I’ve tried creating a test case in which I’d like to verify that the rate limiter actually limits the requests per second to 10. The test case looks like this:
public class AppTest {
@Rule
public WireMockRule wireMockRule = new WireMockRule();
@Test
public void whyDoesntThisWorkQuestionMarkQuestionMark() {
// Given
int numberOfRequests = 30;
int rps = 10;
App app = new App(wireMockRule.port(), rps);
wireMockRule.addStubMapping(stubFor(post(urlPathEqualTo("/testing"))
.willReturn(aResponse().withStatus(200).withBody("hello hello!"))));
// When
ParallelFlux<String> flux = Flux.range(0, numberOfRequests)
.parallel()
.runOn(Schedulers.parallel())
.flatMap(___ -> app.makeRequest());
long startTime = new Date().getTime();
StepVerifier.create(flux).expectNextCount(numberOfRequests).verifyComplete();
long endTime = new Date().getTime();
assertThat(endTime - startTime)
.describedAs("I don't understand why this is not taking longer than this")
.isGreaterThanOrEqualTo((numberOfRequests / rps) * 1000);
}
}
But this fails with:
whyDoesntThisWorkQuestionMarkQuestionMark(se.haleby.AppTest) Time elapsed: 2.376 sec <<< FAILURE!
java.lang.AssertionError: [I don't understand why this is not taking longer than this]
Expecting:
<1466L>
to be greater than or equal to:
<3000L>
at se.haleby.AppTest.whyDoesntThisWorkQuestionMarkQuestionMark(AppTest.java:43)
I don’t really understand why the test is only taking roughly 1,5 seconds instead of the expected 3 seconds (since I’m making 30 requests with an RPS of 10 requests per second). I’m probably missing something quite essential here and I’m ready to be schooled 😃.
My code can be found here: https://github.com/johanhaleby/resilience4j-ratelimiter-question
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 24 (22 by maintainers)
I’ve created a PR where I test a delay of a subscription.
@martin-tarjanyi
Please see: https://github.com/resilience4j/resilience4j/blob/0ab113cb7caa4c60a93e13dcc97d2461c0653569/resilience4j-reactor/src/main/java/io/github/resilience4j/reactor/ratelimiter/operator/MonoRateLimiter.java#L37-L47
@RobWin Oh, that probably explains it. I think I get it now and it makes sense. Thanks a lot for taking the time to explain this, really appreciated. And thanks for all the work on resilience4j.
The timeout seems to be a quite essential part of the rate limiter. In most of our use cases we don’t want to reject calls, but wait instead until we have permission again. Wouldn’t it be possible to implement the waiting in a non-blocking way in the reactor module?
For this use case with the current implementation you need to do some retry for the not permitted exception (like it’s done in @johanhaleby’s example code) which is not a fair implementation, since new calls can get ahead of older calls.
You can see this better when you increase the
limitRefreshPeriodto 5 or 10 seconds.When I use 5 seconds, the total measured time is: ~13 seconds (~4+5+4) When I use 10 seconds, the total measured time is: ~23 seconds (~5+10+8)
Just to make sure that there is no misunderstand. Our RateLimiter is not an implementation of a leaky bucket algorithm which leaks out at a constant rate. It uses a fixed window algorithm which can have burst effects at time window boundaries. The RateLimiter allows 10 requests per second in your case and rejects every further call with a
RequestNotPermitted. Since reactive streams must be non-blocking, thetimeoutDurationis not taken into account, but instead it’s always 0.The implementation is described here: https://resilience4j.readme.io/docs/ratelimiter
This is a nice blog post in general: https://konghq.com/blog/how-to-design-a-scalable-rate-limiting-algorithm/
I debug your test app tomorrow and come back to you.
The behavior changes in 0.15.0 because of #311 The rate limiter will only rate limit subscriptions and not events anymore. Your test should work as expected with the new upcoming release.