micrometer: ObservationThreadLocalAccessor.restore() generating errors in tests
Describe the bug
When upgrading from spring-boot 3.0.4 (micrometer 1.10.4) to spring-boot 3.0.8 or 3.1.1 (micrometer 1.11.1), some of my tests are now failing with a message comparing a NoopObservation
with null :
Observation
<io.micrometer.observation.NoopObservation@390cd91e>
to which we’re restoring is not the same as the one set as this scope’s parent observation<null>
. Most likely a manually created Observation has a scope opened that was never closed. This may lead to thread polluting and memory leaks
It seems related to #3831 which introduced NullObservation
and the verification in ObservationThreadLocalAccessor.restore()
.
Environment
- Micrometer version 1.11.1
- OS: Ubuntu
- Java version: openjdk version “17.0.1” 2021-10-19
To Reproduce
I’m trying to extract a test case from my own code to show how to reproduce it. I will post it here once I’m able to reproduce it on a sample application.
For the moment, it seems related to observations created by Mono.tap(Micrometer.observation(registry))
.
Exemple of test that fails with the new version :
@Test
@DisplayName("Save when the bucket does not exist. Must terminate with errors.")
void saveWhenBucketDoesNotExist() throws IOException {
// GIVEN
final Resource fileResource = ClasspathResourceLoader.fromFile("input/ID-card.png");
// WHEN
final Mono<FileReference> responseFileReferenceMono =
s3Repository.save(fileResource.getFile().toPath());
// THEN
StepVerifier.create(responseFileReferenceMono)
.expectError(NoSuchBucketException.class)
.verify();
}
Code under test :
public Mono<FileReference> save(final Path path) {
return save(AsyncRequestBody.fromFile(path), path.toFile().length());
}
private Mono<FileReference> save(
final AsyncRequestBody asyncRequestBody,
final long length) {
return Mono.fromFuture(() -> putObject(asyncRequestBody, fileReference, length))
.name("s3 put-object")
.doOnError(NoSuchBucketException.class, e -> LOGGER.error("Unknown bucket"))
.tap(Micrometer.observation(observationService.registry()))
.thenReturn(fileReference);
}
private CompletableFuture<PutObjectResponse> putObject(
final AsyncRequestBody asyncRequestBody,
final long length) {
final PutObjectRequest request =
PutObjectRequest.builder()
.bucket("test")
.key("myKey")
.contentLength(length)
.contentType(MediaType.APPLICATION_OCTET_STREAM.getType())
.build();
return s3client.putObject(request, asyncRequestBody);
}
Expected behavior Test should not fail when testing an error happening inside a reactive stream which is observed.
Additional context
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 15 (14 by maintainers)
Commits related to this issue
- NoOpRegistry and NoOpObservation respect scopes without this change there was a problem with OTLA, default ObservationRegistry and NoOpRegistry interop. They didn't see each other scopes with this c... — committed to micrometer-metrics/micrometer by marcingrzejszczak a year ago
- NoOpRegistry and NoOpObservation respect scopes (#3947) * NoOpRegistry and NoOpObservation respect scopes without this change there was a problem with OTLA, default ObservationRegistry and NoOpReg... — committed to micrometer-metrics/micrometer by marcingrzejszczak a year ago
Yes, I merged it forward
Fantastic! Thank you for checking it out. I’ll close the issue for now and will reopen it if there are any issues
Can someone check the latest micrometer snapshots if they helped?
I’m starting to think that OTLA should take into account whether we’re using a NOOP observation registry or sth like that. I’ll also look into this topic