spring-framework: Aspect Not Triggered After Restart in Spring Boot 3.4.x (But Works in 3.3.10)
In my Spring Boot 3.4.0 based application, I have an @Aspect named TenantServiceAspect that intercepts repository method calls to apply multi-tenancy filters using Hibernate filters.
This aspect works perfectly fine in Spring Boot 3.3.10.
However, during load testing, sometimes when the application is restarted under heavy load, the beforeExecution() method of the aspect is not triggered for certain repository calls, especially this one
The problem is not due to the load test. Since the load test constantly tries to make a request while the project is closed, when I restart the project, it probably causes this error because the request comes when the Spring Boot project is not fully ready.
Spring Boot version: 3.4.0
Java version: 21
Batch rest service
@RestController
@RequiredArgsConstructor
@RequestMapping("batch")
public class BatchService {
private final BatchBusinessService batchBusinessService;
@GetMapping(value = "/findBatchByCode", produces = {MediaType.APPLICATION_JSON_VALUE})
ResponseEntity<ServiceResponse<BatchDto>> findBatchByCode(@RequestParam("code") String code) {
return buildResponseEntity(batchBusinessService.findBatchByCode(code));
};
}
BatchBusinessService class
package com.example.batch.service;
import com.example.batch.data.domain.Batch;
import com.example.batch.data.dto.BatchDto;
import com.example.batch.data.repository.BatchRepository;
import com.example.batch.mapper.BatchMapper;
import lombok.RequiredArgsConstructor;
import org.springframework.stereotype.Component;
@Component
@RequiredArgsConstructor
public class BatchBusinessService {
private final BatchRepository batchRepository;
private final BatchMapper mapper;
public BatchDto findBatchByCode(String code) {
Batch batch = batchRepository.findByCode(code).orElse(null);
return mapper.toDto(batch);
}
}
BatchRepository interface
package com.example.batch.data.repository;
import com.example.batch.data.domain.Batch;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.JpaSpecificationExecutor;
import org.springframework.data.jpa.repository.Query;
import org.springframework.data.repository.query.Param;
import org.springframework.stereotype.Repository;
import java.util.Optional;
@Repository
public interface BatchRepository extends JpaRepository<Batch, Long>, JpaSpecificationExecutor<Batch> {
Optional<Batch> findByCode(String code);
}
Aspect class
package com.example.framework.multitenancy.aspect;
import com.example.logger.LogFactory;
import com.example.logger.Logger;
import jakarta.persistence.EntityManager;
import jakarta.persistence.PersistenceContext;
import org.aspectj.lang.JoinPoint;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Before;
import org.aspectj.lang.annotation.Pointcut;
import org.hibernate.Filter;
import org.hibernate.Session;
import org.springframework.stereotype.Component;
@Aspect
@Component
public class TenantServiceAspect {
private static final Logger LOGGER = LogFactory.getLogger(TenantServiceAspect.class);
@PersistenceContext
public EntityManager entityManager;
public TenantServiceAspect() {
}
@Pointcut("execution(* org.springframework.data.jpa.repository.JpaRepository.*(..))")
void isRepository() {
}
@Pointcut("execution(public * com.example..*.repository..*(..))")
void isCustomRepository() {
}
@Before("(isCustomRepository() || isRepository())")
public void beforeExecution(JoinPoint joinPoint) {
LOGGER.warn("TenantServiceAspect triggered for method: {}", joinPoint.getSignature());
Filter filter = ((Session)this.entityManager.unwrap(Session.class)).enableFilter("companyFilter").setParameter("companyId", 1L);
filter.validate();
}
}
Note: While investigating the root cause, I noticed something strange in Spring Boot’s managed dependencies:
- Spring Boot 3.3.10 uses
aspectjweaverversion 1.9.23 - Spring Boot 3.4.0+ (including 3.4.3) uses
aspectjweaverversion 1.9.22.1
3.3.10
3.4.0
While I haven’t confirmed the exact root cause yet, I’ve noticed that starting from Spring Boot 3.4.0, the spring-boot-starter-data-jpa no longer includes the spring-boot-starter-aop dependency by default:
Release Note: “Remove spring-boot-starter-aop dependency from spring-boot-starter-data-jpa and spring-boot-starter-integration” https://github.com/spring-projects/spring-boot/issues/42934
I’m wondering if this change might somehow be related to the issue I’m facing — specifically, that sometimes my @Aspect (TenantServiceAspect) does not get triggered in version 3.4.0, even though it works fine in 3.3.10.
It’s possible that under some conditions (e.g., app restarts, load), the absence of AOP support could lead to such inconsistent behavior.
Just sharing this in case it helps identify the root cause or helps someone else running into similar issues after upgrading to 3.4.0.
Intellij IDEA Profiler Fail and Successfull Output
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 43 (32 by maintainers)
Commits related to this issue
- Restore synchronization against AspectJ race condition behind PointcutExpression Closes gh-34735 — committed to spring-projects/spring-framework by jhoeller a year ago
Hi, we got the same problem in our application, after spring boot application is running, hitting endpoints in parallel can reproduce this problem.
Enabling this debugging log:
<logger name="org.springframework.aop.aspectj.AspectJExpressionPointcut" level="DEBUG" />revealed this error:
after this exception, aspect stops working. Probably because this line caches shadowMatch that always returns false, so every new aspect match will hit this cache and skip shadowMatch evaluation, this is probably the reason why aspect stays in the “corrupted” state and is always skipped.
I found this change set which maybe caused this problem.
Can someone who is more familiar with springframework check this out please.
@jhoeller Even though I cannot say for certain that it is related to the issue here, we observed strange behavior today with one of our applications where a security-relevant aspect, which should protect the execution of a method, was not called on one of several instances. I was able to connect to all instances of the application via the debugger and can see in the stack trace that on one instance the authorization-relevant aspect is not called, while it is called as expected on other instances.
According to our testing department, authorization-related errors seem to occur sporadically since a few months, which would generally coincide with the dependency upgrades to Spring Boot 3.4.
What I want to say with this is that if there is a currently unresolved problem with aspects in Spring, then this can have security-relevant consequences in the worst case, for example, if people rely on aspects to protect certain execution paths in the code.
Against this background, these aspect-related problems should be investigated with high priority.
Unfortunately, I cannot currently identify a real trigger on our end. We have been seeing the mentioned faulty behavior for some time in the context of automated tests, which probably send a large number of requests to the application in a short period of time. However, that is the only parallel I can see to this issue here.
@lgp547 good catch, it looks like that
ReferenceType.getDeclaredMethods()implementation in AspectJ is not thread-safe for parameterized methods since it initializes theparameterizedMethodsarray first and then assigns each element in a loop - which leaves a race condition where the initialized array may be accessed by other threads which might seenullelements then. That aside, there seems to be novolatilemarker orsynchronizedguard either. This should be reported to the AspectJ project but I’m not sure how long it will take them to revise this.In any case, I’ll restore a degree of locking for adding entries to Spring’s ShadowMatch cache for 6.2.8, for defensiveness against concurrency issues in AspectJ’s current implementation. I expect this to resolve the problem in all scenarios reported here.
@bpavin good point, this may also be a race condition within AspectJ’s matching infrastructure that we are uncovering through hitting it concurrently (and then ignoring the re-built
ShadowMatchinstances). Aside from staleShadowMatchinstances on restart, that’s actually an even more likely cause.@bpavin this looks like a potential cause indeed: a race condition in AspectJ
ShadowMatchcache handling, caused by our revised caching in 6.2 - and for some reason falling into a workaround for an old AspectJ bug (“PointcutExpression matching rejected target method”) which silently swallows the exception and unfortunately skips the affected aspect. Thanks for digging into it so deeply!@msievers @otbe we are aiming to resolve this for next week’s 6.2.8 release. If you could confirm that the same debug log message happens in your scenario as well, that would help to focus our efforts accordingly.
@jhoeller I tested the snapshot, and I am no longer able to reproduce the issue, so we can consider it fixed. Thank you for the help and fast fix.
@msievers @bpavin @lgp547 a revision is available in the latest 6.2.8 snapshot now, synchronizing on the AspectJ
PointcutExpressioninstance itself (where the actual AspectJ race condition is hidden). Please give it a try in your scenario before the 6.2.8 release (scheduled for Thursday next week)…We can see the same behaviour with SB 3.4.5 (but most likely also earlier versions). An aspect that should trigger the execution of another method is not being executed on one out of three instances in a production deployment of the same version. Rotating the affected container(s) solves this issues most of the times. In our case that aspect is triggered within a controller method when invoking a service bean method (around aspect for this method). Theres no Spring Data involved here.
Ah yes, good point. A concurrency level of 50 is pretty high in any case, and somehow able to trigger a race condition that otherwise remains unnoticed.
The key question is how and when that aspect is being applied to that repository bean. There must be some laziness involved in the initialization there, otherwise it couldn’t be exposed to that concurrency race condition and result in permanent application state.