antlr4: Golang target extremly slow, while Java target is fast on same input
I need to parse MySQL for a project and use https://github.com/antlr/grammars-v4/tree/master/mysql this grammar with the newest ANTLR from the master branch (4.7.1-SNAPSHOT). On certain queries the parser spends a lot of time in the golang target (25-27sec), while the Java target is much faster (700ms-1sec). The IntelliJ Plugin even parses faster (~300ms). Example query:
SELECT SUM((IFNULL(X, 0) - IFNULL(Y, 0) - IFNULL(Z, 0) - (IFNULL(XX, 0) - IFNULL(YY, 0) - IFNULL(ZZ, 0))) * IFNULL(ASDF, 0)) AS `REVENUE` FROM SALES_FLAT
I used SLL PredicionMode on both targets, but that does not help. I also ran the query multiple times, but warming up also didn’t help much. Afterwards I profiled the Go target and got the following output for my query mentioned above:
flat flat% sum% cum cum%
5980ms 14.28% 14.28% 16020ms 38.26% runtime.scanobject C:\dev\Go\src\runtime\mgcmark.go
4970ms 11.87% 26.15% 5430ms 12.97% runtime.heapBitsForObject C:\dev\Go\src\runtime\mbitmap.go
3280ms 7.83% 33.99% 8070ms 19.27% github.com/antlr/antlr4/runtime/Go/antlr.(*BaseSingletonPredictionContext).hash C:\dev\projects\go\src\github.com\antlr\antlr4\runtime\Go\antlr\prediction_context.go
2620ms 6.26% 40.24% 2620ms 6.26% github.com/antlr/antlr4/runtime/Go/antlr.murmurUpdate C:\dev\projects\go\src\github.com\antlr\antlr4\runtime\Go\antlr\utils.go (inline)
2070ms 4.94% 45.19% 2070ms 4.94% github.com/antlr/antlr4/runtime/Go/antlr.murmurFinish C:\dev\projects\go\src\github.com\antlr\antlr4\runtime\Go\antlr\utils.go (inline)
1240ms 2.96% 48.15% 1240ms 2.96% runtime.markBits.isMarked C:\dev\Go\src\runtime\mbitmap.go (inline)
980ms 2.34% 50.49% 1540ms 3.68% runtime.evacuate C:\dev\Go\src\runtime\hashmap.go
960ms 2.29% 52.78% 960ms 2.29% runtime.spanClass.noscan C:\dev\Go\src\runtime\mgcmark.go (inline)
870ms 2.08% 54.86% 3650ms 8.72% runtime.greyobject C:\dev\Go\src\runtime\mgcmark.go
840ms 2.01% 56.87% 1000ms 2.39% runtime.heapBitsSetType C:\dev\Go\src\runtime\mbitmap.go
Of course the grammar is not optimized, but still the difference between the targets is tremendous.
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Comments: 20 (10 by maintainers)
@jjeffcaii Thanks,It is effective. For the example in antlr/grammars-v4#2327, it is completed in 0s. For the large files in my project, change from 1000s to 200s.
@kirito41dd I did some tuning work, see https://github.com/antlr/antlr4/pull/3243, but the PR hasn’t been merged yet. You can try my fork repo instead, I believe that will be faster. 🚀