antlr4: Golang target extremly slow, while Java target is fast on same input

I need to parse MySQL for a project and use https://github.com/antlr/grammars-v4/tree/master/mysql this grammar with the newest ANTLR from the master branch (4.7.1-SNAPSHOT). On certain queries the parser spends a lot of time in the golang target (25-27sec), while the Java target is much faster (700ms-1sec). The IntelliJ Plugin even parses faster (~300ms). Example query:

SELECT SUM((IFNULL(X, 0) - IFNULL(Y, 0) - IFNULL(Z, 0) - (IFNULL(XX, 0) - IFNULL(YY, 0) - IFNULL(ZZ, 0))) * IFNULL(ASDF, 0)) AS `REVENUE` FROM SALES_FLAT

I used SLL PredicionMode on both targets, but that does not help. I also ran the query multiple times, but warming up also didn’t help much. Afterwards I profiled the Go target and got the following output for my query mentioned above:

  flat  flat%   sum%        cum   cum%
5980ms 14.28% 14.28%    16020ms 38.26%  runtime.scanobject C:\dev\Go\src\runtime\mgcmark.go
4970ms 11.87% 26.15%     5430ms 12.97%  runtime.heapBitsForObject C:\dev\Go\src\runtime\mbitmap.go
3280ms  7.83% 33.99%     8070ms 19.27%  github.com/antlr/antlr4/runtime/Go/antlr.(*BaseSingletonPredictionContext).hash C:\dev\projects\go\src\github.com\antlr\antlr4\runtime\Go\antlr\prediction_context.go
2620ms  6.26% 40.24%     2620ms  6.26%  github.com/antlr/antlr4/runtime/Go/antlr.murmurUpdate C:\dev\projects\go\src\github.com\antlr\antlr4\runtime\Go\antlr\utils.go (inline)
2070ms  4.94% 45.19%     2070ms  4.94%  github.com/antlr/antlr4/runtime/Go/antlr.murmurFinish C:\dev\projects\go\src\github.com\antlr\antlr4\runtime\Go\antlr\utils.go (inline)
1240ms  2.96% 48.15%     1240ms  2.96%  runtime.markBits.isMarked C:\dev\Go\src\runtime\mbitmap.go (inline)
 980ms  2.34% 50.49%     1540ms  3.68%  runtime.evacuate C:\dev\Go\src\runtime\hashmap.go
 960ms  2.29% 52.78%      960ms  2.29%  runtime.spanClass.noscan C:\dev\Go\src\runtime\mgcmark.go (inline)
 870ms  2.08% 54.86%     3650ms  8.72%  runtime.greyobject C:\dev\Go\src\runtime\mgcmark.go
 840ms  2.01% 56.87%     1000ms  2.39%  runtime.heapBitsSetType C:\dev\Go\src\runtime\mbitmap.go

Of course the grammar is not optimized, but still the difference between the targets is tremendous.

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Comments: 20 (10 by maintainers)

Most upvoted comments

@jjeffcaii Thanks,It is effective. For the example in antlr/grammars-v4#2327, it is completed in 0s. For the large files in my project, change from 1000s to 200s.

@kirito41dd I did some tuning work, see https://github.com/antlr/antlr4/pull/3243, but the PR hasn’t been merged yet. You can try my fork repo instead, I believe that will be faster. 🚀

module xxxxxx

go 1.16

require(
	github.com/antlr/antlr4/runtime/Go/antlr v0.0.0-20210803070921-b358b509191a
)

replace github.com/antlr/antlr4/runtime/Go/antlr v0.0.0-20210803070921-b358b509191a => github.com/jjeffcaii/antlr4/runtime/Go/antlr v0.0.0-20210805132115-88c2159aab9e