iree: Not able to fold set_encoding(constant) -> constant in data tiling path
Coming from https://github.com/iree-org/iree/pull/11284, the set_encoding(constant) is not folded into a constant. This happens automatically in legacy mmt4d path because the set_encoding op is expand_reshape + transpose. The fusion on tensors can fold expand_reshape(constant) into a constant; a pattern transposes the constant.
If we can fold set_encoding(constant) -> constant, we can save ~360 flow.dispatch calls for FP32 mobilebert. There are ~2700 kernel launches in total, which means that we can save ~13% of kernel launch.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 45 (29 by maintainers)
Commits related to this issue
- Enable fusion for elementwise Linalg op + pack op (#11284) It also updates the LinalgExtVectorization to use tile+fuse, so we can tile+fuse the generic ops. Some metric data w/ mobilebert fp32: ... — committed to iree-org/iree by hanhanW 2 years ago
- Enable fusion for elementwise Linalg op + pack op (#11284) It also updates the LinalgExtVectorization to use tile+fuse, so we can tile+fuse the generic ops. Some metric data w/ mobilebert fp32: The... — committed to hanhanW/iree by hanhanW 2 years ago
- Enable fusion for elementwise Linalg op + pack op (#11284) It also updates the LinalgExtVectorization to use tile+fuse, so we can tile+fuse the generic ops. Some metric data w/ mobilebert fp32: ... — committed to harsh-nod/iree by hanhanW 2 years ago
- Enable fusion for elementwise Linalg op + pack op (#11374) It also updates the LinalgExtVectorization to use tile+fuse, so we can tile+fuse the generic ops. Some metric data w/ mobilebert fp32: ... — committed to iree-org/iree by hanhanW 2 years ago
- Enable ConstEval for CPU data tiling path. (#14792) - Moves SetEncoding pass to GlobalOptimization stage, and renames `iree-flow-enable-data-tiling` to `iree-opt-enable-data-tiling` - Introduces Mat... — committed to iree-org/iree by hanhanW 10 months ago
- Enable ConstEval for CPU data tiling path. (#14792) - Moves SetEncoding pass to GlobalOptimization stage, and renames `iree-flow-enable-data-tiling` to `iree-opt-enable-data-tiling` - Introduces Mat... — committed to jinchen62/iree by hanhanW 10 months ago
Could we discuss this on GVC sometime… They might be a different solution than what is being discussed here.
We’ll need to fix it properly (no flag) but I don’t think we need to do that all in one step to start - I say go for the workaround and we make sure we’re tracking removing it as part of the larger data tiling work.
I got an idea to unblock the actual users while we figure this out. Current users have dimensions that are static and are multiples of sufficiently large powers of two that they could use a flag of the form
--iree-flow-data-tiling-assume-all-tile-sizes-divide=16that would allow SetEncoding to eschew padding.This should be working like this already: we should always use the HALExecutableTargetAttr::lookup helper, which should check for a target attribute first before walking up to the enclosing variant.