amgcl: Using null-space vectors with block relaxation
Hi, I use amgcl for large linear elasticity models. The problems only use translational degrees of freedom (3 degrees of freedom per node). Models tend to calculate fastest with block 3x3 aggregation and ilu relaxation. This settings fail to converge for some models, and overall providing null-space vectors seems to help with convergence.
When using null-space vectors, the overall calculation time increases by a lot. Looking at the profiler, it seems that relaxation step is taking a lot more time. Is there any possibility of adjusting (or editing in code) ilu relaxation, so that it behaves similarly to situation, when block solve is being run? If I understand correctly, that would give me best of both routes - coarsening using null-space vectors (better convergence) and block relaxation (faster calculation).
Here are is an example from both runs.
Solver: CG, preconditioner: AMG, coarsener: Aggregation, smoother: ILU0
Solver
======
Type: CG
Unknowns: 1772994
Memory footprint: 54.11 M
Preconditioner
==============
Number of levels: 4
Operator complexity: 1.11
Grid complexity: 1.08
Memory footprint: 3.70 G
level unknowns nonzeros memory
---------------------------------------------
0 1772994 136267422 3.36 G (90.43%)
1 127518 13873716 340.45 M ( 9.21%)
2 6288 539856 13.49 M ( 0.36%)
3 294 11700 122.57 K ( 0.01%)
Convergence status: true
Number of iterations run: 59
Residual error: 7.16936e-07
[Profile: 18.801 s] (100.00%)
[ self: 8.342 s] ( 44.37%)
[ CSR copy: 0.468 s] ( 2.49%)
[ axpby: 0.011 s] ( 0.06%)
[ clear: 0.021 s] ( 0.11%)
[ coarse: 1.530 s] ( 8.14%)
[ coarse operator: 0.699 s] ( 3.72%)
[ coarsest level: 0.001 s] ( 0.01%)
[ copy: 0.009 s] ( 0.05%)
[ inner_product: 1.639 s] ( 8.72%)
[ move to backend: 1.124 s] ( 5.98%)
[ relax: 0.114 s] ( 0.61%)
[ axpby: 0.046 s] ( 0.24%)
[ residual: 0.033 s] ( 0.18%)
[ vmul: 0.035 s] ( 0.18%)
[ relaxation: 3.681 s] ( 19.58%)
[ residual: 0.023 s] ( 0.12%)
[ spmv: 0.049 s] ( 0.26%)
[ transfer operators: 1.089 s] ( 5.79%)
[ self: 0.104 s] ( 0.55%)
[ aggregates: 0.405 s] ( 2.15%)
[ interpolation: 0.580 s] ( 3.09%)
[ tentative: 0.571 s] ( 3.04%)`
Solver: CG, preconditioner: AMG, coarsener: Aggregation, smoother: ILU0
Solver
======
Type: CG
Unknowns: 590998
Memory footprint: 54.11 M
Preconditioner
==============
Number of levels: 4
Operator complexity: 1.03
Grid complexity: 1.04
Memory footprint: 1.37 G
level unknowns nonzeros memory
---------------------------------------------
0 590998 15142730 1.34 G (97.35%)
1 22107 385465 35.99 M ( 2.48%)
2 1774 25390 2.42 M ( 0.16%)
3 192 1974 339.36 K ( 0.01%)
Convergence status: true
Number of iterations run: 118
Residual error: 8.33269e-07
[Profile: 12.139 s] (100.00%)
[ self: 5.816 s] ( 47.91%)
[ CSR copy: 0.574 s] ( 4.73%)
[ axpby: 0.011 s] ( 0.09%)
[ clear: 0.006 s] ( 0.05%)
[ coarse: 1.623 s] ( 13.37%)
[ coarse operator: 0.146 s] ( 1.20%)
[ coarsest level: 0.004 s] ( 0.03%)
[ copy: 0.006 s] ( 0.05%)
[ inner_product: 1.691 s] ( 13.93%)
[ move to backend: 0.506 s] ( 4.17%)
[ relax: 0.135 s] ( 1.11%)
[ axpby: 0.034 s] ( 0.28%)
[ residual: 0.058 s] ( 0.48%)
[ vmul: 0.042 s] ( 0.34%)
[ relaxation: 1.307 s] ( 10.77%)
[ residual: 0.023 s] ( 0.19%)
[ spmv: 0.065 s] ( 0.54%)
[ transfer operators: 0.225 s] ( 1.85%)
[ aggregates: 0.208 s] ( 1.71%)
[ interpolation: 0.005 s] ( 0.04%)
[ tentative: 0.004 s] ( 0.03%)
On an unrelated note, I am also struggling with using smoothed aggregation with null-space vectors. They calculate much longer that standard aggregation, which seems counter-intuitive.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 22 (14 by maintainers)
Sure, I will do it next week.
So far this is only an extension of the builtin backend, but similar approach should work for vexcl.