amgcl: Using null-space vectors with block relaxation

Hi, I use amgcl for large linear elasticity models. The problems only use translational degrees of freedom (3 degrees of freedom per node). Models tend to calculate fastest with block 3x3 aggregation and ilu relaxation. This settings fail to converge for some models, and overall providing null-space vectors seems to help with convergence.

When using null-space vectors, the overall calculation time increases by a lot. Looking at the profiler, it seems that relaxation step is taking a lot more time. Is there any possibility of adjusting (or editing in code) ilu relaxation, so that it behaves similarly to situation, when block solve is being run? If I understand correctly, that would give me best of both routes - coarsening using null-space vectors (better convergence) and block relaxation (faster calculation).

Here are is an example from both runs.

Solver: CG, preconditioner: AMG, coarsener: Aggregation, smoother: ILU0
Solver
======
Type:             CG
Unknowns:         1772994
Memory footprint: 54.11 M
Preconditioner
==============
Number of levels:    4
Operator complexity: 1.11
Grid complexity:     1.08
Memory footprint:    3.70 G
level     unknowns       nonzeros      memory
---------------------------------------------
    0      1772994      136267422      3.36 G (90.43%)
    1       127518       13873716    340.45 M ( 9.21%)
    2         6288         539856     13.49 M ( 0.36%)
    3          294          11700    122.57 K ( 0.01%)
Convergence status: true
Number of iterations run: 59
Residual error: 7.16936e-07
[Profile:                 18.801 s] (100.00%)
[ self:                    8.342 s] ( 44.37%)
[  CSR copy:               0.468 s] (  2.49%)
[  axpby:                  0.011 s] (  0.06%)
[  clear:                  0.021 s] (  0.11%)
[  coarse:                 1.530 s] (  8.14%)
[  coarse operator:        0.699 s] (  3.72%)
[  coarsest level:         0.001 s] (  0.01%)
[  copy:                   0.009 s] (  0.05%)
[  inner_product:          1.639 s] (  8.72%)
[  move to backend:        1.124 s] (  5.98%)
[  relax:                  0.114 s] (  0.61%)
[    axpby:                0.046 s] (  0.24%)
[    residual:             0.033 s] (  0.18%)
[    vmul:                 0.035 s] (  0.18%)
[  relaxation:             3.681 s] ( 19.58%)
[  residual:               0.023 s] (  0.12%)
[  spmv:                   0.049 s] (  0.26%)
[  transfer operators:     1.089 s] (  5.79%)
[   self:                  0.104 s] (  0.55%)
[    aggregates:           0.405 s] (  2.15%)
[    interpolation:        0.580 s] (  3.09%)
[      tentative:          0.571 s] (  3.04%)`
Solver: CG, preconditioner: AMG, coarsener: Aggregation, smoother: ILU0
Solver
======
Type:             CG
Unknowns:         590998
Memory footprint: 54.11 M
Preconditioner
==============
Number of levels:    4
Operator complexity: 1.03
Grid complexity:     1.04
Memory footprint:    1.37 G
level     unknowns       nonzeros      memory
---------------------------------------------
    0       590998       15142730      1.34 G (97.35%)
    1        22107         385465     35.99 M ( 2.48%)
    2         1774          25390      2.42 M ( 0.16%)
    3          192           1974    339.36 K ( 0.01%)
Convergence status: true
Number of iterations run: 118
Residual error: 8.33269e-07
[Profile:                 12.139 s] (100.00%)
[ self:                    5.816 s] ( 47.91%)
[  CSR copy:               0.574 s] (  4.73%)
[  axpby:                  0.011 s] (  0.09%)
[  clear:                  0.006 s] (  0.05%)
[  coarse:                 1.623 s] ( 13.37%)
[  coarse operator:        0.146 s] (  1.20%)
[  coarsest level:         0.004 s] (  0.03%)
[  copy:                   0.006 s] (  0.05%)
[  inner_product:          1.691 s] ( 13.93%)
[  move to backend:        0.506 s] (  4.17%)
[  relax:                  0.135 s] (  1.11%)
[    axpby:                0.034 s] (  0.28%)
[    residual:             0.058 s] (  0.48%)
[    vmul:                 0.042 s] (  0.34%)
[  relaxation:             1.307 s] ( 10.77%)
[  residual:               0.023 s] (  0.19%)
[  spmv:                   0.065 s] (  0.54%)
[  transfer operators:     0.225 s] (  1.85%)
[    aggregates:           0.208 s] (  1.71%)
[    interpolation:        0.005 s] (  0.04%)
[      tentative:          0.004 s] (  0.03%)

On an unrelated note, I am also struggling with using smoothed aggregation with null-space vectors. They calculate much longer that standard aggregation, which seems counter-intuitive.

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 22 (14 by maintainers)

Commits related to this issue

Most upvoted comments

Sure, I will do it next week.

So far this is only an extension of the builtin backend, but similar approach should work for vexcl.