rook: EC filesystem fails to deploy with latest release of Rook-Ceph

Is this a bug report or feature request?

• Bug Report

Deviation from expected behavior: EC filesystem is not created

Expected behavior: Everything correctly created

How to reproduce it (minimal and precise): I created on AKS a K8s cluster with 6 nodes. 3 OSDs. The deploy of normal filesystem works just fine, I used the example crds.yaml, common.yaml,

File(s) to submit:

• Cluster CR (custom resource), typically called cluster.yaml, if necessary

apiVersion: ceph.rook.io/v1
kind: CephCluster
metadata:
  name: rook-ceph
  namespace: rook-ceph # namespace:cluster
spec:
  cephVersion:
    image: ceph/ceph:v16.2.4
    allowUnsupported: false
  skipUpgradeChecks: false
  continueUpgradeAfterChecksEvenIfNotHealthy: false
  waitTimeoutForHealthyOSDInMinutes: 10
  mon:
    count: 3
    allowMultiplePerNode: false
  mgr:
    count: 1
    modules:
      - name: pg_autoscaler
        enabled: true
  dashboard:
    enabled: true
    ssl: true
  monitoring:
    enabled: false
    rulesNamespace: rook-ceph
  network:
  crashCollector:
    disable: false
  cleanupPolicy:
    confirmation: ""
    sanitizeDisks:
      method: quick
      dataSource: zero
      iteration: 1
    allowUninstallWithVolumes: false
  annotations:
  labels:
  resources:
  removeOSDsIfOutAndSafeToRemove: false
  storage: # cluster level storage configuration and selection
    storageClassDeviceSets:
    - name: set1
      count: 3
      portable: false
      placement:
        tolerations:
        - key: storage-node
          operator: Exists
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: agentpool
                operator: In
                values:
                - npstorage
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - rook-ceph-osd
                - key: app
                  operator: In
                  values:
                  - rook-ceph-osd-prepare
              topologyKey: kubernetes.io/hostname
      volumeClaimTemplates:
      - metadata:
          name: data
        spec:
          resources:
            requests:
              storage: 30Gi
          storageClassName: managed-premium
          volumeMode: Block
          accessModes:
            - ReadWriteOnce
    useAllNodes: true
    useAllDevices: true
    allowMultiplePerNode: false
    config:
  disruptionManagement:
    managePodBudgets: true
    osdMaintenanceTimeout: 30
    pgHealthCheckTimeout: 0
    manageMachineDisruptionBudgets: false
    machineDisruptionBudgetNamespace: openshift-machine-api
  healthCheck:
    daemonHealth:
      mon:
        disabled: false
        interval: 45s
      osd:
        disabled: false
        interval: 60s
      status:
        disabled: false
        interval: 60s
    livenessProbe:
      mon:
        disabled: false
      mgr:
        disabled: false
      osd:
        disabled: false

• Operator’s logs, if necessary 2021-06-28 07:36:15.478740 I | ceph-file-controller: creating filesystem "myfs-ec" 2021-06-28 07:36:15.478782 I | cephclient: creating filesystem "myfs-ec" with metadata pool "myfs-ec-metadata" and data pools [myfs-ec-data0] 2021-06-28 07:36:17.223857 E | ceph-file-controller: failed to reconcile failed to create filesystem "myfs-ec": failed to create filesystem "myfs-ec": failed enabling ceph fs "myfs-ec": Error EINVAL: pool 'myfs-ec-data0' (id '3') is an erasure-coded pool. Use of an EC pool for the default data pool is discouraged; see the online CephFS documentation for more information. Use --force to override.

Environment:

• OS (e.g. from /etc/os-release): Azure Kubernetes Service
• Kernel (e.g. uname -a):
• Cloud provider or hardware configuration: Azure
• Rook version (use rook version inside of a Rook Pod): v1.6.2
• Storage backend version (e.g. for ceph do ceph -v): 16.2.2
• Kubernetes version (use kubectl version): 1.20.7
• Kubernetes cluster type (e.g. Tectonic, GKE, OpenShift): AKS
• Storage backend status (e.g. for Ceph use ceph health in the Rook Ceph toolbox): HEALTH_OK