backstage: 🐛 Bug Report: Renamed entities are not being picked up after renaming and deleting the orphan

📜 Description

Today as an experiment I have changed the name of the catalog which is the source of component called hyperion.

Before renaming: hyperion.yaml

After renaming: hyperion-test.yaml

Backstage has picked that up, logging errors of having a conflict, now hyperion-test.yaml has hyperion but hyperion.yaml also has hyperion so backstage can’t tell which one is the one, which is expected behaviour 👍

Knowing that the previous hyperion.yaml is gone, going to the UI , to the hyperion component it tells me the source is gone and I can delete. Seeing the errors I though, ok, let’s delete.

So now hyperion is gone. I waited like 1 hour, still gone. I’ve redeployed, still gone…

So, how is supposed this to come back? I found related issue and it mentions that

he old entity remains with an orphan marker, and if you delete it (in the three-dots top right menu of the entity page, and note - delete, not unregister) the new one will take its place as expected.

But the new entity doesn’t appear(

Important ❗❗❗

All our catalogs are stored in a single github repository and we use GithubEntityProvider to fetch them. this is our catalogs setup from app-config.yaml

catalog:
  readonly: true
  rules:
    - allow: [Component, Domain, System, API, Group, User, Resource, Location]
  providers:
    github:
      generatedFactbooks:
        organization: '...'
        catalogPath: 'blabla-path/generated/*.yaml'
        filters:
          branch: 'develop'
          repository: 'repository-name'

👍 Expected behavior

After we delete the orphan entity, the new one should be picked up by the provider and appear in the catalog.

👎 Actual Behavior with Screenshots

Renamed entity doesn’t appear in the catalog after re-running the provider.

👟 Reproduction steps

Rename any catalog .yaml file of an entity to blabla-test.yaml.
Re-deploy, wait for the conflict error to appear
Delete the old entity
Re-deploy

📃 Provide the context for the Bug.

No response

🖥️ Your Environment

OS:   Darwin 21.3.0 - darwin/x64
node: v18.7.0
yarn: 1.22.19
cli:  0.22.9 (installed)
backstage:  1.15.0

Dependencies:
  @backstage/app-defaults                               1.4.1
  @backstage/backend-app-api                            0.4.5
  @backstage/backend-common                             0.19.1
  @backstage/backend-dev-utils                          0.1.1
  @backstage/backend-plugin-api                         0.5.3, 0.5.4
  @backstage/backend-tasks                              0.5.4
  @backstage/catalog-client                             1.4.3
  @backstage/catalog-model                              1.4.1
  @backstage/cli-common                                 0.1.12
  @backstage/cli-node                                   0.1.2
  @backstage/cli                                        0.22.9
  @backstage/config-loader                              1.3.2
  @backstage/config                                     1.0.8
  @backstage/core-app-api                               1.9.0
  @backstage/core-components                            0.12.5, 0.13.3
  @backstage/core-plugin-api                            1.5.3
  @backstage/dev-utils                                  1.0.17
  @backstage/errors                                     1.2.1
  @backstage/eslint-plugin                              0.1.3
  @backstage/integration-aws-node                       0.1.5
  @backstage/integration-react                          1.1.15
  @backstage/integration                                1.5.1
  @backstage/plugin-adr-common                          0.2.11
  @backstage/plugin-adr                                 0.6.3
  @backstage/plugin-api-docs                            0.9.6
  @backstage/plugin-app-backend                         0.3.47
  @backstage/plugin-auth-backend                        0.18.5
  @backstage/plugin-auth-node                           0.2.16
  @backstage/plugin-catalog-backend-module-aws          0.2.2
  @backstage/plugin-catalog-backend-module-github       0.3.2
  @backstage/plugin-catalog-backend                     1.11.0
  @backstage/plugin-catalog-common                      1.0.15
  @backstage/plugin-catalog-graph                       0.2.32
  @backstage/plugin-catalog-node                        1.4.0
  @backstage/plugin-catalog-react                       1.8.0
  @backstage/plugin-catalog                             1.12.0
  @backstage/plugin-events-node                         0.2.8
  @backstage/plugin-fossa                               0.2.52
  @backstage/plugin-github-actions                      0.6.1
  @backstage/plugin-github-pull-requests-board          0.1.14
  @backstage/plugin-home-react                          0.1.1
  @backstage/plugin-home                                0.5.4
  @backstage/plugin-kubernetes-backend                  0.11.2
  @backstage/plugin-kubernetes-common                   0.6.5
  @backstage/plugin-kubernetes                          0.9.3
  @backstage/plugin-org                                 0.6.10
  @backstage/plugin-permission-backend                  0.5.22
  @backstage/plugin-permission-common                   0.7.7
  @backstage/plugin-permission-node                     0.7.10
  @backstage/plugin-permission-react                    0.4.14
  @backstage/plugin-proxy-backend                       0.2.41
  @backstage/plugin-scaffolder-common                   1.3.2
  @backstage/plugin-search-backend-module-catalog       0.1.3
  @backstage/plugin-search-backend-module-elasticsearch 1.3.2
  @backstage/plugin-search-backend-module-techdocs      0.1.3
  @backstage/plugin-search-backend-node                 1.2.3
  @backstage/plugin-search-backend                      1.3.3
  @backstage/plugin-search-common                       1.2.5
  @backstage/plugin-search-react                        1.6.3
  @backstage/plugin-search                              1.3.3
  @backstage/plugin-sonarqube-backend                   0.2.1
  @backstage/plugin-sonarqube-react                     0.1.7
  @backstage/plugin-sonarqube                           0.7.1
  @backstage/plugin-tech-insights-backend-module-jsonfc 0.1.31
  @backstage/plugin-tech-insights-backend               0.5.13
  @backstage/plugin-tech-insights-common                0.2.11
  @backstage/plugin-tech-insights-node                  0.4.5
  @backstage/plugin-tech-insights                       0.3.12
  @backstage/plugin-techdocs-backend                    1.6.4
  @backstage/plugin-techdocs-module-addons-contrib      1.0.15
  @backstage/plugin-techdocs-node                       1.7.3
  @backstage/plugin-techdocs-react                      1.1.8
  @backstage/plugin-techdocs                            1.6.5
  @backstage/plugin-user-settings                       0.7.5
  @backstage/release-manifests                          0.0.9
  @backstage/test-utils                                 1.4.1
  @backstage/theme                                      0.2.19, 0.4.1
  @backstage/types                                      1.1.0
  @backstage/version-bridge                             1.0.4

👀 Have you spent some time to check if this bug has been raised before?

I checked and didn’t find similar issue

🏢 Have you read the Code of Conduct?

I have read the Code of Conduct

Are you willing to submit PR?

Yes I am willing to submit a PR!

About this issue

Original URL
State: closed
Created a year ago
Comments: 15 (7 by maintainers)

Most upvoted comments

I ran into something similar recently, although it was about renaming entities inside a YAML file: #18054 If I understand correctly, you did not rename an entity inside a YAML file, but you renamed the YAML file itself.

I think this happened: Backstage ingests hyperion-test.yaml as a location entity and detects that hyperion.yaml is no longer there. The ingestion pipeline then tries to read entities from the hyperion-test.yaml, and there is a conflict because an orphaned entity already exists from hyperion.yaml. You then deleted the orphaned entity and expected Backstage to pick up the entity from thehyperion-test.yaml. That does not happen, and I think the reason is that Backstage already processed hyperion-test.yaml when it detected the conflict. Since the hyperion-test.yaml has not changed, there is no reason for Backstage to reprocess the location; thus, the entity is not read from the renamed file.

If this is the case, then forcing a refresh of the hyperion-test.yaml location entity should fix it:

Go to https://<your-backstage>/catalog?filters[kind]=location&filters[user]=all and find the location entity for hyperion-test.yaml.
On the entity page, in the about card, click the Schedule entity refresh button:

I don’t know if this is a bug or just how the ingestion pipeline works, I’ll leave that up to the maintainers. It would be nice if Backstage could somehow detect that the conflict is no longer there and reprocess the relevant files.

sanderaernouts on Aug 4, 2023

I believe that this may be another symptom of the state described in this comment.

Short summary: the new location DID try to emit the entity in a new place, which threw away the entity due to the conflict, and then due to the hashing optimization, didn’t try emitting again. This is as of yet unresolved.

It’s possible that changing the result_hash to some random string in the new location will make the entity magically appear again. Could you try that to confirm?

freben on Aug 7, 2023