backstage: Onboarding wizard for repos that do not have a catalog-info.yaml

Feature Suggestion

A lot of users will start their Backstage journey with an empty catalog but a lot of existing source code repos that they would like to get in there. The current workflows require them to manually create catalog-info.yaml files for each one of them and register that. This is tricky to get right from the start and requires low level fiddling with YAML files whose format you are not yet familiar with.

Backstage already has a “Create…” sidebar entry and a “Register existing component” functionality. It should be extended with an “import” wizard that lets you onboard an existing repo of code as easily as possible.

Possible Implementation

Illustrating the idea with an example user journey:

1. There is already a Java backend repo in a GitHub enterprise installation, and the user wishes to add it to the catalog.
2. The user clicks “Create…” in the sidebar and then “Import existing component”.
3. A wizard appears, which asks for what repository and path you want to import. This could be as simple as a text box into which you can paste for example https://ghe.mycompany.net/myorg/myrepo. Or it could be helpful and presents a list of repos that you have access to. Then the user presses Next.
4. A spinner appears while the wizard analyses the target repository. It searches for a long list of well known things. It finds a .github/CODEOWNERS file, it finds a pom.xml file, it finds an openapi.json file nearby, it loads the teams present in the org to see which ones the user is a member of.
5. A page of findings is presented. It suggests that the owner is managed by codeowners, but the user can choose to either select from a list of teams they are a member of, or to type a string manually. It suggests the component name and description from the contents of the pom file. It could not find any data on the lifecycle of the component, so it asks the user to provide one. It said it found a nearby OpenAPI declaration and suggests making an API entity for that too and link it up to the service. When the user is satisfied with the wizard, they click Create.
6. The wizard uses the GitHub API to create a pull request in the target repo, that creates a catalog-info.yaml file. The file contains the service component and the api entity, wired up and with the right data entered. The file is automatically registered in the catalog with a special flag that tells the catalog that it’s OK that it isn’t present yet, because it will be at a later time.
7. The user is presented with an “all done” final state, with a link to the generated file and an offering to import one more repo.

Context

The onboarding process is pretty technical and requires a fair amount of work before seeing the full value of the system. Adding a wizard like this would also make it easier to distribute the effort, so that individual teams more easily can take care of their own onboarding in a point-and-click interface instead of manually learning and writing the catalog info format.

Future work

In the future, it may not even be necessary to create the yaml file at all; instead, we could register the repo as the actual location. With the right set of processors, it could be possible to get all the information needed without having to tether to a specific catalog-info file. The refresh loops would then re-examine the entire repo state on each run and adapt as necessary.