Catalog API: Could not read location from Gitlab repo ($text placeholder)

See original GitHub issue

Expected Behaviour

In my org we have OpenAPI definitions (swagger.yaml), hosted in a private GitLab repository, accessible only by authenticated users.

API definitions are using raw GitLab URLs, with $text placeholder:

apiVersion: backstage.io/v1alpha1
kind: API
metadata:
  name: service-name
  description:  |
    Service Description
spec:
  type: openapi
  owner: project-owner
  lifecycle: production
  system: production
  definition:
    $text: https://gitlab.host.tld/org/group-name/subgroup-name/service-name/raw/master/swagger.yaml

GitLab discovery is enabled and correctly working for catalog-info.yaml files ingestion.

I would then expect also swagger.yaml files to be correctly read from the given definitions.

Actual Behaviour

Unfortunately, an error is occurring while creating the API catalog entities. It seems $text placeholder doesn’t work with the above provided raw URLs.

Steps to Reproduce

  1. Provide a swagger.yml file on a private repos (self-hosted GitLab installation).
  2. Create an API resource on catalog-info.yaml using $text placeholder.
  3. Register the component by using backstage UI.

Context

{"error":{"name":"Error","message":"InputError: Processor PlaceholderProcessor threw an error while preprocessing; caused by Error: Placeholder $text could not read location https://gitlab.host.tld/org/group-name/subgroup-name/service-name/raw/master/swagger.yaml, Error: Incorrect url: https://gitlab.host.tld/org/group-name/subgroup-name/service-name/raw/master/swagger.yaml, Error: Wrong GitLab URL","level":"error","service":"backstage"},"request":{"method":"POST","url":"/locations?dryRun=true"},"response":{"statusCode":500}}

An attempt has also been made by using regular “blob” urls.

Your Environment

Gitlab discovery is correctly configured and working for catalog entities creation.

Adding the gitlab instance hostname to backend.reading.allow config setting (as documented here https://backstage.io/docs/features/software-catalog/descriptor-format#kind-api) didn’t work as expected.

OS:   Darwin 21.2.0 - darwin/x64
node: v16.4.0
yarn: 1.22.15
cli:  0.11.0 (installed)

Dependencies:
  @backstage/app-defaults                                  0.1.4
  @backstage/backend-common                                0.10.3
  @backstage/backend-tasks                                 0.1.3
  @backstage/catalog-client                                0.5.4
  @backstage/catalog-model                                 0.9.9
  @backstage/cli-common                                    0.1.6
  @backstage/cli                                           0.11.0
  @backstage/config-loader                                 0.9.2
  @backstage/config                                        0.1.12
  @backstage/core-app-api                                  0.4.0
  @backstage/core-components                               0.8.4
  @backstage/core-plugin-api                               0.5.0
  @backstage/dev-utils                                     0.2.17
  @backstage/errors                                        0.2.0
  @backstage/integration-react                             0.1.18
  @backstage/integration                                   0.7.1
  @backstage/plugin-api-docs                               0.6.22
  @backstage/plugin-app-backend                            0.3.21
  @backstage/plugin-auth-backend                           0.6.2
  @backstage/plugin-catalog-backend                        0.20.0
  @backstage/plugin-catalog-common                         0.1.0
  @backstage/plugin-catalog-import                         0.7.9
  @backstage/plugin-catalog-react                          0.6.11
  @backstage/plugin-catalog                                0.7.8
  @backstage/plugin-github-actions                         0.4.31
  @backstage/plugin-org                                    0.3.34
  @backstage/plugin-permission-common                      0.3.1
  @backstage/plugin-permission-node                        0.3.0
  @backstage/plugin-permission-react                       0.2.2
  @backstage/plugin-proxy-backend                          0.2.15
  @backstage/plugin-scaffolder-backend-module-cookiecutter 0.1.8
  @backstage/plugin-scaffolder-backend                     0.15.20
  @backstage/plugin-scaffolder-common                      0.1.2
  @backstage/plugin-scaffolder                             0.11.18
  @backstage/plugin-search-backend-node                    0.4.4
  @backstage/plugin-search-backend                         0.3.0
  @backstage/plugin-search                                 0.5.5
  @backstage/plugin-sentry                                 0.3.33
  @backstage/plugin-tech-radar                             0.5.2
  @backstage/plugin-techdocs-backend                       0.12.3
  @backstage/plugin-techdocs                               0.12.14
  @backstage/plugin-user-settings                          0.3.16
  @backstage/search-common                                 0.2.1
  @backstage/techdocs-common                               0.11.3
  @backstage/test-utils                                    0.2.2
  @backstage/theme                                         0.2.14
  @backstage/types                                         0.1.1
  @backstage/version-bridge                                0.1.1
✨  Done in 3.39s.

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:9 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
Rugvipcommented, Feb 14, 2022

Alright, glad it got resolved!

1reaction
mscamcommented, Feb 14, 2022

@Rugvip you are right, thanks. I think we discovered my issue.

Both of these URLs can be considered valid:

https://gitlab.host.tld/group-name/sub-groupname/project-name/blob/main/openapi.yaml https://gitlab.host.tld/group-name/sub-groupname/project-name/-/blob/main/openapi.yaml

In my catalog-info I had some records using /-/blob, some others using /blob. Please feel free to close this ticket if no action is required 🙏 (not an unicode impostor, but actually a URL that was not using “-”)

Read more comments on GitHub >

github_iconTop Results From Across the Web

Descriptor Format of Catalog Entities - Backstage.io
Documentation on Descriptor Format of Catalog Entities which describes the default data shape and semantics of catalog entities.
Read more >
Repositories API - GitLab Docs
Get a list of repository files and directories in a project. This endpoint can be accessed without authentication if the repository is publicly...
Read more >
How do I add an empty directory to a Git repository?
The suggested workaround is to put a .gitignore file in the empty directory. With the file in place the directory is no longer...
Read more >
Writing scaffolder templates - Roadie.io
Templates are defined by a Backstage Entity YAML file with a Template kind and imported into the Backstage catalog. You can create multiple ......
Read more >
Changelog - Cypress Documentation
Fixed a regression introduced in the Electron browser in Cypress 10.8.0 where the CYPRESS_EVERY_NTH_FRAME environment variable was not being set appropriately ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found