concourse: can not login after upgrading to 4

Hey @vito I just setup v4.1, but when I try to login as local user or github user, the login fails due to the same error. Is this issue fixed and verified? The relevant logging when I try to login as local user: {"timestamp":"1536016247.228306532","source":"atc","message":"atc.dex.event","log_level":1,"data":{"fields":{},"message":"login successful: connector \"local\", username=\"concourse\", email=\"concourse\", groups=[]","session":"5"}} {"timestamp":"1536016249.150294065","source":"atc","message":"atc.sky.userinfo.failed-to-parse-authorization-header","log_level":1,"data":{"session":"4.104"}} {"timestamp":"1536016254.152165890","source":"atc","message":"atc.sky.userinfo.failed-to-parse-authorization-header","log_level":1,"data":{"session":"4.105"}} {"timestamp":"1536016255.191825628","source":"atc","message":"atc.build-tracker.track.start","log_level":0,"data":{"session":"19.48"}} {"timestamp":"1536016255.195375443","source":"atc","message":"atc.build-tracker.track.done","log_level":0,"data":{"session":"19.48"}} {"timestamp":"1536016257.772825003","source":"atc","message":"atc.sky.callback.failed-to-fetch-dex-token","log_level":2,"data":{"error":"Post https://externalurl/sky/issuer/token: net/http: TLS handshake timeout","session":"4.103"}} {"timestamp":"1536016259.153844595","source":"atc","message":"atc.sky.userinfo.failed-to-parse-authorization-header","log_level":1,"data":{"session":"4.106"}}

Just setup v4.2.1 and login via github/local is working perfectly now!! Thank you so much for sorting the issue out!!! 💯

For posterity: 4.2.0 fixes my login issue.

The main problem here is that when the ATC can’t reach the external_url then login doesn’t work. This happens because after the authorization_code gets issued the token exchange happens using the external_url even though both components are running on the same host. We’re going to use the loopback address for the token exchange. This fix is going through our pipeline now.

Hey there, I keep getting redirected to 127.0.0.1 right when I click the login button with a fresh install of concourse 4.2.1.

Could you please clarify if I need to specify an external URL for this not to happen? While some users seem to be happy, they haven’t specified their setup, and I’m still having problems.

Here’s my situation:

• I’m currently trying to use local user auth (--add-local-user and --main-team-local-user).
• My users are accessing concourse through multiple domain names, so I haven’t set the external url.
• Some or all of those domain names are unresolvable from within the concourse web container.
• This was working before I tried to upgrade from 3.14.1.

If in v4 I must specify a single external URL, I’ll have to stay on 3.14.1.

@pivotal-jamie-klassen Is this actually done? I see the code still using the external URL here and don’t see any commits pushed.

Moving back to the backlog. I thought this was in 4.1.0 but hadn’t done acceptance on it yet so I’m not sure. We may need to do another release soon to fix this. 😕

@eedwards-sk I don’t think this issue applies to the TSA.

Please open new issues instead of commenting on this one - there was a very specific thing to fix, and we fixed it, and received feedback from those affected by this issue that it was indeed fixed. The title of this GitHub issue is very open-ended and we’ll keep getting comments on here as long as anyone is having login issues. 😃 (We should maybe lock it eventually.)

@jeffawang An external URL must be configured for auth to work properly and securely. If you have a use case for multiple external URLs, we’d like to hear about it (but again, in a separate issue), but it at least sounds a bit strange.

I deployed 4.2.1 today and it works for me! Thanks @vito and team.

Hooray! Works for me too! Thanks guys!

To double tap on this, I am using Concourse behind a reverse proxy that doesn’t use authentication of any form, it just performs a TLS/SSL termination.

I’m still seeing errors:

{"timestamp":"1536690818.222700834","source":"atc","message":"atc.sky.callback.failed-to-fetch-dex-token","log_level":2,"data":{"error":"Post https://externalUrl/sky/issuer/token: dial tcp publicIP:443: i/o timeout","session":"4.4"}}

Reading @vito’s comments about this not making it into 4.1.1 is a bit disheartening, but I wanted to validate some assumptions around the external URLs so there was more visibility into the issue. For me, externalUrl is resolvable both within and external to my network, and publicIP is reachable by everyone.

Here are my deployed versions:

releases:
- name: concourse
  sha1: 513e3a88d135e6e2cd8a974702e2e63caa0cb82b
  url: https://bosh.io/d/github.com/concourse/concourse?v=4.1.0
  version: 4.1.0
- name: garden-runc
  sha1: 2a7c813e7e4d862e19334addf022916fb6b91eb0
  url: https://bosh.io/d/github.com/cloudfoundry/garden-runc-release?v=1.16.3
  version: 1.16.3
- name: postgres
  sha1: 24d2e2887a45258b71bc40577c0f406180e47701
  url: https://bosh.io/d/github.com/cloudfoundry/postgres-release?v=29
  version: "29"

We investigated doing an internal redirect for all of the auth components, but that doesn’t work because of the way Dex is designed. To perform an internal redirect, the issuer URL in Dex needs to be set to the internal URL. This breaks for the following reasons:

• Dex uses the issuer URL to issue tokens. Setting this URL to the internal URL (e.g. 127.0.0.1) would mean that the tokens issued are valid for any instance with that internal URL, which is too generic.

• Dex also checks that the callback URL used by the external auth provider matches the issuer URL. If the issuer URL is the internal URL, this check would fail for external auth providers.

@UniqueElphie Correct, this ended up not making it in to 4.1. We’ve picked it up again and plan to push a 4.1.1 out this week.

Woops, we really shouldn’t be relying on the external URL internally.

I think the problem is just this one line:

https://github.com/concourse/atc/blob/8819682a06f74aa284de084b14955f42919a64cb/atccmd/command.go#L518

If we were to change that to the bind IP/port (except 127.0.0.1 if it binds to 0.0.0.0), this should be fixed.

I’ll prioritize this highly somewhere. Sorry for the turbulence everyone! @r-chris I’m also gonna prioritize https://github.com/concourse/concourse/issues/2519 which should make changing the ports work a lot more smoothly.

@phynias @UniqueElphie give the concourse server ip with port or dns name in below parameter

eg:

• name: CONCOURSE_EXTERNAL_URL value: http:/ap-south-1.compute.amazonaws.com:38686/

In latest version of concourse by default it will redirecting to localhost:8080 if you do not specify CONCOURSE_EXTERNAL_URL.

use below parameters in latest version instead of basic authentication parameters and password should be in bcrypted format:

• name: CONCOURSE_ADD_LOCAL_USER value: test:$2y$10$yE9dQn0P1KvpynigktO5neqvb/dZQricFZH1d1PhwmGfUMVJoac0y
• name: CONCOURSE_MAIN_TEAM_LOCAL_USER value: test

for reference: https://github.com/concourse/concourse/issues/2421

Our DNS was not working correctly. We updated the resolv.conf to use 8.8.8.8 and were able to login.