wallabag: API shouldn't require a username and password

Issue details

The API currently requires new “API client” to be created to operate, which includes a client_id and a client_secret.

But to actually work, the API also needs an access_token and refresh_token. And those are very short lived - one hour maximum.

This means to implement a client, you need to not only ask the user for an incomprehensible client_id and client_secret, but also for their username and password. Every client then needs to store those username and passwords indefinitely to regenerates the access tokens.

Environment

  • wallabag version: 2.1.0 API documentation
  • How did you install wallabag? irrelevant.
  • Last wallabag version that did not exhibit the issue (if applicable): N/A
  • php version: N/A
  • OS: N/A
  • type of hosting (shared or dedicated): N/A
  • which storage system you choose at install (SQLite, MySQL/MariaDB or PostgreSQL): N/A

Steps to reproduce/test case

N/A.

Suggested solution

The username and password should be used only to create access tokens, which should be permanent. This would make the whole API way more efficient because then a single HTTP request would be require for API calls instead of two (assuming the worst case of less than one request per hour per client).

Alternatively, client tokens should be user-specific. I believe there may already be an issue about this, but right now, anyone can see (and delete?) anyone else’s tokens on the server. That seems like a Real Bad Idea.

Clients should only ask the user’s username and password as a transient secret, that can be used to create client tokens on the fly, which then can be used to create access tokens.

About this issue

  • Original URL
  • State: open
  • Created 7 years ago
  • Reactions: 36
  • Comments: 68 (19 by maintainers)

Most upvoted comments

my app can’t ask for credentials. it’s automated.

most apps require username and password only when setting up, not after an arbitrary timeout. as far as i know, twitter Oauth credentials are permanent, for example.

the title of the issue i opened here is “API shouldn’t require a username and password”, and I meant it. it’s not “API shouldn’t require a username and password most of the time”. i want to have a token i can use until revoked.

why are you expiring the tokens anyways? what’s the use case for this?

if we are worried that users will lose that device, then the proper solution is not to expire the tokens, because then we are vulnerable for a long time after the device is lost: if i lose the device N days after then tokens are used, then I am vulnerable for full year. the proper solution is, as done elsewhere, to have a clear interface to see what devices have access and allow revocation.

Clients should never even see a user’s login credentials, tbh. That’s the whole point of OAuth, after all.

The client should redirect the user to the Wallabag server where they log in and grant the client permission (or not). The client then receives the tokens it needs from the server without ever seeing the user’s password.

That’s perhaps not practical for some kinds of clients that can’t start a local webserver to receive the tokens, but that should be the default mode of operation. EDIT: It seems there’s now an API for this that extensions can use without requiring a webserver.

Requiring users to give clients their passwords essentially defeats the whole purpose of using OAuth in the first place.

In that case, it would be better (simpler and more secure) to just let users generate (and revoke) an API key for each client, which is then used in place of username+password. Then users would only have to enter a single value into their client, instead of a client ID, client secret, username and password.

I’m here because Firefox add-on for Wallabag requires 4 pieces of information, including my username and password for the instance of Wallabag. WHAT? The whole purpose of client_id and client_secret is NOT to require credentials. I DO NOT want to provide my credentials to any client app that wants to connect - that makes zero sense. My account is vulnerable if I keep leaving my credentials in all apps. And @anarcat has expressed exactly that.

So, what’s the outcome of such a long discussion? It seems this is not going to change? Would like to better understand what you guys are planning for this.

This is how it currently works:

http POST http://v2.wallabag.org/oauth/v2/token \
    grant_type=password \
    client_id=12_5um6nz50ceg4088c0840wwc0kgg44g00kk84og044ggkscso0k \
    client_secret=3qd12zpeaxes8cwg8c0404g888co4wo8kc4gcw0occww8cgw4k \
    username=yourUsername \
    password=yourPassw0rd

This is how it should work:

http POST http://v2.wallabag.org/oauth/v2/token \
    grant_type=password \
    client_id=12_5um6nz50ceg4088c0840wwc0kgg44g00kk84og044ggkscso0k \
    username=yourUsername \
    password=yourPassw0rd

Notice the absence of client_secret - grant_type=password shouldn’t require it. Sources: simplified, RFC.

In theory API clients (read: client_id and client_secret pairs) should not be user-specific, but since they already are in recent versions, this is what may be added:

http POST http://v2.wallabag.org/oauth/v2/token \
    grant_type=client_credentials \
    client_id=12_5um6nz50ceg4088c0840wwc0kgg44g00kk84og044ggkscso0k \
    client_secret=3qd12zpeaxes8cwg8c0404g888co4wo8kc4gcw0occww8cgw4k

Sources: simplified, RFC.

For f*ck’s sake, people, read the OAuth specs before inventing more crap.

Hej everyone, very interesting topic here. To sum-up, the ideal situation would be to have an authentication à la Nextcloud or Mastodon: when your client needs to log to any instance, you never need to type in a client_id or a client_secret, but you only need to type in your credentials.

How OAuth2 works

Basically, OAuth2 needs to authenticate both the client application and the user to deliver an OAuth2 Token. The client application is authenticated thanks to the client_id and client_secret fields (as defined in the OAuth2 standard), and the user with its username and password (although OAuth2 does not define how the user is authenticated). A client application can use one of several flows to provide those applications to the Authorization Server (i.e. the server part that issues tokens)

  • Authorization Code + PKCE
  • Implicit (deprecated)
  • Resource Owner Password Credential (deprecated)
  • Client Credential

How Wallabag uses OAuth2

The way Wallabag works at the moment, it uses the client credentials flow. This means the user gives his crendentials to his client app. Then the client app sends all the credentials (user credentials and client_id/client_secret couple). In exchange for this, the server answers with an Access Token, which is the one actually use to authenticate API calls; and a Refresh Token, used to renew Access Tokens when they expire, without needing to ask the user his usernam or password again.

Two main issues

Refresh Token lifetime

What seems to be an issue right now is that the Refresh Token is too short lived. Extending the Refresh Token life to a few months doesn’t seem indecent to me: Wallabag is not a critical app and shouldn’t be the main target in case of threat.

client_id and client_secret configuration

The second most notable issue is that the user needs to create a client_id and client_secret in his administration interface, and provide it manually to the client application. This can be a real usability problem, especially with mobile clients.

The client_id and client_secret creation work can be delegated to the client application itself. This can be achieved thanks to the RFC7591 - Dynamic Client Registration.

I need to dig the specifications a bit more in depth, but in my opinion it is not useful to switch from the Client Credentials flow to the Authorization Code + PKCE flow. Indeed, the Authorization Code + PKCE flow relies on a browser/web view to let the user input his username and password directly on the Authorization Server. The main reason for that is so the client application never sees the username and password, and so the user can delegate only a subset of his authorizations to his client application.

In the case of Wallabag, I don’t see any issue if the open source clients see the username and password, and functionally speaking “delegating a subset of its authorization to the client app” does not make any sense.

Same. I had to build an entire agent workflow to go back and forth with Wallabag a couple of times to get a token. Why not just have an API token that can be used in API calls like everybody else?

Hej everyone, so a couple of weeks after stopping the discussion, I’m back with the best practices regarding OAuth2 and what should be done.

Types of clients

The first step is to know which are the types of clients that will be using OAuth2 Access Tokens. The short answer is: all of them. The longer answer is:

  • The web-based interface of the instance itself. While it does not use OAuth2 today (or does it?), it would be a nice to have. It would be useful to simplify the process of companion apps. More on that below
  • The native mobile apps (iOS, Android) or GTK Desktop and Mobile app
  • A potential evolution of the frontend of the Wallabag instance frontend, if it ever were to become a single page application (SPA)

The IETF’s best practices for Native Apps and best practices for Browser Based Apps recommend using the Authorization Code + PKCE flow for these client apps.

Registration of the client app

In order for any client app to be able to use OAuth2 flows on Wallabag, it needs to be registered on the instance. Today, that registration is manually handled by the end-user, ho creates new client apps. Wallabag generates a client_id (which is basically the client app’s username) and a client_secret (which used to be the client app’s password, but is kinda legacy now).

What we want now is the ability to use dynamic client registration. This will allow the client app to reach the instance, and ask it to generate a client_id and a client_secret. During the process, the client app tells the server to bind its client_id to a redirect_URI. The instance does not do any check with that URI, it simply stores it and binds it to the client_id. More on that redirect URI below.

Authorization Code + PKCE

Authorization Request

Let’s put aside the PKCE bit for the sake of clarity. The Authorization Code flow makes use of the end-user’s browser to handle the actual end-user authentication. The following describes how to handle this authorization flow for a native app (most complicated case). We’ll discuss other cases later.

image

Mermaid source for this diagram:

sequenceDiagram
    Wallabapp->>+Browser: Open user browser on Wallabag server's authentication page
    Browser->>Wallaserver: Get login page
    Wallaserver->>Browser: Send login page
    Browser->>Wallaserver: Send username and password
    Wallaserver->>Browser: Send Authorization Code
    Browser->>-Wallabapp: Retrieve Authorization Code
    Wallabapp->>Wallaserver: Exchange Auth Code against tokens
    Wallaserver->>Wallabapp: Send Access Token and Refresh Token

This clever use of the user’s browser allows the client app to retrieve an Access Token without ever being aware of the end-user’s credentials. We’ll discuss the use of the Access and Refresh Token below. We can see that the end-user’s browser seems to call the native app. While this may seem odd, this can be performed thanks to the redirect_URI.

A redirect_URI is a URI bound to the client app. Wallabag client apps should ask to use wb://redirect as a redirect_URI. When the Wallabag instance uses an http redirection in the end-user’s browser to the redirect_URI, it uses query parameters. As an example, if the instance redirects the browser to wb://redirect?lorem=ipsum, the end-user’s browser will parse the url, open the app associated with the wb:// scheme and pass it the URL. Our client app will be associated with the wb:// scheme, get the URL, and be able to determine the value of lorem to be equal to ipsum.

Each client app is bound to the redirect_URI it specified during its registration against the server.

Now let’s get a bit more technical with the following diagram.

image

sequenceDiagram
    Wallabapp->>+Browser: Open user browser on Wallabag server's authentication page
    Browser->>Wallaserver: (1) GET /authorize
    Wallaserver->>Browser: Send login page
    Browser->>Wallaserver: (2) POST /authorize
    Wallaserver->>Browser: (3) Send Authorization Code to redirect URI
    Browser->>-Wallabapp: Retrieve Authorization Code
    Wallabapp->>Wallaserver: (4) POST /token
    Wallaserver->>Wallabapp: (5) Send Access Token and Refresh Token

The app displays a field for the end-user to type in the instance URL. Once the end-user has clicked/tapped the button, the flow (1) starts: the app launches the user’s browser but not an web widget/web view with the following request

GET /authorize?response_type=code&client_id=client_id&state=state_parameter

For the sake of simplicity, let’s consider that the state_parameter is a random string generated and stored by the client app, which will be used later. See Auth0’s article on that matter. The actual best practices are a bit different. See also the IETF specification for that flow (1), called the Authorization Request.

The Wallabag instance answers with an authentication page, displaying two fields: one for the user login, and one for the user password, plus a validation button. Once the end-user has filled the fields, he sends the form, which sends the following request:

POST /authorize`wb://`_redirect_URI_`

username=endusername&password=enduserpassword

Authorization Response

The Wallabag instance checks the user credentials in its internal database, and if the credentials are valid, it generates an authorization code and redirects the end-user’s browser to the redirect_URI bound to the client app that initiated the Authorization Request. It adds the code and state query parameters to that URI. As an example, the response could be a redirect to wb://redirect?code=1bsf4gqa&state=a2fs7d35

See the IETF specification for the flow (3), called the Authorization Response

Access Token Request

The client app retrieves the url from the redirect of the Authorization Response by parsing the query parameters. It checks that state is consistent with the state sent in the Authorization Request, and retrieves the code parameter.

It then calls the /token endpoint of the instance with the following query parameters

  • grant_type which must be equal to `authorization_code"
  • code which must be equal to the code parameter retrieved from the Authorization Reponse
  • redirect_uri which must be the same redirect URI as used when registering on the server
  • client_id which must be its own client_id

See the IETF specification for this flow (4) called the Access Token Request

Access Token Response

The Wallabag instance checks all the parameters, and if successful replies with an Access Token and a Refresh Token, along with the Access Token’s time to live.

See the IETF specification for this flow (5) called the Access Token Response.


I’ll get a bit more in details in the dynamic registration of the client app in another comment. I can assist anyone willing to implement the Authorization Code flow, but my knowledge of PHP and Symfony is too low at the moment to produce actually clean and maintainable code.

quite frankly, if the Oauth implementation in Wallabag is going to stay this way, it might as well be removed, because it just makes writing clients needlessly hard at absolutely no security advantage.

Just wanted to say: I’m evaluating wallabag as a pocket alternative and this security issue is the largest barrier to adoption for me right now.

this issue is the root of all client apps issues : https://github.com/wallabag/wallabagger/issues/145 and https://github.com/wallabag/android-app/issues/788

There must be a way 😭

Long time lurker here: exactly this! I have built a small service which grabs articles from my wallabag, converts them and send them to my kindle. Its quite alien to me to have my user credentials addtionally to my api secret in that configuration.

Would be nice if it were possible to use only api secrets for automated services.

I don’t see any issue if the open source clients see the username and password

I do: it doesn’t fix the core problem with the current implementation. Having to give someone your password is precisely what tokens (including OAuth) are intended to avoid. It might not be a big deal giving the open-source iOS app your password, but what if you want to integrate your Wallabag with some other service?

The current OAuth implementation is broken precisely because it requires you to give clients your password, so any replacement that still requires giving clients your password isn’t a fix.

The goal should be sensible, secure API authentication, not doing the best we can with OAuth. I feel that by concentrating on OAuth, we’re not seeing the wood for the trees.

OAuth itself is fundamentally a poor solution for Wallabag because it’s not really intended for this kind of application. The advantages of OAuth are:

  1. Very simple for users. The application provider jumps through a few extra hoops (getting & setting authentication endpoints, client IDs & secrets) so users only need to click a link and then “Allow” to authorise a client.
  2. Allows applications to use third-party auth providers (“Log in with Facebook” etc.)
  3. Client applications use tokens. Permissions can be granular and they never see users’ passwords.
  4. Separate access and refresh tokens lighten the load on authentication/authorisation services.

(2) and (4) are irrelevant here because Wallabag is its own auth provider.

(1) becomes a disadvantage with Wallabag because its self-hosted nature means the user is also the provider. All complexity accrues to the user, which the current implementation demonstrates very well.

(3) is the only real advantage OAuth would have for Wallabag, and it’s defeated when the user has to give a client their password.

And (3) can be achieved more simply (in terms of both implementation and user workflow) with simple API keys.

The user generates an API key in their Wallabag account and then gives it (and the URL of their Wallabag instance) to the client application. It’s slightly more complex than giving the client your username and password, but far more secure.

In the context of Wallabag, it has all the advantages of OAuth without the disadvantages/complexity. The API keys are tokens in exactly the same way as OAuth tokens are, with all the same advantages, but without the complexity of OAuth (which has no additional benefit for Wallabag).

The problem arises from the fact that the only allowed grant_type (grant_type=password) configured to require a weird set of parameters: it takes both username and password (ok for this grant type) and client_id and client_secret. client_id and client_secret are supposed to be used with grant_type=client_credentials which doesn’t need username and password.

If grant_type=client_credentials is allowed, all the other problems discussed here are gone.

See: OAuth 2 Simplified (or read other OAuth-related info).

It may be worth mentioning that OAuth’s most common flow, the “authorization code” flow, is actually perfectly usable for command-line applications and the like. There’s a standard redirect URI that these types of apps should use: urn:ietf:wg:oauth:2.0:oob, with “oob” meaning “out of band”.

When this special URI is used, rather than actually redirecting anywhere, a supporting authorisation server generates a page displaying the authorisation code to the user directly. The code can then be copy-pasted into the command-line app, allowing the usual OAuth flow to proceed from there. It’s equivalent in security to using a normal redirect URI, since the app must still provide its client ID and secret when consuming the new authorisation code. (In other words, pasting the code into the wrong place by accident doesn’t actually give your access token to the wrong app.)

There is also the “loopback” approach: the command-line app simply starts up a tiny HTTP server on localhost:some_random_port, and then that server is used as the redirect URI. This allows the command-line app to grab the authorisation code directly from the redirected request without any user intervention, but it’s more work on the CLI app dev’s part and usually isn’t much more convenient than a copy-paste job for the typical user of command-line apps.

If refresh tokens worked properly, either one of these OAuth approaches could be applied as a fast one-time configuration step when first setting up your command-line app - the app would simply save the access and refresh tokens in its configuration and use those later on, refreshing and replacing the access token when required.

Either way it’s much better than giving your command-line app both its own client ID and client secret, and your username and password.

You already have two sensible, and much better, solutions suggested. What more do you need to start fixing it?

This is not useful. @tcitworld was pretty clear about what’s needed: they need someone to do a PR and clean that code up. Snarky comments won’t help anyone here…

And I’m sorry if i seemed to be in the snarky crew. 😉

Hey folks. For what it’s worth absolutely nobody is happy with the current situation, but let’s keep productive in the comments so that we can fix this properly. Of course everyone is welcomed to propose some pull request to fix the current authentication flow.

I’ve added this to the 2.3 milestone and dropped a bounty, cause it’s a very important issue to me. Hopefully someone will find the time to fix it.

Bountysource

Looks like @tcitworld won the bounty 🙂 💰

Did add a bounty too since I’m not sure about having the time.

Then we agree.

Not entirely. I think that solution is acceptable. I still prefer plain API keys because they are much easier to implement (vs OAuth client) and much better suited to CLI applications.

API Key rotation could be something desirable, since an API Key can be leaked.

True. But it’s hardly a great problem to regenerate an API key in such rare situations.

the scope claim in OAuth2 Access Tokens has everything you need.

It’s a small part of what you need. Associating a few permissions with a key is simple. 95% of the work is managing the permissions and implementing the authorisation, which OAuth does not help with.

Functionnally speaking and from an end-user perspective […]

Functionally, there’s little difference. I agree it’s better for connecting some kinds of applications, but it’s rather more complex for others. It also makes implementing clients much more complex because OAuth is complex.

Something functionally simple, like a script that adds a URL to Wallabag, becomes far more difficult to write and use because of OAuth. A dozen or so lines of code becomes a mess of OAuth libraries.

If this requires a significant revamp of the authentication handling maybe it would also be a good time to look into scopes. An example use case would be a service like ifttt. Such a service would need the permission to add new articles, but not permissions to read or modify my list of existing entries. For reference: https://www.brandur.org/oauth-scope

Edit: See #3005.

There is authorization_code grant_type which should also be a good choice in case of wallabag, but:

  1. There is no endpoint to get that authorization code.
  2. It has pretty much the same disadvantages discussed here (it allows to get tokes once; if the tokens expire, the procedure needs to be repeated).

Update: I included the first point because authorization_code is present in the “Grant type allowed” list in wallabag UI, but there is no way to actually use it.