dbal: Postgres: public should not be considered a "application schema" and should be ignored the same as other system schema from `getSchemaNames`

Bug Report

For now 10 years public has been returned by getSchemaNames generating a lot of bug and weird behaviour in higher level libraries relying on it.

Q A
Version 3.x

Summary

For now 10 years since commit https://github.com/doctrine/dbal/commit/b89490a557584b61d575aaa67f8b386d833d5a0f (before that it was not working at all )

public is returned by PostgreSQLSchemaManager::getSchemaNames()

Current behaviour

How to reproduce

(a reproducer is soon coming)

  1. connect to any postgresql (a vanilla postgres docker is fine)
  2. call PostgreSQLSchemaManager::getSchemaNames -> public is returned among other user-defined schema

Expected behaviour

all application schema are returned , but not public (as well as other postgres-defined schema are also not returned

More context

This bug is at least the root cause of these issues

the more visible issue of it is with doctrine migration:

  1. all migrations contains a down migration trying to CREATE SCHEMA public that EVERYBODY deletes or workaround ( 💯 if you have done that too ) because it fails miserably (as it already exists)
  2. to avoid dropping it , we don’t drop schemas at all cf https://github.com/doctrine/dbal/pull/5604/files , so if automated generated migrations are not bijectives because DROP SCHEMA will never be generated.

Why public should be considered a “system schema”

  1. public is not created by the user
  2. everybody assume that it is present
  3. even postgres assumes it is present
  4. it has a behaviour that no user-defined schema has (i.e omitting the name of the schema default to ‘public’ , i.e when you do CREATE TABLE foobar you’re actually doing CREATE TABLE public.foobar
  5. if a ill-advised user drop it, a lot of libraries get broken ( doctrine migration by default create is own internal table in the public namespace , so as this SQL runs BEFORE the first migration, you can’t even create back public in the first migration as it will fail before)
  6. weak argument: after 10 years of managing postgresql I never ever seen people dropping and even less (re)creating the public schema, the only case I could hypothetically see for it is a extremly ordered DBA that have severa applications on the same instance , and want each of them in its own schema to ease maintenance (but in that case it’s the “infrastrcture” responsability to drop it once and for all, not one of the applications )

@morozov I understand the issue is “scary” because this library is a high profile one and after a time even bugs become part of the API , so I understand we may want to be careful on that one, so maybe we can find a deprecation strategy and changing this behaviour only on a minor/major release (at least not just a bug fix one)

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 1
  • Comments: 23 (7 by maintainers)

Most upvoted comments

Indeed, I’ve taken a look, and although it could be fixed in the ORM by tweaking SchemaTool::getSchemaFromMetadata(), that would make ORM schema creation attempt to create the public PG schema.

I see that DBAL has a schemaAssetsFilter configuration node, which I could not find documentation on but seems to be about making the DBAL ignore assets it should not manage. This leads me to formulating the following problem:

While it is possible to filter out assets in a schema, it’s not possible to filter out schemas in a database.

A solution could be to add another configuration node just for this, and let the user configure it. That way we don’t need to make any assumptions about what the user thinks about public. Assumptions should be made in Symfony recipes. That assumption should be made here, in the form of a commented out configuration node.

“everybody assume that it is present” doesn’t have a proof

  1. postgresql default value for search_path is public , so if you drop public select * from articles will break
  2. postgresql documentation states that By default such tables (and other objects) are automatically put into a schema named “public”. Every new database contains such a schema. https://www.postgresql.org/docs/current/ddl-schemas.html#DDL-SCHEMAS-PUBLIC so it read as … well “it is present in every new database” , i don’t know where’s the subjective interpertration you can
  3. doctrine migrations breaks if you drop the public schema because it is the default one it uses

I’m really sorry but what kind of proof more than that do you need , if I rephrase it as "major projects, including the database itself use it as a default value because except if you’re doing very specific stuff, it will be there " would a more objective statement ? if not how do you describe my points ?

also what’s really frustrating is that these points are from my above comments, I really took the time to gather them, and you decide to discard them

i feel a bit disappointed , the stated issues are clear not a opinion, the “opinion” is a consequence of these issues, not the other way around,

it doesn’t state an objective problem.

facts:

  • right now dbal can not handle dropping schema
  • right now dbal always report it will create the public schema (causing the “issue in an other repo” it’s because it’s the only repo using this feature in a way that is tangible for the end user , it does not take a long time to see doctrine migration is just doing a very thin layer on top of this, so there’s no way to fix it except here) making people needing to edit their down migration to remove it

when you tried to consider public as a normal schema while adding the possibly for dbal to report SQL to drop schema , you had issues

as a consequence of that i see that the way to solve this is to consider public as special schema

I’m not some kind of militan that think public should be considered special out of the blue and here find a way to spread my propaganda , i’m just a pragmatic people who try to find solution to problem

so please first discard the facts before focusing on the proposed solution

It’s only because it’s in the user’s/session’s search_path.

thanks for pointing that out, I was looking if it was overridable and it was right under my nose.

However my other point still stands and this one is slighltly amended , as in the case of a DBA requesting the search_path to be overrided to avoid accidental use of publc , it means even more that public should not be touched at all by the application layer