postgres_exporter: exclude-databases does not properly work
What did you do?
I ran an exporter process including the auto-discover-databases
and exclude-databases
flags. Also pulling from a custom query files:
exec postgres_exporter --web.listen-address :9189 --disable-default-metrics --auto-discover-databases --exclude-databases postgres,template0,template1 --extend.query-path /tmp/queries.yaml
/tmp/queries.yaml
file includes two different queries:
pg_database:
query: "SELECT pg_database.datname, pg_database_size(pg_database.datname) as size_bytes FROM pg_database"
master: true
metrics:
- datname:
usage: "LABEL"
description: "Name of the database"
- size_bytes:
usage: "GAUGE"
description: "Disk space used by the database"
pg_database_2:
query: "SELECT pg_database.datname, pg_database_size(pg_database.datname) as size_bytes FROM pg_database WHERE datname = current_database()"
master: true
metrics:
- datname:
usage: "LABEL"
description: "Name of the database"
- size_bytes:
usage: "GAUGE"
description: "Disk space used by the database"
DBs on my instance:
postgres=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+----------+----------+------------+------------+-----------------------
nsoengas | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =Tc/postgres +
| | | | | postgres=CTc/postgres+
| | | | | nsoengas=C/postgres
postgres | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
template0 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +
| | | | | postgres=CTc/postgres
(4 rows)
postgres=#
What did you expect to see?
Since I am using exclude-databases
flag and then excluding postgres,template0,template1
DBs, then my expectation is postgres exporter connecting to nsoengas
DB and performing the queries on my custom file (/tmp/queries.yaml).
According to those queries, pg_database
should retrieve datname and size for every single database on given instance. pg_database_2
should ONLY show datname and size for the database I am connected to. I’ve included the proper query filter to do that.
So, pg_database_2
should connect to “nsoengas” DB and report its datname and size.
What did you see instead? Under which circumstances?
pg_database
seems to be “showing” what it suppose to show. However, pg_database_2
is showing “postgres” as datname. Since I explicitly included the filter WHERE datname = current_database()
AFAIK it is connecting to “postgres” database and retrieving its size
It could be seen while scrapping on the exporter port:
[root@nsoengas-db-0 /]# curl http://localhost:9189/metrics | grep pg_database
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0# HELP pg_database_2_size_bytes Disk space used by the database
# TYPE pg_database_2_size_bytes gauge
pg_database_2_size_bytes{datname="postgres",server="localhost:5432"} 7.526936e+06
# HELP pg_database_size_bytes Disk space used by the database
# TYPE pg_database_size_bytes gauge
pg_database_size_bytes{datname="nsoengas",server="localhost:5432"} 7.58428e+06
pg_database_size_bytes{datname="postgres",server="localhost:5432"} 7.526936e+06
pg_database_size_bytes{datname="template0",server="localhost:5432"} 7.414276e+06
pg_database_size_bytes{datname="template1",server="localhost:5432"} 7.414276e+06
100 51180 0 51180 0 0 4543k 0 --:--:-- --:--:-- --:--:-- 4543k
[root@nsoengas-db-0 /]#
Environment
- System information:
[root@nsoengas-db-0 ~]$ uname -srm
Linux 5.4.17-2102.202.5.el7uek.x86_64 x86_64
[root@nsoengas-db-0 ~]$
- postgres_exporter version:
[root@nsoengas-db-0 /]# postgres_exporter --version
postgres_exporter, version 0.10.0 (branch: HEAD, revision: 57719ba53cac428769aaf3c4c0bb742df3cfca98)
build user: root@4dcb2c7f1315
build date: 20210709-11:49:20
go version: go1.16.5
platform: linux/amd64
[root@nsoengas-db-0 /]#
- postgres_exporter flags:
exec postgres_exporter --web.listen-address :9189 --disable-default-metrics --auto-discover-databases --exclude-databases postgres,template0,template1 --extend.query-path /tmp/queries.yaml
- PostgresSQL version:
postgres=# SELECT version();
version
-----------------------------------------------------------------------------------------------------------
PostgreSQL 9.6.15 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36), 64-bit
(1 row)
postgres=#
About this issue
- Original URL
- State: open
- Created 3 years ago
- Reactions: 1
- Comments: 16 (4 by maintainers)
Commits related to this issue
- vm-builder: Turn off --auto-discover-databases It causes exporter connection to every database in the Postgres and it was disabled for pods a while ago, but I forgot that VMs have a different build p... — committed to neondatabase/autoscaling by ololobus 8 months ago
- vm-builder: Turn off --auto-discover-databases (#571) It causes exporter connection to every database in the Postgres and it was disabled for pods a while ago, but I forgot that VMs have a differen... — committed to neondatabase/autoscaling by ololobus 8 months ago
Hello,
I’ve the same behavior. I use v0.10 in docker. It return all databases.
+1
same on 0.11.1 and azure postgresql. Last version it worked for me was v0.10.0.
Thx a lot, I reverted to 0.8.0 since trying 0.13.1 breaks some queries due to breaking changes - but then the exclude databases works properly. I will fix the queries at some point and migrate to 0.13+.
sad to see that exclude databases is deprecated though…
@The-Seyed I don’t think you’re in the same situation as the one describe in this report (see my comment above for what I think is the explanation of the observed behaviour, which is not a bug). It seems to me that you’re facing the issue of the newly introduced pg_database collector (https://github.com/prometheus-community/postgres_exporter/releases/tag/v0.11.0) not respecting the
--exclude-databases
option. A PR is open which should fix this: https://github.com/prometheus-community/postgres_exporter/pull/697.+1
AWS RDS
rdsadmin
database is not accessible and it is not excluded when using--exclude-databases rdsadmin
parameter.