airflow: Bad terminal handling of `airflow db shell` command with Postgres DB
Apache Airflow version
main (development)
What happened
This happens in both iTerm 2 and the built-in Terminal.app in macOS when running ~breeze start-airflow~. airflow db shell
when using postgres as a backend.
Within psql
(run via airflow db shell
):
- Scrollback editing (eg: hitting up arrow to edit a previous query) is messed up for multi-line queries. I don’t have a better description. I recorded a video of what happens.
- Some things require an additional keypress (eg: quitting the pager)
- Some things, like ctrl+P to scrollback needing an additional keypress happen in a normal Docker environment. This is different and distinct from that.
- SIGINT isn’t registered when trying to cancel editing a query and reset the SQL prompt
I suspect that there’s a lot of interaction happening between iTerm/Terminal.app, ZSH, Python/Breeze, Bash, Docker, tmux, Python/Airflow CLI, and psql, and some keypresses are being filtered or queued along the way. So there are multiple fun rabbit holes to chase down to solve this.
What you think should happen instead
- Scrollback editing should work. Your text cursor should be in the correct place and the text should be rendered correctly.
- An additional keypress to exit the pager should not be necessary
- SIGINT should cancel editing the query and reset the edit line (eg: it should return to an empty SQL prompt:
airflow=#
instead ofairflow-#
)
How to reproduce
-
Scrollback editing is messed up - indentation and text wrapping are off/wrong
- This happens even when I don’t adjust the tmux pane sizes at all
- Example:
-
Run
breeze start-airflow
in a terminal -
Run
docker exec -it $CONTAINER_ID /bin/bash
with the breeze container that is running -
Run
airflow db shell
within the container -
Then this happens (note the up arrow keypresses aren’t called out visually):
https://user-images.githubusercontent.com/597113/188750191-2b3e8ef3-ee89-4b2e-872d-1451807bdbc7.mov
-
-
Some things require an additional keypress Example:
- Run a SQL query:
airflow=# SELECT * FROM dag_schedule_dataset_reference
- it displays in the pager
dataset_id | dag_id | created_at | updated_at ------------+-----------------------+-------------------------------+------------------------------- 3 | consumer_dag_2 | 2022-09-06 20:49:23.719275+00 | 2022-09-06 20:49:23.719295+00 2 | consumer_dag_2 | 2022-09-06 20:49:23.7193+00 | 2022-09-06 20:49:23.719302+00 4 | taskflow_consumer_dag | 2022-09-06 20:49:23.719305+00 | 2022-09-06 20:49:23.719308+00 5 | taskflow_consumer_dag | 2022-09-06 20:49:23.719311+00 | 2022-09-06 20:49:23.719313+00 6 | taskflow_consumer_dag | 2022-09-06 20:49:23.719316+00 | 2022-09-06 20:49:23.719319+00 1 | consumer_dag_1 | 2022-09-06 20:49:23.719322+00 | 2022-09-06 20:49:23.719324+00 (6 rows) (END)
- Usually you just hit
q
and it exits immediately, but the pager sticks around:dataset_id | dag_id | created_at | updated_at ------------+-----------------------+-------------------------------+------------------------------- 3 | consumer_dag_2 | 2022-09-06 20:49:23.719275+00 | 2022-09-06 20:49:23.719295+00 2 | consumer_dag_2 | 2022-09-06 20:49:23.7193+00 | 2022-09-06 20:49:23.719302+00 4 | taskflow_consumer_dag | 2022-09-06 20:49:23.719305+00 | 2022-09-06 20:49:23.719308+00 5 | taskflow_consumer_dag | 2022-09-06 20:49:23.719311+00 | 2022-09-06 20:49:23.719313+00 6 | taskflow_consumer_dag | 2022-09-06 20:49:23.719316+00 | 2022-09-06 20:49:23.719319+00 1 | consumer_dag_1 | 2022-09-06 20:49:23.719322+00 | 2022-09-06 20:49:23.719324+00 (6 rows) (END)
- The
q
keypress is registered upon your next keypress, whatever that is (in this case,6
). And the second keypress is echoed in the SQL editor:airflow=# SELECT * FROM dag_schedule_dataset_reference; airflow=# 6
- Run a SQL query:
-
SIGINT isn’t registered to cancel editing a query Example:
- Write a query:
airflow=# SELECT * FROM dag_messedupspelling
- Then try to cancel/reset it with Ctrl+C:
(nothing happens) In normal psql this will reset the query editor (you can tell because theairflow=# SELECT * FROM dag_messedupspelling airflow-#
airflow-#
prompt changes back toairflow=#
).
- Write a query:
Operating System
macOS 11.6.8 Big Sur
Versions of Apache Airflow Providers
N/A
Deployment
Other Docker-based deployment
Deployment details
Breeze
Anything else
This happens every time.
Apologies if this isn’t the place to report Breeze issues.
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project’s Code of Conduct
About this issue
- Original URL
- State: open
- Created 2 years ago
- Comments: 15 (14 by maintainers)
I updated the description to clarify it. Fancy to attempt to fix it @blag ? I think it is the matter of how subprocess is used to run
psql