[CT-668] [Bug] dbt-core capturing KeyboardInterrupt Event with significant delay after pressing ctrl-c
See original GitHub issueIs there an existing issue for this?
- I have searched the existing issues
Current Behavior
When pressing ctrl-c while dbt model executing, dbt-core waiting to capture KeyboardInterrupt Event until current model execution completion and also its waiting even after that with significant delay.
My output: 14:48:40 1 of 8 START view model dbt_test.stg_customers… [RUN] 14:48:55 1 of 8 OK created view model dbt_test.stg_customers… [OK in 14.96s] 14:48:55 2 of 8 START view model dbt_test.stg_orders… [RUN] 14:49:15 2 of 8 OK created view model dbt_test.stg_orders… [OK in 19.97s] 14:49:15 dbt-core adapter: KeyboardInterrupt Event Captured!!! 14:49:15 3 of 8 START view model dbt_test.stg_payments… [RUN] 14:49:15 teradata adapter: Canceling Open connections from teradata adapter 14:49:15 teradata adapter: Connection Name : list_schemas, connection state is: closed 14:49:15 teradata adapter: Connection Name : list_None_dbt_test, connection state is: closed 14:49:15 teradata adapter: Connection Name : model.jaffle_shop.stg_payments, connection state is: closed
Here the run was cancelled with ctrl+c after 1 second from starting but the model was running for the full almost 15 seconds despite this and also next model ran for 20 secs.
Expected Behavior
When pressing ctrl-c while dbt model exeucting, dbt-core should capture KeyboardInterrupt Event immediately and it should not wait until model execution.
Steps To Reproduce
- Create dbt project with dbt-teradata adapter
- Create model with a long runtime, materialised as a table (my example runs about 400 seconds)
- Run the model with dbt run and model selector
- Wait for model to start executing
- Cancel the run with ctrl+c in the terminal
- Check the console or log file
Relevant log output
$ dbt run
14:47:54 Running with dbt=1.0.1
14:47:54 Found 8 models, 20 tests, 0 snapshots, 0 analyses, 197 macros, 0 operations, 3 seed files, 0 sources, 0 exposures, 0 metrics
14:47:54
14:48:40 Concurrency: 1 threads (target='dev')
14:48:40
14:48:40 1 of 8 START view model dbt_test.stg_customers.................................. [RUN]
14:48:55 1 of 8 OK created view model dbt_test.stg_customers............................. [OK in 14.96s]
14:48:55 2 of 8 START view model dbt_test.stg_orders..................................... [RUN]
14:49:15 2 of 8 OK created view model dbt_test.stg_orders................................ [OK in 19.97s]
14:49:15 dbt-core adapter: Satish: KeyboardInterrupt Event Captured!!!
14:49:15 3 of 8 START view model dbt_test.stg_payments................................... [RUN]
14:49:15 teradata adapter: Canceling Open connections from teradata adapter
14:49:15 teradata adapter: Connection Name : list_schemas, connection state is: closed
14:49:15 teradata adapter: Connection Name : list_None_dbt_test, connection state is: closed
14:49:15 teradata adapter: Connection Name : model.jaffle_shop.stg_payments, connection state is: closed
14:49:26 teradata adapter: calling teradata adapter cancel method to cancel connection : model.jaffle_shop.stg_payments as state of this connection is open
14:49:26 teradata adapter: Closing connection name: model.jaffle_shop.stg_payments
14:49:26 teradata adapter: returning all connection names to dbt core ['list_schemas', 'list_None_dbt_test', 'model.jaffle_shop.stg_payments']
14:49:26 CANCEL query PrintCancelLine from dbt-core list_schemas......................... [CANCEL]
14:49:26 CANCEL query PrintCancelLine from dbt-core list_None_dbt_test................... [CANCEL]
14:49:26 CANCEL query PrintCancelLine from dbt-core model.jaffle_shop.stg_payments....... [CANCEL]
14:49:26 Unhandled error while executing model.jaffle_shop.stg_payments
6 is not a valid connection pool handle
14:49:26 3 of 8 ERROR creating view model dbt_test.stg_payments.......................... [ERROR in 11.10s]
14:49:26
14:49:26 Exited because of keyboard interrupt.
14:49:26
14:49:26 Done. PASS=2 WARN=0 ERROR=0 SKIP=0 TOTAL=2
14:49:28 ctrl-c
Environment
- OS: Windows 10
- Python:Python 3.9.11
- dbt:1.0.1
Plugins:
- teradata: 1.0.0a
What database are you using dbt with?
other (mention it in “Additional Context”)
Additional Context
database : teradata connector : dbt-teradata (https://github.com/Teradata/dbt-teradata Parent Issue: https://github.com/Teradata/dbt-teradata/issues/35
dbt-teradata adapter supports cancelling connections. this extended dbt sql adatper functionality and it inherits the is_cancelable
Issue Analytics
- State:
- Created a year ago
- Comments:9 (7 by maintainers)
Top Related StackOverflow Question
@jtcohen6 Yes, I added
dbt-core adapter: KeyboardInterrupt Event Captured!!!log line in dbt-core’s code base at the spot linked above as below.Please let me know if you any additional information required. Looking forward to provide more information.
Although we are closing this issue as stale, it’s not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.