[CT-668] [Bug] dbt-core capturing KeyboardInterrupt Event with significant delay after pressing ctrl-c

See original GitHub issue

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

When pressing ctrl-c while dbt model executing, dbt-core waiting to capture KeyboardInterrupt Event until current model execution completion and also its waiting even after that with significant delay.

My output: 14:48:40 1 of 8 START view model dbt_test.stg_customers… [RUN] 14:48:55 1 of 8 OK created view model dbt_test.stg_customers… [OK in 14.96s] 14:48:55 2 of 8 START view model dbt_test.stg_orders… [RUN] 14:49:15 2 of 8 OK created view model dbt_test.stg_orders… [OK in 19.97s] 14:49:15 dbt-core adapter: KeyboardInterrupt Event Captured!!! 14:49:15 3 of 8 START view model dbt_test.stg_payments… [RUN] 14:49:15 teradata adapter: Canceling Open connections from teradata adapter 14:49:15 teradata adapter: Connection Name : list_schemas, connection state is: closed 14:49:15 teradata adapter: Connection Name : list_None_dbt_test, connection state is: closed 14:49:15 teradata adapter: Connection Name : model.jaffle_shop.stg_payments, connection state is: closed

Here the run was cancelled with ctrl+c after 1 second from starting but the model was running for the full almost 15 seconds despite this and also next model ran for 20 secs.

Expected Behavior

When pressing ctrl-c while dbt model exeucting, dbt-core should capture KeyboardInterrupt Event immediately and it should not wait until model execution.

Steps To Reproduce

  1. Create dbt project with dbt-teradata adapter
  2. Create model with a long runtime, materialised as a table (my example runs about 400 seconds)
  3. Run the model with dbt run and model selector
  4. Wait for model to start executing
  5. Cancel the run with ctrl+c in the terminal
  6. Check the console or log file

Relevant log output

$ dbt  run
14:47:54  Running with dbt=1.0.1
14:47:54  Found 8 models, 20 tests, 0 snapshots, 0 analyses, 197 macros, 0 operations, 3 seed files, 0 sources, 0 exposures, 0 metrics
14:47:54
14:48:40  Concurrency: 1 threads (target='dev')
14:48:40
14:48:40  1 of 8 START view model dbt_test.stg_customers.................................. [RUN]
14:48:55  1 of 8 OK created view model dbt_test.stg_customers............................. [OK in 14.96s]
14:48:55  2 of 8 START view model dbt_test.stg_orders..................................... [RUN]
14:49:15  2 of 8 OK created view model dbt_test.stg_orders................................ [OK in 19.97s]
14:49:15  dbt-core adapter: Satish: KeyboardInterrupt Event Captured!!!
14:49:15  3 of 8 START view model dbt_test.stg_payments................................... [RUN]
14:49:15  teradata adapter:  Canceling Open connections from teradata adapter
14:49:15  teradata adapter:  Connection Name :  list_schemas, connection state is: closed
14:49:15  teradata adapter:  Connection Name :  list_None_dbt_test, connection state is: closed
14:49:15  teradata adapter:  Connection Name :  model.jaffle_shop.stg_payments, connection state is: closed
14:49:26  teradata adapter:  calling teradata adapter cancel method to cancel connection : model.jaffle_shop.stg_payments as state of this connection is open
14:49:26  teradata adapter:  Closing connection name: model.jaffle_shop.stg_payments
14:49:26  teradata adapter: returning all connection names to dbt core ['list_schemas', 'list_None_dbt_test', 'model.jaffle_shop.stg_payments']
14:49:26  CANCEL query PrintCancelLine from dbt-core list_schemas......................... [CANCEL]
14:49:26  CANCEL query PrintCancelLine from dbt-core list_None_dbt_test................... [CANCEL]
14:49:26  CANCEL query PrintCancelLine from dbt-core model.jaffle_shop.stg_payments....... [CANCEL]
14:49:26  Unhandled error while executing model.jaffle_shop.stg_payments
6 is not a valid connection pool handle
14:49:26  3 of 8 ERROR creating view model dbt_test.stg_payments.......................... [ERROR in 11.10s]
14:49:26
14:49:26  Exited because of keyboard interrupt.
14:49:26
14:49:26  Done. PASS=2 WARN=0 ERROR=0 SKIP=0 TOTAL=2
14:49:28  ctrl-c

Environment

- OS: Windows 10 
- Python:Python 3.9.11
- dbt:1.0.1
    Plugins:
     - teradata: 1.0.0a

What database are you using dbt with?

other (mention it in “Additional Context”)

Additional Context

database : teradata connector : dbt-teradata (https://github.com/Teradata/dbt-teradata Parent Issue: https://github.com/Teradata/dbt-teradata/issues/35

dbt-teradata adapter supports cancelling connections. this extended dbt sql adatper functionality and it inherits the is_cancelable

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:9 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
SatishChGitcommented, May 20, 2022

@jtcohen6 Yes, I added dbt-core adapter: KeyboardInterrupt Event Captured!!! log line in dbt-core’s code base at the spot linked above as below.

 except KeyboardInterrupt:
            logger.info("KeyboardInterrupt Event Captured!!!")
            self._cancel_connections(pool)
            print_run_end_messages(self.node_results, keyboard_interrupt=True)
            raise

Please let me know if you any additional information required. Looking forward to provide more information.

0reactions
github-actions[bot]commented, Dec 8, 2022

Although we are closing this issue as stale, it’s not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.

Read more comments on GitHub >

github_iconTop Results From Across the Web

KeyboardInterrupt doesn't work when arcgisscripting module ...
The KeyboardInterrupt exception -- thrown when Ctrl-C, or Ctrl-Break on some machines, are pressed in a Python console window -- is commonly ...
Read more >
Catch Keyboard Interrupt in program that is waiting on an Event
pause() is a blocking function and gets unblocked when a signal is received by the process. In this case, when ^C is pressed,...
Read more >
dbt - bytemeta
[CT-668] [Bug] dbt-core capturing KeyboardInterrupt Event with significant delay after pressing ctrl-c. miro-ur. miro-ur OPEN · Updated 2 months ago ...
Read more >
[REQUEST]: Could be use compiler-explorer in sublime text?
[CT-668] [Bug] dbt-core capturing KeyboardInterrupt Event with significant delay after pressing ctrl-c, 6, 2022-05-19, 2022-11-04.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found