Can we launch accelerate from a Python script instead of CLI?

See original GitHub issue

I am trying to do hyperparameter optimization using Ray Tune and want to use accelerate to handle multi-gpu training.

I am fairly new to accelerate and could only find the CLI method (accelerate launch) to launch the script. Do we have something like accelerate.launch or something similar to notebook_launcher that can launch the training from the script?

I am looking to see if I can pass a function that takes a config as an argument - something like this: accelerate.launch(my_train_func, ags=[config], ...)

Thanks in advance.

Issue Analytics

  • State:open
  • Created a year ago
  • Comments:7 (1 by maintainers)

github_iconTop GitHub Comments

2reactions
sguggercommented, Nov 1, 2022

This would require to run independent subprocesses, not too sure how Ray Tune is going to like that since it won’t be able to inspect how training is going.

0reactions
pacman100commented, Dec 6, 2022

Hello @samiede and @ayulockin, have you looked at this test: https://github.com/huggingface/accelerate/blob/main/tests/deepspeed/test_deepspeed.py#L657-L699 wherein accelerate is launched from a python script instead of cli? Let us know if that solves the issue.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Running launch command without python script #256 - GitHub
Currently, the main argument to the accelerate launch command is a python script. For some projects, it would be convenient for me to...
Read more >
Quick tour — accelerate documentation - Hugging Face
Now that this is done, you can run your script with the following command: accelerate launch path_to_script.py --args_for_the_script.
Read more >
speed python script on command line vs launched in IDLE shell
I noticed several times that a simple python script (with some straightforward algebraic computation in a loop) runs considerably faster (up ...
Read more >
accelerate - PyPI
Accelerate supports training on single/multiple GPUs using DeepSpeed. To use it, you don't need to change anything in your training code; you can...
Read more >
Running Python script on GPU. - GeeksforGeeks
First, make sure that Nvidia drivers are upto date also you can install cudatoolkit explicitly from here. then install Anaconda add anaconda to...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found