Can we launch accelerate from a Python script instead of CLI?
See original GitHub issueI am trying to do hyperparameter optimization using Ray Tune and want to use accelerate to handle multi-gpu training.
I am fairly new to accelerate and could only find the CLI method (accelerate launch) to launch the script. Do we have something like accelerate.launch or something similar to notebook_launcher that can launch the training from the script?
I am looking to see if I can pass a function that takes a config as an argument - something like this: accelerate.launch(my_train_func, ags=[config], ...)
Thanks in advance.
Issue Analytics
- State:
- Created a year ago
- Comments:7 (1 by maintainers)
Top Results From Across the Web
Running launch command without python script #256 - GitHub
Currently, the main argument to the accelerate launch command is a python script. For some projects, it would be convenient for me to...
Read more >Quick tour — accelerate documentation - Hugging Face
Now that this is done, you can run your script with the following command: accelerate launch path_to_script.py --args_for_the_script.
Read more >speed python script on command line vs launched in IDLE shell
I noticed several times that a simple python script (with some straightforward algebraic computation in a loop) runs considerably faster (up ...
Read more >accelerate - PyPI
Accelerate supports training on single/multiple GPUs using DeepSpeed. To use it, you don't need to change anything in your training code; you can...
Read more >Running Python script on GPU. - GeeksforGeeks
First, make sure that Nvidia drivers are upto date also you can install cudatoolkit explicitly from here. then install Anaconda add anaconda to...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
This would require to run independent subprocesses, not too sure how Ray Tune is going to like that since it won’t be able to inspect how training is going.
Hello @samiede and @ayulockin, have you looked at this test: https://github.com/huggingface/accelerate/blob/main/tests/deepspeed/test_deepspeed.py#L657-L699 wherein accelerate is launched from a python script instead of cli? Let us know if that solves the issue.