History

Patrick Esser eee8df53b5 test gpu scripts and other launchers and configs		2022-07-22 09:56:22 +00:00
..
resume_512	slurmy	2022-07-09 22:08:16 +00:00
resume_512_improvedaesthetic	more launch scripts	2022-07-09 22:10:20 +00:00
resume_768_hr	more launch scripts	2022-07-09 22:10:20 +00:00
v1_iahr_torch111	add v1 hr subset of aesthetics training, resume v3	2022-07-22 09:53:43 +00:00
v1_improvedaesthetics	final v1 restart	2022-07-18 23:47:36 +00:00
v1_improvedaesthetics_torch111	test gpu scripts and other launchers and configs	2022-07-22 09:56:22 +00:00
v1_laionhr_torch111	test gpu scripts and other launchers and configs	2022-07-22 09:56:22 +00:00
v2_laionhr1024	v2 on laionhr 1024	2022-07-14 23:36:08 +00:00
v3_pretraining	add v1 hr subset of aesthetics training, resume v3	2022-07-22 09:53:43 +00:00
README.md	ready to slurm	2022-07-06 22:52:52 +00:00

Example

Resume f8 @ 512 on Laion-HR

sbatch scripts/slurm/resume_512/sbatch.sh

Reuse

To reuse this as a template, copy sbatch.sh and launcher.sh somewhere. In sbatch.sh, adjust the lines

#SBATCH --job-name=stable-diffusion-512cont
#SBATCH --nodes=24

and the path to your launcher.sh in the last line,

srun bash /fsx/stable-diffusion/stable-diffusion/scripts/slurm/resume_512/launcher.sh

In launcher.sh, adjust CONFIG and EXTRA. Maybe give it a test run with debug flags uncommented and a reduced number of nodes.