Skip to content
Snippets Groups Projects

Lightning acc

Merged Gokhan OZBULAK requested to merge lightning-acc into main
1 unresolved thread
1 file
+ 8
9
Compare changes
  • Side-by-side
  • Inline
@@ -83,15 +83,14 @@ def reusable_options(f):
"-c",
help="Number of chunks in every batch (this parameter affects "
"memory requirements for the network). The number of samples "
"loaded for every iteration will be batch-size/batch-chunk-count. "
"batch-size needs to be divisible by batch-chunk-count, otherwise an "
"error will be raised. This parameter is used to reduce the number of "
"samples loaded in each iteration, in order to reduce the memory usage "
"in exchange for processing time (more iterations). This is especially "
"interesting when one is training on GPUs with limited RAM. The "
"default of 1 forces the whole batch to be processed at once. Otherwise "
"the batch is broken into batch-chunk-count pieces, and gradients are "
"accumulated to complete each batch.",
"loaded for every iteration will be batch-size*batch-chunk-count. "
"This parameter is used to reduce the number of samples loaded in each "
"iteration, in order to reduce the memory usage in exchange for "
"processing time (more iterations). This is especially interesting "
"when one is training on GPUs with limited RAM. The default of 1 forces "
"the whole batch to be processed at once. Otherwise the batch is "
"multiplied by batch-chunk-count pieces, and gradients are accumulated "
"to complete each batch.",
required=True,
show_default=True,
default=1,
Loading