Skip to content

Ak/fix train feeding#5

Open
AlexeyKruglov wants to merge 3 commits into
spro:masterfrom
AlexeyKruglov:ak/fix-train-feeding
Open

Ak/fix train feeding#5
AlexeyKruglov wants to merge 3 commits into
spro:masterfrom
AlexeyKruglov:ak/fix-train-feeding

Conversation

@AlexeyKruglov

Copy link
Copy Markdown
  1. Fix index out of range error.
  2. About 8x (for me) training speed-up by feeding whole sample sequence through CuDNN, not char-by-char.
  3. Inference: GPU memory requirement not growing with generated seq length anymore -- by dropping Variable history in inference.

... by feeding the whole sequence to CuDNN RNN, as opposed to character-by-character.
@codeman38

codeman38 commented Jan 10, 2018

Copy link
Copy Markdown

For what it's worth, using the pre-built pytorch 1.3 on macOS (where CUDA is not available), on an early-2015 MacBook Pro, this patch doesn't seem to improve performance on CPU-- instead, it makes training more than twice as slow. [Edited to add: Tested with CPU on Linux, and got similar results there as well.]

After 100 iterations using the default hyperparameters:
master: ~1.09s/iteration
fix-train-feeding: ~3.71s/iteration

Not sure whether this is an issue with this specific CPU, an inefficiency in PyTorch itself, or something that could be further improved in the char-rnn code, but it's definitely worth pointing out either way.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants