projects / platform / upstream / pytorch.git / commit
? search:
summary | shortlog | log | commit | commitdiff | tree
(parent: a5e7b1d) | patch
Remove chunk count check on the ChunkBuffer (#16868)
authorJaliya Ekanayake <jaliyaek@microsoft.com>
Wed, 13 Feb 2019 18:26:15 +0000 (10:26 -0800)
committerFacebook Github Bot <facebook-github-bot@users.noreply.github.com>
Wed, 13 Feb 2019 19:09:42 +0000 (11:09 -0800)
commitbc39cf4d5e90410f00d8654216ae8060888cf380
tree3a93fb58b8167bc75d9eec7bbafca687f952fd0ftree | snapshot
parenta5e7b1d03268cc2afbce4109206dcf6a57b0f4fdcommit | diff
Remove chunk count check on the ChunkBuffer (#16868)

Summary:
Previously, the ChunkBuffer depends on the remaining chunk count to signal end of dataloading. This does not work with distributed samplers where each sampler only loads a subset of  chunks. This refactor remove the dependency on the remaining chunk count at the ChunkBuffer.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/16868

Differential Revision: D14066517

Pulled By: goldsborough

fbshipit-source-id: 293dfe282ceff326dff0876c2f75c2ee4f4463e2
test/cpp/api/dataloader.cpp diff | blob | history
torch/csrc/api/include/torch/data/datasets/chunk.h diff | blob | history
torch/csrc/api/include/torch/data/samplers/base.h diff | blob | history
Domain: Machine Learning / ML Framework;