Iterable Dataset Multithreaded - data - PyTorch Forums
discuss.pytorch.org › t › iterable-datasetDec 21, 2021 · Hello there, I want to make custom dataset or dataloader, just don’t know which is the best way to do it. This is a self supervised task, where you want to blank out part of the input and then use it as a “label”. Let’s say I have .npy files, each one of shape (600,30000), and I want to do the following: Read the file, pick a row from the 600, take a slice around that row, say (32 ...
Using iterable datasets - PyTorch Forums
discuss.pytorch.org › t › using-iterable-datasetsSep 19, 2019 · Hello, Im trying to set up a processes to deal with streaming sensor data. I understand this is the exact usecase for iterable datasets. Unfortunately it seems to be a relatively new feature and tutorials on how to make and use iterable datasets appear to be rare. To start I am mocking by making my iter function grab from a CSV. I am not sure exactly what my iter function should be returning ...
Using iterable datasets - PyTorch Forums
https://discuss.pytorch.org/t/using-iterable-datasets/5637419.09.2019 · Hello, Im trying to set up a processes to deal with streaming sensor data. I understand this is the exact usecase for iterable datasets. Unfortunately it seems to be a relatively new feature and tutorials on how to make and use iterable datasets appear to be rare. To start I am mocking by making my iter function grab from a CSV. I am not sure exactly what my iter function should be …