deeplearning4j: Error when customizing PathLabelGenerator while setting up data pipeline for semantic segmentation
Issue Description
Hello guys, I am trying to implement semantic image segmentation with Unet network architecture, and i am having a hard time concerning how to set the data pipeline. I have managed to get the training data arranged in the following way:
- One directory in which i have the training images
- Another directory in which i put the labels (segmented images)
- I named the files in such a way that any image file in the training image directory has as corresponding label an image with same file name in the label directory.
Now i need to customize PathLabelGenerator to map the input images to their corresponding label images, and i intend to link the data by file name, meaning i want to link any training image and its corresponding label by making them have the same file name.
Here is my customized PathLabelGenerator code which i called CustomPathLabelGenerator:
public class CustomPathLabelGenerator implements org.datavec.api.io.labels.PathLabelGenerator {
public CustomPathLabelGenerator() {
}
NativeImageLoader imageLoader = new NativeImageLoader(512, 512, 3);
File labelsDir = new File(System.getProperty("user.dir"), "/src/main/resources/data/trainLabels/");
public Writable getLabelForPath(String path) {
NDArrayWritable ndArrayWritable = null;
try {
Collection<File> files = FileUtils.listFiles(labelsDir, null, true);
for(Iterator<File> iter = files.iterator(); iter.hasNext();) {
File file = (File) iter.next();
if(file.getName().equals(new File(path).getName())) {
ndArrayWritable = new NDArrayWritable(imageLoader.asMatrix(new File(path).getName()));
}
}
}catch (Exception e) {
e.printStackTrace();
}
return ndArrayWritable;
}
public Writable getLabelForPath(URI uri) {
System.out.println("URL in first getLabelForPath function: " + new File(uri).toString());
return getLabelForPath(new File(uri).toString());
}
public boolean inferLabelClasses() {
return true;
}
}
And here is my code for for the data pipeline:
CustomPathLabelGenerator labelMaker = new CustomPathLabelGenerator();
File mainTrainDataPath = new File(System.getProperty("user.dir"), "/src/main/resources/data/trainData/");
FileSplit trainFileSplit = new FileSplit(mainTrainDataPath, NativeImageLoader.ALLOWED_FORMATS, rng);
int numExamples = toIntExact(trainFileSplit.length());
RandomPathFilter trainPathFilter = new RandomPathFilter(rng, null, numExamples);
InputSplit[] trainInputSplit = trainFileSplit.sample(trainPathFilter);
InputSplit trainData = trainInputSplit[0];
DataNormalization scaler = new ImagePreProcessingScaler(0, 1);
ImageRecordReader trainRecordReader = new ImageRecordReader(height, width, channels, labelMaker);
trainRecordReader.initialize(trainData);
But i am having the following error while just trying to check if the mapping works well:
java.io.FileNotFoundException: train.tif (The system cannot find the file specified)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(Unknown Source)
at java.io.FileInputStream.<init>(Unknown Source)
at org.datavec.image.loader.NativeImageLoader.asMatrix(NativeImageLoader.java:231)
at org.datavec.image.loader.NativeImageLoader.asMatrix(NativeImageLoader.java:226)
at com.neuronalstructuressegmentation.CustomPathLabelGenerator.getLabelForPath(CustomPathLabelGenerator.java:43)
at com.neuronalstructuressegmentation.CustomPathLabelGenerator.getLabelForPath(CustomPathLabelGenerator.java:56)
at org.datavec.image.recordreader.BaseImageRecordReader.initialize(BaseImageRecordReader.java:143)
at com.neuronalstructuressegmentation.Segmentation.execute(Segmentation.java:90)
at com.neuronalstructuressegmentation.Segmentation.main(Segmentation.java:380)
Exception in thread "main" java.lang.NullPointerException
at org.datavec.image.recordreader.BaseImageRecordReader.initialize(BaseImageRecordReader.java:143)
at com.neuronalstructuressegmentation.Segmentation.execute(Segmentation.java:90)
at com.neuronalstructuressegmentation.Segmentation.main(Segmentation.java:380)
I made sure that the files are present in the respective derectories.
I currently have no idea about the origin or the cause of this error.
Version Information
Please indicate relevant versions, including, if relevant:
- Deeplearning4j version 1.0.0-beta3
- Platform information windows 8.1
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 16 (5 by maintainers)
Thanks. I did that. Now its working fine.