-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathindex.html
More file actions
731 lines (675 loc) · 41.1 KB
/
index.html
File metadata and controls
731 lines (675 loc) · 41.1 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<meta name="description"
content="IACS Capstone Project: Deep Learning for Nematode Detection and Classification (Partner: BASF)">
<meta name="author" content="Jiejun Lu, Hongxiang Qiu, Weidong Xu, Zeyu Zhao">
<!-- TODO add favicon -->
<link rel="icon" href="images/favicon.png">
<title>Deep Learning for Nematode Detection and Classification</title>
<link rel="stylesheet" href="css/bootstrap.css">
<link rel="stylesheet" href="css/site.css">
<link rel="stylesheet" href="css/fontawesome-all.css">
</head>
<body data-spy="scroll" data-target="#nav-content" data-offset="20">
<div class="container-fluid">
<div class="row">
<nav id="nav" class="col-lg-2 col-md-2 col-sm-3 position-fixed">
<div id="sidebar-header">
<div>
<img src="images/iacs.png"/>
</div>
<h3><i class="fas fa-list-alt" style="padding-right: 0.4em; font-size: 0.8em"></i>Table of Contents</h3>
</div>
<nav id="topbar" class="navbar navbar-light">
<div class="navbar-brand">
<a href="#">
<img src="images/iacs.png"/>
</a>
</div>
<div class="float-right ml-auto">
<button class="navbar-toggler float-right" type="button" data-toggle="collapse"
data-target="#nav-content"
aria-controls="navbarNav" aria-expanded="false" aria-label="Toggle navigation">
<span class="navbar-toggler-icon"></span>
</button>
</div>
</nav>
<div id="nav-content" class="collapse navbar-collapse">
<ul class="nav nav-pills flex-column">
<li class="nav-item ">
<a class="nav-link first active" href="#introduction">
<i class="fas fa-clipboard-list"></i>Introduction
</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#data">
<i class="fas fa-database"></i>Data
</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#literature_review">
<i class="fas fa-book"></i>Literature Review
</a>
</li>
<li class="nav-item"><!-- Link with dropdown items -->
<a class="nav-link"
data-target="#modelMenu" data-toggle="collapse"
aria-expanded="false" href="#model"><i class="fas fa-superscript"></i>Modeling</a>
<ul class="nav collapse" id="modelMenu">
<li class="nav-item"><a class="nav-link" href="#conventional">Conventional Models</a></li>
<li class="nav-item"><a class="nav-link" href="#rcnn">Deep Learning (R-CNN)</a></li>
<li class="nav-item"><a class="nav-link" href="#sarcnn">Adding Scale Information</a></li>
</ul>
</li>
<li class="nav-item">
<a class="nav-link" href="#results">
<i class="fas fa-edit"></i>Results
</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#deliverable">
<i class="fas fa-box-open"></i>Deliverable
</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#interpretation">
<i class="fas fa-table"></i>Empirical Interpretation
</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#conclusions">
<i class="fas fa-compass"></i>Conclusions & Future Work
</a>
</li>
</ul>
</div>
</nav>
<div id="main" class="col">
<div id="content">
<div id="header" class="row col">
<div class="col">
<h1>
A Biological Challenge in the Billions
</h1>
<h2>
Deep Learning for Nematode Detection and Classification
</h2>
<p style="color: OLIVEDRAB;">By: Jiejun Lu, Hongxiang Qiu, Weidong Xu and Zeyu Zhao</p>
<hr class="row hr-heavy">
</div>
</div>
<div id="introduction" class="col section">
<div class="row">
<h2>Introduction</h2>
</div>
<hr class="row hr-light"/>
<div class="row">
<p>
<img src="images/BASF.png" class="image-surround"/>
The organic food industry has been growing rapidly over the past few years. An important task in the organic food industry is to find alternatives to chemical pesticides for protecting crops and plants from pests due to the detrimental consequences of pesticides. An alternative to chemical pesticides is
to
encourage naturally occurring organisms in the soil to destroy pests such as slugs, weevils
and
caterpillars. Nematodes are naturally occurring microscopic worms already present in the
soil
that
actively seek out and destroy pests. Biologicals for pest control is one of the focus areas
within
BASF's Agricultural Solutions business. They develop unique formulations of beneficial
nematodes
and
their storage media to provide optimum stability and product performance. The nematodes in
the
products selectively target problematic insect species, while remaining harmless to
beneficial
insects (e.g. ladybugs) and nearby wildlife. Finding an efficient way to use nematodes to
protect
crops could potentially reduce the need to introduce artificial chemicals in to the food
supply. BASF is the only mass producer of nematodes used for slug protection, utilizing a plant in
Littlehampton, UK equipped with 20 vessels that provide over 190,000 L of fermentation
capacity
and
can hold ~ 38 × 10<sup>12</sup> nematodes. Product quality is continually tested and
maintained
through
refrigeration and efficient shipping logistics to all their customers, which is a key BASF
skill.
</p>
<figure class="figure text-center figure-center">
<img src="images/life_stages.png" class="figure-img img-fluid"/>
<figcaption class="figure-caption">Different life stages of nematodes.</figcaption>
</figure>
<p>
As shown in the figure above, nematodes have different efficacy at different stages of their life cycles. Automation of
identification of whether a nematode is infective juvenile or not enables efficient quality
control. This
project
applied deep learning techniques to microscope image data of nematode populations to
determine
whether nematodes are at infective juvenile stage <b>(interested)</b> or not <b>(non-interested)</b>.
We implemented and trained a scale-aware (SA) version of
faster
region-based convolutional neural network (faster R-CNN) model for the identification and
classification of nematodes from microscope images, and built a software package in a
virtual machine image for the automation of nematode classification task. The expected input and output is shown in the figure below.
</p>
<figure class="figure">
<img src="images/input_output.png" class="figure-img img-fluid"/>
<figcaption class="figure-caption">Expected input and output.</figcaption>
</figure>
</div>
</div>
<div id="data" class="col section">
<hr class="row hr-heavy"/>
<div class="row">
<h2>Data</h2>
</div>
<hr class="row hr-light"/>
<div class="row">
<h3>First batch of data</h3>
<p>
BASF didn't have a digital way to label the nematodes in the microscope images. Initially,
we only had some sample microscope images and a slide briefing the characteristics of each
life stage of nematodes. A sample image is shown below.
</p>
<figure class="figure text-center figure-center">
<img src="images/life_stages.png" class="figure-img img-fluid"/>
<figcaption class="figure-caption">Labeling in the first batch of data.</figcaption>
</figure>
</div>
<div class="row">
<h3>Labeling tool</h3>
<p>
We implemented a labeling tool and delivered it to BASF. BASF used the labeling tool we
provided to generate digitalized labels of nematodes in microscope images. A screen shot of our labeling tool is shown below.</p>
<figure class="figure text-center figure-center">
<img src="images/labeling_tool.png" class="figure-img img-fluid"/>
<figcaption class="figure-caption">Screen shoot of our labeling tool.</figcaption>
</figure>
<p>
After that, we have received several batches of data from BASF. Currently (May 2018), we
have 406 valid labeled microscope images, with unbalanced distribution as shown below.
</p>
<figure class="figure text-center figure-center">
<img src="images/nematode_distribution.png" class="figure-img img-fluid"
style="width:300px"/>
<figcaption class="figure-caption">Nematode distribution.</figcaption>
</figure>
</div>
<div class="row">
<h3>Challenges</h3>
<p>We found many challenges during our data analysis and processing, including bad illumination, noise, overlapping nematodes, nematodes with weired shape, and mislabeled data (in the example image below, all nematodes were labeled as "interested"; however, it is impossible in this case), as shown below:</p>
<div class="container-fluid container-image">
<figure class="figure text-center">
<img src="images/c1.png" class="figure-img img-fluid" width="300"/>
<figcaption class="figure-caption">Bad illumination.</figcaption>
</figure>
<figure class="figure text-center">
<img src="images/c2.png" class="figure-img img-fluid"/>
<figcaption class="figure-caption">Noise.</figcaption>
</figure>
<figure class="figure text-center">
<img src="images/c3.png" class="figure-img img-fluid"/>
<figcaption class="figure-caption">Overlapping nematodes.</figcaption>
</figure>
<figure class="figure text-center">
<img src="images/c4.png" class="figure-img img-fluid"/>
<figcaption class="figure-caption">Nematodes with weird shape.</figcaption>
</figure>
<figure class="figure text-center">
<img src="images/c5.png" class="figure-img img-fluid"/>
<figcaption class="figure-caption">
Mislabeled data.
</figcaption>
</figure>
</div>
</div>
<div class="row">
<h3>Data augmentation</h3>
<p>
Typically, we need much more samples to train a deep neural network such as faster R-CNN.
Considering we don't have enough data, we have applied aggressive data augmentation.
For each original image, we generate its flipped versions and illumination adjusted
versions, which grows our training set by 11 times.
</p>
</div>
</div>
<div id="literature_review" class="col section">
<hr class="row hr-heavy"/>
<div class="row">
<h2>Literature Review</h2>
</div>
<hr class="row hr-light"/>
<div class="row">
<p>
Conventional computer vision methods rely on the extraction of feature vectors from images
or image patches by using the feature descriptor. Histogram of oriented gradients
(HOG) is a popular feature descriptor, which uses the distribution of directions of
gradients as
features
[<a href="https://ieeexplore.ieee.org/document/1467360/">1</a>].
Gradients of an image are large around edges and corners, which carry a lot of information
about object
shape, and are thus useful for classification. The HOG descriptors can be fed into a
conventional classification algorithm, such as support vector machine (SVM). An example of HOG feature descriptor is shown below.
</p>
<figure class="figure text-center figure-center">
<img src='images/lr_HOG.png' class="figure-img img-fluid">
<figcaption class="figure-caption">HOG feature descriptor (source:
https://www.learnopencv.com/histogram-of-oriented-gradients/).
</figcaption>
</figure>
<p>
Deep learning methods usually outperform conventional methods when there are enough data.
Faster
R-CNN
is the state-of-the-art deep learning algorithm for object detection [<a
href="https://arxiv.org/abs/1506.01497">2</a>]. The basic idea is
first
apply convolutional layers to get the feature maps. The region proposal network (RPN)
then takes different
sizes and positions of anchor boxes from the feature map, and learns to classify bounding
boxes
into
background and foreground as well as to refine the bounding boxes. The foreground
boxes from
RPN
goes
through region of interest pooling. It will resize the feature map corresponding to the
proposed bounding boxes into same
shape, and the classifier will predict the class of each bounding box. The architecture scheme of faster R-CNN is shown below.
</p>
<figure class="figure text-center figure-center">
<img src='images/faster_rcnn.png' class="figure-img img-fluid">
<figcaption class="figure-caption">Faster R-CNN (source:
https://arxiv.org/pdf/1506.01497.pdf).
</figcaption>
</figure>
</div>
</div>
<div id="model" class="col section">
<hr class="row hr-heavy"/>
<div class="row">
<h2>Modeling</h2>
</div>
<hr class="row hr-light"/>
<div class="row section" id="conventional">
<h3>Conventional Models</h3>
<p>
From a machine learning point of view, the problem could be divided into two parts:
detection and classification. Detection is to find individual nematodes in the
microscope image. Classification is to predict whether each nematode is in interested stage.
</p>
<p>
The steps of our baseline model is as follows:
</p>
<ol>
<li>
Extract images of individual nematodes from a microscope image. Our segmentation uses
conventional computer vision techniques. We set area threshold and average color
threshold to distinguish large noise contours from nematodes.
</li>
<li>
Apply HOG feature extraction algorithm to obtain feature vectors for extracted
nematodes.
</li>
<li>
Apply classification on the feature vectors. The classifier could be anything like
support vector machine or
random forest.
</li>
</ol>
<p>
The scheme of baseline models pipeline is shown below.
</p>
<figure class="figure text-center figure-center">
<img src='images/baseline.png' class="figure-img img-fluid">
<figcaption class="figure-caption">Baseline models pipeline.</figcaption>
</figure>
</div>
<div class="row section" id="rcnn">
<h3>Deep Learning (R-CNN)</h3>
<p>
Our baseline models can not handle situations such as overlapped nematodes. They also seem
unable to capture details which could be very important in classification.
Deep learning models usually outperform conventional methods when there are enough data. We
applied an end-to-end faster R-CNN model for nematode detection and classification.
</p>
<p>
We applied a pre-trained RES-152 [<a href="https://arxiv.org/abs/1512.03385">3</a>] network
to get the feature maps. The region proposal network
(RPN) takes different sizes and positions of anchor boxes from the feature map, and learns
to classify bounding boxes into background and foreground; it also learns to refine the
bounding boxes. The output of RPN goes through region of interest (RoI) pooling. It resizes
the feature map corresponding to the proposed bounding boxes into same shape, and the
classifier will predict the class of
each bounding box. The architecture scheme of the faster R-CNN model is shown below.
</p>
<figure class="figure text-center figure-center">
<img src='images/faster_rcnn1.png' class="figure-img img-fluid">
<figcaption class="figure-caption">Faster R-CNN.</figcaption>
</figure>
</div>
<div class="row section" id="sarcnn">
<h3>Adding Scale Information</h3>
<p>
The sizes of bounding boxes are not explicitly considered when faster R-CNN does
classification, since RoI layer resizes feature maps into same dimensions. This makes sense
in most applications
because objects appear smaller when they are further away. For example, the cars have different sizes in the example image below. When we do object detection in
such cases, size of bounding boxes (or feature maps) shouldn’t be considered.
</p>
<figure class="figure text-center figure-center">
<img src="images/distance_to_camera.jpg" height="1000" width="1500"/>
<figcaption class="figure-caption">The cars in the image have different sizes. (source:
http://greatplacesincanada.ca/gpic_places/historic-main-street/)
</figcaption>
</figure>
<p>
However, in microscope images, the distance to camera is fixed. And when we classify
nematodes, size matters. Nematodes that are either too small or too big are unlikely to be
interested. To take the size information into account, we implemented a modified version of
faster R-CNN, which we call scale-aware R-CNN (SA-R-CNN). We get the size information from
RPN, which is then passed into the classifier together with the results from RoI pooling. The architecture scheme of SA-R-CNN is shown below.
</p>
<figure class="figure text-center figure-center">
<img src='images/faster_rcnn2.png' class="figure-img img-fluid"
style="max-width: 60% !important;">
<figcaption class="figure-caption">SA-R-CNN.</figcaption>
</figure>
</div>
</div>
<div id="results" class="col section">
<hr class="row hr-heavy"/>
<div class="row">
<h2>Results</h2>
</div>
<hr class="row hr-light"/>
<p>
We recieved 4 batches of data, which contained 406 labeled images in total. We trained our
models on
the first 3 batches and evaluated on the last batch (42 images).
</p>
<p>
The bounding box detection error for SA-R-CNN is summarized below:
</p>
<table class="table">
<thead>
<tr>
<th>Detection</th>
<th>Actual label</th>
<th>No Actual Label</th>
</tr>
</thead>
<tr>
<th>Predicted Label</th>
<td>85</td>
<td>78 (only 1 interested)</td>
</tr>
<tr>
<th> No Predicted Label</th>
<td>0</td>
<td>N/A</td>
</tr>
</table>
<p>
The classification results are summarized below (AUC/accuracy is evaluated by matching each box
predicted to each box labeled and ignoring those don’t match):
</p>
<table class="table">
<thead>
<tr>
<th>Model</th>
<th>AUC</th>
<th>Accuracy</th>
</tr>
</thead>
<tr>
<td>SVM</td>
<td>0.66</td>
<td>0.75</td>
</tr>
<tr>
<td>Random Forest</td>
<td>0.69</td>
<td>0.75</td>
</tr>
<tr>
<td>Faster R-CNN (res152)</td>
<td>0.82</td>
<td>0.74</td>
</tr>
<tr>
<td><b>SA-R-CNN (res152)</b></td>
<td><b>0.88</b></td>
<td><b>0.75</b></td>
</tr>
</table>
<p>
From the results, we see our SA-R-CNN is better than the default faster R-CNN and both of them
are
better than conventional models (since conventional models are worse on detection, the
statistics
shown for conventional models are higher than the actual values). For detection, our SA-R-CNN
predicts many nematode-like objects such as dead nematodes. However, since almost all those
predictions are non-interested, we think it’s not an issue (dead nematodes are not interested
anyway).
</p>
<figure class="figure w-100 text-center">
<table class="table">
<thead>
<tr>
<th></th>
<th>Predicted as Non-interested</th>
<th>Predicted as Interested</th>
</tr>
</thead>
<tr>
<th>Labeled as Non-interested</th>
<td>17</td>
<td>1</td>
</tr>
<tr>
<th>Labeled as Interested</th>
<td>20</td>
<td>47</td>
</tr>
</table>
<figcaption class="figure-caption">Classification confusion matrix for SA-R-CNN</figcaption>
</figure>
<p>
From the confusion matrix we can see we have few false positives but many false negatives.
As discussed, we think our models are not trained enough due to the lack of training data. With
more
data, we believe our models can do better.
</p>
</div>
<div id="deliverable" class="col section">
<hr class="row hr-heavy"/>
<div class="row">
<h2>Deliverable</h2>
</div>
<hr class="row hr-light"/>
<p>
We built a software that automatically outputs the predictions and generates a CSV for counts of
interested/non-interested nematodes in each image. An example output image by our software is shown below.
</p>
<figure class="figure-center figure">
<img src='images/success1.png' style="width: 400px !important;">
<figcaption>An example output image by our software.
</figcaption>
</figure>
<p>
We delivered a virtual machine with all models and GUI to BASF (screenshots are shown below).
</p>
<br>
<figure class="figure-center figure">
<img src='images/vm1.png' style="width: 600px !important;">
<figcaption>Virtual machine.
</figcaption>
</figure>
<br>
<br>
<figure class="figure-center figure">
<img src='images/vm2.png' style="width: 400px !important;">
<figcaption>GUI (export counts, update, etc.).
</figcaption>
</figure>
<br>
<p>The flowchart for using the software is:</p>
<figure class="figure-center figure">
<img src="images/gui_fc.png" style="width: 300px !important;">
</figure>
<p>
The source code of our models and software can be found here: <a
href="https://github.com/BASFProject-2018spring">https://github.com/BASFProject-2018spring</a>
</p>
<p>
The VM can be downloaded from our Google Drive:
<a href="https://drive.google.com/file/d/1-VdYoSuK8tQ_kuPHHLNH-PFjvQa4caqY/view?usp=sharing">https://drive.google.com/file/d/1-VdYoSuK8tQ_kuPHHLNH-PFjvQa4caqY/view?usp=sharing</a>
</p>
<p>
The video tutorial of our software is here:
</p>
<iframe class="video"
src="https://www.youtube.com/embed/2c5iCQSVKvE?rel=0"
allow="autoplay; encrypted-media" allowfullscreen></iframe>
<p>
In case BASF wants to retrain the model in the future with more available data, we wrote a
documented <a href="https://github.com/BASFProject-2018spring/ABOUT">training pipeline</a>.
</p>
<p>
All code can be found in our <a href="https://github.com/BASFProject-2018spring">project
organization</a> (public)
</p>
<p>
We also made a video tutorial for the training:
</p>
<iframe class="video"
src="https://www.youtube.com/embed/mN6CnejkdtE?rel=0"
allow="autoplay; encrypted-media" allowfullscreen></iframe>
<p>In the latest meeting, BASF told us they would like to add more classes of nematodes in the
future (such as dead nematode class). So we have included information on places to change for
our model working for more classes. This documentation is also in our training pipeline.</p>
<p>Once BASF have new data labled (with more classes), they could make changes to our code
accordingly and train a new model.</p>
</div>
<div id="interpretation" class="col section">
<hr class="row hr-heavy"/>
<div class="row">
<h2>Empirical Interpretation</h2>
</div>
<hr class="row hr-light"/>
<p>
BASF is also interested in how the model makes the prediction (i.e. what features the network
learns). However, deep learning models are hard to interpret.
</p>
<p>
One way to do this is to fit a simpler model to simulate the deep learning model, using the
desired features (like size, angle and etc.). The fitted simpler model can partially explain the
deep neural network. The problem is that some desired features are hard to define mathematically
(for example, the angle or the shape) and some features are hard to extract (for example,
accurately obtaining size of a nematode is hard). We leave this method as a future work.
</p>
<p>
Another simpler empirical way is to get an interested nematode, change its features and see
whether our model changes the prediction. We tried this method and the result is shown in the
following images.
</p>
<div class="row">
<div class="container-fluid container-image">
<figure class="figure text-center">
<img src="images/int_org.jpg" class="figure-img img-fluid" width="300">
<figcaption class="figure-caption">Nematodes</figcaption>
</figure>
<figure class="figure text-center">
<img src="images/int_org.png" class="figure-img img-fluid" width="300">
<figcaption class="figure-caption">Detection and classification results</figcaption>
</figure>
</div>
</div>
<p>
We made the first image based on an interested nematode (top-left corner). We used PhotoShop to
rotate the nematode, change its color, resize it and change its shape. All generated fake
nematodes were then combined into a single image whose background is from a training image. The
second image is generated by passing the first image into our model. We observed the following:
</p>
<ol>
<li>
The model predicts all rotated nematodes as interested. So rotation doesn't affect the
model.
</li>
<li>
With mild shape change, the nematodes are still predicted as interested. However, if we
straighten the nematode, it will become non-interested. So the model does look at the shape
of the nematode, but with mild shape change, its prediction won't change.
</li>
<li>
Relative darkness (brightness) of nematodes matters a lot. We changed the brightness of the
nematode and it becomes non-intersted. For more aggressive change (let the nematode
transparent), our model will think the nematode is part of the background.
</li>
<li>
Size matters a lot. Letting the nematode larger or smaller makes it non-interested. If the
nematode is too small, our model will think it's part of the background.
</li>
<li>
Noise does not matter. There're different debris around the rotated nematodes and the
original nematodes. But the model does not change the prediction nor the prediction
confidence level.
</li>
</ol>
<p>
We conducted such experiment on some nematodes and the conclusions are similar. So we hope our
conclusions above is generalizable for all nematode images because that means our model works as
expected. In order to get insight about how exactly the model uses those features to make
decisions, we need to use more sophisticated methods.
</p>
</div>
<div id="conclusions" class="col section">
<hr class="row hr-heavy"/>
<div class="row">
<h2>Conclusions & Future Work</h2>
</div>
<hr class="row hr-light"/>
<p>
We have implemented both conventional models and deep learning models for the
nematode detection and classification tasks. We have created a new network
architecture to make faster R-CNN model scale aware for nematode classification. Our deep
learning
models outperform baseline models, although our models suffer from overfitting due to the
unavailability of enough training data. Also, we have delivered our models and GUI (with tools
like CSV generator) within a virtual machine to BASF.
</p>
<p>
Our models can be improved in the following aspects:
</p>
<ol>
<li>
Deep learning models typically require hugh amount of training data. We believe our model
can perform better with more data.
</li>
<li>
Currently our models assume the input images are at a fixed scale. For different scale
images, users have to use <a
href="https://github.com/BASFProject-2018spring/VOC_format_builder/blob/master/image_rescale.py">our
script</a> to transfer the images into the required scale. We should be able to
handle images at different scales in the future.
</li>
<li>
More efforts are required to improve the interpretability of our models.
</li>
</ol>
</div>
</div>
</div>
</div>
</div>
<script> window.jQuery || document.write('<script src="./js/jquery.js"><\/script>')</script>
<script src="js/bootstrap.js"></script>
<script src="js/site.js"></script>
</body>
</html>