To protect your data, the CISO officer has suggested users to enable GitLab 2FA as soon as possible.

README.md 7.21 KB
Newer Older
Xinqi Zhu's avatar
Xinqi Zhu committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
# Deep Neural Network for Lane Detection  
Lane detection: a semantic segmentation approach.  
Use tensorflow to implement a deep neural network for lane detection based on the paper “ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation.” You can refer this for more details: https://arxiv.org/pdf/1606.02147v1.pdf ​. This model is an encoder-decoder network for general road scene semantic segmentation. It is modified to suit out task.

Notes: The environment is `python3` and `tensorflow` under `anaconda`. We suggest you use it as well, the below instructions to run the training or prediction is based on this environment. 

## Result demo:

* [Slides](./CV_Project.pptx)
* [Video at Youtube](https://youtu.be/tDCkRfYBk4U)
* [Report](./Report.pdf)

## Members:

**Xinqi Zhu**, **Xuyang Shen**, **Alisdair Cameron**

## Environment setup
* This software has only been tested on Win10(64-bit), python3.7.1, Cuda-10.0 with a GTX-1060 6GB GPU.
* Currently, our codes can only run on the tensorflow-gpu, which means it may raise exceptions if run on CPU

Please install all the required package in your `anaconda` environment: (we also provide the `conda bash` command below)
> tensorflow: v1.13.1 `conda install -c conda-forge tensorflow`  
> tensorflow-gpu: v1.13.1 `conda install -c anaconda tensorflow-gpu`  
> tensorboard: v1.13.1 `conda install -c conda-forge tensorboard`  
> opencv: v3.4.1 or v4.1.0 `conda install -c conda-forge opencv`  
> Argparse: v1.3.0 `conda install -c anaconda argparse`  
> imageio: v1.5.0 `conda install -c menpo imageio`  

## Train Model
The task is the TuSimple lane detection competition, we only tested the dataset provided by TuSimple.
[The download URL for the whole dataset](https://github.com/TuSimple/tusimple-benchmark/issues/3). 

* Note: Due to the offline generator(for train labels), it is a bit tricky(complex) to guide the training instructions. To make  it clear, we provide two main methods to train the model.
The first main method only requires to download the TuSimple train set, while the second one requires to download another training component `label_set-tfRecord` [Download Address](https://github.com/XuyangSHEN/lane_detection_enet/raw/master/train_set/labels.zip).

* training commands:
```bash
$ python training.py -h
usage: training.py [-h] [-i INPUT-PATH] [-m MODEL-PATH] [-l yes/no] [-gt PATH]
                   [-e iteration] [-b per iteration] [-f buffer size]

optional arguments:
  -h, --help            show this help message and exit
  -i INPUT-PATH         The path to the train_set (include the train_set).
                        (default: sample_data/train_set/)
  -m MODEL-PATH         The path to store the training model. (default:
                        tmp/train_01/)
  -l yes/no             if it is the first time to run the train, please
                        select yes [offline generator]
  -gt PATH, --ground_truth PATH
                        input the json file of the labels
  -e iteration, --epochs iteration
                        input the number of epochs need to run (default: 40)
  -b per iteration, --batch_size per iteration
                        input the batch size (default: 4)
  -f buffer size, --buffer_size buffer size
                        input the buffer size (default: 150)

```

### run a small training
Apart from the large training set, we also provide one sample training set to run a small training.

```bash
simple command: 
$ python training.py
full command:
$ python training.py -i  sample_data/train_set/ -m tmp/train_01/ -l no
```

### Main Method1: run training with TuSimple Dataset
Requirement:
* Download the train set: [The URL for the whole dataset](https://github.com/TuSimple/tusimple-benchmark/issues/3)
* Please select one Json to train(among the `_0301`, `_0501`, `_0601`). We also kindly provide the combination Json to train all of them. [JSON Download URL](https://github.com/XuyangSHEN/lane_detection_enet/raw/master/train_set/total_label.zip)
* Please check whether the `xxxlabel.Json` is inside the `train_set` folder

Sample command:
```bash
$ python training.py -i (data set path) -m (model storage path) -l yes -gt (xxx.json) 
```

### Main Method2: run training with TuSimple Dataset plus the downloaded tfRecord
Requirement:
* Download the train set:  [The URL for the whole dataset](https://github.com/TuSimple/tusimple-benchmark/issues/3)
* Download the labels represented by tfRecord: [The URL for the whole label set](https://github.com/XuyangSHEN/lane_detection_enet/raw/master/train_set/labels.zip)
* Download the combination Json file: [JSON Download URL](https://github.com/XuyangSHEN/lane_detection_enet/raw/master/train_set/total_label.zip)
* Move the downloaded `label` folder into the `train_set` folder
* Please check whether the `xxxlabel.Json` is inside the `train_set` folder

Sample command:
```bash
$ python training.py -i (data set path) -m (model storage path) -l no
```

## Predict Model
We have uploaded the pre-trained model inside the `pre_trained_model`folder, which is trained based on [tusimple benchmark](http://benchmark.tusimple.ai/#/).
You can also follow the `Train Model` instruction to train your own model and make a prediction.

* Please download the Pre-trained model before run the prediction [pre_trained_model](https://drive.google.com/file/d/1GMlc6KFTYyubuj6n9zGhhniW1Q8At7oH/view?usp=sharing)

* prediction commands:
```bash
$ python prediction.py -h
usage: prediction.py [-h] [-i INPUT-PATH] [-o OUTPUT-PATH] [-m MODEL-PATH]
                     [-d DISPLAY]

optional arguments:
  -h, --help      show this help message and exit.
  -i INPUT-PATH   The path to the prediction data folder. [Default: 'sample_data/test_set/']
  -o OUTPUT-PATH  The path to put the prediction result folder. [Default:'tmp_pred/']
  -m MODEL-PATH   The path to the model storage. [Default: 'pre_trained_model/']
  -d DISPLAY      whether to display prediction result or not. [Options: yes/no, Default: no]

```
* sample command:
```bash
$ python prediction.py -i sample_data/test_set/ -o tmp_pred/prediction01/ -m pre_trained_model/ -d no
```

## Contents
* `data_provider` folder:  
> data_dataset.py: provide data for training. tf.Dataset version  
> data_np.py:provide data for training. numpy version  
> label2tfrecord.py: offline label generator  
> line_connect.py: to fix the gaps for the ground truth  
> one_hot_coding.py: one hot labeling (differ from tf.one_hot)  

* `enet` folder:
> ENet.py: netural network layers 
> ENet_Components: components for the ENet model

* `main root:`
> config.py: command line parser  
> training.py: to train the ENet  
> prediction.py: to predict the result  

* `pre_trained_model` folder:
> store the pre-trained model

* `sample_data` folder:
> including some sample data and results

## Reference:
* [ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation](https://arxiv.org/pdf/1606.02147v1.pdf)
* [TensorFlow-ENet](https://github.com/kwotsin/TensorFlow-ENet)
* [Implementation of Max Unpooling](https://github.com/tensorflow/tensorflow/issues/2169)


## Feedback:
We are happy to receive any feedback or bug report. Please email us:

Xinqi Zhu's avatar
Xinqi Zhu committed
152
- xinqi.zhu@anu.edu.au: first choice for issue regarding the ENet model or its implementation
Xinqi Zhu's avatar
Xinqi Zhu committed
153
- xuyangshen@yahoo.com: first choice for issue regarding training and testing