Added base structure of dataset wiki with much more documentation abi…

…lity and specified legacy datasets. Also torrents for new data releases today!

README.md

@@ -22,21 +22,6 @@ Each challenge will contain awesome prizes (cash and others) for the most effect
 
 **You can find a current list of challenges, with lots of information, on the [Udacity self-driving car page](http://udacity.com/self-driving-car). This is the primary way to contribute to this open source self-driving car project.**
 
-## Datasets
-
-####Driving Data
-| Date | Lighting Conditions | Duration | Compressed Size | Uncompressed | Direct Download | Torrent | MD5 |
-| ---- | :------------------:| --------:| ---------------:| ------------:|:---------------:|:-------:|:---:|
-| 09/29/2016 | Sunny | 00:12:40 | 25G | 40G | [HTTP](http://bit.ly/udacity-dataset-2-1) | [Torrent](datasets/dataset.bag.tar.gz.torrent)| `33a10f7835068eeb29b2a3274c216e7d` |
-| 10/03/2016 | Overcast | 00:58:53 | 124G | 183G | [HTTP](http://bit.ly/udacity-dataset-2-2) | [Torrent](datasets/dataset-2-2.bag.tar.gz.torrent) | `34362e7d997476ed972d475b93b876f3` |
-| 10/10/2016 | Sunny | 03:20:02 | 21G | 23.3G |  | [Torrent](http://bit.ly/2dZTOcq) | `156fb6975060f60c452a9fa7c4121195` |
-| 10/20/2016 | Sunny | 03:30:00 | 30G | 40G |  | [Torrent](http://bit.ly/2epl7Ir ) | `13f107727bed0ee5731647b4e114a545` |
-
-Check out [udacity-driving-reader](https://github.com/rwightman/udacity-driving-reader) for some easy-to-use scripts to read or export to CSV or TensorFlow.
-
-####Isolated and Trimmed Driving Data
-With the help of [Auro Robotics](http://www.auro.ai/), compression, and selective recording, we now have considerably smaller datasets that will be available shortly. You can find instructions for these datasets, when available, in the /datasets/udacity_launch readme.
-
 ## Contributions
 Here's a list of the projects we've open sourced from crowdsourced contributions:
 

challenges/challenge_2/README.md

@@ -10,7 +10,7 @@ Massive thanks to Machine Learning wizard [Mikel Bober-Irizar (@anokas on the Ud
 | Rank | Team Name        |     Public Score     |    Private Score     |
 | ---- | :---------------:| --------------------:| --------------------:|
 
-## Leaderboard
+## Round 1 Leaderboard
 
 For reference, an example file filled with '0' for every steering prediction scores 0.035665505598782017.
 

datasets/CH2/README.md

@@ -0,0 +1,20 @@
+##CH2_001
+The '/center' folder contains JPG images for each test frame, similar to the Round 1 evaluation set. The decision was made to move to JPG from PNG to save file space, and should not cause any artifacting due to the size of the images. If you would like to ge tthe image set in PNG, you can either convert the folder in batch using a tool like IRFANVIEW, or use rwightman's Udacity Reader docker tool (https://github.com/rwightman/udacity-driving-reader) using the PNG flag. I suggest using a batch convert process to avoid complications with not having the original BAG file with all topics, as you will likely need to modify the code in the docker tool. 
+
+The file 'final_example.csv' includes a template CSV file with false values for the steering angle column, but the frame IDs are correct. You should submit you results EXACTLY as provided, as I will not be fixing submissions for the final round. When you have output from your model, please send your results to [email protected] with the csv attached, named as 'teamName.csv' and your team members in the body of the email.
+
+The leaderboard can be found at the same location as before (https://github.com/udacity/self-driving-car/tree/master/challenges/challenge_2) and every attempt will be made ot update as often as possible. If someone in the community wants to build an automated submission system, please contact me and I'd be happy to work with you on this. You'll get massive props on the Udacity github.
+
+HMB_3.bag is not included, as it is the test dataset, but I've included a filtered HMB_3_release.bag which only includes the center camera imagery, camera info topic, and timing information. I know many of you asked for this last time.
+
+The HMB_3_release.bag file was created using the following filter rules: 
+
+rosbag filter HMB_3.bag HMB_3_release.bag "topic == '/center_camera/camera_info' or topic == '/center_camera/image_color/compressed'"
+
+#CH2_002
+HMB_1: 221 seconds, direct sunlight, many lighting changes. Good turns in beginning, discontinuous shoulder lines, ends in lane merge, divided highway
+HMB_2: 791 seconds, two lane road, shadows are prevalent, traffic signal (green), very tight turns where center camera can't see much of the road, direct sunlight, fast elevation changes leading to steep gains/losses over summit. Turns into divided highway around 350s, quickly returns to 2 lanes
+HMB_3: 281 seconds, two lane road in sunlight and shadows. Ends when divided highway begins
+HMB_4: 99 seconds, divided highway segment of return trip over the summit
+HMB_5: 212 seconds, guardrail and two lane road, shadows in beginning may make training difficult, mostly normalizes towards the end
+HMB_6: 371 seconds, divided multi-lane highway with a fair amount of traffic

datasets/CH3/README.md

@@ -0,0 +1,3 @@
+##CH03_001
+el_camino_north: Northbound drive on El Camino with enhanced IMU-based positioning available in '/fix' topic (Does not make it all the way to SF)
+el_camino_south: Southbound drive on El Camino with enhanced IMU-based positioning available in '/fix' topic (Does not start in SF)

datasets/CHX/README.md

@@ -0,0 +1,2 @@
+##CHX_001
+Lap around block at Udacity office with new HDL-32E LIDAR from George Hotz. Can almost create a loop-closure using NDT mapping, more sophisticated methods of SLAM should be able to create a high-fidelity map

datasets/README.md

@@ -1 +1,39 @@
+# Udacity Self Driving Car Open Source Data
+In an attempt to cleanup the data release practices of the Udacity Self-Driving Car team, we will start maintaining this wiki of data we have uploaded. Issues persist throughout many of the datasets, so we will be working backwards to catalog legacy data and unify the naming methodology. Please feel free to add to this list and submit a PR.
 
+Check out [udacity-driving-reader](https://github.com/rwightman/udacity-driving-reader) for some easy-to-use scripts to read or export to CSV or TensorFlow.
+
+##Current Releases
+#### These releases should be issue/error free and comply with the new naming schema.
+
+#### Challenge 2 Driving Data
+| Name | Purpose |
+|:----:|:-------:|
+| [CH2_001]() | Final testing data for the last round of Challenge 2 |
+| [CH2_002]() | Training data with very similar driving conditions to Ch2_001 |
+
+#### Challenge 3 Driving Data
+| Name | Purpose |
+|:----:|:-------:|
+| [CH3_001]() | Northbound and Southbound drives on El Camino with IMU positioning |
+
+#### Misc. Driving Data
+| Name | Purpose |
+|:----:|:-------:|
+| [CHX_001]() | Lap around block at Udacity office with new HDL-32E LIDAR |
+
+##Legacy Data
+
+[All torrent releases from Udacity can be found on our AcademicTorrents page with associated descriptions.](http://academictorrents.com/userdetails.php?id=5125)
+
+#### These releases are old and likely have issues that make them unsuitable for training. Please use only as a reference.
+####Driving Data
+| Date | Lighting Conditions | Duration | Compressed Size | Uncompressed | Direct Download | Torrent | MD5 |
+| ---- | :------------------:| --------:| ---------------:| ------------:|:---------------:|:-------:|:---:|
+| 09/29/2016 | Sunny | 00:12:40 | 25G | 40G | [HTTP](http://bit.ly/udacity-dataset-2-1) | [Torrent](datasets/dataset.bag.tar.gz.torrent)| `33a10f7835068eeb29b2a3274c216e7d` |
+| 10/03/2016 | Overcast | 00:58:53 | 124G | 183G | [HTTP](http://bit.ly/udacity-dataset-2-2) | [Torrent](datasets/dataset-2-2.bag.tar.gz.torrent) | `34362e7d997476ed972d475b93b876f3` |
+| 10/10/2016 | Sunny | 03:20:02 | 21G | 23.3G |  | [Torrent](http://bit.ly/2dZTOcq) | `156fb6975060f60c452a9fa7c4121195` |
+| 10/20/2016 | Sunny | 03:30:00 | 30G | 40G |  | [Torrent](http://bit.ly/2epl7Ir ) | `13f107727bed0ee5731647b4e114a545` |
+
+####Isolated and Trimmed Driving Data
+With the help of [Auro Robotics](http://www.auro.ai/), compression, and selective recording, we now have considerably smaller datasets.