add v1.5

bionicdl-sustech · May 8, 2020 · a607d94 · a607d94
1 parent ee550ba
commit a607d94
Show file tree

Hide file tree

Showing 256 changed files with 42,085 additions and 850 deletions.
diff --git a/README.md b/README.md
@@ -2,12 +2,9 @@
 
 paper | poster | video
 
-Establishing a reproducible and shareable benchmarking for dexterous manipulation has been a significant challenge since the diversity of robot systems, the complexity of manipulation tasks, and a wide selection of metrics. To reduce the entry barrier, we propose **DeepClaw** - a standardized dexterous manipulation protocol, which comprises four common operations to streamline the manipulation process: *localization*, *identification*, *multiple points motion planning*, and *execution*. 
-
-Robot can learning skills that applicable for the similar tasks, called the *task familiy*[1]. We have implemented several manipulation tasks in three task families representing assembly tasks, reasoning tasks and bin-picking tasks separately. You can find them <a href="#tasks">here</a>. 
-
-In addition, we propose certain metrics measuring manipulations in many dimensions, such as xxx.
+The DeepClaw benchmark is a framework for establishing a reproducible and shareable benchmarking for dexterous manipulation. DeepClaw benchmark provides a standardized dexterous manipulation pipeline consisting of four subtasks: localization, recognition, grasp planning, and motion planning. It also provide necessary components to benchmark manipulations including hardware drivers, data I/O utilities, baseline algorithm modules and evaluation metrics.
 
+The DeepClaw has been used extensively to benchmark a series of manipulation tasks including claw machine, jigsaw game and TicTacToe. The source codes of these experiments are placed under /examples.
 ![](https://github.com/bionicdl-sustech/DeepClawBenchmark/blob/master/Documents/Figs/deepclaw-framework.png)
 
 ## Quick Start
@@ -19,19 +16,19 @@ DeepClaw framework has only been tested with *Python 2.7* and *Ubuntu 16.04 LTS*
 Install virtualenv.
 
 ```shell
-$ sudo pip install -U virtualenv
+$ pip install -U virtualenv
 ```
 
 Create a new virtual environment.
 
 ```shell
-$ virtualenv --system-site-packages -p python2.7 ./venv
+$ virtualenv -p /usr/bin/python2.7 ./DCvenv
 ```
 
 Activate or retreat from virtual environment.
 
 ```shell
-$ source ./venv/bin/activate # activate virtual environment
+$ source ./DCvenv/bin/activate # activate virtual environment
 $ deactivate # retreat from virtual environment
 ```
 
@@ -44,36 +41,42 @@ $ git clone https://github.com/bionicdl-sustech/DeepClawBenchmark.git
 $ cd ./DeepClawBenchmark
 ```
 
-Run the DeepClaw installation helper script:
+Install Prerequisites:
 
 ```shell
-$ sudo sh install.sh realsense ur
+$ pip install -r requirements.txt
 ```
+Build libfranka server
+```shell
+$ cd ./DeepClawBenchmark/driver/arms/Franka/libfraka_server
+$ mkdir build
+$ cd build
+$ cmake ..
+$ make
+```
+### Verify Installation
+Run calibration task with your drivers, for example, UR10e, HandE, Kinect and so on.
 
-The brackets indicate optional arguments to switch installation methods.
-
-The first argument specifies the version:
-
-- **realsense**: RealSense D435 support.
-
-The second argument specifies the installation mode:
-
-- **ur**: UNIVERSAL ROBOT arm series support (UR5 and UR10e).
-- **franka**: FRANKA arm support (update later).
-- **aubo**: AUBO arm support (update later).
-- **denso**: DENSO Cobotta arm support (update later).
+```shell
+$ python main.py ur10e hande kinect-azure calibration true
+```
 
-There are some test cases for testing your installation and calibration.
+There also are some test cases for testing your installation and calibration.
 
 [Test cases](https://github.com/bionicdl-sustech/DeepClawBenchmark/blob/master/Documents/TestCases.md)
 
 ## <a name="tasks">Tasks</a>
-We have implemented some task families with DeepClaw:
+We have implemented some tasks using DeepClaw with classical algorithm modules:
 - Task Family 1: [Jigsaw](https://github.com/bionicdl-sustech/DeepClawBenchmark/blob/master/Documents/Jigsaw_task/task_description.md)
-- Task Family 2: Tictactoe Game
-- Task Family 3: Toy-Claw
+- Task Family 2: Tic-tac-toe Game
+- Task Family 3: Toy Claw Machine
 
-Find the task desription template [here](https://github.com/bionicdl-sustech/DeepClawBenchmark/blob/master/Documents/Task-Description-Template.md).
+Find the task description template [here](https://github.com/bionicdl-sustech/DeepClawBenchmark/blob/master/Documents/Task-Description-Template.md).
+And we encourage developers to create new tasks ([how to create](https://github.com/bionicdl-sustech/DeepClawBenchmark/blob/python2.7/documents/How-to-Create-Task.md)).
+## Algorithm Modules
+We also provide modules pool for developers to assembly their own manipulation tasks.
 
+Find all modules description [here](https://github.com/bionicdl-sustech/DeepClawBenchmark/tree/python2.7/modules).
+And how to create a new module.
 ## References
 [1] O. Kroemer, S. Niekum, and G. Konidaris, “A review of robot learning for manipulation: Challenges, representations, and algorithms,”arXiv preprintarXiv:1907.03146, 2019.
diff --git a/config/arms/franka.yaml b/config/arms/franka.yaml
@@ -1,23 +1,9 @@
 # initial params
+robot_ip: "192.168.1.100"
+
 HOME_JOINTS:
   -
-    - ''
-    - ''
-    - ''
-  -
-    - ''
-    - ''
-    - ''
-
 HOME_POSE:
-  -
-    - 0.352
-    - 0.325
-    - 0.25
-  -
-    - 3.14
-    - 0
-    - 0
-
-CALIBRATION_DIR: "/data/calibration_data/franka.npz"
+  - 
+CALIBRATION_DIR: "/data/calibration_data/franka-realsense.npz"
 
diff --git a/config/arms/ur10e.yaml b/config/arms/ur10e.yaml
@@ -1,27 +1,27 @@
 # initial params
 SOCKET_CONFIGURATION:
-  robot_ip: "192.168.31.10"
+  robot_ip: "192.168.1.10"
   port_number: 30003
 
 HOME_JOINTS:
-  -
-    - -72.94
-    - -80.84
-    - -130.1
-  -
-    - -51.87
-    - 111.72
-    - -161.44
+  - -62.78
+  - -91.07
+  - -121.67
+  - -46.41
+  - 115.78
+  - 95.66
 
 HOME_POSE:
-  -
-    - 0.03
-    - -0.54
-    - 0.4
-  -
-    - 3.14
-    - -0.4
-    - 0
+  - 0.179
+  - -0.615
+  - 0.404
+  - 3.14
+  - 0
+  - 0
+
+JOINT_ACC: 1.6
+JOINT_VEL: 0.7
+
 PICK_Z: 0.
 PLACE_Z: 0.08
-
+CALIBRATION_DIR: "/data/calibration_data/ur10e-realsense.npz"
diff --git a/config/arms/ur5.yaml b/config/arms/ur5.yaml
@@ -4,25 +4,24 @@ SOCKET_CONFIGURATION:
   port_number: 30003
 
 HOME_JOINTS:
-  -
-    - -72.94
-    - -80.84
-    - -130.1
-  -
-    - -51.87
-    - 111.72
-    - -161.44
+  - -72.58
+  - -74.45
+  - -94.91
+  - -100.72
+  - 89.25
+  - 107.34
 
 HOME_POSE:
-  -
-    - 0.03
-    - -0.54
-    - 0.4
-  -
-    - 3.14
-    - -0.4
-    - 0
-PICK_Z: 0.
-PLACE_Z: 0.08
-CALIBRATION_DIR: "/data/calibration_data/ur5.npz"
+  - 0.012
+  - -0.3085
+  - 0.46689
+  - 3.14
+  - 0
+  - 0
 
+JOINT_ACC: 1.6
+JOINT_VEL: 0.7
+
+PICK_Z: 0.34
+PLACE_Z: 0.40
+CALIBRATION_DIR: "/data/calibration_data/ur5-realsense.npz"
diff --git a/config/modules/calibration.yaml b/config/modules/calibration.yaml
@@ -1,14 +1,27 @@
 initial_position:
-  -
-    - 0.4
-    - -0.5
-    - 0.35
-  -
-    - 3.14
-    - 0
-    - 0
+  - 0.300
+  - 0.100
+  - 0.350
+  - 1.5707963267948966
+  - 1.5707963267948966
+  - 0
 # the numebr of sampled points is cube_size*cube_size*cube_size
 cube_size: 4
-x_stride: 0.05
-y_stride: 0.075
-z_stride: 0.01
+x_stride: 0.050 
+y_stride: -0.050
+z_stride: 0.030
+#cube_size: 2
+#x_stride: 0.1
+#y_stride: 0.1
+#z_stride: 0.04
+checkerboard_type: 'chessboard' # 'aruco' 'chessboard'
+
+aruco_settings: # only be used when checkerboard type is aruco
+ aruco_size: 0.045 # unit: (m)
+ aruco_type: 'original' # 'original' '4x4' '5x5' '6x6'  ''7x7
+
+CALIBRATION_DIR: "/data/calibration_data/franka-realsense.npz"
+OFFSET: # unit: (m)
+  - 0
+  - 0.0085
+  - 0.0172
diff --git a/config/modules/calibration_ur5.yaml b/config/modules/calibration_ur5.yaml
@@ -0,0 +1,18 @@
+initial_position:
+  - 0.0
+  - -0.65
+  - 0.415
+  - 3.14
+  - 0
+  - 0
+# the numebr of sampled points is cube_size*cube_size*cube_size
+cube_size: 4
+x_stride: 0.05
+y_stride: 0.05
+z_stride: 0.02
+#cube_size: 2
+#x_stride: 0.1
+#y_stride: 0.1
+#z_stride: 0.04
+CALIBRATION_DIR: "/DeepClawBenchmark/data/calibration_data/ur5.npz"
+OFFSET: 0.14
diff --git a/config/sensors/AKinect.json b/config/sensors/AKinect.json
@@ -0,0 +1,11 @@
+{
+	"camera_fps" : "K4A_FRAMES_PER_SECOND_30",
+	"color_format" : "K4A_IMAGE_FORMAT_COLOR_MJPG",
+	"color_resolution" : "K4A_COLOR_RESOLUTION_720P",
+	"depth_delay_off_color_usec" : "0",
+	"depth_mode" : "K4A_DEPTH_MODE_WFOV_2X2BINNED",
+	"disable_streaming_indicator" : "false",
+	"subordinate_delay_off_master_usec" : "0",
+	"synchronized_images_only" : "false",
+	"wired_sync_mode" : "K4A_WIRED_SYNC_MODE_STANDALONE"
+}
diff --git a/config/sensors/AKinect_intrinsics/1080_color.txt b/config/sensors/AKinect_intrinsics/1080_color.txt
@@ -0,0 +1,20 @@
+
+===== Device 0: 000647492212 =====
+resolution width: 1920
+resolution height: 1080
+principal point x: 955.35
+principal point y: 545.277
+focal length x: 903.547
+focal length y: 903.589
+radial distortion coefficients:
+k1: 0.514646
+k2: -2.67247
+k3: 1.57664
+k4: 0.3961
+k5: -2.50126
+k6: 1.50402
+center of distortion in Z=1 plane, x: 0
+center of distortion in Z=1 plane, y: 0
+tangential distortion coefficient x: -0.00021904
+tangential distortion coefficient y: -2.78752e-05
+metric radius: 0
diff --git a/config/sensors/AKinect_intrinsics/1440_color.txt b/config/sensors/AKinect_intrinsics/1440_color.txt
@@ -0,0 +1,20 @@
+
+===== Device 0: 000647492212 =====
+resolution width: 2560
+resolution height: 1440
+principal point x: 1273.97
+principal point y: 727.203
+focal length x: 1204.73
+focal length y: 1204.79
+radial distortion coefficients:
+k1: 0.514646
+k2: -2.67247
+k3: 1.57664
+k4: 0.3961
+k5: -2.50126
+k6: 1.50402
+center of distortion in Z=1 plane, x: 0
+center of distortion in Z=1 plane, y: 0
+tangential distortion coefficient x: -0.00021904
+tangential distortion coefficient y: -2.78752e-05
+metric radius: 0
diff --git a/config/sensors/AKinect_intrinsics/1536_color.txt b/config/sensors/AKinect_intrinsics/1536_color.txt
@@ -0,0 +1,20 @@
+
+===== Device 0: 000647492212 =====
+resolution width: 2048
+resolution height: 1536
+principal point x: 1019.07
+principal point y: 773.662
+focal length x: 963.783
+focal length y: 963.829
+radial distortion coefficients:
+k1: 0.514646
+k2: -2.67247
+k3: 1.57664
+k4: 0.3961
+k5: -2.50126
+k6: 1.50402
+center of distortion in Z=1 plane, x: 0
+center of distortion in Z=1 plane, y: 0
+tangential distortion coefficient x: -0.00021904
+tangential distortion coefficient y: -2.78752e-05
+metric radius: 0
diff --git a/config/sensors/AKinect_intrinsics/2160_color.txt b/config/sensors/AKinect_intrinsics/2160_color.txt
@@ -0,0 +1,20 @@
+
+===== Device 0: 000647492212 =====
+resolution width: 3840
+resolution height: 2160
+principal point x: 1911.2
+principal point y: 1091.05
+focal length x: 1807.09
+focal length y: 1807.18
+radial distortion coefficients:
+k1: 0.514646
+k2: -2.67247
+k3: 1.57664
+k4: 0.3961
+k5: -2.50126
+k6: 1.50402
+center of distortion in Z=1 plane, x: 0
+center of distortion in Z=1 plane, y: 0
+tangential distortion coefficient x: -0.00021904
+tangential distortion coefficient y: -2.78752e-05
+metric radius: 0
diff --git a/config/sensors/AKinect_intrinsics/3072_color.txt b/config/sensors/AKinect_intrinsics/3072_color.txt
@@ -0,0 +1,20 @@
+
+===== Device 0: 000647492212 =====
+resolution width: 4096
+resolution height: 3072
+principal point x: 2038.65
+principal point y: 1547.82
+focal length x: 1927.57
+focal length y: 1927.66
+radial distortion coefficients:
+k1: 0.514646
+k2: -2.67247
+k3: 1.57664
+k4: 0.3961
+k5: -2.50126
+k6: 1.50402
+center of distortion in Z=1 plane, x: 0
+center of distortion in Z=1 plane, y: 0
+tangential distortion coefficient x: -0.00021904
+tangential distortion coefficient y: -2.78752e-05
+metric radius: 0