Offline
Here, we report reproduced final and best scores for all datasets and offline algorithms considered. Note that they differ by a significant
margin, and some papers may use different approaches, not making it always explicit which reporting methodology they chose.
Tip
If you want to re-collect our results in a more structured/nuanced manner, see [how to reproduce](repro.md) section.
Last Scores
Gym-MuJoCo
| Task-Name |
BC |
10% BC |
TD3+BC |
AWAC |
CQL |
IQL |
ReBRAC |
SAC-N |
EDAC |
DT |
| halfcheetah-medium-v2 |
42.40 ± 0.19 |
42.46 ± 0.70 |
48.10 ± 0.18 |
50.02 ± 0.27 |
47.04 ± 0.22 |
48.31 ± 0.22 |
64.04 ± 0.68 |
68.20 ± 1.28 |
67.70 ± 1.04 |
42.20 ± 0.26 |
| halfcheetah-medium-replay-v2 |
35.66 ± 2.33 |
23.59 ± 6.95 |
44.84 ± 0.59 |
45.13 ± 0.88 |
45.04 ± 0.27 |
44.46 ± 0.22 |
51.18 ± 0.31 |
60.70 ± 1.01 |
62.06 ± 1.10 |
38.91 ± 0.50 |
| halfcheetah-medium-expert-v2 |
55.95 ± 7.35 |
90.10 ± 2.45 |
90.78 ± 6.04 |
95.00 ± 0.61 |
95.63 ± 0.42 |
94.74 ± 0.52 |
103.80 ± 2.95 |
98.96 ± 9.31 |
104.76 ± 0.64 |
91.55 ± 0.95 |
| hopper-medium-v2 |
53.51 ± 1.76 |
55.48 ± 7.30 |
60.37 ± 3.49 |
63.02 ± 4.56 |
59.08 ± 3.77 |
67.53 ± 3.78 |
102.29 ± 0.17 |
40.82 ± 9.91 |
101.70 ± 0.28 |
65.10 ± 1.61 |
| hopper-medium-replay-v2 |
29.81 ± 2.07 |
70.42 ± 8.66 |
64.42 ± 21.52 |
98.88 ± 2.07 |
95.11 ± 5.27 |
97.43 ± 6.39 |
94.98 ± 6.53 |
100.33 ± 0.78 |
99.66 ± 0.81 |
81.77 ± 6.87 |
| hopper-medium-expert-v2 |
52.30 ± 4.01 |
111.16 ± 1.03 |
101.17 ± 9.07 |
101.90 ± 6.22 |
99.26 ± 10.91 |
107.42 ± 7.80 |
109.45 ± 2.34 |
101.31 ± 11.63 |
105.19 ± 10.08 |
110.44 ± 0.33 |
| walker2d-medium-v2 |
63.23 ± 16.24 |
67.34 ± 5.17 |
82.71 ± 4.78 |
68.52 ± 27.19 |
80.75 ± 3.28 |
80.91 ± 3.17 |
85.82 ± 0.77 |
87.47 ± 0.66 |
93.36 ± 1.38 |
67.63 ± 2.54 |
| walker2d-medium-replay-v2 |
21.80 ± 10.15 |
54.35 ± 6.34 |
85.62 ± 4.01 |
80.62 ± 3.58 |
73.09 ± 13.22 |
82.15 ± 3.03 |
84.25 ± 2.25 |
78.99 ± 0.50 |
87.10 ± 2.78 |
59.86 ± 2.73 |
| walker2d-medium-expert-v2 |
98.96 ± 15.98 |
108.70 ± 0.25 |
110.03 ± 0.36 |
111.44 ± 1.62 |
109.56 ± 0.39 |
111.72 ± 0.86 |
111.86 ± 0.43 |
114.93 ± 0.41 |
114.75 ± 0.74 |
107.11 ± 0.96 |
| locomotion average |
50.40 |
69.29 |
76.45 |
79.39 |
78.28 |
81.63 |
89.74 |
83.52 |
92.92 |
73.84 |
Maze2d
| Task-Name |
BC |
10% BC |
TD3+BC |
AWAC |
CQL |
IQL |
ReBRAC |
SAC-N |
EDAC |
DT |
| maze2d-umaze-v1 |
0.36 ± 8.69 |
12.18 ± 4.29 |
29.41 ± 12.31 |
65.65 ± 5.34 |
-8.90 ± 6.11 |
42.11 ± 0.58 |
106.87 ± 22.16 |
130.59 ± 16.52 |
95.26 ± 6.39 |
18.08 ± 25.42 |
| maze2d-medium-v1 |
0.79 ± 3.25 |
14.25 ± 2.33 |
59.45 ± 36.25 |
84.63 ± 35.54 |
86.11 ± 9.68 |
34.85 ± 2.72 |
105.11 ± 31.67 |
88.61 ± 18.72 |
57.04 ± 3.45 |
31.71 ± 26.33 |
| maze2d-large-v1 |
2.26 ± 4.39 |
11.32 ± 5.10 |
97.10 ± 25.41 |
215.50 ± 3.11 |
23.75 ± 36.70 |
61.72 ± 3.50 |
78.33 ± 61.77 |
204.76 ± 1.19 |
95.60 ± 22.92 |
35.66 ± 28.20 |
| maze2d average |
1.13 |
12.58 |
61.99 |
121.92 |
33.65 |
46.23 |
96.77 |
141.32 |
82.64 |
28.48 |
Antmaze
| Task-Name |
BC |
10% BC |
TD3+BC |
AWAC |
CQL |
IQL |
ReBRAC |
SAC-N |
EDAC |
DT |
| antmaze-umaze-v2 |
55.25 ± 4.15 |
65.75 ± 5.26 |
70.75 ± 39.18 |
56.75 ± 9.09 |
92.75 ± 1.92 |
77.00 ± 5.52 |
97.75 ± 1.48 |
0.00 ± 0.00 |
0.00 ± 0.00 |
57.00 ± 9.82 |
| antmaze-umaze-diverse-v2 |
47.25 ± 4.09 |
44.00 ± 1.00 |
44.75 ± 11.61 |
54.75 ± 8.01 |
37.25 ± 3.70 |
54.25 ± 5.54 |
83.50 ± 7.02 |
0.00 ± 0.00 |
0.00 ± 0.00 |
51.75 ± 0.43 |
| antmaze-medium-play-v2 |
0.00 ± 0.00 |
2.00 ± 0.71 |
0.25 ± 0.43 |
0.00 ± 0.00 |
65.75 ± 11.61 |
65.75 ± 11.71 |
89.50 ± 3.35 |
0.00 ± 0.00 |
0.00 ± 0.00 |
0.00 ± 0.00 |
| antmaze-medium-diverse-v2 |
0.75 ± 0.83 |
5.75 ± 9.39 |
0.25 ± 0.43 |
0.00 ± 0.00 |
67.25 ± 3.56 |
73.75 ± 5.45 |
83.50 ± 8.20 |
0.00 ± 0.00 |
0.00 ± 0.00 |
0.00 ± 0.00 |
| antmaze-large-play-v2 |
0.00 ± 0.00 |
0.00 ± 0.00 |
0.00 ± 0.00 |
0.00 ± 0.00 |
20.75 ± 7.26 |
42.00 ± 4.53 |
52.25 ± 29.01 |
0.00 ± 0.00 |
0.00 ± 0.00 |
0.00 ± 0.00 |
| antmaze-large-diverse-v2 |
0.00 ± 0.00 |
0.75 ± 0.83 |
0.00 ± 0.00 |
0.00 ± 0.00 |
20.50 ± 13.24 |
30.25 ± 3.63 |
64.00 ± 5.43 |
0.00 ± 0.00 |
0.00 ± 0.00 |
0.00 ± 0.00 |
| antmaze average |
17.21 |
19.71 |
19.33 |
18.58 |
50.71 |
57.17 |
78.42 |
0.00 |
0.00 |
18.12 |
Adroit
| Task-Name |
BC |
10% BC |
TD3+BC |
AWAC |
CQL |
IQL |
ReBRAC |
SAC-N |
EDAC |
DT |
| pen-human-v1 |
71.03 ± 6.26 |
26.99 ± 9.60 |
-3.88 ± 0.21 |
76.65 ± 11.71 |
13.71 ± 16.98 |
78.49 ± 8.21 |
103.16 ± 8.49 |
6.86 ± 5.93 |
5.07 ± 6.16 |
67.68 ± 5.48 |
| pen-cloned-v1 |
51.92 ± 15.15 |
46.67 ± 14.25 |
5.13 ± 5.28 |
85.72 ± 16.92 |
1.04 ± 6.62 |
83.42 ± 8.19 |
102.79 ± 7.84 |
31.35 ± 2.14 |
12.02 ± 1.75 |
64.43 ± 1.43 |
| pen-expert-v1 |
109.65 ± 7.28 |
114.96 ± 2.96 |
122.53 ± 21.27 |
159.91 ± 1.87 |
-1.41 ± 2.34 |
128.05 ± 9.21 |
152.16 ± 6.33 |
87.11 ± 48.95 |
-1.55 ± 0.81 |
116.38 ± 1.27 |
| door-human-v1 |
2.34 ± 4.00 |
-0.13 ± 0.07 |
-0.33 ± 0.01 |
2.39 ± 2.26 |
5.53 ± 1.31 |
3.26 ± 1.83 |
-0.10 ± 0.01 |
-0.38 ± 0.00 |
-0.12 ± 0.13 |
4.44 ± 0.87 |
| door-cloned-v1 |
-0.09 ± 0.03 |
0.29 ± 0.59 |
-0.34 ± 0.01 |
-0.01 ± 0.01 |
-0.33 ± 0.01 |
3.07 ± 1.75 |
0.06 ± 0.05 |
-0.33 ± 0.00 |
2.66 ± 2.31 |
7.64 ± 3.26 |
| door-expert-v1 |
105.35 ± 0.09 |
104.04 ± 1.46 |
-0.33 ± 0.01 |
104.57 ± 0.31 |
-0.32 ± 0.02 |
106.65 ± 0.25 |
106.37 ± 0.29 |
-0.33 ± 0.00 |
106.29 ± 1.73 |
104.87 ± 0.39 |
| hammer-human-v1 |
3.03 ± 3.39 |
-0.19 ± 0.02 |
1.02 ± 0.24 |
1.01 ± 0.51 |
0.14 ± 0.11 |
1.79 ± 0.80 |
0.24 ± 0.24 |
0.24 ± 0.00 |
0.28 ± 0.18 |
1.28 ± 0.15 |
| hammer-cloned-v1 |
0.55 ± 0.16 |
0.12 ± 0.08 |
0.25 ± 0.01 |
1.27 ± 2.11 |
0.30 ± 0.01 |
1.50 ± 0.69 |
5.00 ± 3.75 |
0.14 ± 0.09 |
0.19 ± 0.07 |
1.82 ± 0.55 |
| hammer-expert-v1 |
126.78 ± 0.64 |
121.75 ± 7.67 |
3.11 ± 0.03 |
127.08 ± 0.13 |
0.26 ± 0.01 |
128.68 ± 0.33 |
133.62 ± 0.27 |
25.13 ± 43.25 |
28.52 ± 49.00 |
117.45 ± 6.65 |
| relocate-human-v1 |
0.04 ± 0.03 |
-0.14 ± 0.08 |
-0.29 ± 0.01 |
0.45 ± 0.53 |
0.06 ± 0.03 |
0.12 ± 0.04 |
0.16 ± 0.30 |
-0.31 ± 0.01 |
-0.17 ± 0.17 |
0.05 ± 0.01 |
| relocate-cloned-v1 |
-0.06 ± 0.01 |
-0.00 ± 0.02 |
-0.30 ± 0.01 |
-0.01 ± 0.03 |
-0.29 ± 0.01 |
0.04 ± 0.01 |
1.66 ± 2.59 |
-0.01 ± 0.10 |
0.17 ± 0.35 |
0.16 ± 0.09 |
| relocate-expert-v1 |
107.58 ± 1.20 |
97.90 ± 5.21 |
-1.73 ± 0.96 |
109.52 ± 0.47 |
-0.30 ± 0.02 |
106.11 ± 4.02 |
107.52 ± 2.28 |
-0.36 ± 0.00 |
71.94 ± 18.37 |
104.28 ± 0.42 |
|
|
|
|
|
|
|
|
|
|
|
| adroit average |
48.18 |
42.69 |
10.40 |
55.71 |
1.53 |
53.43 |
59.39 |
12.43 |
18.78 |
49.21 |
Best Scores
Gym-MuJoCo
| Task-Name |
BC |
10% BC |
TD3+BC |
AWAC |
CQL |
IQL |
ReBRAC |
SAC-N |
EDAC |
DT |
| halfcheetah-medium-v2 |
43.60 ± 0.14 |
43.90 ± 0.13 |
48.93 ± 0.11 |
50.81 ± 0.15 |
47.62 ± 0.03 |
48.84 ± 0.07 |
65.62 ± 0.46 |
72.21 ± 0.31 |
69.72 ± 0.92 |
42.73 ± 0.10 |
| halfcheetah-medium-replay-v2 |
40.52 ± 0.19 |
42.27 ± 0.46 |
45.84 ± 0.26 |
46.47 ± 0.26 |
46.43 ± 0.19 |
45.35 ± 0.08 |
52.22 ± 0.31 |
67.29 ± 0.34 |
66.55 ± 1.05 |
40.31 ± 0.28 |
| halfcheetah-medium-expert-v2 |
79.69 ± 3.10 |
94.11 ± 0.22 |
96.59 ± 0.87 |
96.83 ± 0.23 |
97.04 ± 0.17 |
95.38 ± 0.17 |
108.89 ± 1.20 |
111.73 ± 0.47 |
110.62 ± 1.04 |
93.40 ± 0.21 |
| hopper-medium-v2 |
69.04 ± 2.90 |
73.84 ± 0.37 |
70.44 ± 1.18 |
95.42 ± 3.67 |
70.80 ± 1.98 |
80.46 ± 3.09 |
103.19 ± 0.16 |
101.79 ± 0.20 |
103.26 ± 0.14 |
69.42 ± 3.64 |
| hopper-medium-replay-v2 |
68.88 ± 10.33 |
90.57 ± 2.07 |
98.12 ± 1.16 |
101.47 ± 0.23 |
101.63 ± 0.55 |
102.69 ± 0.96 |
102.57 ± 0.45 |
103.83 ± 0.53 |
103.28 ± 0.49 |
88.74 ± 3.02 |
| hopper-medium-expert-v2 |
90.63 ± 10.98 |
113.13 ± 0.16 |
113.22 ± 0.43 |
113.26 ± 0.49 |
112.84 ± 0.66 |
113.18 ± 0.38 |
113.16 ± 0.43 |
111.24 ± 0.15 |
111.80 ± 0.11 |
111.18 ± 0.21 |
| walker2d-medium-v2 |
80.64 ± 0.91 |
82.05 ± 0.93 |
86.91 ± 0.28 |
85.86 ± 3.76 |
84.77 ± 0.20 |
87.58 ± 0.48 |
87.79 ± 0.19 |
90.17 ± 0.54 |
95.78 ± 1.07 |
74.70 ± 0.56 |
| walker2d-medium-replay-v2 |
48.41 ± 7.61 |
76.09 ± 0.40 |
91.17 ± 0.72 |
86.70 ± 0.94 |
89.39 ± 0.88 |
89.94 ± 0.93 |
91.11 ± 0.63 |
85.18 ± 1.63 |
89.69 ± 1.39 |
68.22 ± 1.20 |
| walker2d-medium-expert-v2 |
109.95 ± 0.62 |
109.90 ± 0.09 |
112.21 ± 0.06 |
113.40 ± 2.22 |
111.63 ± 0.38 |
113.06 ± 0.53 |
112.49 ± 0.18 |
116.93 ± 0.42 |
116.52 ± 0.75 |
108.71 ± 0.34 |
| locomotion average |
70.15 |
80.65 |
84.83 |
87.80 |
84.68 |
86.28 |
93.00 |
95.60 |
96.36 |
77.49 |
Maze2d
| Task-Name |
BC |
10% BC |
TD3+BC |
AWAC |
CQL |
IQL |
ReBRAC |
SAC-N |
EDAC |
DT |
| maze2d-umaze-v1 |
16.09 ± 0.87 |
22.49 ± 1.52 |
99.33 ± 16.16 |
136.96 ± 10.89 |
92.05 ± 13.66 |
50.92 ± 4.23 |
162.28 ± 1.79 |
153.12 ± 6.49 |
149.88 ± 1.97 |
63.83 ± 17.35 |
| maze2d-medium-v1 |
19.16 ± 1.24 |
27.64 ± 1.87 |
150.93 ± 3.89 |
152.73 ± 20.78 |
128.66 ± 5.44 |
122.69 ± 30.00 |
150.12 ± 4.48 |
93.80 ± 14.66 |
154.41 ± 1.58 |
68.14 ± 12.25 |
| maze2d-large-v1 |
20.75 ± 6.66 |
41.83 ± 3.64 |
197.64 ± 5.26 |
227.31 ± 1.47 |
157.51 ± 7.32 |
162.25 ± 44.18 |
197.55 ± 5.82 |
207.51 ± 0.96 |
182.52 ± 2.68 |
50.25 ± 19.34 |
|
|
|
|
|
|
|
|
|
|
|
| maze2d average |
18.67 |
30.65 |
149.30 |
172.33 |
126.07 |
111.95 |
169.98 |
151.48 |
162.27 |
60.74 |
Antmaze
| Task-Name |
BC |
10% BC |
TD3+BC |
AWAC |
CQL |
IQL |
ReBRAC |
SAC-N |
EDAC |
DT |
| antmaze-umaze-v2 |
68.50 ± 2.29 |
77.50 ± 1.50 |
98.50 ± 0.87 |
70.75 ± 8.84 |
94.75 ± 0.83 |
84.00 ± 4.06 |
100.00 ± 0.00 |
0.00 ± 0.00 |
42.50 ± 28.61 |
64.50 ± 2.06 |
| antmaze-umaze-diverse-v2 |
64.75 ± 4.32 |
63.50 ± 2.18 |
71.25 ± 5.76 |
81.50 ± 4.27 |
53.75 ± 2.05 |
79.50 ± 3.35 |
96.75 ± 2.28 |
0.00 ± 0.00 |
0.00 ± 0.00 |
60.50 ± 2.29 |
| antmaze-medium-play-v2 |
4.50 ± 1.12 |
6.25 ± 2.38 |
3.75 ± 1.30 |
25.00 ± 10.70 |
80.50 ± 3.35 |
78.50 ± 3.84 |
93.50 ± 2.60 |
0.00 ± 0.00 |
0.00 ± 0.00 |
0.75 ± 0.43 |
| antmaze-medium-diverse-v2 |
4.75 ± 1.09 |
16.50 ± 5.59 |
5.50 ± 1.50 |
10.75 ± 5.31 |
71.00 ± 4.53 |
83.50 ± 1.80 |
91.75 ± 2.05 |
0.00 ± 0.00 |
0.00 ± 0.00 |
0.50 ± 0.50 |
| antmaze-large-play-v2 |
0.50 ± 0.50 |
13.50 ± 9.76 |
1.25 ± 0.43 |
0.50 ± 0.50 |
34.75 ± 5.85 |
53.50 ± 2.50 |
68.75 ± 13.90 |
0.00 ± 0.00 |
0.00 ± 0.00 |
0.00 ± 0.00 |
| antmaze-large-diverse-v2 |
0.75 ± 0.43 |
6.25 ± 1.79 |
0.25 ± 0.43 |
0.00 ± 0.00 |
36.25 ± 3.34 |
53.00 ± 3.00 |
69.50 ± 7.26 |
0.00 ± 0.00 |
0.00 ± 0.00 |
0.00 ± 0.00 |
| antmaze average |
23.96 |
30.58 |
30.08 |
31.42 |
61.83 |
72.00 |
86.71 |
0.00 |
7.08 |
21.04 |
Adroit
| Task-Name |
BC |
10% BC |
TD3+BC |
AWAC |
CQL |
IQL |
ReBRAC |
SAC-N |
EDAC |
DT |
| pen-human-v1 |
99.69 ± 7.45 |
59.89 ± 8.03 |
9.95 ± 8.19 |
119.03 ± 6.55 |
58.91 ± 1.81 |
106.15 ± 10.28 |
127.28 ± 3.22 |
56.48 ± 7.17 |
35.84 ± 10.57 |
77.83 ± 2.30 |
| pen-cloned-v1 |
99.14 ± 12.27 |
83.62 ± 11.75 |
52.66 ± 6.33 |
125.78 ± 3.28 |
14.74 ± 2.31 |
114.05 ± 4.78 |
128.64 ± 7.15 |
52.69 ± 5.30 |
26.90 ± 7.85 |
71.17 ± 2.70 |
| pen-expert-v1 |
128.77 ± 5.88 |
134.36 ± 3.16 |
142.83 ± 7.72 |
162.53 ± 0.30 |
14.86 ± 4.07 |
140.01 ± 6.36 |
157.62 ± 0.26 |
116.43 ± 40.26 |
36.04 ± 4.60 |
119.49 ± 2.31 |
| door-human-v1 |
9.41 ± 4.55 |
7.00 ± 6.77 |
-0.11 ± 0.06 |
17.70 ± 2.55 |
13.28 ± 2.77 |
13.52 ± 1.22 |
0.27 ± 0.43 |
-0.10 ± 0.06 |
2.51 ± 2.26 |
7.36 ± 1.24 |
| door-cloned-v1 |
3.40 ± 0.95 |
10.37 ± 4.09 |
-0.20 ± 0.11 |
10.53 ± 2.82 |
-0.08 ± 0.13 |
9.02 ± 1.47 |
7.73 ± 6.80 |
-0.21 ± 0.10 |
20.36 ± 1.11 |
11.18 ± 0.96 |
| door-expert-v1 |
105.84 ± 0.23 |
105.92 ± 0.24 |
4.49 ± 7.39 |
106.60 ± 0.27 |
59.47 ± 25.04 |
107.29 ± 0.37 |
106.78 ± 0.04 |
0.05 ± 0.02 |
109.22 ± 0.24 |
105.49 ± 0.09 |
| hammer-human-v1 |
12.61 ± 4.87 |
6.23 ± 4.79 |
2.38 ± 0.14 |
16.95 ± 3.61 |
0.30 ± 0.05 |
6.86 ± 2.38 |
1.18 ± 0.15 |
0.25 ± 0.00 |
3.49 ± 2.17 |
1.68 ± 0.11 |
| hammer-cloned-v1 |
8.90 ± 4.04 |
8.72 ± 3.28 |
0.96 ± 0.30 |
10.74 ± 5.54 |
0.32 ± 0.03 |
11.63 ± 1.70 |
48.16 ± 6.20 |
12.67 ± 15.02 |
0.27 ± 0.01 |
2.74 ± 0.22 |
| hammer-expert-v1 |
127.89 ± 0.57 |
128.15 ± 0.66 |
33.31 ± 47.65 |
129.08 ± 0.26 |
0.93 ± 1.12 |
129.76 ± 0.37 |
134.74 ± 0.30 |
91.74 ± 47.77 |
69.44 ± 47.00 |
127.39 ± 0.10 |
| relocate-human-v1 |
0.59 ± 0.27 |
0.16 ± 0.14 |
-0.29 ± 0.01 |
1.77 ± 0.84 |
1.03 ± 0.20 |
1.22 ± 0.28 |
3.70 ± 2.34 |
-0.18 ± 0.14 |
0.05 ± 0.02 |
0.08 ± 0.02 |
| relocate-cloned-v1 |
0.45 ± 0.31 |
0.74 ± 0.45 |
-0.02 ± 0.04 |
0.39 ± 0.13 |
-0.07 ± 0.02 |
1.78 ± 0.70 |
9.25 ± 2.56 |
0.10 ± 0.04 |
4.11 ± 1.39 |
0.34 ± 0.09 |
| relocate-expert-v1 |
110.31 ± 0.36 |
109.77 ± 0.60 |
0.23 ± 0.27 |
111.21 ± 0.32 |
0.03 ± 0.10 |
110.12 ± 0.82 |
111.14 ± 0.23 |
-0.07 ± 0.08 |
98.32 ± 3.75 |
106.49 ± 0.30 |
| adroit average |
58.92 |
54.58 |
20.51 |
67.69 |
13.65 |
62.62 |
69.71 |
27.49 |
33.88 |
52.60 |
Visual summary
