Datasets
Daily Interactive Manipulation (DIM) Dataset
Robots that succeed in factories stumble to complete the simplest daily task humans take for granted, for the change of environment makes the task exceedingly difficult. Aiming to teach robots perform daily interactive manipulation in a changing environment using human demonstrations, we collected our own data of interactive manipulation. The dataset focuses on position, orientation, force, and torque of objects manipulated in daily tasks. The dataset includes 1,603 trials of 32 types of daily motions and 1,751 trials of pouring alone, as well as helper code. We present our dataset to facilitate the research on task-oriented interactive manipulation.
We present a dataset of daily interactive manipulation. Specifically, we record daily performed fine motion in which an object is manipulated to interact with another object. We refer to the person who executes the motion as subject, the manipulated object as tool, and the interactive object as object. We focus on recording the motion of the tool. In some cases, we also record the motion of the object.
The dataset consists of two parts. The first part contains 1,603 trials that cover 32 types of motions. We choose fine motions that people commonly perform in daily life which involve interaction with a variety of objects. Different subsets of the motions are found in multiple different motion-related datasets. The motions we collect include those that are most frequently executed in cooking scenarios except that we do not include pick-and-place because it barely involve change of orientation.
The second part contains the pouring motion alone. We collect it to help with motion generalization to different environments. We chose pouring because 1) pouring is found to be the second frequently executed motion in cooking, right after pick-and-place and 2) we can vary the environment setup of the pouring motion easily by switching different materials, cups, and containers. The pouring data contains 1,751 trials of pouring 3 materials from 6 cups into 10 containers. We collect the two parts of the data using the same system.
The dataset provides position and orientation (PO) with 100% coverage, and force and torque (FT) with 100% coverage, and vision with 50% coverage. The less-than-perfect coverage of vision results from filming restrictions.
To cite this dataset:
Huang, Y. and Sun, Y. (2019), A Dataset of Daily Interactive Manipulation, International Journal of Robotics Research (IJRR), 38(8): 879-886. [pdf]
Bibtex:
@article{huang2018dataset,
title={A dataset of daily interactive manipulation},
author={Huang, Yongqiang and Sun, Yu},
journal={The International Journal of Robotics Research},
pages={8790886},
volume = 38,
number = 8,
year={2019},
publisher={SAGE Publications Sage UK: London, England}
}
| Motion Code | Motion Name | Total | PO | FT | Vision | Download |
|---|---|---|---|---|---|---|
| m2 | stir with spatula | 25 | 25 | 25 | 25 | |
| m3 | sprinkle, shake pepper | 40 | 40 | 40 | 40 | |
| m4 | spread/oil | 25 | 25 | 25 | 25 | |
| m6 | vertical cut | 25 | 25 | 25 | 25 | |
| m7 | use spoon to pick up | 98 | 98 | 98 | 35 | |
| m8 | pizza wheel | 25 | 25 | 25 | 25 | |
| m10 | use black brush | 25 | 25 | 25 | 25 | |
| m11 | spear object using fork | 30 | 30 | 30 | 30 | |
| m12 | stir water using spoon | 25 | 25 | 25 | 25 | |
| m13 | fasten screw with screwdriver | 40 | 40 | 40 | 40 | |
| m14 | loosen screw with screwdriver | 35 | 35 | 35 | ||
| m15 | unlock lock with key | 165 | 165 | 165 | 75 | |
| m16 | fasten nut with wrench | 40 | 40 | 40 | 15 | |
| m17 | use paint brush to dip and spread | 25 | 25 | 25 | 25 | |
| m18 | use hammer to hammer in nail | 25 | 25 | 25 | 25 | |
| m19 | brush teeth | 50 | 50 | 50 | 25 | |
| m20 | use file to file wooden thing | 125 | 125 | 125 | 25 | |
| m21 | comb hair | 25 | 25 | 25 | 25 | |
| m22 | scrape substance from surface | 25 | 25 | 25 | 25 | |
| m23 | peel cucumber/potato | 30 | 30 | 30 | 30 | |
| m24 | slice cucumber | 25 | 25 | 25 | 25 | |
| m25 | flip bread | 124 | 124 | 124 | 74 | |
| m26 | use spoon to scoop and pour | 25 | 25 | 25 | 25 | |
| m27 | shave object | 30 | 30 | 30 | 30 | |
| m28 | use roller to roll out dough | 30 | 30 | 30 | 30 | |
| m30 | loosen nut with wrench | 46 | 46 | 46 | ||
| m31 | scoop and pour with measuring spoon/cup | 30 | 30 | 30 | 30 | |
| m32 | insert peg into pegboard | 140 | 140 | 140 | ||
| m33 | brush powder accross grey tray | 80 | 80 | 80 | ||
| m34 | insert straw through to-go cup lid | 25 | 25 | 25 | ||
| m35 | m34 with eyes closed | 25 | 25 | 25 | 25 | |
| m36 | m31 without pour | 120 | 120 | 120 |