Research Themes


| Main Themes

There are two main research themes in our lab.

  1. Knowledge representation, recognition and manipulation of deformable objects (cloth, paper, string, etc.)
  2. Acquisition of robot skills by means of observation and trial and error

Theme 1: Tasks involving deformable objects can be seen in various environments such as household environments, factory environments, distribution site, etc. However, automating such tasks is far from easy. Our laboratory aims to research and systematize knowledge representation, recognition, and manipulation methods for deformable objects. Some of our research results are:

  •  Recognition and classification of cloth products
  •  Recognition of cloth products’ states during manipulation
  •  Generation of cloth manipulation plans on basis of shape change predictions

Theme 2 is aimed at facilitating the introduction of robotic systems into various workspaces. We adopt methods from imitation learning and reinforcement learning. In imitation learning, skills are extracted from observations of human movements. In reinforcement learning, skills are acquired through trial and error by the robot itself. We are also studying a practical approach that combines the two. Example results are:

  • Automatic acquisition of folding skills by a humanoid robot using its own body as a support surface
  • A method for acquisition of tool manipulation trajectories from human demonstration with a stand-in object (Nikkan Kogyo Shimbun, online edition)
  • Acquisition of grasping routines for cloth products (Joint research using Toyota Motor Corporation’s HSR


| Other Research Task

In addition, we study the following research subjects.

  1. Object recognition and environment recognition for autonomous robots
  2. Motion planning for robots with multiple joints and degrees-of-freedom
  3. Visual information processing for field robotics
  4. Creation of suitable end effectors for object manipulation in manufacturing
  5. Intelligent systems for autonomous robots performing tasks

Theme 1 is aimed at providing robots operating in complex environments with capacities for knowing the surrounding environment and operation targets. Research topics include construction of environmental maps, discovery of operation targets, recognition of human movement, etc. Examples of our research in these areas are as follows.

  • Automatic generation of 3D shape models of small objects
  • Recognition and Modeling of structures such as drawers
  • Object tracking and estimation of superposition relations in human tool operation
  • Efficient environmental map representations for dynamic environments
  • Classification of consumer electronics parts

Theme 2 is aimed at providing robots operating in various environments with smarter ways of moving around those environments. This covers various form factors such as wheeled robots and robot arms, as well as robots combining the two. The following research projects has been conducted on this topic.

  • Grasp planning on basis of automatically generated object models
  • Grasp posture planning with consideration of movement error
  • Obstacle avoidance using a virtual arm

Theme 3 aims to contribute to the automation of outdoor work, where labor shortages are a concern. Examples are disaster response and infrastructure inspection. We have worked on recognition abilities to support search, inspection, and crop harvesting operations. In addition, we have research experience in robot mobility for public buildings and urban areas.

  • An image recognition system for supporting search and situation assessment operations (press release)
  • Detection of cracks and delamination for bridge inspection robots (introduction article on platform)
  • Crop detection for automated harvesting of headed vegetables
  • Research on pedestrian detection and tracking (Tokyo University JSK pageRobot Watch article)
  • Environmental mapping, self-localization, and traversable route detection for autonomous robots

In the 4th theme, we explore outside-the-box designs for new robot hands and sensors in the context of automating product manufacturing. The following research has been conducted so far.

  • End effectors for fabric manipulation using winding motions
  • End effectors for cabling

Theme 5 addresses system construction for robots performing various tasks. With a focus on robots performing operations that involve object manipulation, we study the design of the necessary elements as well as methods for their integration. We are also studying mechanisms for failure detection and recovery.