Selecting suitable grasps on three-dimensional objects is a challenging visuomotor computation, which involves combining information about an object (e.g., its shape, size, and mass) with information about the actor\'s body (e.g., the optimal grasp aperture and hand posture for comfortable manipulation). Here, we used functional magnetic resonance imaging to investigate brain networks associated with these distinct aspects during grasp planning and execution. Human participants of either sex viewed and then executed preselected grasps on L-shaped objects made of wood and/or brass. By leveraging a computational approach that accurately predicts human grasp locations, we selected grasp points that disentangled the role of multiple grasp-relevant factors, that is, grasp axis, grasp size, and object mass. Representational Similarity Analysis revealed that grasp axis was encoded along dorsal-stream regions during grasp planning. Grasp size was first encoded in ventral stream areas during grasp planning then in premotor regions during grasp execution. Object mass was encoded in ventral stream and (pre)motor regions only during grasp execution. Premotor regions further encoded visual predictions of grasp comfort, whereas the ventral stream encoded grasp comfort during execution, suggesting its involvement in haptic evaluation. These shifts in neural representations thus capture the sensorimotor transformations that allow humans to grasp objects.SIGNIFICANCE STATEMENT Grasping requires integrating object properties with constraints on hand and arm postures. Using a computational approach that accurately predicts human grasp locations by combining such constraints, we selected grasps on objects that disentangled the relative contributions of object mass, grasp size, and grasp axis during grasp planning and execution in a neuroimaging study. Our findings reveal a greater role of dorsal-stream visuomotor areas during grasp planning, and, surprisingly, increasing ventral stream engagement during execution. We propose that during planning, visuomotor representations initially encode grasp axis and size. Perceptual representations of object material properties become more relevant instead as the hand approaches the object and motor programs are refined with estimates of the grip forces required to successfully lift the object.

The primary motivation of this paper is to present a compliant and cost-effective solution for object transfer between human and robot. The application prospect of this study is robot-human collaboration in manufacturing. To achieve above goals, a novel modular 3-axis force sensor is proposed for the grasping system to achieve interactive force sensing. Compliant object transfer control strategy, which is composed of incremental force control mode and gravity balance control mode, is proposed for object transfer between human and robot. A prototype of underactuated grasping system which is mounted on the proposed modular 3-axis force sensor is fabricated to investigate the effectiveness of the proposed interactive control strategy. Experimental results reveal that the incremental force control mode is suitable for the lighter objects with a higher interactive sensitivity. For transferring heavier objects, the gravity balance control mode is more suitable. In gravity balance control mode, the human hand could achieve a quasi-static equilibrium with the object, and achieve a compliant transfer operation. Due to the above characteristic, the proposed control strategy has the potentials to enhance the object transfer compliance and safety in the human-robot object transfer process.

Obtaining accurate depth information is key to robot grasping tasks. However, for transparent objects, RGB-D cameras have difficulty perceiving them owing to the objects\' refraction and reflection properties. This property makes it difficult for humanoid robots to perceive and grasp everyday transparent objects. To remedy this, existing studies usually remove transparent object areas using a model that learns patterns from the remaining opaque areas so that depth estimations can be completed. Notably, this frequently leads to deviations from the ground truth. In this study, we propose a new depth completion method [i.e., ClueDepth Grasp (CDGrasp)] that works more effectively with transparent objects in RGB-D images. Specifically, we propose a ClueDepth module, which leverages the geometry method to filter-out refractive and reflective points while preserving the correct depths, consequently providing crucial positional clues for object location. To acquire sufficient features to complete the depth map, we design a DenseFormer network that integrates DenseNet to extract local features and swin-transformer blocks to obtain the required global information. Furthermore, to fully utilize the information obtained from multi-modal visual maps, we devise a Multi-Modal U-Net Module to capture multiscale features. Extensive experiments conducted on the ClearGrasp dataset show that our method achieves state-of-the-art performance in terms of accuracy and generalization of depth completion for transparent objects, and the successful employment of a humanoid robot grasping capability verifies the efficacy of our proposed method.

The increasing demand for grasping diverse objects in unstructured environments poses severe challenges to the existing soft/rigid robotic fingers due to the issues in balancing force, compliance, and stability, and hence has given birth to several hybrid designs. These hybrid designs utilize the advantages of rigid and soft structures and show better performance, but they are still suffering from narrow output force range, limited compliance, and rarely reported stability. Owing to its rigid-soft coupling structure with flexible switched multiple poses, human finger, as an excellent hybrid design, shows wide-range output force, excellent compliance, and stability. Inspired by human finger, we propose a hybrid finger with multiple modes and poses, coupled by a soft actuator (SA) and a rigid actuator (RA) in parallel. The multiple actuation modes formed by a pneumatic-based rigid-soft collaborative strategy can selectively enable the RA\'s high force and SA\'s softness, whereas the multiple poses derived from the specially designed underactuated RA skeleton can be flexibly switched with tasks, thus achieving high compliance. Such hybrid fingers also proved to be highly stable under external stimuli or gravity. Furthermore, we modularize and configure these fingers into a series of grippers with excellent grasping performance, for example, wide graspable object range (diverse from 0.1 g potato chips to 27 kg dumbbells for a 420 g two-finger gripper), high compliance (tolerate objects with 94% gripper span size and 4 cm offset), and high stability. Our study highlights the potential of fusing rigid-soft technologies for robot development, and potentially impacts future bionics and high-performance robot development.

While reaching and grasping are highly prevalent manual actions, neuroimaging studies provide evidence that their neural representations may be shared between different body parts, i.e., effectors. If these actions are guided by effector-independent mechanisms, similar kinematics should be observed when the action is performed by the hand or by a cortically remote and less experienced effector, such as the foot. We tested this hypothesis with two characteristic components of action: the initial ballistic stage of reaching, and the preshaping of the digits during grasping based on object size. We examined if these kinematic features reflect effector-independent mechanisms by asking participants to reach toward and to grasp objects of different widths with their hand and foot. First, during both reaching and grasping, the velocity profile up to peak velocity matched between the hand and the foot, indicating a shared ballistic acceleration phase. Second, maximum grip aperture and time of maximum grip aperture of grasping increased with object size for both effectors, indicating encoding of object size during transport. Differences between the hand and foot were found in the deceleration phase and time of maximum grip aperture, likely due to biomechanical differences and the participants\' inexperience with foot actions. These findings provide evidence for effector-independent visuomotor mechanisms of reaching and grasping that generalize across body parts.

Previous studies have shown that our perception of stimulus properties can be affected by the emotional nature of the stimulus. It is not clear, however, how emotions affect visually-guided actions toward objects. To address this question, we used toy rats, toy squirrels, and wooden blocks to induce negative, positive, and neutral emotions, respectively. Participants were asked to report the perceived distance and the perceived size of a target object resting on top of one of the three emotion-inducing objects; or to grasp the same target object either without visual feedback (open-loop) or with visual feedback (closed-loop) of both the target object and their grasping hand during the execution of grasping. We found that the target object was perceived closer and larger, but was grasped with a smaller grip aperture in the rat condition than in the squirrel and the wooden-block conditions when no visual feedback was available. With visual feedback present, this difference in grip aperture disappeared. These results showed that negative emotion influences both perceived size and grip aperture, but in opposite directions (larger perceived size but smaller grip aperture) and its influence on grip aperture could be corrected by visual feedback, which revealed different effects of emotion to perception and action. Our results have implications on the understanding of the relationship between perception and action in emotional condition, which showed the novel difference from previous theories.

The proposal of postural synergy theory has provided a new approach to solve the problem of controlling anthropomorphic hands with multiple degrees of freedom. However, generating the grasp configuration for new tasks in this context remains challenging. This study proposes a method to learn grasp configuration according to the shape of the object by using postural synergy theory. By referring to past research, an experimental paradigm is first designed that enables the grasping of 50 typical objects in grasping and operational tasks. The angles of the finger joints of 10 subjects were then recorded when performing these tasks. Following this, four hand primitives were extracted by using principal component analysis, and a low-dimensional synergy subspace was established. The problem of planning the trajectories of the joints was thus transformed into that of determining the synergy input for trajectory planning in low-dimensional space. The average synergy inputs for the trajectories of each task were obtained through the Gaussian mixture regression, and several Gaussian processes were trained to infer the inputs trajectories of a given shape descriptor for similar tasks. Finally, the feasibility of the proposed method was verified by simulations involving the generation of grasp configurations for a prosthetic hand control. The error in the reconstructed posture was compared with those obtained by using postural synergies in past work. The results show that the proposed method can realize movements similar to those of the human hand during grasping actions, and its range of use can be extended from simple grasping tasks to complex operational tasks.

Collecting seafood animals (such as sea cucumbers, sea echini, scallops, etc.) cultivated in shallow water (water depth: ~30 m) is a profitable and an emerging field that requires robotics for replacing human divers. Soft robotics have several promising features (e.g., safe contact with the objects, lightweight, etc.) for performing such a task. In this paper, we implement a soft manipulator with an opposite-bending-and-extension structure. A simple and rapid inverse kinematics method is proposed to control the spatial location and trajectory of the underwater soft manipulator\'s end effector. We introduce the actuation hardware of the prototype, and then characterize the trajectory and workspace. We find that the prototype can well track fundamental trajectories such as a line and an arc. Finally, we construct a small underwater robot and demonstrate that the underwater soft manipulator successfully collects multiple irregular shaped seafood animals of different sizes and stiffness at the bottom of the natural oceanic environment (water depth: ~10 m).

The unique morphological bases of human hands, which are distinct from other primates, endow them with excellent grasping and manipulative abilities. However, the lack of understanding of human hand morphology and its parametric features is a major obstacle in the scientific design of prosthetic hands. Existing designs of prosthetic hand morphologies mostly adopt engineering-based methods, which depend on human experience, direct measurements of human hands, or numerical simulation/optimization. This paper explores for the first time a science-driven design method for prosthetic hand morphology, aiming to facilitate the development of prosthetic hands with human-level dexterity. We first use human morphological, movement, and postural data to quantitatively cognize general morphological characteristics of human hands in static, dynamic, functional, and non-functional perspectives. Taking these cognitions as bases, we develop a method able to quickly transfer human morphological parameters to prosthetic hands and endow the prosthetic hands with great grasping/manipulative potential at the same time. We apply this method to the design of an advanced prosthetic hand (called X-hand II) embedded with compact actuating systems. The human-size prosthetic hand can reach wide grasping/manipulative ranges close to those of human hands, replicate various daily grasping types and even execute dexterous in-hand manipulation. This science-driven method may also inspire other artificial limb and bionic robot designs.

Small and manipulable objects (tools) preferentially evoke a network of brain regions relative to other objects, including the lateral occipitotemporal cortex (LOTC), which is assumed to process tool shape information. Given the correlation between various object properties, the exact type of information being represented in the LOTC remains debated. In three fMRI experiments, we examined the effects of multiple levels of shape (whole vs. object parts) and motor-related (grasping; manipulation) information. Combining representational similarity analysis and commonality analysis allowed us to partition the unique and shared effects of correlated dimensions. We found that grasping manner (for pickup), not the overall object shape or manner of manipulation, uniquely explained the LOTC neural activity pattern (Experiments 1 and 2). Experiment 3 tested tools composed of two parts to understand better how grasping manner was computed from object visual inputs. Support vector machine analysis revealed that the LOTC activity could decode different shapes of the tools\' handle parts but not the tools\' head parts. Together, these results suggest that the LOTC parses tool shapes by how it maps onto grasping programs; such parsing is not fully based on the whole-object shape but rather an interaction between the whole (where to grasp) and its parts (distinguishing the shape for the grasping part for specific grasping manners).