In various experimental settings it has been observed that human motions are highly stereotyped. A common assumption is that they are (approximately) optimal with respect to an unknown cost function. We use an bilevel optimization approach to investigate the following inverse problem: Which cost function out of a parameterized family composed from basic cost functions suggested in literature reproduces the recorded human motions best? The lower level problem is an optimal control problem governed by a nonlinear model of the human dynamics and the upper level problem is the data matching problem comparing the optimal control output with the recorded human data. We propose a solution technique for this class of inverse optimal control problems that is based on a collocation approach and a reformulation of the resulting bilevel problem using optimality conditions. Selected modeling aspects and numerical results are discussed for three application examples: human arm motions, human-steered lane changes of a car and human navigation problems.