We consider the problem of packing ellipsoids of different size and shape into an ellipsoidal container so as to minimize a measure of total overlap. The motivating application is chromosome organization in the human cell nucleus. We describe a bilevel optimization formulation, together with an algorithm for the general case and a simpler algorithm for the special case in which all ellipsoids are in fact spheres. We prove convergence to stationary points of this non-convex problem, and describe computational experience, including applications to sphere packing and results from the chromosome packing application.