In theory, the successive gradients generated by the conjugate gradient method applied to a quadratic should be orthogonal. However, for some ill-conditioned problems, orthogonality is quickly lost due to rounding errors, and convergence is much slower than expected. A limited memory version of the conjugate gradient method will be presented. The memory is used to both detect the loss of orthogonality and to restore orthogonality. Numerical comparisons to the limited memory BFGS method (L-BFGS) will be also discussed.