In this talk we are concerned with the second-order methods for optimization (which naturally include interior point algorithms). Many large-scale problems cannot be solved with methods which rely on exact directions obtained by factorizing matrices. For such problems, the search directions have to be computed using iterative methods. We address the problem of how much of inexactness is allowed without noticeably slowing down the convergence compared with the exact second-order method. We argue that (except for some very special problems) matrix-free approaches have to be applied to successfully tackle truly large scale problems. We provide new theoretical insights and back them up with computational experience.