Singular value decompositions form a basic building block in numerical linear algebra and appear in several optimization algorithms, notably in nuclear norm minimization problems and in optimization problems that occur in the context of machine learning models. In several cases the SVD calculations form the computational bottleneck that dominates the computational costs. We discuss a loosely coupled, communication poor parallel algorithm for computing the leading part singular value decomposition of very large scale matrices and its convergence, through theoretical analysis, numerical experiments and comparison with other competitive approaches. Applications in optimization problems are also presented.