The positive semidefinite rank of a nonnegative $m\times n$-matrix $S$ is the minimum number $q$ such that there exist positive semidefinite $q\times q$ matrices $A_1, ..., A_m, B_1, ..., B_n$ such that $S(k,\ell) = tr A_k^* B_\ell$. Just as the nonnegative rank characterizes the minimum size of formulations of combinatorial optimization problems as linear programs, the positive semidefinite rank characterizes their minimum size as positive semidefinite programs. The most important lower bound technique on nonnegative rank only uses the zero/non-zero pattern of the matrix. We characterize the power of lower bounds on positive semidefinite rank based on the zero/non-zero pattern. We then use this characterization to prove lower bounds on the positive semidefinite ranks of families of matrices which arise from the Traveling Salesman and Max-Cut problems.