Matrix Transpose on RowMatrix in Spark -
suppose have rowmatrix.
- how can transpose it. api documentation not seem have transpose method.
- the matrix has transpose() method. not distributed. if have large matrix greater memory how can transpose it?
i have converted rowmatrix densematrix follows
densematrix mat = new densematrix(m,n,matarr);
which requires converting rowmatrix javardd , converting javardd array.
is there other convenient way conversion?
thanks in advance
you correct: there no
rowmatrix.transpose()
method. need operation manually.
here non-distributed/local matrix versions:
def transpose(m: array[array[double]]): array[array[double]] = { (for { c <- m(0).indices } yield m.map(_(c)) ).toarray }
the distributed version along following lines:
origmatrdd.rows.zipwithindex.map{ case (rvect, i) => rvect.zipwithindex.map{ case (ax, j) => ((j,(i,ax)) }.groupbykey .sortby{ case (i, ax) => } .foldbykey(new densevector(origmatrdd.numrows())) { case (dv, (ix,ax)) => dv(ix) = ax }
caveat: have not tested above: will have bugs. basic approach valid - , similar work had done in past small linalg library spark.
Comments
Post a Comment