Matrix Transpose on RowMatrix in Spark -


suppose have rowmatrix.

  1. how can transpose it. api documentation not seem have transpose method.
  2. the matrix has transpose() method. not distributed. if have large matrix greater memory how can transpose it?
  3. i have converted rowmatrix densematrix follows

    densematrix mat = new densematrix(m,n,matarr); 

    which requires converting rowmatrix javardd , converting javardd array.

is there other convenient way conversion?

thanks in advance

you correct: there no

 rowmatrix.transpose() 

method. need operation manually.

here non-distributed/local matrix versions:

def transpose(m: array[array[double]]): array[array[double]] = {     (for {       c <- m(0).indices     } yield m.map(_(c)) ).toarray } 

the distributed version along following lines:

    origmatrdd.rows.zipwithindex.map{ case (rvect, i) =>         rvect.zipwithindex.map{ case (ax, j) => ((j,(i,ax))     }.groupbykey     .sortby{ case (i, ax) => }     .foldbykey(new densevector(origmatrdd.numrows())) { case (dv, (ix,ax))  =>               dv(ix) = ax      } 

caveat: have not tested above: will have bugs. basic approach valid - , similar work had done in past small linalg library spark.


Comments

Popular posts from this blog

angularjs - ADAL JS Angular- WebAPI add a new role claim to the token -

php - CakePHP HttpSockets send array of paramms -

node.js - Using Node without global install -