How to read and write Map<String, Object> from/to parquet file in Java or Scala? -


looking concise example on how read , write map<string, object> from/to parquet file in java or scala?

here expected structure, using com.fasterxml.jackson.databind.objectmapper serializer in java (i.e. looking equivalent using parquet):

public static map<string, object> read(inputstream inputstream) throws ioexception {     objectmapper objectmapper = new objectmapper();      return objectmapper.readvalue(inputstream, new typereference<map<string, object>>() {      }); }  public static void write(outputstream outputstream, map<string, object> map) throws ioexception {     objectmapper objectmapper = new objectmapper();      objectmapper.writevalue(outputstream, map);         } 

i'm not quite parquet but, here:

schema schema = new schema.parser().parse(resources.getresource("map.avsc").openstream());      file tmp = file.createtempfile(getclass().getsimplename(), ".tmp");     tmp.deleteonexit();     tmp.delete();     path file = new path(tmp.getpath());      avroparquetwriter<genericrecord> writer =          new avroparquetwriter<genericrecord>(file, schema);      // write record empty map.     immutablemap emptymap = new immutablemap.builder<string, integer>().build();     genericdata.record record = new genericrecordbuilder(schema)         .set("mymap", emptymap).build();     writer.write(record);     writer.close();      avroparquetreader<genericrecord> reader = new avroparquetreader<genericrecord>(file);     genericrecord nextrecord = reader.read();      assertnotnull(nextrecord);     assertequals(emptymap, nextrecord.get("mymap")); 

in situation change immutablemap (google collections) default map below:

schema schema = new schema.parser().parse( resources.getresource( "map.avsc" ).openstream() );          file tmp = file.createtempfile( getclass().getsimplename(), ".tmp" );         tmp.deleteonexit();         tmp.delete();         path file = new path( tmp.getpath() );          avroparquetwriter<genericrecord> writer = new avroparquetwriter<genericrecord>( file, schema );          // write record empty map.         map<string,object> emptymap = new hashmap<string, object>();          // not empty more         emptymap.put( "something", new something() );         genericdata.record record = new genericrecordbuilder( schema ).set( "mymap", emptymap ).build();         writer.write( record );         writer.close();          avroparquetreader<genericrecord> reader = new avroparquetreader<genericrecord>( file );         genericrecord nextrecord = reader.read();          assertnotnull( nextrecord );         assertequals( emptymap, nextrecord.get( "mymap" ) ); 

i didn't test code, give try..


Comments

Popular posts from this blog

angularjs - ADAL JS Angular- WebAPI add a new role claim to the token -

php - CakePHP HttpSockets send array of paramms -

node.js - Using Node without global install -