How do I build up a Neo4j graph from hundreds of thousands of events? -


we have event sourced application , thinking adding neo4j question view relations. think use case good, problem if replay every event 1 one take time build graph. what's best rebuild graph ~1.000.000 events? resulting graph approx. 100.000 nodes , lot of relations. first naive attempt 1 one takes way time since might want more 1 time.

is best approach build inmemory list of nodes , build large cypher query nodes , 1 relationships? or best way use batch api?

in ideal situation logic rebuilding should same when handle event 1 one live handling "commit" should done after each event.

if requirement keep same logic in application lifecycle events, , use cypher in application, you'll have no choice handle transactional cypher endpoint.

it barely doable, while load csv , batch api give more performance.

i have application write ~1250 cypher statements per cypher transaction, each transaction takes approximately between 0.2 , 0.3 seconds.

some little tips achieve time rate :

do small statements : if need write lot, better eg have 10 small statements match (n1), (n2) merge (n1)-[:follow]->(n2) big statement 10 merges.

profile queries , make sure indexes correct

stay between 500-1000 statements per transaction.

allow sufficient memory database time of bulk load.


Comments

Post a Comment

Popular posts from this blog

angularjs - ADAL JS Angular- WebAPI add a new role claim to the token -

i have created angular spa webapi backend using adal js authentication.as there no roles in ad, need manually add role claims in order give users access different api controllers. the roles stored in database. expecting inject claim through call webapi after authentication ad. webapi code might this. identity.addclaim(new claim("role", "user")); var ticket = new authenticationticket(identity, props); var accesstoken = startup.oauthbeareroptions.accesstokenformat.protect(ticket); is possible replace adal idtoken new token? is viable solution or there other better way handle this? as initial token generated azuread, possible edit token add new claim? appreciated. graph api supporting group claims. see here: http://justazure.com/azure-active-directory-part-4-group-claims/ if @ examples on page, users assigned groups , app can check group in claims. in current version of portal, need app manifest , modify it.
Read more

node.js - Using Node without global install -

i'm working on few node & electron (atom shell) projects , i'm curious how them work without user having install node globally. i've seen few applications including node directory in project contains node.exe binary. how go setting node project use binary? edit: thought should add, i've searched everywhere answer this, unrelated results.
Read more

php - CakePHP HttpSockets send array of paramms -

i problem when make request (i can request). $link = 'http://uri/path/?param1=1&sid=2&sid=3&sid=4' $http = $httpsocket = new httpsocket(); $res = $httpsocket->get($link); if check request pr($http); i see : [line] => /url/path/?param1=1&sid%5b0%5d=2&sid%5b1%5d=3&sid%5b2%5d=4 http/1.1 and i'm getting empty response because server doesn't know parameters sid%5b0%5d=2 when try send parameters array : $data = array('param1' => '1', 'sid' => array('2','3', '4')); i see same changes %5b0%5d - additional indexes. how fix it? unfortunately , can not change server side, if send request in browser, i'll normal response in json format. when make request, can parameters this: $param1 = $_get['param1']; $sid= $_get['sid']; tip: use diferente names parameters. have multiples sid. use sid1, sid2, sid3... update: can parameters this:...
Read more