json - How to handle large documents? -


i wrote import program reads xml files disk, converts them json , stores them documentdb. there index files stored using node.js sdk.

a second type of files contains chapters. these chapters inserted index files using stored procedure. stored procedure gets index document database, inserts chapter document index document , writes updated document database.

after third document error stored procedure saying execution of stored procedure has been blocked, because exceeded resource limits several times. how can handle situation?

providing answer original question here instead of in comments , addressing second issue in comments.

first answer original question. inside of stored procedure, whenever false in response database operation, need wrap store procedure or risk stored procedure being blacklisted. however, there bug causing stored procedure blacklisting when terminate stored procedure correctly. around this, can delete , recreate stored procedure. have created npm package, documentdb-utils, automatically you.

in comments indicated once got past that, hit limitation, "request rate large", code 429. happens when ignore 'x-ms-retry-after-ms' header response. supposed wait many milliseconds before submitting next request.

here code in documentdb-utils deals this:

 delay = (ms, func) ->    settimeout(func, ms)   processerror = (err, header, toretryif429, nextifnot429 = null) ->    debug('processerror()')    if err.code 429      retryafter = number(header['x-ms-retry-after-ms'])      timelosttothrottling += retryafter      debug("throttled. retrying after delay of #{retryafter}ms")      delay(retryafter, toretryif429)    else if nextifnot429?      nextifnot429()    else      callcallback(err) 

Comments