erlang - How should I auto-expire entires in an ETS table, while also limiting its total size? -


i have lot of analytics data i'm looking aggregate every (let's 1 minute.) data being sent process stores in ets table, , every timer sends message process table , remove old data.

the problem amount of data comes in varies wildly, , need 2 things it:

  • if amount of data coming in big, drop oldest data , push new data in. viewed fixed size queue, if amount of data hits limit, queue start dropping things front new data comes back.
  • if queue isn't full, data has been sitting there while, automatically discard (after fixed timeout.)

if these 2 conditions kept, assume table has constant size, , in newer x.

the problem haven't found efficient way these 2 things together. know use match specs delete entires older x, should pretty fast if index timestamp. though i'm not sure if best way periodically trim table.

the second problem keeping total table size under limit, i'm not sure how do. 1 solution comes mind use auto-increment field wich each insert, , when table being trimmed, @ first , last index, calculate difference , again, use match specs delete below threshold.

having said this, feels might using ets table wasn't designed do. there better way store data this, or approaching problem correctly?

you can determine amount of data occupied using ets:info(tab, memory). result in number of words. there catch. if storing binaries heap binaries included. if storing normal erlang terms can use , timestamp described, way go. size in bytes multiply erlang:system_info(wordsize).


Comments

Post a Comment

Popular posts from this blog

node.js - Using Node without global install -

i'm working on few node & electron (atom shell) projects , i'm curious how them work without user having install node globally. i've seen few applications including node directory in project contains node.exe binary. how go setting node project use binary? edit: thought should add, i've searched everywhere answer this, unrelated results.
Read more

How to access a php class file from PHPFox framework into javascript code written in simple HTML file? -

i've website developed using phpfox v3.0.7 . now i'm trying implement 'sse(server sent events)' 1 of php files. reference i'm giving below necessary code part of file. file titled process.class.php , present @ location /var/www/module/notification/include/service/process.class.php present on server having ip http://34.124.40.142/ . <?php /** * [phpfox_header] */ header('content-type: text/event-stream'); header('cache-control: no-cache'); defined('phpfox') or exit('no dice!'); /** * * * @copyright [phpfox_copyright] * @author raymond benc * @package phpfox_service * @version $id: service.class.php 67 2009-01-20 11:32:45z raymond_benc $ */ class notification_service_process extends phpfox_service { /** * class constructor */ public function __construct() { $this->_stable = phpfox::gett('notification'); } public function add($...
Read more

java - Null response to php query in android, even though php works properly -

Image
table i'm trying retrieve above data using 2 php files below. want pass email parameter , return bets user = email. work fine when tested advanced rest client, problem not php. however, when try through android 2 java classes below, null pointer exception (logcat below) , json object returned null. i'm not sure error is, think i'm not passing parameter php , that's why it's producing null response. logcat 05-31 15:10:50.227 26074-26074/com.example.albert.betterapp e/androidruntime﹕ fatal exception: main process: com.example.albert.betterapp, pid: 26074 java.lang.runtimeexception: unable start activity componentinfo{com.example.albert.betterapp/com.example.albert.betterapp.displayallbets}: java.lang.nullpointerexception @ android.app.activitythread.performlaunchactivity(activitythread.java:2292) @ android.app.activitythread.handlelaunchactivity(activitythread.java:2350) @ android.app.activitythread.acce...
Read more