performance - Perl creates objects very slowly -

i have perl script reads ~50,000 rows database , stores them in array of hashes. standard dbi code. rather work directly on hashes, prefer put data objects can pass other code modules cleanly. table i'm reading has 15+ columns in it. code looks like:

my $db = dbi->connect(); # pretend see proper dbi connect here $resultset = $db->selectall_arrayref($sql); $db->disconnect();  # here's problem starts. %objects; $row (@{$resultset}) {     ($col1, $col2, ..., $col15) = @{$row};     %inputhash;     $inputhash{col1} = $col1 if $col1;     ...     $inputhash{col15} = $col1 if $col15;     $obj = model::object->new(%inputhash);     $objects{$col1} = $obj; } return values %objects; 

it collects stuff hash eliminate dups select. problem starts in loop below comment says "here's problem starts". i've put message in loop log line every 100 objects created. first 100 objects created in 5 secs. next 100 took 16 secs. getting 300 took 30 more secs. it's 9000 objects , taking 12+ minutes create 100 objects. didn't think 50,000 objects large enough create these kinds of issues.

the model::object that's being created class getters , setters each of properties. has new method , serialize method (essentially tostring) , that's it. there's no logic it.

i'm running activestate perl 5.16 on windows laptop 8 gb of ram, i7 processor (3 yrs old) , ssd drive reasonable space. i've seen on linux machine same version of perl, don't think it's hardware thing. need stay on 5.16 of perl. advice how improve performance appreciated. thanks.

first of all: profile program! have narrowed down 1 sub, devel::nytprof (for example) can narrow down line culprit.

here general considerations side:

just glancing on it, probable slowing factors spring mind, can't sure if don't profile program:

mayhe hashing-allocation takes long. %objects hash grows, perl steadily allocate more memory. pre-set size of $objects hash. feature documented here. since memory allocation problem, wouldn't recognize this, if profile small data set.

# somewhere outside of loop keys(%objects) = $number_of_rows * 1.2; # hash should little bigger objects stored in 

secondly, object-creation takes long. take @ model::object. don't know what's in there can't make comment that. should consider passing %inputhash reference. model::object->new(%inputhash);, put keys , values on stack, , retrieve it, in worst case my %options = @_;. move, recompute hash every key.

maybe can come way rid of small $inputhash completely. can come ways, based on definednes, checking truthyness (are sure that's right, btw? "0" false, example).

but again, importantly: profile program. maybe take smaller data set, wouldn't able see memory allocation problems then. profiling see, @ point program takes time.

the perldoc has speeding program. has nice chapter profiling, too.