Memory efficient datastructure for hashmap (c++) -
the scenario kind of simple.
i value, in range between 0 , 2^x (x~27)
. use value key hashmap. in hashmap store index (source of value). x may greater 27, have use memory efficient data structure.
first tried unordered_multimap, there big overhead, disqualifying it. tried unordered_map of vectors. increasing number of vectors in map, overhead big. thought of using 2d array reallocating dynamic size.
learned here on stackoverflow calling 2^27 times "malloc()" creates overhead, tried this:
uint64_t length = (uint64_t) pow(2.0,27); uint64_t ** hashmap; hashmap = (uint64_t **) malloc(sizeof * hashmap * length); uint64_t * values = (uint64_t *) malloc(sizeof * values * 3 * length); for(int = 0;i<length;i++) hashmap[i] = values + 3 * i; //destroys whole datastructure hashmap[0] = (uint64_t *) realloc(hashmap[0],sizeof*hashmap[0]*4);
i allocate 3 * siezof * values
keep track of actual length , maximal length of bucket.
comment says reallocating destroys whole arrray, maybe because there no bookkeeping (via malloc) on pointer stores 3 elements? there way realloc on structure? or u know better structure intend?
edit cause of dau_sama's answer:
while using following code, i'm excpierencing performance problems (runtime , memory):
std::unordered_map <uint64_t, std::vector<uint64_t>> m; uint64_t length = 1ul<<22; for(int = 0 ; i<length;i++) { m.emplace(i,vector<uint64_t>()); m.at(i).push_back(i); }
i reduced length 2^22 cause aborted 2^27 implementation @ runtime of 7 minutes , memory usage of ~8gb.
snippet has runtime of 60 seconds , memory usage of ~1.7gb. compared above array implementation that's alot, array took ~4gb of memory , runtime of 1.7 seconds (at 2^27 elements). maybe i'm doing wrong?
it's simple as: don't reinvent wheel, have std::unordered_map<int, int>
maps need. it's nice understand pointers, don't need call malloc
directly of cases.
Comments
Post a Comment