Ordset/orddict vs sets / dict

[russelldb]

Evaluate the cost and benefit of ord* vs * for internal data structures. Is it possible to be adaptive and move from one to the other when size dictates it is optimal (maybe as part of from_binary, if we store a size in the binary encoding?)

[lenary]

How soon do you need results/code for this?

[lenary]

I've begun working on a benchmark to see what the difference is. As to whether we can be adaptive, I've not started working on that.

[russelldb]

And that's why you're the best.

[seancribbs]

I read Elixir's HashDict last night, might be worth converting (and easy). Basically buckets are ordered and it starts with one bucket, splitting/rehashing when each bucket reaches 8 elements. So at small sizes it is basically orddict, scaling well up to sizes comparable with dict.

[lenary]

I'm being rather slow.

I've written something to allow you to swap which set/dict you use, depending on compile-time macros. Probably rather ugly. in branch feature/lenary/dict-bm

I also started working on a basho_bench so i could work out which one is faster. I didn't get this finished, mostly because I tried to do too much in one module. https://gist.github.com/lenary/51f0ae9bb9ae1944fd27

I realise you're trying to get me to use HashDict. This would probably be sensible as i can see us running into issues with orddict soon, and dict itself has some major issues. Let's not think about porting it until we actually run into issues though.

[seancribbs]

@lenary Not trying to get you to use HashDict, it was just a suggestion. I have it finished btw. 😄

[lenary]

So @seancribbs converted HashDict already: https://gist.github.com/seancribbs/7912656

Also he may be converting elixir's HashSet.

We need to take a closer look at riak_dt_orswot.erl's types.

[seancribbs]

Since this will be an internal format change, we'll need code to convert/upgrade after a cluster has upgraded with capabilities and such. Assigning to 2.1