Scoring Profile Results
This is a running log of my results when profiling the scoring algorithm.
Initial trace shows CNT=34007830, ACT=138591.002, OWN=138450.474. Surprisingly, the biggest problem turns out to be botbrackets_bracket:random_bracket/1 with CNT=5000, ACT=112721.667, OWN=46.771, which is 81.4%. Inside of that, you have:
| Method | CNT | ACT | OWN | Caller | Overall % |
|---|---|---|---|---|---|
| ncaab_bracket:add_winner/3 | 315000 | 73345.611 | 3171.307 | botbrackets_bracket:add_random_winner/3 | 52.9% |
| ncaab_bracket:set_result/4 | 315000 | 49642.942 | 941.127 | ncaab_bracket:add_winner/3 | 35.8% |
| gb_trees:insert/3 | 315000 | 38247.469 | 1615.587 | gb_trees:enter/3 (from add_winner) | 27.6% |
| ncaab_bracket:get_players/3 | 315000 | 14508.047 | 3316.906 | botbrackets_bracket:add_random_winner/3 | 10.5% |
| ncaab_bracket:get_result/3 | 310000 | 7915.894 | 1667.671 | ncaab_bracket:get_players/3 | 5.7% |
| ncaab_bracket:seed/1 | 630000 | 3259.728 | 1612.486 | ncaab_bracket:get_players/3 | 2.4% |
| ncaab_bracket:get_result/3 | 315000 | 10063.795 | 1985.449 | botbrackets_bracket:add_random_winner/3 | 7.3% |
| gb_trees:lookup/2 | 1250000 | 27957.882 | 3444.765 | ncaab_bracket:get_result/3 (from a few places) | 20.2% |
From this, it looks like almost half the time is in gb_trees. Perhaps I should look at other data structures.
The times for a single trace of random_bracket is CNT=5613, ACC=63.297, OWN=63.295....
| Method | CNT | ACT | OWN | Caller | Overall % |
|---|---|---|---|---|---|
| botbrackets_bracket:add_random_winner/3 | 63 | 46.926 | 0.378 | botbrackets_bracket:random_bracket/1 (via lists:foldl) | 74.1% |
| gb_trees:insert/3 | 63 | 16.645 | 0.126 | ncaab_bracket:add_winner/3 | 26.3% |
| ncaab_bracket:get_result/3 | 63 | 14.871 | 0.126 | botbrackets_bracket:add_random_winner/3 | 23.5% |
| ncaab_bracket:get_result/3 | 125 | 12.412 | 0.250 | ncaab_bracket:check_winner/4 | 19.6% |
| botbrackets_bracket:'-random_bracket/1-lc$^1/1-1-'/3 | 69 | 16.088 | 16.059 | botbrackets_bracket:random_bracket/1 | 25.4% |
The ncaab_bracket module has conditional compilation support for using a dict instead of gb_trees. Using that, I get ACC=22.325 of which add_random_winner is ACC=6.345... a marked improvement! Plugging that into the original test gives ACC=121447.622, though, with dict:find/3 taking up 30% and dict:store/3 taking up 17.1%. That implies that perhaps I should remove the extra checks of the internal structure and just assume it's valid. That could shave off perhaps 15% of the runtime.
Well, that was an underestimate! Ran it again with calling an "unsafe" version of the function and it gives me ACC=91292.002 with botbrackets_bracket:add_random_winner/3 accounting for 62210.208 (68.1%):
| Method | CNT | ACT | OWN | Caller | Overall % |
|---|---|---|---|---|---|
| ncaab_bracket:add_winner_unsafe/4 | 315000 | 23467.847 | 1721.984 | botbrackets_bracket:add_random_winner/3 | 25.7% |
| ncaab_bracket:get_players/3 | 315000 | 18243.628 | 3700.723 | botbrackets_bracket:add_random_winner/3 | 20.0% |
| ncaab_bracket:get_result/3 | 315000 | 11757.958 | 1797.329 | botbrackets_bracket:add_random_winner/3 | 12.9% |
| botbrackets_bracket:score/3 | 5000 | 22991.711 | 75.364 | botbrackets_bracket:monte_carlo/5 | 25.2% |
Moreover, changing score/3 to not track game results and to use the streaming Kahan summation logic resulted in a run of ACC=59295.682! The breakdown is as follows:
| Method | CNT | ACT | OWN | Caller | Overall % |
|---|---|---|---|---|---|
| botbrackets_bracket:random_bracket/1 | 5000 | 47875.779 | 31.937 | botbrackets_bracket:monte_carlo/5 | 80.7% |
| ncaab_bracket:add_winner_unsafe/4 | 315000 | 16471.375 | 1203.372 | botbrackets_bracket:add_random_winner/3 | 27.8% |
| ncaab_bracket:get_players/3 | 315000 | 12942.726 | 2827.163 | botbrackets_bracket:add_random_winner/3 | 21.8% |
| ncaab_bracket:get_result/3 | 315000 | 7675.235 | 1275.407 | botbrackets_bracket:add_random_winner/3 | 12.9% |
| random:uniform/0 | 315000 | 2233.380 | 1598.797 | botbrackets_bracket:add_random_winner/3 | 3.8% |
| botbrackets_bracket:score_quick/3 | 5000 | 10888.898 | 54.286 | botbrackets_bracket:monte_carlo/5 | 18.4% |
