other (cheap) ideas

[timm]

@vivekaxl and @ginfung: stop coding. pause and think before doing.

here are 2 dozen ways to improve gale, they are all cheap++ to code

i really forward to @vivekaxl and @ginfung extending this list then working this list

caution to @ginfung:

• optimizing the time required to create valid candidates is not useful is those candidates underperform other optimizers

note that, to be on this list, you need to be some minimal delta on top of gale. also, all items on this list should address some known drawback with the current gale

also, to explore this list, sort it by coding effort then work simplest to hardest

ideas (latest ideas, on top)

normalization

Kamvar, Table1, five normalization functions http://ijcai.org/Past%20Proceedings/IJCAI-2003/PDF/083.pdf

• Mincut
• Symmetric divisive
• None
• Normalized additive

hyper parameter optimization

if whatever GALE-variant we use has magic parameters, set those via a secondary DE

GALE experiments slow to run

• get the multi-core stuff working (some success there already I think)
• make the report generation simpler. nothing wrong with those little ascii xtiles charts.

GALE under-performing compared to other optimizers

• try different #goals (Joe's results seemed to show that

GALE's results from vivek different to GALE's results from joe

• vivek talks to joe about those differences. tries to isolate the cause.

if GALE has worse hypervolumes and spreads

the following should volume and spread

• instead of mutating towards best, mutate away from worst
  • x=candidate
  • w=worst
  • for each attribute value, x's mutant is r()*a*(x-w) where a is some acceleration constant; eg a=1,

mutation improvements

if the current gale mutator adds no value, then fix it

• try the mutator described above
• for each mutant generated GALE style, find (b,a) which is the distance to the worse,best pole
  and reject any mutant where b > n*a (and at n=1, your just ensuring mutant is nearer best than rest)
• if few/none of GALE's mutants appear in generation i+1 i+2, then make more mutants. right now
  GALE drowns sqrt(N) candidates in 1-sqrt(N) randoms. why not
  • over-mutate so sqrt(N) mutates up to N/2
  • take the sqrt(N) mutants then do something like SMOTE (take the nearest neighbor and
    generate some new items between that mutant and its neighbor_
• take the X% nearest the best pole and do
  • cross over with mutation (GA style)
  • smearing (DE style)

treat GALE as a sub-routine (e.g. the crowdpruner) of other optimizers

my thinking is that the following stuff is harder than the above so should be done second

maybe the future of GALE is to augment, not replace other optimisers

and in the following, its not so much GALE as WHERE

• given a population of size np
• given a budget of m evals per generation (m < np)
• generate the m items in population0
  • e.g. the DE way (smearing 3 things)
  • e.g. the PS0 way (extrapolating between local and global best)
  • e.g. the GA way (crossover, mutate)
• for each generation, create tmp
  • i.e. np new candidates moved the DE/PSO/GA way
  • evaluate m of them
• Create a space of 2*np items containing 2*m evaluated items
  • Cluster them with WHERE
  • prune using NOT the east west pairs but the median of the 2*m evaluated items in each half
  • stopping at sqrt(2_np) or the variance of the 2_m items starts increasing in sub trees or there
    are not evaluated items in a subtree
• the population pruned in this way becomes the population

As to GALE plus MOEA/D or NSGA-III. don't know