This is a great example of why having earth in the Goldilocks zone is great. Mutation Rate approximates solar radiation.
It is all about having the right amount of radiation. Have too much and everybody gets cancer and dies, have too little and the first plague to come along kills everybody because they were all alike. Just right and you have the ideal mix, enough diversity to survive challenges, but most people do not die of cancer before breeding.
Actually, it's not surprising that the mutation rate is 'just right'.
If mutations caused by (say) solar activity were much higher, then DNA itself would incorporate more redundancy to protect itself (it looks like there's around a 3-to-1 mapping of letters to proteins already). Every part of the replication process is simultaneously optimizing...
Genetic Programming (Koza, etc) is a lot more interesting (IMHO) than hand-coded Genetic Algorithm mappings for this reason.
One of the problems the original post was battling was the 'fact' that the x86 instruction set was too brittle to do actual mutations on, and so were searching for a way of coding programs with instructions that didn't self-destruct at the first sign of bit-twiddling.
For straight-forward Genetic Algorithms, there's also an issue of how to code the problem space in a way that the evolution operations (mutation, and more importantly cross-over) have a chance of getting hold of the problem.
But searching for a representation that is amenable to mutation is also a search that could be done 'genetically' - by evolving the representation language.
For that reason, I'm no fan of 'straight coding' of the representation. Neither, it seems is evolution... DNA codes for proteins (etc) that builds structures, and frameworks, and other 'stuff' (eg: there are more neurons than DNA letters : there must be some meta-coding going on).
Genetic Programming (Koza, etc) seems like a step in the right direction (since manipulating trees can lead to structures with as much complexity as 'desired' by the evolution bit). But it's still not quite right IMHO, since it's all operating at the 'language' level (mutation of symbols that themselves perform the computation). For an example from nature : Our brains don't work through the interaction of DNA, they work through the interaction of stuff whose precursor (during the person-building stage) was DNA.
If standard GA manipulates the parameters of a problem, then GP is manipulating a tree that codes for the parameters of a problem. The next step up would be manipulating a language that builds the tree. How many meta steps should we make? Isn't that a search problem too?
People have these odd ideas about what a "GA" is. The Genetic Algorithm is an algorithm, not a particular representation of candidate solutions (vectors, lists, trees, whatever). Likewise Koza-style Genetic Programming isn't an algorithm at all: but largely a representation.
That seems pedantic, since of course GAs have certain classic representations and GP has a certain traditional algorithm. But it's important in this case, because (1) non-tree representations can be used handily to evolve code: for example, see machine-code GP or Grammatical Evolution, both of which use GA-style lists. And (2) because to my knowledge there's little in GP which would support the claim that "every part of the replication process is simultaneously optimizing".
Or maybe he meant that Koza-style tree GP was providing "redundancy" (in the form of introns), which is odd since this is hardly viewed as a positive: but at any rate, introns exist in all sorts of GA representations, such as machine-code GP lists.
At any rate, there seems to be a lot of confusion there.
Ok - just to be clear, I was talking about GA in its standard bit-twiddling setting, rather than as an overall approach.
Also, I'm not really saying GP (again, the original Koza-style tree manipulation type) is a magic cure-all : I just like the way it's more 'meta' than bit-twiddling GA.
Are introns bad? An outside observer might think so, but something about the overall algorithm must like them...
As to being confused : I'm surprised that anyone would think that this whole representation problem has any kind of cut-and-dried answer yet. There's still a lot of room for innovation here - and which directions to explore are pretty much a question of taste.
You make an interesting case, which I think is backed up by two things, in the special case of the x86 instruction set:
- Instruction opcodes and arguments don't occupy a fixed space or position. This leads to a significant increase in lethality of mutations. Some other factors that contribute to this is the large instruction set, and the fact that opcodes and arguments are directly connected (context-specific). Straight-up GA does provably not perform well, unless there's a meta-language involved, as the article shows. I'd dare say this is very close to Koza's approach, and it also seems to be close to Michael Korns' approach, which is more recent.
- "Instruction set" put aside, mutations in a x86 system (or in any other computing environment, for that matter) happens on a bit granularity. The sort of natural redundancy that would allow GA to operate on a sufficient population for robust mutations to appear is not there, without a metalanguage. (even ECC in memory tends to not be favoured, so resistance against bit flips, ie. point mutations)
Also, it seems to me that without a "natural" redundancy mechanism within an instruction set, it is impossible for other types of mutations, namely deletion/insertion/duplication, to produce survivable offspring. As a (probably not very good) metaphor one should consider what would happen to aminoacids if we were to consider codons as their building blocks (disregarding nucleotides).
It is all about having the right amount of radiation. Have too much and everybody gets cancer and dies, have too little and the first plague to come along kills everybody because they were all alike. Just right and you have the ideal mix, enough diversity to survive challenges, but most people do not die of cancer before breeding.