Rightfully so, many MIPS members are concerned about curve fitting, data mining, etc., in timing models on the market today (and, or course, in the MIPS models themselves). I can tell you point blank that we do not curve fit in the development of the MIPS models. I can also say with a high degree of certainly that any engineer with an advanced degree from a good school who has written software for control systems (say for nuclear power plants, space rockets, fighter jets, commercial aircraft, etc.), where failure is expensive or disastrous, knows the difference between curve fitting and the development of new algorithms to improve their software's performance.
I say "rightfully so" above, because many (or most) model developers do curve fit in the "development" of their models (if you want to call it "development"). The problem is that "curve fitting" is not really "development". It is merely forcing a bad model to look good using a certain set of data. And yes, done correctly, that model may produce good results after curve fitting with a certain set of data, but will almost certainly fail with any other set of data. I can also tell you that I know several model "developers" that curve fit in some form or fashion in their models, and they don't even know that they are doing so.
In reality, there can be a fine line between:
(1) curve fitting an existing model to perform better or
(2) introducing new algorithms to make a revised model perform better than its predecessor.
Let's see if I can come up with an example of this. In many cases, trying to come up with an example to explain something is more complicated than describing the original.
Allow me to try to explain fixing a fighter jet's control system software to better adhere to the design specs as to how fast the jet should climb in relation to how far back the pilot pulls "the stick". In any decent software of this type, the formula would be based on the physical fact that, because of the curvature of the top of the wing compared to the flat bottom, the air traveling over the top of the wing goes faster than that along the bottom, and hence the air pressure on the top of the wing is lower than that on the bottom. This pressure difference, of course, is what "pushes" the jet up into the lower pressure zone. This is why the wing is designed and built the way it is.
Now, let's say the fighter jet isn't working the way is should and two teams set out to fix it. Team #1 applies curve fitting and team #2 chooses to redesign.
Team #1
Curve fitting in this case could be "developers" tweaking certain parameters in the existing formula to force the jet to climb at a certain rate depending upon the speed of the jet. They may make this one jet perform better, but their "design" will likely fail for all others.
Team #2
Real development in this control system software would be where the design engineers that developed the formulas for how fast the wind over the wing should travel, realize that these formulas were developed for Mach1 speeds, but the current jets travel at Mach3. They also know that, at Mach3, the faster wind speed heats up the air going over the wing more than Mach1, and this hotter air get "lighter" (lower pressure). So, lighter air on the top makes the pressure from the bottom more effective, and the jet moves up faster.
Therefore, rather than one-time random "adjustments" which are also unexplainable from Team #1 (the curve fitters), the "real" design engineers in Team #2 introduced new mathematical algorithms that take the speed of the jet into consideration and adjust accordingly and automatically, and the software then works again at all speeds.
The main difference between these two approaches, of course, is that, when the software controlling of the jet needs to be improved:
Team #1 members "need and use" the raw data to adjust/develop their model, whereas
Team #2 members introduce new mathematical algorithms to closer resemble the performance of the jet itself (and they use the data
only to "prove" that the new algorithms/equations did improve the performance of the jet in the way in which it was designed).
Is this understandable?
...
