Function learning was used to test the functionality of the new weight smoothing CMAC. It was expected that the new CMAC would have advantages approximating smooth functions like a sinusoidal waveform. Also, with the better approximation, a better derivative approximation could be found. Test functions were programmed into Matlab, and CMAC and weight smoothing CMAC were compared for their approximation error, weight spaces, and other factors.

CMAC’s function learning performs little mathematics giving it a high performance speed; this is a huge advantage for real-time applications. CMAC uses an error learning update method to adjust its weights. The error is developed from the actual function value and CMAC’s function approximation.

                                                                                      Eqn 3.1

where d is the function value and a is the actual CMAC approximation. The CMAC approximation comes from the sum of the weights in a pre-specified generalization region. The CMAC input is the same as the function index. For example, consider a sinusoidal function that has been sampled 10 times.

 

                                         

                             Fig. 3.1 Sampled sinusoid of amplitude one.

Assuming a generalization of two, the weight space would update 5 weights per input. The total number of weights for this problem becomes the number of samples plus 2*generalization. During all of the simulations, the CMAC weights were initialized to zero; all the networks started at the same point. The initialized CMAC weight space would look like the following.

The sampled sinusoid is sampled randomly. Assume that sample four is the first sample for the network to learn. The CMAC equation with a learning rate of 0.5 would find the weight updates.

                                                            Eqn 3.2

                                                                          Fig. 3.3 Updated CMAC weights.       

Notice how CMAC puts a rectangular window into the weights. This example illustrates how CMAC updates in the function learning programs for this thesis. Weight smoothing CMAC learns in a similar way, but it uses equation 2.14 developed in the previous chapter. Before making an approximation, weight smoothing CMAC smoothes over the weights corresponding to the input. The CMAC does this by using equation 2.14. The edges and the sum of the weights are held to their previous values. The

purpose of this step is to smooth the weights; it does not change the error the network will have because the sum of the weights is kept the same. Once the weights are smoothed, the error is found exactly as CMAC did in eqn. 3.1. The error multiplied by the learning rate is the training signal for CMAC. This error would be the same if the weights were never smoothed; the purpose of smoothing is to keep adjacent weights close in value.

After finding the training signal, the update window for the weights can be found. Once again, for this thesis, equation 2.14 is used to find the window. The edges are held to zero for the update window to put more penalty on the center weight. The sum of the D w is set equal to the training signal. For the error found in equation 3.2, the new weight smoothed weights are described below.

Notice that the edge weights are held closer to zero (the previous value) than the other weights; however, they do not equal zero. This is a result of fulfilling the other smoothing criteria of getting the correct sum and holding adjacent weights close in value. The update window can be manipulated by changing adjacent weight penalties in the Q matrix as described in Chapter 2. This window is added over the previously smoothed windows to provide the newly updated weights.

The new CMAC algorithm was tested learning a sinusoidal function in simulation. Fifty-one samples of one cycle of a sinusoid were available to CMAC. A sinusoid was chosen because of its smoothness, an advantage for the new algorithm. Each CMAC (original and weight smoothing) was given 100 random sample points of the function. The CMACs were set with a generalization of three and a learning rate of ½. The results from original CMAC are shown in Figure 3.5.

Fig. 3.5 One-Dim. CMAC function learning a sinusoidal waveform.

CMAC does a great job approximating the sinusoidal function. There is very little error in the approximation. However, the approximation does waver above and below the actual function value which causes problems with the derivative approximation. The derivative approximation wavers above and below the true derivative making the approximation unsmooth. The final plot shows how the weights developed by the end of training. The weights have had small spikes introduced affecting the approximation; this is what weight smoothing should eliminate. The weight smoothing results are shown in Fig. 3.6.

Fig. 3.6 One-Dim. Weight Smoothing CMAC function learning a sinusoidal waveform.

The weight smoothing result is close to the same in terms of the approximation. However, the approximation is close; weight smoothing seems to have trouble reaching the peaks of the sinusoid. Notice the difference in the derivative approximation. The derivative approximation follows the actual derivative much closer than original CMAC’s approximation. There is a problem with the endpoints of the approximation,

but this is believed to be caused by Matlab’s derivative function. The weights are also smoother proving that the method is working as expected. There are not any spikes introduced into the weight space with the new update algorithm so the weights remain smooth.

An analysis of the sum squared error between the function approximation and the derivative approximation was performed to see if either method (original CMAC and weight smoothing CMAC) was consistently more precise. The analysis results showed that in terms of sum squared error of the approximations, both methods were nearly identical. Neither method was consistently better at approximating the function. In terms of derivative approximations, weight smoothing CMAC holds a significant advantage. Figures 3.7 and 3.8 show two resulting plots of the sum squared error for the sinusoid above through 250 samples.

Fig. 3.7 Sum Squared error for the function approximation of a sinusoid

using CMAC and weight smoothing CMAC.

As figure 3.7 shows, both CMACs get to zero error by approximately the 200^th sample. Weight smoothing CMAC gets to zero sooner in this particular case; however, neither CMAC was consistent at learning in fewer samples than the other. For the same situation, the sum-squared error for the derivative approximation is shown in Fig. 3.8.

Fig. 3.8 Sum-squared error for derivative approximation of a sinusoid

using CMAC and weight smoothing CMAC.

Weight smoothing CMAC’s derivative approximation is significantly better than traditional CMAC’s. The sum-squared error reaches its minimum after approximately 50 cycles while CMAC doesn’t reach that point until the full 250 cycles. A lot of training is necessary to smooth out the function. From the earlier plots in Fig. 3.5 and 3.6, it is obvious that CMAC has a choppier approximation that moves above and below the function derivative. Weight smoothing holds the smoothness of the function derivative but has trouble getting to the function peaks.

Weight smoothing CMAC performed as expected when learning a smooth function like the sinusoid described previously. With these results, comparisons were made between the two CMACs for a function that was not smooth. It was expected that CMAC would be able to learn a non-smooth function much easier than weight smoothing CMAC. A unit step function was chosen to test the networks.

Fig. 3.9 CMAC approximating a unit step function.

Figure 3.9 shows that CMAC does a decent job of approximating the unit step function. CMAC used 600 random samples of the function to learn the above approximation. The learning rate was set to ½. A quick transition from zero to one is difficult to make in a one-dimensional weight space with generalization set to three; however, CMAC makes the transition rather quickly. To approximate the function correctly, CMAC has introduced spikes into the weight space. This is something weight smoothing will not allow.

Fig. 3.10 Weight smoothing CMAC approximating a unit step function.

Weight smoothing attempts to follow the function as best it can, but it is limited to learning functions without quick transitions. The weight smoothing CMAC was set up exactly as the CMAC that approximated the unit step function. Once again, Figure 3.10 shows weight smoothing CMAC results in a sinusoidal looking function. Notice, the weight smoothing weight space has no spikes introduced. Weight smoothing is doing the job it has been asked to do; however, it has not been designed to follow functions with quick transitions.

The two CMACs were further analyzed by increasing the dimensionality of the function being learned. A two dimensional half sphere was programmed into Matlab to see how the two CMACs would learn the function. The same learning method was used as described for the one-dimensional case. The half-sphere was chosen because of its inherent smoothness, a benefit for weight smoothing. Both CMACs had their generalization set to 3 and learning rate set to ½. The function being learned is shown in Fig. 3.11.

Fig. 3.11 Half-sphere, the function to be learned.

The function to be learned was a half-sphere of radius 3. The results of the approximations by weight smoothing CMAC and original CMAC are described in Fig. 3.12-3.17.

Fig. 3.12 Weight smoothing approximation to the half-sphere, generalization = 3, randomly sampled 500 times.

Fig. 3.13 CMAC approximation to the half-sphere, generalization = 3, randomly sampled 500 times.

Due to CMAC’s rough approximation, the generalization was increased to see what result it may have on the approximation. The larger generalization increases the cycle time for learning, but averages more weights for the approximation. Fig. 3.14 shows the new approximation.

Fig. 3.14 CMAC approximation to the half-sphere, generalization = 7, randomly sampled 500 times.

The new CMAC approximation is much smoother than the one with the generalization set to 3. Weight smoothing CMAC’s approximation is better than both of the CMAC approximations. The entire surface of the approximation is smooth with no pyramid-shaped bumps at the peak. Both CMAC approximations have pyramid-shaped growths at the peaks. The bumps are a result of spikes in CMAC’s weight space. A look at the weight spaces that produced these approximations are shown in Figures 3.15-3.17.

Fig. 3.15 Weight smoothing CMAC weight space for half-sphere approximation.

Fig. 3. 16 CMAC weight space for half-sphere approximation, gen=3.

Fig. 3.17 CMAC weight space for half-sphere approximation, gen. = 7.

The weight spaces show why the approximations resulted the way that they did. CMAC’s weights spaces have spikes in them which show through in the approximation. Weight smoothing does not allow these spikes to be introduced. Keeping these spikes to a minimum, weight smoothing is able to come up with a smooth function approximation that is better than CMACs. The weight smoothing method worked as expected.

The next step was to apply the new CMAC to a two-dimensional function that is not completely smooth. A two-dimensional pyramid was formed in Matlab to test weight smoothing CMAC’s ability to learn a function with a sharp point. Weight smoothing was limited when learning the one dimensional step function, so it is expected that the new method will not perform well with the pyramid function. The function to be learned is shown in Fig. 3.18.

                                                                                    Fig. 3.18 Pyramid, the function to be learned.

Fig. 3.19 Weight smoothing approximation to the pyramid, generalization = 3, randomly sampled 4000 times.

Fig. 3.20 CMAC approximation to the pyramid, generalization = 3, randomly sampled 4000 times.

Fig. 3.21 CMAC approximation to the pyramid, generalization = 7, randomly sampled 4000 times.

Figures 3.20 and 3.21 show that CMAC does a much better job of learning this type of function. In both CMAC approximations, the peaks of the pyramid come close to a sharp point. The approximation with generalization set to seven is rounded at the top, but it has formed the function closely. The approximation with generalization set to three has almost perfectly duplicated the function; with further training, the function could probably be learned. The weight smoothing approximation sown in Figure 3.19 is the worst of the approximations. The peak is rounded off, and it will never reach a point because of the weight smoothness. Figures 3.22-3.24 show the weight spaces for the approximations.

Fig. 3.22 Weight smoothing CMAC weight space for pyramid approximation.

Figure 3.22 shows the weight smoothing weights. It is obvious the weights are attempting to remain close in value; however, due to the steep slope of the pyramid function, this forces the weight space to form a staircase shape during training. Since adjacent weights are being held together, this function cannot be approximated well.

Fig. 3.23 CMAC weight space for pyramid approximation, gen. = 3.

Fig. 3.24 CMAC weight space for pyramid approximation, gen = 7.

The CMAC weights have spikes introduced again but the spikes in the weight space are necessary for this particular approximation. All of the CMACs were trained with the same learning rate, ½, and the same number of training samples. Generalization was the only parameter changed and it was changed for the two CMACs shown in figures 3.23 and 3.24.

Weight smoothing CMAC does its job in terms of keeping adjacent weights close in value. This can be an advantage or disadvantage dependent upon the function being learned. With a smooth function such as a sinusoid, weight smoothing provides a smooth approximation making the derivative approximation more accurate than original CMAC. Weight smoothing CMAC has difficulties with any function that is not inherently smooth. This was shown by the poor approximations of the unit step function and the pyramid function. Original CMAC can provide a much more accurate approximation of non-smooth functions assuming generalization size is set correctly. The next step is to analyze how weight smoothing compares to CMAC in a closed loop situation.