Fitting nonlinear (small aircraft) models from experimental data

My labmate Ani Majumdar wrote up some useful notes from fitting models for our experimental data (bolded text is mine). See also Table 3.1 from my thesis:

I made progress on this. The model seems quite good now (comparing simulations using matab’s sysid toolbox for experimental flight trials, and looking at tracking performance on some preliminary experiments). Here are the things I tried in chronological order (and some lessons I learned along the way):

(1) Get parametric model from textbook (Aircraft Control and Simulation [Stevens], and Flight Dynamics [Stengel]), then do physical experiments on the plane to determine the parameters, and hope that F = ma.

The following parameters had to be measured/estimated:

– Physical dimensions (mass, moments of inertia, wing span/area, rudder/elevator dimensions, etc…)
– These are easy to just measure directly

– Relationship between servo command (0-255) and deflection angle of control surfaces
– This is simple to measure with a digital inclinometer/protractor (the reason this is not trivial is that the servo deflection gets transmitted through the wires to the actual control surface deflection.. so you actually do have to measure it)

– Relationship between throttle command and airspeed over wings
– I measured this using a hot-wire anemometer placed above the wings for different values of throttle commands (0-255). The relationship looks roughly like airspeed = sqrt(c*throttle_command), which is consistent with theory.

– Relationship between throttle command and airspeed over elevator/rudder
– Same experiment as above (the actual airspeed is different though).

– Relationship between throttle command and thrust produced
– I put the plane on a low-friction track and used a digital force-meter (really a digital scale) to measure the thrust for different throttle commands. The plane pulls on the force-meter and you can read out the force values. This is a scary experiment because there’s the constant danger of the plane getting loose and flying off the track! You also have to account for static friction. You can either just look at the value of predicted thrust at 0 and just subtract this off, or you can also tilt the track to see when the plane starts to slide (this can be used to compute the force: m*g*sin(theta)). In my case, these were very close. The relationship was linear, but the thrust saturates at around a throttle command of 140.

– Aerodynamic parameters (e.g. lift/drag coefficients, damping terms, moment induced by angle derivatives).
– I could have put the plane in a wind-tunnel for some of these, but decided not to. I ended up using a flat plate model.

(2) Use matlab’s sysid toolbox to fit stuff

Approach (1) wasn’t giving me particularly good results. So, I tried just fitting everything with matlab’s sysid toolbox (prediction error minimization, pem.m). I collected a bunch of experimental flight trials with sinusoid-like inputs (open-loop, of course).

This didn’t work too well either.

(3) Account for delay.

Finally, I remembered that Tim and Andy noticed a delay of about 50 ms when they were doing their early prophang experiments (they tried to determine this with some physical experiments). So, I took the inputs from my experimental trials and shifted the control input tapes by around 50 ms (actually 57 ms).

When I used matlab’s sysid toolbox to fit parameters after shifting the control input commands to account for delay, the fits were extremely good!

I noticed this when I was fitting a linear model to do LQR for prophang, back in October 2011. The fits are not good if you don’t take delay into account (duh). Got to remember this the next time.

Here is a summary of what worked, and how I would go about doing it if I had to do it again (which I probably will have to at some point on the new hardware):

(1) Do some physical experiments to determine the stuff that is easy to determine. And to get the parameteric form of the dependences (e.g. thrust is linear with the 0-255 prop command, and it saturates around 140).

(2) Use matlab’s sysid toolbox to fit parameters with bounds on the parameters after having accounted for delay. The bounds are important. If some of your outputs are not excited (e.g. yaw on our plane and pitch to some degree), pem will make your parameters aphysical (e.g. extremely large) in order to eke out a small amount of prediction performance. As an extreme example, let’s say you have a system:

xdot = f(x) + c1*u1 + c2*u2.

Let’s say all the control inputs you used for sysid had very small u2 (let’s say 0 just for the sake of arguments). Then, any value of c2 is consistent with the data.. so you can just make c2 = 10^6 for example, which is physically meaningless. If u2 is small (but non-zero) and is overshadowed by the effect that u1 has, then some non-physical values of c2 could lead to very slight improvements in prediction errors and could be preferred over physical values.

So, bounding parameters to keep them physically meaningful is a good idea in my experience. Of course, ideally you would just collect enough data that this is not a problem, but this can be hard (especially for us since the experimental arena is quite confined).

Another good thing to do is to make sure you can overfit with the model you have. This sounds stupid, but is actually really important. If the parametric model you have is incorrect (or if you didn’t account for delay), then your model is incapable of explaining even small amounts of data. So, as a sanity check, take just a small piece of data and see if you can fit parameters to explain that piece of data perfectly. If you can’t, something is wrong. I was seeing this before I accounted for delay and this tipped me off that I was missing something fundamental. (it took me a little while longer to realize that it was delay :). I also tried this on the Acrobot on the first day I tried to use Elena’s model to do sysid with. Something was off again (in this case, it was a sign error – my fault, not hers).

(3) Finally, account for delay when you run the controller by predicting the state forwards in time to compute the control input. This works well for Joe, and seems to be working well for me so far.