In __getattr__ we check to see if the object has been lazy loaded or not
before we try to do attribute lookups. However, once the attributes have
been loaded, we never record that.

So this changes _init_attrs, which actually does the loading, to record
that it's done so.

This change took a loop I had that was taking 8-10s to finish down to
.5-1s.