For the other parts, the question is what we actually want to let
user space configure.
Being able to specify "Very secure" "maximum secure" "average
secure" all doesn't really make sense to me.
Well, it doesn't to me either unless the user feels a cost/benefit, so
if max cost $100 per invocation and average cost nothing, most people
would chose average unless they had a very good reason not to. In your
migratable model, if we had separate limits for non-migratable and
migratable, with non-migratable being set low to prevent exhaustion,
max secure becomes a highly scarce resource, whereas average secure is
abundant then having the choice might make sense.
The discussion regarding migratability only really popped up because
this is a user-visible thing and not being able to migrate can be a
real problem (fragmentation, ZONE_MOVABLE, ...).
I think the biggest use will potentially come from hardware
acceleration. If it becomes simple to add say encryption to a secret
page with no cost, then no flag needed. However, if we only have a
limited number of keys so once we run out no more encrypted memory then
it becomes a costly resource and users might want a choice of being
backed by encryption or not.