That seems to be where gradient descent points, as it usually only runs a gradient through the choice of action, leaving the context (including reactions to the action) fixed. Though, to be clear, I also expect that CDT would lead to self-modification away from CDT under current training methods, unless that is disincentivized such as through myopia.
Why do you think causal decision theory is likely under current training methods?
That seems to be where gradient descent points, as it usually only runs a gradient through the choice of action, leaving the context (including reactions to the action) fixed. Though, to be clear, I also expect that CDT would lead to self-modification away from CDT under current training methods, unless that is disincentivized such as through myopia.