Several papers have presented the rules to apply to a straight line program to differentiate it in direct or reverse mode. In this paper, we first recall these rules, and we try to specify the different possible strategies for the differentiation of a multi-level program in direct or reverse mode. The strategy to apply in direct mode is straight forward and can be directly extended from the straight line case. For the reverse mode, the computation of derivatives in reverse order (ie. the computation of initial variables) makes the problem much more complicated. We show that, a lot of strategies can be applied, between storing recursively the variables and recomputing them all from the initial point. In order to make the comparison of those strategies possible, we show the complexities in terms of memory requirement and execution time. The first section describes basics of automatic differentiation, the second one contains some notations and the graphical representation of a program we are using in the rest of the paper. The third section is dedicated to the description of different strategies applicable on a real code.
展开▼
