Short Course on Algorithmic Differentiation

Project Information

Discipline

Mathematics and Applied Mathematics (402) 

Subdiscipline

27.03 Applied Mathematics 

Orientation

Research 

Algorithmic differentiation (AD) is a process for producing derivative computations from computer programs. The resulting derivatives are accurate to machine precision with respect to the original computation and can be used in many contexts, including uncertainty quantification, numerical optimization, nonlinear partial differential equation solvers, or the solution of inverse problems using least squares.

Established tools provide AD for various programming languages, including Fortran, C, Matlab, and C++. The tools differ in their support for the basic algorithmic approaches for AD, known as tangent-linear and adjoint mode, and also their capabilities for advanced uses such as sparsity exploitation and the computation of higher order derivatives. AD tools generally adopt one of two implementations approaches: operator overloading and source transformation. Depending on the code/problem that requires derivatives, the choice between the basic algorithmic approach, the AD tool, and the exploitation of higher-level properties such as sparsity are important to obtain the most efficiently computed derivatives especially for computationally complex numerical models.

The course will introduce the AD theory needed to motivate the choices mentioned above. The majority of the course will explore the use and illustrate the theoretical concept using four AD tools. The tools ADIC2, ADOL-C, OpenAD, and Rapsodia have all been co-authored by the presenters. Each tool is appropriate for a particular class of code/problem requiring derivatives. The goal of the course is to allow an attendee to to identify opportunities to use AD. The attendee should come away with the ability to apply a tool that we present, know the basic aspects of the AD theory that have a practical impact on the derivative computation, and understand how these theoretical aspects are related to applying an AD tool.

The course will consist of the following sections:
1) An introduction to why AD is possible
2) Operator overloading AD with an ADOL-C
3) Source transformation AD with ADIC2 and OpenAD
4) Reverse mode operator overloading AD with ADOL-C
5) Reverse mode source to source transformation with OpenAD and ADIC2
6) Exploiting sparsity in Jacobians with ADOL-C and ADIC2
7) Higher order derivatives with Rapsodia

Each section will feature an explanation of relevant AD theory and offer the chance for participants to run tools on examples provided. Attendees will be required to install a virtual machine image containing pre-installed software on their laptops in order to run the examples. Practical issues such as pre-processing input code and writing drivers will also be discussed.

The use of futuregrid allows the course attendants to access preinstalled tools in a tested environment. This avoids the typical complications associated with tool installations in varying environments and, once established, yields a basis for future courses to be held on the same topic.

The dissemination of theory and practice of algorithmic differentiation(AD) makes more potential users in the science and engineering community aware of its benefits. While the benefits, particularly for gradient computations can be tremendous, there still is a steep learning curve. A short course with examples setup in a ready-to-use environment will remove some of the obstacles potential users face, especially if they do not have an extensive computer science background. The setup on futuregrid follows some simple recipe that then permits jumping into the relevant AD topic without much delay.

Project Contact

Project Lead

Jean Utke (utke) 

Project Manager

Jean Utke (utke) 

Project Members

Sri Hari Krishna Narayanan  

Resource Requirements

Hardware System

• hotel (IBM iDataPlex at U Chicago)

 

short courses and demonstrations for individual users

a few VMs at the time of the short course

Project Timeline

Submitted

02/24/2014 - 09:58