World Library  
Flag as Inappropriate
Email this Article

Fréchet derivative

Article Id: WHEBN0002735727
Reproduction Date:

Title: Fréchet derivative  
Author: World Heritage Encyclopedia
Language: English
Subject: Gradient, Functional analysis, Differential of a function, H-derivative, Banach space
Collection: Banach Spaces, Generalizations of the Derivative
Publisher: World Heritage Encyclopedia

Fréchet derivative

In mathematics, the Fréchet derivative is a derivative defined on Banach spaces. Named after Maurice Fréchet, it is commonly used to generalize the derivative of a real-valued function of a single real variable to the case of a vector-valued function of multiple real variables, and to define the functional derivative used widely in the calculus of variations.

Generally, it extends the idea of the derivative from real-valued functions of one real variable to functions on Banach spaces. The Fréchet derivative should be contrasted to the more general Gâteaux derivative which is a generalization of the classical directional derivative.

The Fréchet derivative has applications to nonlinear problems throughout mathematical analysis and physical sciences, particularly to the calculus of variations and much of nonlinear analysis and nonlinear functional analysis.


  • Definition 1
  • Properties 2
  • Finite dimensions 3
  • Relation to the Gâteaux derivative 4
  • Higher derivatives 5
  • Generalization to topological vector spaces 6
  • See also 7
  • Notes 8
  • References 9
  • External links 10


Let V and W be Banach spaces, and U\subset V be an open subset of V. A function f : UW is called Fréchet differentiable at x \in U if there exists a bounded linear operator A:V\to W such that

\lim_{h \to 0} \frac{ \| f(x + h) - f(x) - Ah \|_{W} }{ \|h\|_{V} } = 0.

The limit here is meant in the usual sense of a limit of a function defined on a metric space (see Functions on metric spaces), using V and W as the two metric spaces, and the above expression as the function of argument h in V. As a consequence, it must exist for all sequences \langle h_n\rangle_{n=1}^{\infty} of non-zero elements of V which converge to the zero vector h_n{\rightarrow}0. Equivalently, the first-order expansion holds, in Landau notation

f(x + h) = f(x) + Ah +o(h).

If there exists such an operator A, it is unique, so we write Df(x)=A and call it the (Fréchet) derivative of f at x. A function f that is Fréchet differentiable for any point of U is said to be C1 if the function

Df:U\to B(V,W) ; x \mapsto Df(x)

is continuous. Note that this is not the same as requiring that the map Df(x) : V \to W be continuous for each value of x (which is assumed).

This notion of derivative is a generalization of the ordinary derivative of a function on the real numbers f : RR since the linear maps from R to R are just multiplication by a real number. In this case, Df(x) is the function t \mapsto tf'(x) .


A function differentiable at a point is continuous at that point.

Differentiation is a linear operation in the following sense: if f and g are two maps VW which are differentiable at x, and r and s are scalars (two real or complex numbers), then rf + sg is differentiable at x with D(rf + sg)(x) = rDf(x) + sDg(x).

The chain rule is also valid in this context: if f : UY is differentiable at x in U, and g : YW is differentiable at y = f(x), then the composition g o f is differentiable in x and the derivative is the composition of the derivatives:

D(g \circ f)(x) = Dg(f(x))\circ Df(x).

Finite dimensions

The Fréchet derivative in finite-dimensional spaces is the usual derivative. In particular, it is represented in coordinates by the Jacobian matrix.

Suppose that f is a map, f:URnRm with U an open set. If f is Fréchet differentiable at a point aU, then its derivative is

Df(a) : \mathbf{R}^n \to \mathbf{R}^m \quad\mbox{with}\quad Df(a)(v) = J_f(a) \, v,

where Jf(a) denotes the Jacobian matrix of f at a.

Furthermore, the partial derivatives of f are given by

\frac{\partial f}{\partial x_i}(a) = Df(a)(e_i) = J_f(a) \, e_i,

where {ei} is the canonical basis of Rn. Since the derivative is a linear function, we have for all vectors hRn that the directional derivative of f along h is given by

Df(a)(h) = \sum_{i=1}^{n} h_i \frac{\partial f}{\partial x_i}(a).

If all partial derivatives of f exist and are continuous, then f is Fréchet differentiable (and, in fact, C1). The converse is not true: the function

f(x, y)= \begin{cases} (x^2+y^2)\sin (\frac{1}{\sqrt{x^2+y^2}}) & \mbox{ if } (x, y)\ne (0, 0)\\ 0 & \mbox{ if } (x, y)=(0, 0) \end{cases}

is Fréchet differentiable and yet fails to have continuous partial derivatives at (0,0).

Relation to the Gâteaux derivative

A function f : UVW is called Gâteaux differentiable at x ∈ U if f has a directional derivative along all directions at x. This means that there exists a function g : VW such that

g(h)=\lim_{t \to 0} \frac{ f(x + th) - f(x) }{ t }

for any chosen vector h in V, and where t is from the scalar field associated with V (usually, t is real).[1] If f is Fréchet differentiable at x, it is also Gâteaux differentiable there, and g is just the linear operator A = Df(x). However, not every Gâteaux differentiable function is Fréchet differentiable.

For example, the real-valued function f of two real variables defined by

f(x, y)= \begin{cases} \frac{x^3}{x^2+y^2} & \mbox{ if } (x, y)\ne (0, 0)\\ 0 & \mbox{ if } (x, y)=(0, 0) \end{cases}

is continuous and Gâteaux differentiable at (0, 0), with its derivative being

g(a, b)=\begin{cases} \frac{a^3}{a^2+b^2}& \mbox{ if } (a, b)\ne (0, 0)\\ 0 & \mbox{ if } (a, b)=(0, 0). \end{cases}

The function g is not a linear operator, so this function is not Fréchet differentiable.

More generally, any function of the form f(x,y) = g (r) h (\phi), where r and φ are the polar coordinates of (x,y), is continuous and Gâteaux differentiable at (0,0) if g is differentiable at 0 and h(\phi + \pi) = -h(\phi), but the Gâteaux derivative is only linear and the Fréchet derivative only exists if h is sinusoidal.

In another situation, the function f given by

f(x, y)= \begin{cases} \frac{x^3y}{x^6+y^2} & \mbox{ if } (x, y)\ne (0, 0)\\ 0 & \mbox{ if } (x, y)=(0, 0) \end{cases}

is Gâteaux differentiable at (0, 0), with its derivative there being g(ab) = 0 for all (ab), which is a linear operator. However, f is not continuous at (0, 0) (one can see by approaching the origin along the curve (t, t3)) and therefore f cannot be Fréchet differentiable at the origin.

A more subtle example is

f(x, y)= \begin{cases} \frac{x^2y}{x^4+y^2}\sqrt{x^2+y^2} & \mbox{ if } (x, y)\ne (0, 0)\\ 0 & \mbox{ if } (x, y)=(0, 0) \end{cases}

which is a continuous function that is Gâteaux differentiable at (0, 0), with its derivative being g(ab) = 0 there, which is again linear. However, f is not Fréchet differentiable. If it were, its Fréchet derivative would coincide with its Gâteaux derivative, and hence would be the zero operator; hence the limit


would have to be zero, whereas approaching the origin along the curve (t, t2) shows that this limit does not exist.

These cases can occur because the definition of the Gâteaux derivative only requires that the difference quotients converge along each direction individually, without making requirements about the rates of convergence for different directions. Thus, for a given ε, although for each direction the difference quotient is within ε of its limit in some neighborhood of the given point, these neighborhoods may be different for different directions, and there may be a sequence of directions for which these neighborhoods become arbitrarily small. If a sequence of points is chosen along these directions, the quotient in the definition of the Fréchet derivative, which considers all directions at once, may not converge. Thus, in order for a linear Gâteaux derivative to imply the existence of the Fréchet derivative, the difference quotients have to converge uniformly for all directions.

The following example only works in infinite dimensions. Let X be a Banach space, and φ a linear functional on X that is discontinuous at x = 0 (a discontinuous linear functional). Let

f(x) = \|x\|\varphi(x).\,

Then f(x) is Gâteaux differentiable at x = 0 with derivative 0. However, f(x) is not Fréchet differentiable since the limit

\lim_{x\to 0}\varphi(x)

does not exist.

Higher derivatives

If f : UVW is a differentiable function at all points in an open subset U of V, it follows that its derivative

D f : U \to L(V, W) \,

is a function from U to the space L(V, W) of all bounded linear operators from V to W. This function may also have a derivative, the second order derivative of f, which, by the definition of derivative, will be a map

D^2 f : U \to L\big(V, L(V, W)\big).

To make it easier to work with second-order derivatives, the space on the right-hand side is identified with the Banach space L2(V×V, W) of all continuous bilinear maps from V to W. An element φ in L(V, L(V, W)) is thus identified with ψ in L2(V×V, W) such that for all x and y in V

\varphi(x)(y)=\psi(x, y)\,

(intuitively: a function φ linear in x with φ(x) linear in y is the same as a bilinear function ψ in x and y).

One may differentiate

D^2 f : U \to L^2(V\times V, W) \,

again, to obtain the third order derivative, which at each point will be a trilinear map, and so on. The n-th derivative will be a function

D^n f : U \to L^n(V\times V\times \cdots \times V, W),

taking values in the Banach space of continuous multilinear maps in n arguments from V to W. Recursively, a function f is n+1 times differentiable on U if it is n times differentiable on U and for each x in U there exists a continuous multilinear map A of n+1 arguments such that the limit

\lim_{h_{n+1} \to 0} \frac{ \| D^nf(x + h_{n+1})(h_1, h_2, \dots, h_n) - D^nf(x)(h_1, h_2, \dots, h_n) - A(h_1, h_2, \dots, h_n, h_{n+1}) \| }{ \|h_{n+1}\| } = 0

exists uniformly for h1, h2, ..., hn in bounded sets in V. In that case, A is the n+1st derivative of f at x.

Moreover, we may obviously identify a member of the space L^n(V\times V\times \cdots \times V, W) with a linear map L(\bigotimes_{j=1}^n V_j, W) through the identification f(x_1,x_2, \ldots, x_n) = f(x_1 \otimes x_2 \otimes \cdots \otimes x_n) , thus viewing the derivative as a linear map.

Generalization to topological vector spaces

The notion of the Fréchet derivative can be generalized to arbitrary topological vector spaces (TVSs) X and Y. Letting U be an open subset of X that contains the origin and given a function f : U \to Y such that f(0) = 0, we first define what it means for this function to have 0 as its derivative. We say that this function f is tangent to 0 if for every open neighborhood of 0, W \sub Y there exists an open neighborhood of 0, V \sub X and a function o : \mathbb{R} \to \mathbb{R} such that \lim_{t \to 0}\frac{o(t)}{t} = 0, \, and for all t, f(t V) \sub o(t) W .

We can now remove the constraint that f(0) = 0 by defining f to be Fréchet differentiable at a point x_0 \in U if there exists a continuous linear operator \lambda : X \to Y such that f(x_0 + h) - f(x_0) - \lambda h, considered as a function of h, is tangent to 0. (Lang p. 6)

If the Fréchet derivative exists then it is unique. Furthermore, the Gâteaux derivative must also exist and be equal the Fréchet derivative in that for all v \in X, \lim_{\tau \to 0}\frac{f(x_0 + \tau v) - f(x_0)}{\tau} = f'(x_0) v, where f'(x_0) is the Fréchet derivative. A function that is Fréchet differentiable at a point is necessarily continuous there and sums and scalar multiples of Fréchet differentiable functions are differentiable so that the space of functions that are Fréchet differentiable at a point form a subspace of the functions that are continuous at that point. The chain rule also holds as does the Leibniz rule whenever Y is an algebra and a TVS in which multiplication is continuous.

See also


  1. ^ It is common to include in the definition that the resulting map g must be a continuous linear operator. We do not adopt this convention here so that the widest possible class of pathologies can be examined.


  • Cartan, Henri (1967), Calcul différentiel, Paris: Hermann, .  
  • .  
  • Lang, Serge (1995), Differential and Riemannian Manifolds, .  
  • Munkres, James R. (1991), Analysis on manifolds, .  
  • .  
  • Coleman, Rodney, ed. (2012), Calculus on Normed Vector Spaces, Universitext, .  

External links

  • B. A. Frigyik, S. Srivastava and M. R. Gupta, Introduction to Functional Derivatives, UWEE Tech Report 2008-0001.
  • This webpage is mostly about basic probability and measure theory, but there is nice chapter about Frechet derivative in Banach spaces (chapter about Jacobian formula). All the results are given with proof.
This article was sourced from Creative Commons Attribution-ShareAlike License; additional terms may apply. World Heritage Encyclopedia content is assembled from numerous content providers, Open Access Publishing, and in compliance with The Fair Access to Science and Technology Research Act (FASTR), Wikimedia Foundation, Inc., Public Library of Science, The Encyclopedia of Life, Open Book Publishers (OBP), PubMed, U.S. National Library of Medicine, National Center for Biotechnology Information, U.S. National Library of Medicine, National Institutes of Health (NIH), U.S. Department of Health & Human Services, and, which sources content from all federal, state, local, tribal, and territorial government publication portals (.gov, .mil, .edu). Funding for and content contributors is made possible from the U.S. Congress, E-Government Act of 2002.
Crowd sourced content that is contributed to World Heritage Encyclopedia is peer reviewed and edited by our editorial staff to ensure quality scholarly research articles.
By using this site, you agree to the Terms of Use and Privacy Policy. World Heritage Encyclopedia™ is a registered trademark of the World Public Library Association, a non-profit organization.

Copyright © World Library Foundation. All rights reserved. eBooks from World eBook Library are sponsored by the World Library Foundation,
a 501c(4) Member's Support Non-Profit Organization, and is NOT affiliated with any governmental agency or department.