Many problems in analysis (as well as adjacent fields such as combinatorics, theoretical computer science, and PDE) are interested in the order of growth (or decay) of some quantity that depends on one or more asymptotic parameters (such as ) – for instance, whether the quantity grows or decays linearly, quadratically, polynomially, exponentially, etc. in . In the case where these quantities grow to infinity, these growth rates had once been termed “orders of infinity” – for instance, in the 1910 book of this name by Hardy – although this term has fallen out of use in recent years. (Hardy fields are still a thing, though.)
In modern analysis, asymptotic notation is the preferred device to organize orders of infinity. There are a couple of flavors of this notation, but here is one such (a blend of Hardy’s notation and Landau’s notation). Formally, we need a parameter space equipped with a non-principal filter that describes the subsets of parameter space that are “sufficiently large” (e.g., the cofinite (Fréchet) filter on , or the cocompact filter on ). We will use to denote elements of this filter; thus, an assertion holds for sufficiently large if and only if it holds for all in some element of the filter . Given two positive quantities that are defined for sufficiently large , one can then define the following notions:
We caution that in analytic number theory and adjacent fields, the slightly different notation of Vinogradov is favored, in which would denote the concept (i) instead of (ii), and would denote a fourth concept instead of (iii). However, we will use the Hardy-Landau notation exclusively in this blog post.
Anyone who works with asymptotic notation for a while will quickly recognize that it enjoys various algebraic properties akin to the familiar algebraic properties of order on the real line. For instance, the symbols behave very much like , , , , with properties such as the following:
One also has the “tropical” property , making asymptotic notation a kind of “tropical algebra“.
However, in contrast with other standard algebraic structures (such as ordered fields) that blend order and arithmetic operations, the precise laws of orders of infinity are usually not written down as a short list of axioms. Part of this is due to cultural differences between analysis and algebra – as discussed in this essay by Gowers, analysis is often not well suited to the axiomatic approach to mathematics that algebra benefits so much from. But another reason is due to our orthodox implementation of analysis via “epsilon-delta” type concepts, such as the notion of “sufficiently large” used above, which notoriously introduces a large number of both universal and existential quantifiers into the subject (for every epsilon, there exists a delta…) which tends to interfere with the smooth application of algebraic laws (which are optimized for the universal quantifier rather than the existential quantifier).
But there is an alternate approach to analysis, namely nonstandard analysis, which rearranges the foundations so that many of quantifiers (particularly the existential ones) are concealed from view (usually via the device of ultrafilters). This makes the subject of analysis considerably more “algebraic” in nature, as the “epsilon management” that is so prevalent in orthodox analysis is now performed much more invisibly. For instance, as we shall see, in the nonstandard framework, orders of infinity acquire the algebraic structure of a totally ordered vector space that also enjoys a completeness property reminiscent, though not identical to, the completeness of the real numbers. There is also a transfer principle that allows one to convert assertions in orthodox asymptotic notation into logically equivalent assertions about nonstandard orders of infinity, allowing one to then prove asymptotic statements in a purely algebraic fashion. There is a price to pay for this “algebrization” of analysis; the spaces one works with become quite large (in particular, they tend to be “inseparable” and not “countably generated” in any reasonable fashion), and it becomes difficult to extract explicit constants (or explicit decay rates) from the asymptotic notation. However, there are some cases in which the tradeoff is worthwhile. For instance, symbolic computations tend to be easier to perform in algebraic settings than in orthodox analytic settings, so formal computations of orders of infinity (such as the ones discussed in the previous blog post) could benefit from the nonstandard approach. (See also my previous posts on nonstandard analysis for more discussion about these tradeoffs.)
Let us now describe the nonstandard approach to asymptotic notation. With the above formalism, the switch from standard to nonstandard analysis is actually quite simple: one assumes that the asymptotic filter is in fact an ultrafilter. In terms of the concept of “sufficiently large”, this means adding the following useful axiom:
This can be compared with the situation with, say, the Fréchet filter on the natural numbers , in which one has to insert some qualifier such as “after passing to a subsequence if necessary” in order to make the above axiom true.
The existence of an ultrafilter requires some weak version of the axiom of choice (specifically, the ultrafilter lemma), but for this post we shall just take the existence of ultrafilters for granted.
We can now define the nonstandard orders of infinity to be the space of all non-negative functions defined for sufficiently large , modulo the equivalence relation defined previously. That is to say, a nonstandard order of infinity is an equivalence class
of functions defined on elements of the ultrafliter. For instance, if is the natural numbers, then itself is an order of infinity, as is , , , , and so forth. But we exclude ; it will be important for us that the order of infinity is strictly positive for all sufficiently large .
We can place various familiar algebraic operations on :
With these operations, combined with the ultrafilter axiom, we see that obeys the laws of many standard algebraic structures, the proofs of which we leave as exercises for the reader:
- is a totally ordered set.
- In fact, is a totally ordered vector space, with playing the role of the zero vector, multiplication playing the role of vector addition, and scalar exponentiation playing the role of scalar multiplication. (Of course, division would then play the role of vector subtraction.) To avoid confusion, one might refer to as a log-vector space rather than a vector space to emphasize the fact that the vector structure is coming from multiplication (and exponentiation) rather than addition (and multiplication). Ordered log-vector spaces may seem like a strange and exotic concept, but they are actually already studied implicitly in analysis, albeit under the guise of other names such as log-convexity.
- is a semiring (albeit one without an additive identity element), which is idempotent: for all .
- More generally, addition can be described in purely order-theoretic terms: for all . (It would therefore be natural to call a tropical semiring, although the precise axiomatization of this term does not appear to be fully standardized currently.)
The ordered (log-)vector space structure of in particular opens up the ability to prove asymptotic implications by (log-)linear programming; this was implicitly used in my previous post. One can also use the language of (log-)linear algebra to describe further properties of various orders of infinity. For instance, if is the natural numbers, we can form the subspace
of consisting of those orders of infinity which are of polynomial type in the sense that for some ; this is then a (log)-vector subspace of , and has a canonical (log-)linear surjection that assigns to each order of infinity of polynomial type the unique real number such that , that is to say for all one has for all sufficiently large . (The existence of such an follows from the ultrafilter axiom and by a variant of the proof of the Bolzano–Weierstrass theorem; the uniqueness is also easy to establish.) The kernel of this surjection is then the log-subspace of quasilogarithmic orders of infinity – for which for all .
In addition to the above algebraic properties, the nonstandard orders of infinity also enjoy a completeness property that is reminiscent of the completeness of the real numbers. In the reals, it is true that any nested sequence of non-empty closed intervals has a non-empty intersection, which is a property closely tied to the more familiar definition of completeness as the assertion that Cauchy sequences are always convergent. This claim of course fails for open intervals: for instance, for is a nested sequence of non-empty open intervals whose intersection is empty. However, in the nonstandard orders of infinity , we have the same property for both open and closed intervals!
Lemma 1 (Completeness for arbitrary intervals) Let be a nested sequence of non-empty intervals in (which can be open, closed, or half-open). Then the intersection is non-empty.
Proof: For sake of notation we shall assume the intervals are open intervals , although much the same argument would also work for closed or half-open intervals (and then by the pigeonhole principle one can then handle nested sequences of arbitrary intervals); we leave this extension to the interested reader.
Pick an element of each , then we have whenever . In particular, one can find a set in the ultrafilter such that
whenever and , and by taking suitable intersections that these sets are nested: . If we now define to equal for (and leave undefined outside of ), one can check that for all , thus lies in the intersection of all the , giving the claim.
This property is closely related to the countable saturation and overspill properties in nonstandard analysis. From this property one might expect that has better topological structure than say the reals. This is not exactly true, because unfortunately is not metrizable (or separable, or first or second countable). It is perhaps better to view as obeying a parallel type of completeness that is neither strictly stronger nor strictly weaker than the more familiar notion of metric completeness, but is otherwise rather analogous.
