This post was revised with the help of ChatGPT.

This post walks through the continuity equation [1], from physical intuition to the mathematical statement. We derive it step by step and explain every quantity involved.

1. Motivation: conservation in physical systems

Many physical quantities are conserved, for example:

• mass in a closed system;
• charge in electromagnetism;
• energy in an isolated system;
• the number of particles in a fluid.

Although these quantities are conserved globally, they vary locally in space and time. To describe their local behavior, we use density functions. A density tells us how much of the conserved quantity there is per unit volume at each point and time:

• mass density \(\rho(\boldsymbol{x}, t)\);
• charge density \(\rho_q(\boldsymbol{x}, t)\);
• energy density \(\epsilon(\boldsymbol{x}, t)\);
• particle number density \(n(\boldsymbol{x}, t)\).

Each of these is a function \(\mathbb{R}^n \times \mathbb{R}_{\ge 0} \to \mathbb{R}\).

To model how a density evolves, imagine the quantity being carried by a flow. A tiny fluid parcel at position \(\boldsymbol{x}\) and time \(t\) moves with velocity \(\boldsymbol{v}(\boldsymbol{x}, t)\) and carries the local amount of the quantity with it.

Our goal is a local law describing how the density changes over space and time. This law is the continuity equation.

2. Basic definitions

Let:

• \(\rho(\boldsymbol{x}, t): \mathbb{R}^n \times \mathbb{R}_{\ge 0} \to \mathbb{R}\) be the scalar density of a conserved quantity;
• \(\boldsymbol{J}(\boldsymbol{x}, t): \mathbb{R}^n \times \mathbb{R}_{\ge 0} \to \mathbb{R}^n\) be the flux. Its component along a unit vector \(\boldsymbol{n}\) is the amount of the quantity crossing a unit area perpendicular to \(\boldsymbol{n}\) per unit time.

When the quantity is transported by a velocity field \(\boldsymbol{v}(\boldsymbol{x}, t)\), the flux is

In words, flux is density times velocity: how much passes through depends on how much is there and how fast it moves.

Now fix a region \(V \subset \mathbb{R}^n\) with a smooth boundary \(\partial V\).

• The total amount inside \(V\) at time \(t\) is

  \[Q_V(t) = \int_V \rho(\boldsymbol{x}, t) \, d\boldsymbol{x} .\]

• Its rate of change is

  \[\frac{d}{dt} Q_V(t) = \frac{d}{dt} \int_V \rho(\boldsymbol{x}, t) \, d\boldsymbol{x} .\]

  Since the quantity is conserved (there are no sources or sinks), this change can only come from flux across the boundary \(\partial V\).

3. Flux across the boundary

Each point \(\boldsymbol{x} \in \partial V\) has an outward unit normal \(\boldsymbol{n}(\boldsymbol{x})\). Let \(dS\) be the surface area element, and let \(d\boldsymbol{S} = \boldsymbol{n} \, dS\) be the oriented surface element, pointing outward.

The net outflow through \(\partial V\) per unit time is

The dot product keeps only the component of the flux normal to the surface; flow tangent to the surface does not cross it. The integral is positive when more of the quantity leaves than enters.

Conservation says that the amount inside decreases exactly at the rate it flows out:

This is the integral form of the conservation law.

4. Gauss’s divergence theorem

To turn the boundary integral into a volume integral, we use the divergence theorem [2]. It holds for a continuously differentiable field \(\boldsymbol{J}\):

The theorem says that the total outward flux through the boundary equals the total divergence inside:

• the left-hand side adds up what flows out of each patch of the surface;
• the right-hand side adds up the local “spreading out” of the flow at each point inside \(V\).

Substituting into the conservation law gives

5. From integral form to differential form

If \(\rho\) is smooth enough (see Section 8), we may move the time derivative inside the integral:

Thus

and moving everything to one side,

This holds for every region \(V\). If the integrand \(g = \partial_t \rho + \nabla \cdot \boldsymbol{J}\) is continuous, it must vanish pointwise. To see why, suppose \(g(\boldsymbol{x}_0, t) > 0\) at some point. By continuity, \(g > 0\) on a small ball \(B\) around \(\boldsymbol{x}_0\), and then \(\int_B g \, d\boldsymbol{x} > 0\), a contradiction. The same argument works if \(g(\boldsymbol{x}_0, t) < 0\). Therefore

6. The continuity equation

Using \(\boldsymbol{J} = \rho \, \boldsymbol{v}\), we obtain the continuity equation:

It expresses the local conservation of the quantity carried by the flow.

Expanded form. By the product rule, \(\nabla \cdot (\rho \boldsymbol{v}) = \boldsymbol{v} \cdot \nabla \rho + \rho \, \nabla \cdot \boldsymbol{v}\), so

The left-hand side is the material derivative \(D\rho / Dt\), the rate of change of density seen by an observer moving with a parcel. So the density following a parcel changes only when the flow compresses (\(\nabla \cdot \boldsymbol{v} < 0\)) or expands (\(\nabla \cdot \boldsymbol{v} > 0\)). For an incompressible flow (\(\nabla \cdot \boldsymbol{v} = 0\)), the density along each parcel stays constant.

7. Divergence: geometric meaning

The divergence of a vector field is

By the continuity equation, \(\partial_t \rho = -\nabla \cdot \boldsymbol{J}\). So:

• if \(\nabla \cdot \boldsymbol{J} > 0\) at a point, more flows out than in, and \(\rho\) decreases there;
• if \(\nabla \cdot \boldsymbol{J} < 0\), more flows in than out, and \(\rho\) increases;
• if \(\nabla \cdot \boldsymbol{J} = 0\), inflow and outflow balance, and \(\rho\) does not change there.

Geometrically, divergence is a local measure of how much a vector field “spreads out” at a point. This follows from the divergence theorem: shrink \(V\) to a small ball \(B_r(\boldsymbol{x})\) around a point, and

i.e. divergence is the outward flux per unit volume.

8. Why can we swap the derivative and the integral?

We used

By definition of the derivative,

The step \((\star)\), which swaps the limit and the integral, is the only one that needs justification. A sufficient condition is that \(V\) is bounded and \(\partial_t \rho\) is continuous on \(\overline{V} \times [t - \delta, t + \delta]\). By the mean value theorem, the difference quotient equals \(\partial_t \rho(\boldsymbol{x}, t + \theta \Delta t)\) for some \(\theta \in (0, 1)\). This is uniformly bounded, so the dominated convergence theorem applies. (The region \(V\) is fixed in time. For a region that moves with the flow, you would need the Reynolds transport theorem instead.)

9. Summary

We began with:

• a physical idea: a conserved quantity;
• a model: the quantity is carried by a flow with velocity \(\boldsymbol{v}\);
• a density \(\rho(\boldsymbol{x}, t)\) and a flux \(\boldsymbol{J} = \rho \boldsymbol{v}\);
• a balance law: rate of change inside a region = \(-\)(net outflow through its boundary);
• the divergence theorem: surface integral \(\to\) volume integral;
• swapping derivative and integral: \(\frac{d}{dt}\int_V = \int_V \partial_t\);
• “zero integral over every region” \(\Rightarrow\) “zero integrand.”

We arrived at

The continuity equation appears whenever we model a conserved quantity moving through space. It captures one simple idea:

Any local change in the quantity must be explained by its flow into or out of that region.

It is fundamental across physics and mathematics: in fluid mechanics, electromagnetism (conservation of charge), quantum mechanics (conservation of probability), and, more recently, in the flow-matching and diffusion models of machine learning, where it describes how a probability density is transported by a learned velocity field.

References

This is a brief review of some topics from ECE 220. All code below is LC-3 assembly; the code blocks use NASM syntax highlighting, which is close enough.

I. Purpose and description

1. Purpose

This program reviews:

1. LC-3 basics, including I/O;
2. the run-time stack;
3. recursion in LC-3;
4. linked lists in LC-3;
5. binary trees in LC-3.

2. Description

1. The program is a direct LC-3 translation of a C program that prints a directory structure.
2. The directory tree is stored in memory in fixed data structures. The program prints it to the console in a pre-order traversal, in the format shown in Desired output.
3. To show how C is translated into assembly, every user-defined C function uses the run-time stack. I also wrote my own versions of the OUT and PUTS system calls, which C’s printf() would ultimately rely on. They are called PRINT_A_CHAR and PRINT_A_STRING. Like real system calls, they do not use the run-time stack. They work the same way as the OUT and PUTS trap service routines in the LC-3 operating system.
4. The test case is not exhaustive. Gemini generated most of it, and I fixed a few syntax errors so that it assembles. Feel free to add more test cases.

3. Data structure design

(1) Files and folders

A file node (a node of a singly linked list):

• [0] next pointer: the next file in the same folder (NULL if none);
• [1] pointer to the file name (.STRINGZ).

A folder node (a node of a binary tree):

• [0] left pointer: the left child (NULL if none);
• [1] right pointer: the right child (NULL if none);
• [2] file pointer: the head of this folder’s file list (NULL if none);
• [3] pointer to the folder name (.STRINGZ).

Both children of a folder are sub-folders of it. The tree is a plain binary tree, not a “left-child, right-sibling” encoding.

(2) Folder tree (test case)

                         root
                /                   \
          projects                  media
          /      \               /         \
    c_project    blog      documents       photos
     /     \                /      \           \
 utils  python_project  reports  presentations  travel
                                                    \
                                                 sub_travel

(3) Files in each folder (test case)

• root: config.sys
• projects: README.md
• c_project: main.c → Makefile
• utils: helper.c
• python_project: main.py → requirements.txt
• blog: index.md
• media: (empty)
• documents: meeting_notes.docx → todo_list.txt
• reports: q1_report.pdf → budget_summary.xlsx
• presentations: project_kickoff.pptx
• photos: IMG001.jpg → IMG002.jpg
• travel: trip.txt → map.png
• sub_travel: packing_list.md

(4) C structs and function prototypes

typedef struct f {
    struct f* nextfile;
    const char* filename;
} file_node;

typedef struct F {
    struct F* left;
    struct F* right;
    file_node* file;
    const char* foldername;
} folder_node;

// See the comment before each subroutine for its full definition.
void print_space(int n);
void print_file(int num, file_node* file);
void print_folder(int depth, folder_node* folder);

4. Desired output

root
    file: config.sys
    projects
        file: README.md
        c_project
            file: main.c Makefile
            utils
                file: helper.c
            python_project
                file: main.py requirements.txt
        blog
            file: index.md
    media
        documents
            file: meeting_notes.docx todo_list.txt
            reports
                file: q1_report.pdf budget_summary.xlsx
            presentations
                file: project_kickoff.pptx
        photos
            file: IMG001.jpg IMG002.jpg
            travel
                file: trip.txt map.png
                sub_travel
                    file: packing_list.md

5. Conventions

These conventions follow the run-time stack model from our textbook [1], as best I remember it. Please let me know if you spot a mistake.

Registers:

• R4: global data pointer (not needed in this program)
• R5: frame pointer (points to the first local variable)
• R6: stack pointer (points to the top element of the stack)
• R7: return address (written by JSR and TRAP)

The stack frame of one function call (the stack grows toward lower addresses):

Other conventions:

1. Arguments and local variables are accessed through R5 plus an offset, never through R6, because R6 moves whenever something is pushed.
2. The caller pushes arguments from right to left, so the first argument ends up nearest the frame. On return, the callee leaves R6 pointing at the return value, and the caller pops the return value and the arguments.

Notes on this program:

1. Each of print_folder(), print_file(), and print_space() has exactly one local variable (current, next, and i), stored at M[R5 + 0].
2. main() does not build a frame and leaves R5 unset. This is fine here, because main never reads R5, and every callee saves and restores R5 itself.

II. LC-3 translation of the C functions

1. main()

;;; ---------------------------- MAIN ------------------------------------
;;; int main() {
;;;     folder_node* root = get_info();  // get_info() is too complex to write in LC-3;
;;;                                      // we assume the tree is already in memory.
;;;     print_folder(0, root);           // see the next subroutine
;;;     return 0;
;;; }
;;;
;;; R0, R1: temporary registers.
    .ORIG   x3000

    ;; initialize the stack pointer (R6) to the bottom of an empty stack
    LD  R6, STACK_BOTTOM
    ;; main has no stack frame, so R5 is not set here

    ;; push the arguments of print_folder(0, root), right to left
    LEA R1, FOLDER_root
    ADD R6, R6, #-1             ; push root
    STR R1, R6, #0
    AND R0, R0, #0
    ADD R6, R6, #-1             ; push depth = 0
    STR R0, R6, #0

    ;; call print_folder()
    JSR PRINT_FOLDER

    ;; pop the return value (unused, void) and the two arguments
    ADD R6, R6, #3
    HALT

STACK_BOTTOM
    .FILL   x4000               ; empty stack: the first push writes to x3FFF

2. print_folder()

;;; ------------------------- PRINT_FOLDER --------------------------------
;;; void print_folder(int depth, folder_node* node) {
;;;     // This check only matters for the root: the recursive calls below
;;;     // never pass a NULL node.
;;;     if (node == NULL) {
;;;         return;
;;;     }
;;;
;;;     print_space(depth);
;;;     folder_node* current = node;
;;;     printf("%s", current->foldername);
;;;
;;;     // pre-order traversal
;;;     if (current->file != NULL) {
;;;         printf("\n");
;;;         print_file(depth + 1, current->file);
;;;     }
;;;     if (current->left != NULL) {
;;;         printf("\n");
;;;         print_folder(depth + 1, current->left);
;;;     }
;;;     if (current->right != NULL) {
;;;         printf("\n");
;;;         print_folder(depth + 1, current->right);
;;;     }
;;; }
;;;
;;; R0: temporary register (caller-saved).
;;;
PRINT_FOLDER
;;; bookkeeping: build the stack frame
    ADD R6, R6, #-2             ; skip the return-value slot, then
    STR R7, R6, #0              ;   save the return address
    ADD R6, R6, #-1             ; save the caller's frame pointer
    STR R5, R6, #0
    ADD R6, R6, #-1             ; allocate local variable `current`
    ADD R5, R6, #0              ; R5 = new frame pointer

;;; body
    ;; if (node == NULL) return;
    LDR R0, R5, #5
    BRz PRINT_FOLDER_DONE

    ;; print_space(depth);
    LDR R0, R5, #4
    ADD R6, R6, #-1             ; push depth
    STR R0, R6, #0
    JSR PRINT_SPACE
    ADD R6, R6, #2              ; pop the return value and the argument

    ;; folder_node* current = node;
    LDR R0, R5, #5
    STR R0, R5, #0

    ;; printf("%s", current->foldername);
    LDR R0, R5, #0
    LDR R0, R0, #3
    JSR PRINT_A_STRING

PRINT_FOLDER_FILE
;;; if (current->file != NULL) {
;;;     printf("\n");
;;;     print_file(depth + 1, current->file);
;;; }
    LDR R0, R5, #0              ; current->file == NULL ?
    LDR R0, R0, #2
    BRz PRINT_FOLDER_LEFT
    LD  R0, PRINT_FOLDER_NEWLINE
    JSR PRINT_A_CHAR

    LDR R0, R5, #0              ; push current->file
    LDR R0, R0, #2
    ADD R6, R6, #-1
    STR R0, R6, #0
    LDR R0, R5, #4              ; push depth + 1
    ADD R0, R0, #1
    ADD R6, R6, #-1
    STR R0, R6, #0
    JSR PRINT_FILE
    ADD R6, R6, #3              ; pop the return value and two arguments

PRINT_FOLDER_LEFT
;;; if (current->left != NULL) {
;;;     printf("\n");
;;;     print_folder(depth + 1, current->left);
;;; }
    LDR R0, R5, #0              ; current->left == NULL ?
    LDR R0, R0, #0
    BRz PRINT_FOLDER_RIGHT
    LD  R0, PRINT_FOLDER_NEWLINE
    JSR PRINT_A_CHAR

    LDR R0, R5, #0              ; push current->left
    LDR R0, R0, #0
    ADD R6, R6, #-1
    STR R0, R6, #0
    LDR R0, R5, #4              ; push depth + 1
    ADD R0, R0, #1
    ADD R6, R6, #-1
    STR R0, R6, #0
    JSR PRINT_FOLDER            ; recursive call
    ADD R6, R6, #3              ; pop the return value and two arguments

PRINT_FOLDER_RIGHT
;;; if (current->right != NULL) {
;;;     printf("\n");
;;;     print_folder(depth + 1, current->right);
;;; }
    LDR R0, R5, #0              ; current->right == NULL ?
    LDR R0, R0, #1
    BRz PRINT_FOLDER_DONE
    LD  R0, PRINT_FOLDER_NEWLINE
    JSR PRINT_A_CHAR

    LDR R0, R5, #0              ; push current->right
    LDR R0, R0, #1
    ADD R6, R6, #-1
    STR R0, R6, #0
    LDR R0, R5, #4              ; push depth + 1
    ADD R0, R0, #1
    ADD R6, R6, #-1
    STR R0, R6, #0
    JSR PRINT_FOLDER            ; recursive call
    ADD R6, R6, #3              ; pop the return value and two arguments

PRINT_FOLDER_DONE
;;; tear down the frame and return
    ADD R6, R6, #1              ; pop the local variable
    LDR R5, R6, #0              ; restore the caller's frame pointer
    ADD R6, R6, #1
    LDR R7, R6, #0              ; restore the return address
    ADD R6, R6, #1              ; R6 -> return-value slot
    RET

PRINT_FOLDER_NEWLINE
    .FILL   #10

3. print_file()

;;; -------------------------- PRINT_FILE ---------------------------------
;;; void print_file(int num, file_node* file) {
;;;     if (file == NULL) {
;;;         return;
;;;     }
;;;     file_node* next = file;
;;;
;;;     print_space(num);
;;;     printf("file: ");
;;;
;;;     while (next != NULL) {
;;;         printf("%s ", next->filename);
;;;         next = next->nextfile;
;;;     }
;;; }
;;;
;;; R0: temporary register (saved and restored by the callee here).
PRINT_FILE
;;; bookkeeping: build the stack frame
    ADD R6, R6, #-2
    STR R7, R6, #0              ; save the return address
    ADD R6, R6, #-1
    STR R5, R6, #0              ; save the caller's frame pointer
    ADD R6, R6, #-1             ; allocate local variable `next`
    ADD R5, R6, #0              ; R5 = new frame pointer

;;; body
    ST  R0, PRINT_FILE_SAVE_R0

    ;; if (file == NULL) return;  file_node* next = file;
    LDR R0, R5, #5
    BRz PRINT_FILE_DONE
    STR R0, R5, #0

    ;; print_space(num);
    LDR R0, R5, #4
    ADD R6, R6, #-1             ; push num
    STR R0, R6, #0
    JSR PRINT_SPACE
    ADD R6, R6, #2              ; pop the return value and the argument

    ;; printf("file: ");
    LEA R0, PRINT_FILE_PROMPT
    JSR PRINT_A_STRING

PRINT_FILE_LOOP
    LDR R0, R5, #0              ; while (next != NULL)
    BRz PRINT_FILE_DONE
    LDR R0, R0, #1              ; printf("%s ", next->filename);
    JSR PRINT_A_STRING
    LD  R0, PRINT_FILE_SPACE
    JSR PRINT_A_CHAR
    LDR R0, R5, #0              ; next = next->nextfile;
    LDR R0, R0, #0
    STR R0, R5, #0
    BRnzp   PRINT_FILE_LOOP

PRINT_FILE_DONE
;;; tear down the frame and return
    LD  R0, PRINT_FILE_SAVE_R0
    ADD R6, R6, #1              ; pop the local variable
    LDR R5, R6, #0              ; restore the caller's frame pointer
    ADD R6, R6, #1
    LDR R7, R6, #0              ; restore the return address
    ADD R6, R6, #1              ; R6 -> return-value slot
    RET

PRINT_FILE_SPACE
    .FILL   #32
PRINT_FILE_PROMPT
    .STRINGZ "file: "
PRINT_FILE_SAVE_R0
    .FILL   x0000

Saving R0 in a fixed memory location like PRINT_FILE_SAVE_R0 only works because print_file() is not recursive. A recursive function would overwrite the saved value on each nested call, so it would have to save registers on the stack instead.

4. print_space()

;;; -------------------------- PRINT_SPACE --------------------------------
;;; void print_space(int n) {
;;;     int i = 4 * n;
;;;     while (i > 0) {
;;;         printf(" ");
;;;         i--;
;;;     }
;;; }
;;;
;;; R0: temporary register (saved and restored by the callee here).
PRINT_SPACE
;;; bookkeeping: build the stack frame
    ADD R6, R6, #-2
    STR R7, R6, #0              ; save the return address
    ADD R6, R6, #-1
    STR R5, R6, #0              ; save the caller's frame pointer
    ADD R6, R6, #-1             ; allocate local variable `i`
    ADD R5, R6, #0              ; R5 = new frame pointer

;;; body
    ST  R0, PRINT_SPACE_SAVE_R0
    LDR R0, R5, #4              ; int i = 4 * n;
    ADD R0, R0, R0
    ADD R0, R0, R0
    STR R0, R5, #0

PRINT_SPACE_LOOP
    LDR R0, R5, #0              ; while (i > 0)
    BRnz    PRINT_SPACE_DONE
    LD  R0, PRINT_SPACE_SPACE   ; printf(" ");
    JSR PRINT_A_CHAR
    LDR R0, R5, #0              ; i--;
    ADD R0, R0, #-1
    STR R0, R5, #0
    BRnzp   PRINT_SPACE_LOOP

PRINT_SPACE_DONE
;;; tear down the frame and return
    LD  R0, PRINT_SPACE_SAVE_R0
    ADD R6, R6, #1              ; pop the local variable
    LDR R5, R6, #0              ; restore the caller's frame pointer
    ADD R6, R6, #1
    LDR R7, R6, #0              ; restore the return address
    ADD R6, R6, #1              ; R6 -> return-value slot
    RET

PRINT_SPACE_SPACE
    .FILL   #32
PRINT_SPACE_SAVE_R0
    .FILL   x0000

III. PRINT_A_CHAR and PRINT_A_STRING “system calls”

;;; ------------------------- PRINT_A_CHAR --------------------------------
;;; Same as OUT: write the character in R0 to the display.
;;; Polls the display status register (DSR) until the display is ready,
;;; then writes R0 to the display data register (DDR).
PRINT_A_CHAR
    ST  R1, PRINT_A_CHAR_SAVE_R1
PRINT_A_CHAR_POLL
    LDI R1, DSR                 ; bit 15 of DSR = 1 when the display is ready
    BRzp    PRINT_A_CHAR_POLL
    STI R0, DDR
    LD  R1, PRINT_A_CHAR_SAVE_R1
    RET

PRINT_A_CHAR_SAVE_R1
    .FILL   x0000
DSR
    .FILL   xFE04
DDR
    .FILL   xFE06

;;; ------------------------ PRINT_A_STRING -------------------------------
;;; Same as PUTS: write the null-terminated string starting at address R0.
;;; It calls PRINT_A_CHAR, which overwrites R7, so R7 must be saved first.
PRINT_A_STRING
    ST  R0, PRINT_A_STRING_SAVE_R0
    ST  R1, PRINT_A_STRING_SAVE_R1
    ST  R7, PRINT_A_STRING_SAVE_R7
    ADD R1, R0, #0              ; R1 = pointer to the current character

PRINT_A_STRING_LOOP
    LDR R0, R1, #0
    BRz PRINT_A_STRING_DONE     ; stop at the null terminator
    JSR PRINT_A_CHAR
    ADD R1, R1, #1
    BRnzp   PRINT_A_STRING_LOOP

PRINT_A_STRING_DONE
    LD  R0, PRINT_A_STRING_SAVE_R0
    LD  R1, PRINT_A_STRING_SAVE_R1
    LD  R7, PRINT_A_STRING_SAVE_R7
    RET

PRINT_A_STRING_SAVE_R0
    .FILL   x0000
PRINT_A_STRING_SAVE_R1
    .FILL   x0000
PRINT_A_STRING_SAVE_R7
    .FILL   x0000

IV. Data

;;; -------------------------folder_info-----------------------------------
FOLDER_root
    .FILL FOLDER_projects       ; left
    .FILL FOLDER_media          ; right
    .FILL FILE_root             ; file
    .FILL FOLDER_NAME_root      ; foldername
FOLDER_NAME_root
    .STRINGZ "root"

FOLDER_projects
    .FILL FOLDER_c_project
    .FILL FOLDER_blog
    .FILL FILE_projects
    .FILL FOLDER_NAME_projects
FOLDER_NAME_projects
    .STRINGZ "projects"

FOLDER_c_project
    .FILL FOLDER_utils
    .FILL FOLDER_python_project
    .FILL FILE_c_project
    .FILL FOLDER_NAME_c_project
FOLDER_NAME_c_project
    .STRINGZ "c_project"

FOLDER_utils
    .FILL x0000
    .FILL x0000
    .FILL FILE_utils
    .FILL FOLDER_NAME_utils
FOLDER_NAME_utils
    .STRINGZ "utils"

FOLDER_blog
    .FILL x0000
    .FILL x0000
    .FILL FILE_blog
    .FILL FOLDER_NAME_blog
FOLDER_NAME_blog
    .STRINGZ "blog"

FOLDER_media
    .FILL FOLDER_documents
    .FILL FOLDER_photos
    .FILL x0000
    .FILL FOLDER_NAME_media
FOLDER_NAME_media
    .STRINGZ "media"

FOLDER_photos
    .FILL x0000
    .FILL FOLDER_travel
    .FILL FILE_photos
    .FILL FOLDER_NAME_photos
FOLDER_NAME_photos
    .STRINGZ "photos"

FOLDER_travel
    .FILL x0000
    .FILL FOLDER_sub_travel
    .FILL FILE_travel
    .FILL FOLDER_NAME_travel
FOLDER_NAME_travel
    .STRINGZ "travel"

FOLDER_python_project
    .FILL x0000
    .FILL x0000
    .FILL FILE_python_project
    .FILL FOLDER_NAME_python_project
FOLDER_NAME_python_project
    .STRINGZ "python_project"

FOLDER_documents
    .FILL FOLDER_reports
    .FILL FOLDER_presentations
    .FILL FILE_documents
    .FILL FOLDER_NAME_documents
FOLDER_NAME_documents
    .STRINGZ "documents"

FOLDER_reports
    .FILL x0000
    .FILL x0000
    .FILL FILE_reports
    .FILL FOLDER_NAME_reports
FOLDER_NAME_reports
    .STRINGZ "reports"

FOLDER_presentations
    .FILL x0000
    .FILL x0000
    .FILL FILE_presentations
    .FILL FOLDER_NAME_presentations
FOLDER_NAME_presentations
    .STRINGZ "presentations"

FOLDER_sub_travel
    .FILL x0000
    .FILL x0000
    .FILL FILE_sub_travel
    .FILL FOLDER_NAME_sub_travel
FOLDER_NAME_sub_travel
    .STRINGZ "sub_travel"

;;; ------------------------file_info-------------------------------------
FILE_root
    .FILL x0000                 ; nextfile
    .FILL FILE_NAME_root        ; filename
FILE_NAME_root
    .STRINGZ "config.sys"

FILE_projects
    .FILL x0000
    .FILL FILE_NAME_projects
FILE_NAME_projects
    .STRINGZ "README.md"

FILE_c_project
    .FILL FILE_c_project_2
    .FILL FILE_NAME_c_project1
FILE_NAME_c_project1
    .STRINGZ "main.c"

FILE_c_project_2
    .FILL x0000
    .FILL FILE_NAME_c_project2
FILE_NAME_c_project2
    .STRINGZ "Makefile"

FILE_utils
    .FILL x0000
    .FILL FILE_NAME_utils
FILE_NAME_utils
    .STRINGZ "helper.c"

FILE_blog
    .FILL x0000
    .FILL FILE_NAME_blog
FILE_NAME_blog
    .STRINGZ "index.md"

FILE_photos
    .FILL FILE_photos_2
    .FILL FILE_NAME_photos1
FILE_NAME_photos1
    .STRINGZ "IMG001.jpg"

FILE_photos_2
    .FILL x0000
    .FILL FILE_NAME_photos2
FILE_NAME_photos2
    .STRINGZ "IMG002.jpg"

FILE_travel
    .FILL FILE_travel_2
    .FILL FILE_NAME_travel1
FILE_NAME_travel1
    .STRINGZ "trip.txt"

FILE_travel_2
    .FILL x0000
    .FILL FILE_NAME_travel2
FILE_NAME_travel2
    .STRINGZ "map.png"

FILE_python_project
    .FILL FILE_python_project_2
    .FILL FILE_NAME_python_project1
FILE_NAME_python_project1
    .STRINGZ "main.py"

FILE_python_project_2
    .FILL x0000
    .FILL FILE_NAME_python_project2
FILE_NAME_python_project2
    .STRINGZ "requirements.txt"

FILE_documents
    .FILL FILE_documents_2
    .FILL FILE_NAME_documents1
FILE_NAME_documents1
    .STRINGZ "meeting_notes.docx"

FILE_documents_2
    .FILL x0000
    .FILL FILE_NAME_documents_2
FILE_NAME_documents_2
    .STRINGZ "todo_list.txt"

FILE_reports
    .FILL FILE_reports_2
    .FILL FILE_NAME_reports1
FILE_NAME_reports1
    .STRINGZ "q1_report.pdf"

FILE_reports_2
    .FILL x0000
    .FILL FILE_NAME_reports2
FILE_NAME_reports2
    .STRINGZ "budget_summary.xlsx"

FILE_presentations
    .FILL x0000
    .FILL FILE_NAME_presentations1
FILE_NAME_presentations1
    .STRINGZ "project_kickoff.pptx"

FILE_sub_travel
    .FILL x0000
    .FILL FILE_NAME_sub_travel
FILE_NAME_sub_travel
    .STRINGZ "packing_list.md"

    .END

References

This post introduces the Gaussian integral, starting from basic integration and building up to a probabilistic and linear-algebraic view of it.

I. The fundamental integral

The most basic Gaussian integral is

or equivalently, in the form of the standard Gaussian density,

Here is how to compute it. Write \(I\) for the integral.

Step 1: square it

The integrand is positive, so Tonelli’s theorem allows us to combine the two integrals into a double integral.

Step 2: switch to polar coordinates

Let

The Jacobian matrix of this map is

so the area element transforms as

Step 3: integrate

Since \(I > 0\),

Shifting and scaling. Substituting \(x \mapsto (x - b)/\sqrt{a}\) gives, for any \(a > 0\) and \(b \in \mathbb{R}\),

or, in the form of a Gaussian density,

II. From one dimension to many

Now we generalize the univariate Gaussian integral to \(n\) dimensions.

Notation.

1. \(\boldsymbol{x} = (x_1, \dots, x_n)^\top \in \mathbb{R}^n\) is a column vector.
2. \(d\boldsymbol{x}\) denotes the volume element \(dx_1 \cdots dx_n\).
3. For \(f: \mathbb{R}^n \to \mathbb{R}\), \(\displaystyle \int_{\mathbb{R}^n} f(\boldsymbol{x}) \, d \boldsymbol{x} = \idotsint_{\mathbb{R}^n} f(x_1, \dots, x_n) \, dx_1 \cdots dx_n\).

Statement. Let \(A \in \mathbb{R}^{n \times n}\) be symmetric positive definite, and let \(\boldsymbol{b} \in \mathbb{R}^n\). Then

For \(n = 1\) and \(A = a\), this is exactly the shifted and scaled formula above.

Step 1: diagonalize

Since \(A\) is symmetric positive definite, so is \(A^{-1}\). By the spectral theorem [1],

where \(\Lambda = \mathrm{diag}(\lambda_1, \dots, \lambda_n)\) has positive diagonal entries and \(Q\) is orthogonal (\(Q^\top Q = Q Q^\top = I\)).

Change variables to

Then \(\partial y_i / \partial x_k = Q_{ik}\), so the Jacobian matrix is \(\partial \boldsymbol{y} / \partial \boldsymbol{x} = Q\). An orthogonal matrix satisfies \(\det(Q)^2 = \det(Q^\top Q) = 1\), so \(\det Q = \pm 1\). (It can be \(-1\), for a reflection. That is why the absolute value matters.) Hence

Substituting,

Step 2: separate the variables

Because \(\Lambda\) is diagonal,

so the exponential factors into a product, and so does the integral:

Step 3: use the one-dimensional result

Each factor is a one-dimensional Gaussian integral with \(a = 1/\lambda_i\):

Finally, from \(A^{-1} = Q^\top \Lambda Q\),

Putting everything together,

which proves \(\eqref{eq:multivariate}\). \(\blacksquare\)

III. Probability and linear algebra

1. The univariate Gaussian distribution

If \(X \sim \mathcal{N}(\mu, \sigma^2)\), its density integrates to one:

This is just the shifted and scaled formula from Part I: the mean \(\mu\) plays the role of the shift \(b\), and the variance \(\sigma^2\) plays the role of the scale \(a\).

2. The multivariate Gaussian distribution

We write a random vector in bold capitals, \(\boldsymbol{X} = (X_1, \dots, X_n)^\top\). Each component \(X_i\) is a random variable. A specific value of \(\boldsymbol{X}\) is written in bold lowercase, \(\boldsymbol{x}\).

If \(\boldsymbol{X} \sim \mathcal{N}(\boldsymbol{\mu}, \Sigma)\) with a positive definite covariance matrix \(\Sigma\), then by \(\eqref{eq:multivariate}\) with \(A = \Sigma\) and \(\boldsymbol{b} = \boldsymbol{\mu}\),

The interesting part is the change of variables that simplified the quadratic form in the exponent. It has a clear probabilistic meaning.

(1) Diagonalizing \(\Sigma^{-1}\): the PCA view

Write

where the rows of the orthogonal matrix \(Q\) are eigenvectors of \(\Sigma^{-1}\), and \(\Lambda\) holds the corresponding eigenvalues. The eigenvectors of \(\Sigma^{-1}\) are the same as those of \(\Sigma\), but each eigenvalue is inverted: \(\Sigma = Q^\top \Lambda^{-1} Q\).

This is exactly the core idea of principal component analysis (PCA) [2]. We find the principal axes of the distribution and rotate the coordinate system to align with them. The variance along the \(i\)-th axis is \(1/\lambda_i\), so the direction of largest variance belongs to the smallest eigenvalue of \(\Sigma^{-1}\).

(2) The change of variables

Apply

This centers and rotates the coordinate system, and the quadratic form becomes

The covariance of the new random vector is

which is diagonal. So the components \(Y_1, \dots, Y_n\) are uncorrelated, with \(\mathrm{Var}(Y_i) = 1/\lambda_i\). Going from \(\boldsymbol{X}\) to \(\boldsymbol{Y}\) decorrelates the coordinates.

(3) Uncorrelated vs. independent

In general, uncorrelated random variables need not be independent. But for a jointly Gaussian random vector, uncorrelated components are independent. (Marginally Gaussian is not enough: the vector must be jointly Gaussian.) Here \(\boldsymbol{Y}\) is an affine function of a Gaussian vector, so it is jointly Gaussian.

You can see this directly from the density, which factors into a product of one-dimensional Gaussians:

This factorization is exactly what made the integral in Part II easy to compute.

Thanks to ChatGPT, which helped me organize my words.

References

Thanks to 子预. His article helped me a lot in understanding this topic [1].

This post gives a short, mostly self-contained proof of the Central Limit Theorem (CLT). It is not fully rigorous: I state two standard results from analysis without proving them.

The CLT is one of the most important results in probability theory. It says that the normalized sum of many independent random variables tends to a normal distribution, whatever the distribution of each variable is. This is why the bell curve shows up so often in nature: many small, independent sources of randomness add up to a predictable pattern.

The statement

Theorem (Lindeberg–Lévy CLT). Let \(X_1, X_2, \dots\) be i.i.d. (independent and identically distributed) random variables with \(\mathbb{E}[X_j] = \mu\) and \(\mathrm{Var}(X_j) = \sigma^2\), where \(0 < \sigma^2 < \infty\). Define

Then for every \(\tau \in \mathbb{R}\),

or, equivalently, \(Z_n \xrightarrow{D} \mathcal{N}(0, 1)\).

Notes.

• The name Lindeberg–Lévy refers to this version of the theorem, which needs i.i.d. variables with finite variance.
• A more general version, the Lindeberg–Feller CLT, drops the “identically distributed” assumption and replaces it with the weaker Lindeberg condition.

1. The road map

We use characteristic functions. The characteristic function of a random variable \(X\) is

It always exists, because \(\lvert e^{itX} \rvert = 1\). The proof rests on two classical facts, which I state without proof:

1. Uniqueness. The characteristic function determines the distribution: if \(\varphi_X(t) = \varphi_Y(t)\) for all \(t\), then \(X\) and \(Y\) have the same distribution. (This is the Fourier inversion theorem in disguise.)
2. Lévy’s continuity theorem. If \(\varphi_{Z_n}(t) \to \varphi_Z(t)\) for every \(t\), then \(Z_n \xrightarrow{D} Z\). (See Durrett [2], Section 3.3, for proofs of both facts.)

Uniqueness alone is not enough. We never have \(\varphi_{Z_n} = \varphi_N\) exactly for a finite \(n\), so we need the continuity theorem to turn pointwise convergence of characteristic functions into convergence in distribution.

So the plan is:

1. compute the characteristic function of \(N \sim \mathcal{N}(0,1)\);
2. show that \(\varphi_{Z_n}(t) \to \varphi_N(t)\) for every \(t\);
3. apply Lévy’s continuity theorem.

We use two simple properties of characteristic functions, which are proved in the appendix:

1. Affine maps: \(\varphi_{aX + b}(t) = e^{ibt} \, \varphi_X(at)\).
2. Independence: if \(X\) and \(Y\) are independent, then \(\varphi_{X + Y}(t) = \varphi_X(t) \, \varphi_Y(t)\).

2. The characteristic function of \(\mathcal{N}(0, 1)\)

Let \(N \sim \mathcal{N}(0, 1)\). We want to show

The quick (heuristic) way. Complete the square in the exponent:

The remaining integral is the Gaussian integral shifted by the imaginary constant \(it\). It still equals \(\sqrt{2\pi}\), but justifying this requires contour integration (Cauchy’s theorem). A plain real change of variables does not work.

A rigorous way without complex analysis. Since \(\mathbb{E}\lvert N \rvert < \infty\), we may differentiate under the integral sign:

This is a first-order linear ODE with \(\varphi_N(0) = 1\). Its unique solution is \(\varphi_N(t) = e^{-t^2/2}\).

3. The characteristic function of \(Z_n\)

Step 1: standardize

Define the standardized variables

Then

The \(Y_j\) are still i.i.d., so they share one characteristic function, which we write as \(\varphi_Y\).

Step 2: expand \(\varphi_Y\) near zero

The original version of this post expanded \(e^{itY}\) as an infinite Taylor series and swapped \(\mathbb{E}\) with the sum. That is not allowed here, because it needs all moments of \(Y\) to exist, and we only assumed a finite second moment.

The correct tool is a second-order expansion. Since \(\mathbb{E}[Y^2] < \infty\), \(\varphi_Y\) is twice continuously differentiable, with

Taylor’s theorem then gives

Step 3: multiply

By the two properties of characteristic functions (independence, then the affine map with \(a = 1/\sqrt{n}\), \(b = 0\)):

Step 4: take the limit

For fixed \(t\), write \(c_n = n\left(\varphi_Y(t/\sqrt{n}) - 1\right)\). The expansion above shows \(c_n \to -t^2/2\). A standard fact (which also holds for complex \(c_n\)) is

Therefore

4. Conclusion

We have shown that \(\varphi_{Z_n}(t) \to \varphi_N(t)\) for every \(t\), and \(\varphi_N\) is the characteristic function of \(\mathcal{N}(0,1)\). By Lévy’s continuity theorem,

This completes the proof of the CLT for i.i.d. variables with finite, nonzero variance.

Appendix: two properties of characteristic functions

Affine maps. For constants \(a, b \in \mathbb{R}\),

Independence. If \(X\) and \(Y\) are independent with densities \(f_X, f_Y\), their joint density factors as \(f_{X,Y}(x, y) = f_X(x) f_Y(y)\). So for any (bounded, measurable) functions \(\alpha\) and \(\beta\),

The same identity holds without densities, but it is easiest to see in this case. Taking \(\alpha(x) = e^{itx}\) and \(\beta(y) = e^{ity}\),

By induction, the characteristic function of a sum of \(n\) independent variables is the product of their characteristic functions.

References