Variance

Variance in mathematics is a measure of how spread out the values in a set of data are. It quantifies the average squared deviation of each data point from the mean (average) of the dataset. In simpler terms, it tells us how far individual data points are from the center of the data.

The formula for variance is:

Where:

Higher variance means the data points are more spread out, while lower variance indicates they are closer to the mean.

Standard Deviation

Standard deviation tells us how much data points tend to deviate from the mean on average. A small standard deviation indicates that the data points are clustered closely around the mean, whereas a large standard deviation shows that the data points are spread out.

Example

Alright, let’s walk through an example to make the concept of standard deviation clearer!

Imagine you have the test scores of five students in a math exam: 80, 85, 90, 95, and 100. Here’s how we calculate the standard deviation step by step:

Step 1: Find the Mean

The mean ((\mu)) is the average of the data:
$$\mu = \frac{80 + 85 + 90 + 95 + 100}{5} = 90$$

Step 2: Calculate Deviations from the Mean

Subtract the mean from each score to find the deviations:

• (80 – 90 = -10)
• (85 – 90 = -5)
• (90 – 90 = 0)
• (95 – 90 = 5)
• (100 – 90 = 10)

Step 3: Square Each Deviation

Square the deviations to eliminate negative values:

Step 4: Find the Average of the Squared Deviations

Add up the squared deviations and divide by the total number of data points ((n = 5)):
$$\text{Variance} (\sigma^2) = \frac{100 + 25 + 0 + 25 + 100}{5} = 50$$

Step 5: Take the Square Root of the Variance

The square root of the variance gives us the standard deviation:
$$\sigma = \sqrt{50} \approx 7.07$$

Final Result:

The standard deviation is approximately 7.07. This means that, on average, the test scores deviate from the mean by about 7.07 points.

Now, with this example, you see how the standard deviation helps visualize the spread of scores

Moving Averages

The moving averages method is a statistical technique used to analyze time-series data by smoothing out short-term fluctuations and highlighting trends or patterns over time. It’s often applied in fields like finance, economics, and sales forecasting.

Here’s how it works:

1. Determine the Window: Choose the number of data points (known as the “window”) you want to consider for calculating the average. For example, you might decide on a 3-month or 5-day moving average.
2. Calculate the Average: For each period, calculate the average of the chosen number of data points. This average is the moving average for that period.
3. Shift the Window: Move the window one period forward (e.g., from days 1–5 to days 2–6) and calculate the next average.
4. Repeat: Continue shifting the window and calculating averages across the entire dataset.

The result is a new series of data (the moving averages), which smooths out random variations in the original data, allowing you to better understand trends.

There are different types of moving averages:

• Simple Moving Average (SMA): Equal weight is given to all data points in the window.
• Weighted Moving Average (WMA): More weight is given to recent data points.
• Exponential Moving Average (EMA): Uses a formula to give progressively more weight to recent data points.

sample c# code for moving averages

// Simple moving average calculation 
double[] data = { 10, 12, 14, 16, 18, 20, 22, 24, 26, 28 }; 
int windowSize = 3; 
double[] movingAverages = new double[data.Length - windowSize + 1]; 
for (int i = 0; i < movingAverages.Length; i++) { 
              movingAverages[i] = data.Skip(i).Take(windowSize).Average(); 
} 
Console.WriteLine("Moving Averages: " + string.Join(", ", movingAverages));

Explanation

Sure! Let me break down the code for you step by step:

Purpose

This code calculates the Simple Moving Average (SMA) of an array of numbers, given a specified window size.

---

Variables and Initialization

1. double[] data:
   This array holds the input data (in this case, {10, 12, 14, 16, 18, 20, 22, 24, 26, 28}), which represents a time-series or sequential dataset.
2. int windowSize:
   This is the size of the moving average window, set to 3. It determines how many consecutive numbers are used to calculate each average.
3. double[] movingAverages:
   This array will store the computed moving averages. Its length is:
   $$\text{Length of data array} – \text{windowSize} + 1$$
   This ensures that the moving average calculation stops when the remaining data is insufficient to fill the window.

---

Loop Logic

The for loop iterates through the data array to calculate the moving averages:

1. Iteration:
   The loop runs from i = 0 to i < movingAverages.Length. During each iteration, it computes the average of a specific slice (sub-array) of the input data.
2. data.Skip(i).Take(windowSize):

• Skip(i): Skips the first i elements of the data array.
• Take(windowSize): Takes the next windowSize elements starting from the current position.
  For example:
• At i = 0: Takes {10, 12, 14}
• At i = 1: Takes {12, 14, 16}
• At i = 2: Takes {14, 16, 18}, and so on.

1. .Average():
   Computes the average of the selected windowSize elements and stores it in the movingAverages[i] array.

---

Output

After the loop finishes, all the calculated moving averages are stored in the movingAverages array. Finally, the code prints them using:

Console.WriteLine("Moving Averages: " + string.Join(", ", movingAverages));

This will output the moving averages as a comma-separated list.

---

Example

For the given data array and windowSize = 3:

• Moving average for {10, 12, 14} = ((10 + 12 + 14) / 3 = 12)
• Moving average for {12, 14, 16} = ((12 + 14 + 16) / 3 = 14)
• Moving average for {14, 16, 18} = ((14 + 16 + 18) / 3 = 16)
• And so on.

The final output will be:

Moving Averages: 12, 14, 16, 18, 20, 22, 24, 26

---

The Skip(i) function is used to shift the starting position of the window when calculating moving averages. Here’s why it’s necessary:

• In the first iteration (i = 0), we want to start the window at the beginning of the data array, which includes the first three elements (e.g., {10, 12, 14}).
• In the next iteration (i = 1), we want the window to move forward by one position, so it starts at the second element and includes the next three elements (e.g., {12, 14, 16}).
• This shifting process continues for each subsequent iteration, ensuring that each moving average is calculated using the right set of consecutive numbers.

Without Skip(i), the function would always start from the beginning of the array, and you’d end up calculating the same average repeatedly instead of progressively shifting the window. By skipping i elements, the window moves forward as intended, covering all possible sets of data points.

In the first iteration ((i = 0)), nothing is skipped because (Skip(0)) means “skip zero elements”—essentially, it starts at the beginning of the array, so it includes 10, 12, and 14 as intended.

The confusion might stem from interpreting Skip(i) too literally. Here’s what happens step by step:

• At (i = 0), data.Skip(0) takes the array as-is (no skipping), so it starts with {10, 12, 14}.
• At (i = 1), data.Skip(1) skips the first element (10), resulting in {12, 14, 16} being taken.
• At (i = 2), data.Skip(2) skips the first two elements (10, 12), resulting in {14, 16, 18} being taken.

The Skip(i) ensures the window shifts correctly as the loop progresses. But in the first iteration, nothing is skipped, and 10 is included in the moving average calculation.

Convolution

What is Convolution?

Convolution is a mathematical operation that combines two sequences (arrays) to produce a new sequence. It essentially slides one sequence (called the “kernel” or “filter”) across another sequence (the “input data”) and calculates weighted sums at each step.

Simple Example

Let’s use a very basic case:

Input Data:

Imagine you have the sequence:

Kernel (Filter):

The kernel is a smaller sequence:

How Convolution Works:

The kernel slides across the input data. At each position, we multiply the kernel values by the corresponding input values and sum them up.

---

Step-by-Step Calculation

Step 1: First Position

Align the kernel with the first three values of the input:
[ [1, 2, 3] ]
Multiply each input value by the corresponding kernel value:

So, the first result is -2.

---

Step 2: Second Position

Move the kernel one step to the right:
[ [2, 3, 4] ]
Multiply and sum:

The second result is -2.

---

Step 3: Third Position

Move the kernel again:
[ [3, 4, 5] ]
Multiply and sum:

The third result is -2.

---

Final Result:

The output of the convolution is:
[ [-2, -2, -2] ]

---

Why Is This Useful?

Convolution is used in many fields:

• Moving Averages: To smooth data or detect trends.
• Image Processing: To apply filters like blurring or edge detection.
• Machine Learning: In convolutional neural networks for feature extraction.