A statistical model used for classification and estimates the probability of an event occurring - usually binary (dichotomous) outcomes, such as win or loss, Yes or no.

Like linear regression, logistic regression is also used to determine the relationship between one or more independent variables and a dependent variable. However, it is applied to make prediction on a categorical variable as opposed to a continuous one.

In logistic regression, the probabilities are transformed into log(odds) scale to fit a linear model. This transformation allows for a range of values from negative infinity to positive infinity, making it compatible with linear modeling. However, binary outcomes can lead to some data points approaching values near positive and negative infinity, which makes it difficult to calculate the residuals. Consequently, the Ordinary Least Squares (OLS) method is not suitable, as it relies on the summation of residuals to estimate the best-fitting line.

• If $w$ is 0, the log(odds of obesity) is -2.

• For every one unit of weight gained, the log(odds of obesity) increases by 1.5.

The logistic regression is a sigmoid function that compresses the space from 0 to 1. This allows the output to be interpreted as the probability of belonging to a specific class.

$$p(x)=\frac{1}{1+e^{-({\beta }_0+{\beta }_{1}x)}}$$

Maximum Likelihood estimator.

1. Transform the probabilities into log(odds) to enable linear modelling.
2. Fit a line in the logit space (aka log-odds space).
3. Project the original data points onto the line to calculate the log(odds) values.
4. Transform the log(odds) into probabilities.
5. Calculate the log likelihood for the observed status.
6. Repeat the steps 1-5:
   • Iterate until the maximum likelihood is found and that would be the best fitted line.

The purpose is to model the relationship between the independent variables and the binary variable in a way that allows for linear modelling. By taking the logarithm of the odds, we convert the non-linear probability values (ranging from 0 to 1) into a linear scale that can be analysed using linear regression techniques. This linearity simplifies the estimation of coefficients and makes the model interpretable.

The probability is transformed to the log(odds) scale

$$\log (odds)=\log (\frac{p}{1-p})$$

Where, $p$ is the probability observed.

• Exponentiate both sides:

  $$e^{\log (\frac{p}{1-p})}=e^{\log (odds)}$$ $$\frac{p}{1-p}=e^{\log (odds)}$$

  • Using the rule: $e^{ln(x)}=x$, we get $\frac{p}{1-p}=e^{\log (odds)}$

• Rearrange the equation to solve for $p.$

  $$\frac{p}{1-p}=e^{\log (odds)}$$ $$p=(1-p)e^{\log (odds)}$$ $$p=e^{\log (odds)}-p(e^{\log (odds)})$$ $$p+p(e^{\log (odds)})=e^{\log (odds)}$$ $$p(1+e^{\log (odds)})=e^{\log (odds)}$$ $$p=\frac{e^{\log (odds)}}{1+e^{\log (odds)}}$$

$$p=\frac{e^{\log (odds)}}{1+e^{\log (odds)}}$$ $$e^{-3.48}=0.031$$ $$p=\frac{0.031}{1+0.031}=0.03$$

$$p=\frac{e^{\log (odds)}}{1+e^{\log (odds)}}$$

$$p=\frac{1}{1+e^{-\log (odds)}}$$

$$p(x)=\frac{1}{1+e^{-({\beta }_0+{\beta }_{1}x)}}$$

• Knowing that $\frac{1}{1+e^{-x}}=\frac{e^x}{1+e^x}$
• Substitute $\log (odds)$ with ${\beta }_0+{\beta }_{1}x$

$$\log (odds)=\log (\frac{p}{1-p})$$ $$log(\frac{1}{1-1})=\log (\frac{1}{0})=\log (1)-\log (0)$$

$\log (0)$ is defined as negative infinity. $\log (1)--\mathrm{\infty }=\mathrm{\infty }$ so the whole term will be positive infinity.

$$\log (odds)=\log (\frac{p}{1-p})$$ $$log(\frac{0}{1-0})=\log (\frac{0}{1})=\log (0)-\log (1)$$

$\log (0)$ is defined as negative infinity. $-\mathrm{\infty }+\log (1)=-\mathrm{\infty }$ so the whole term will be negative infinity.

The goal of MLE is to determine the optimal parameters for a chosen probability distribution that best explain the observed data. It involves identifying the type of distribution that fits the data and finding the specific parameter values that maximise the likelihood of observing the given data. Simply put, MLE helps us figure out the best-fitting distribution and where to center it, making the observed data most probable under that distribution.

Odds provide a measure of the likelihood of a particular outcome by expressing the likelihood of an event happening compared to the likelihood of it not happening.

$$o=\frac{p}{(1-p)}$$

Probability is a measure of how likely an event is to occur, expressed as a value between 0 and 1. Whereas, odds represent the ratio of the probability of an event occurring to the probability of it not occurring.

• To solve the asymmetric issues in odds. Odds that are against will be bounded to 0 and 1. However odds are in favour can range between 1 to infinity.

• Therefore, using log of odds makes the final values symmetric. making it easier to interpret and perform statistical analysis on the values.

• For example, applying log to the odds are against 1 to 6 and odds are in favour 6 to 1, will have result..

  $$\log (1/6)=-1.79$$ $$log(6/1)=1.79$$

$$e^{ln(x)}=x$$ $$\ln (e^{ln(x)})=\ln (x)$$ $$\ln (x)\times ln(e)=\ln (x)$$ $$\ln (x)\times 1=\ln (x)$$ $$x=x$$

$$\frac{1}{1+e^{-x}}=\frac{1}{1+\frac{1}{e^x}}$$ $$\frac{1}{1+\frac{1}{e^x}}\times \frac{e^x}{e^x}=\frac{e^x}{1+e^x}$$