Strangely of not this is the variance of the data. So much work to get the mean and variance of the data! They are the b

As simple as that, we have our model fitting the data. We can see our two parameters sigma2.irregular (ε_t) and the level component μ_1. We also get several statistical tests that we will learn about in the future.,For small sample sizes, our estimator is unlikely to perfectly represent the data. Using this normalization term is a way of reducing the bias on our estimator. Let’s implement the latter.,Just one more ah ah moment. We already used, in the last post, the least-squares method to estimate our parameters of the linear regression. When the model is assumed to be Gaussian, the MLE estimates are equivalent to the least-squares method.,This means that P(A | B) is proportional to P(B | A) P(A), i.e., P(B) is a normalizing constant, and we don’t need to normalize our result. In fact, this is the hardest component to compute in the Bayes equation, and we will see more about it in the future.,In the equation above, B is the evidence, p(A) is the prior, p(B | A) is the likelihood, and p(A | B) is the posterior. p(A | B) is the probability of A happening if B happened.,Remember that we need to be precise in differentiating P(Y|θ) from P(θ|Y). The first is the likelihood, and the latter is the posterior, and they can be very different. We need to sharpen our eyes to spot the differences constantly.,In this estimation, we will not be focusing so much on our Bayesian workflow and using all the good practices that we learned in the last post. But don’t worry, we will get back to it in the future. The reason behind it is our intention of showing that the MLE and the MAP are indeed the same thing, if we use flat priors. We will be using very vague priors (not actually flat, to help our sampler slightly).,Unobserved Components Results ================================================================================== Dep. Variable: y No. Observations: 192 Model: deterministic constant Log Likelihood 63.314 Date: Wed, 25 Nov 2020 AIC -124.628 Time: 16:24:36 BIC -121.375 Sample: 0 HQIC -123.310 - 192 Covariance Type: opg ==================================================================================== coef std err z P>|z| [0.025 0.975] ------------------------------------------------------------------------------------ sigma2.irregular 0.0294 0.003 8.987 0.000 0.023 0.036 =================================================================================== Ljung-Box (Q): 637.74 Jarque-Bera (JB): 0.73 Prob(Q): 0.00 Prob(JB): 0.69 Heteroskedasticity (H): 2.06 Skew: 0.09 Prob(H) (two-sided): 0.00 Kurtosis: 2.76 =================================================================================== Warnings: [1] Covariance matrix calculated using the outer product of gradients (complex-step).,We discussed the Bayes Theorem in the last post; now it is time to connect it to a new concept: the Maximum a Posteriori (MAP). The MAP is the bayesian equivalent to the MLE.,import statsmodels.api as sm model_ll = sm.tsa.UnobservedComponents(y, level=True) model_fit = model_ll.fit() σ_sq_hat = model_fit.params[0] print(np.round(σ_sq_hat,5)),It wasn’t a swift detour, but we got somewhere. We know that state-space models maximize a log-likelihood function, and we saw how it is defined as well as two different procedures to do this maximization. Using the MLE, we get two estimators, μ (hat) and σ (hat). Let’s calculate these estimators for our problem.,Returning to the problem at hand. The expression above is quite similar to what we saw earlier working our the MLE example. We have one additional element: P(θ). This is our prior knowledge about our parameters and one fundamental idea behind Bayesian statistics. In the MLE case, we were implicitly assuming that all values of our parameters μ and σ were equally likely, i.e., we didn’t have any information to start with. This is the real difference between MLE and MAP. MLE assumes that all solutions are equally likely beforehand. MAP, on the other hand, allows us to accommodate prior information on our calculations. If we define the MAP with a flat prior, then we are basically performing MLE. When using more informative priors, we add a regularizing effect to our MAP estimation; that is why you often see MAP being framed as a regularization of the MLE.,Not getting into the math, we can grasp the idea. In the case of the least-squares parameter estimation, we want to find the line that minimizes the total squared distance between the regression line and the data points. On the other hand, in the maximum likelihood estimation, we want to maximize the total probability of the data. In the case of the Gaussian distribution that happens when the data points are close to the mean value. Due to the symmetric nature of the distribution, this is equivalent to minimizing the distance between the data points and the mean value (see more here [2]).,Almost there. Just a small note on the σ (hat) value. Let’s run the same model with a library that has everything set up for us. We will go through that code, but I need you to see something.,The Promise.any() method is still experimental and in stage 4. But being in stage 4 means that it is in the “Finished Proposals” stage and very much ready for release with an expected release date of 2021. Similar to the Promise.race() method, this method too expects an iterable as an argument.