Remember that we need to be precise in differentiating P(Y|θ) from P(θ|Y). The first is the likelihood, and the latter i

In this estimation, we will not be focusing so much on our Bayesian workflow and using all the good practices that we learned in the last post. But don’t worry, we will get back to it in the future. The reason behind it is our intention of showing that the MLE and the MAP are indeed the same thing, if we use flat priors. We will be using very vague priors (not actually flat, to help our sampler slightly).,Accessibility: The location and access of the interface are one of the most important qualities, and we do not want to give a how_to_use.txt to customers. Exposing our documentation on the cloud where everyone can see it, is the most convenient thing we can do.,Not so exciting! The deterministic level model is constant, and thus it does not vary over time as a result. The residuals are clearly not randomly distributed for this case as they result from the deviations of the observed values from their mean.,We are building momentum and gathering important tools for the journey ahead. I think we are ready to start adding some stochastic behavior to our parameters. See you in the next post!,Almost there. Just a small note on the σ (hat) value. Let’s run the same model with a library that has everything set up for us. We will go through that code, but I need you to see something.,Returning to the problem at hand. The expression above is quite similar to what we saw earlier working our the MLE example. We have one additional element: P(θ). This is our prior knowledge about our parameters and one fundamental idea behind Bayesian statistics. In the MLE case, we were implicitly assuming that all values of our parameters μ and σ were equally likely, i.e., we didn’t have any information to start with. This is the real difference between MLE and MAP. MLE assumes that all solutions are equally likely beforehand. MAP, on the other hand, allows us to accommodate prior information on our calculations. If we define the MAP with a flat prior, then we are basically performing MLE. When using more informative priors, we add a regularizing effect to our MAP estimation; that is why you often see MAP being framed as a regularization of the MLE.,Let’s follow the same approach that we did in the last post. But first, we need to compare our Bayesian estimation of the parameters with the frequentist approach that we used earlier. Remember that we can solely compare pointwise estimations since only the Bayesian framework produces posterior distributions.,Unobserved Components Results ================================================================================== Dep. Variable: y No. Observations: 192 Model: deterministic constant Log Likelihood 63.314 Date: Wed, 25 Nov 2020 AIC -124.628 Time: 16:24:36 BIC -121.375 Sample: 0 HQIC -123.310 - 192 Covariance Type: opg ==================================================================================== coef std err z P>|z| [0.025 0.975] ------------------------------------------------------------------------------------ sigma2.irregular 0.0294 0.003 8.987 0.000 0.023 0.036 =================================================================================== Ljung-Box (Q): 637.74 Jarque-Bera (JB): 0.73 Prob(Q): 0.00 Prob(JB): 0.69 Heteroskedasticity (H): 2.06 Skew: 0.09 Prob(H) (two-sided): 0.00 Kurtosis: 2.76 =================================================================================== Warnings: [1] Covariance matrix calculated using the outer product of gradients (complex-step).,import statsmodels.api as sm model_ll = sm.tsa.UnobservedComponents(y, level=True) model_fit = model_ll.fit() σ_sq_hat = model_fit.params[0] print(np.round(σ_sq_hat,5)),It wasn’t a swift detour, but we got somewhere. We know that state-space models maximize a log-likelihood function, and we saw how it is defined as well as two different procedures to do this maximization. Using the MLE, we get two estimators, μ (hat) and σ (hat). Let’s calculate these estimators for our problem.,Just a quick note on the parameter σ_ε². We’ve been estimating the variance of the ε_t term. Nevertheless, when fitting our likelihood, we are using the most usual way to fit a Gaussian distribution with the mean and the standard deviation. That is why I created a Deterministic variable (nothing to do with our deterministic model), which is the way to keep track of a transformed variable in PyMC3.,And here we have our parameters. For now, focus only on the columns mean and sd from the table above. As you can see, we have posterior distributions for our parameters μ_1 and σ_ε², not only a point estimate. Let’s compare our results with the ones that we got from our implementation using statsmodel.,As simple as that, we have our model fitting the data. We can see our two parameters sigma2.irregular (ε_t) and the level component μ_1. We also get several statistical tests that we will learn about in the future.,For small sample sizes, our estimator is unlikely to perfectly represent the data. Using this normalization term is a way of reducing the bias on our estimator. Let’s implement the latter.,We are building momentum and gathering important tools for the journey ahead. I think we are ready to start adding some stochastic behavior to our parameters. See you in the next post!