checkpointing

Author: mitzimorris

jupyter/radon/radon.ipynb

@@ -39,7 +39,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "d03ed3eb",
+   "id": "23be27b5",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -67,7 +67,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "74391bea",
+   "id": "c21c80d9",
    "metadata": {},
    "source": [
     "The radon data may sometimes trigger spurious warnings from plotnine and CmdStanPy, which we will ignore.  During model development we recommend leaving CmdStanPy logging level at `logging.WARNING`."
@@ -299,7 +299,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "c08867d5",
+   "id": "fccb5170",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -314,7 +314,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "66961dbc",
+   "id": "546c552c",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -420,7 +420,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "67bc90a9",
+   "id": "afb04f4d",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -510,11 +510,12 @@
    "source": [
     "## Best Practice #2: start with a simple model\n",
     "\n",
-    "\n",
-    "We can use a simple linear regression model the relationship between\n",
+    "Starting from a simple model ensures that there is a good baseline\n",
+    "against which to measure performance.\n",
+    "Gelman and Hill start from a simple linear regression which models the relationship between\n",
     "the log radon measurement and the floor on which this measurement was taken.\n",
-    "This is the **complete pooling** model because it pools all measurements from all counties.\n",
     "\n",
+    "The **complete pooling** model pools all measurements from all counties.\n",
     "While it's possible to fit this model in Stan, we can also do this in plotnine by adding\n",
     "[plotnine.geom.geom_smooth(method=\"lm\")](https://plotnine.readthedocs.io/en/stable/generated/plotnine.geoms.geom_smooth.html#plotnine.geoms.geom_smooth)\n",
     "to the earlier scatter plot of radon measurements by floor.\n",
@@ -535,7 +536,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "2b5e6077",
+   "id": "58c552b0",
    "metadata": {},
    "source": [
     "If we add faceting to this plot, we are fitting 85 individual regressions.  Given the sparsity of the data, this is a non-starter. There are 25 counties where all measurements were taken on the basement level; for these counties, the trend line is absent. For the remaining counties, most of which have very few measurments, in several instances, e.g., counties \"Lyon\" and \"Redwood\", the regression line between floor 0 and 1 has a positvie slope; in contrast to the complete pooling model. This goes against what we know about how radon enters the home.\n",
@@ -545,7 +546,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "d18ffec9",
+   "id": "6f3c5760",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -622,7 +623,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "b1977e2e",
+   "id": "a754da06",
    "metadata": {},
    "source": [
     "The CmdStanMCMC object provides accessor methods which return the set of draws for each model variable.\n",
@@ -640,13 +641,13 @@
     "[stan_variables](https://mc-stan.org/cmdstanpy/api.html#cmdstanpy.CmdStanMCMC.stan_variables)\n",
     "returns a Python dict from variable names to numpy.ndarrays.\n",
     "\n",
-    "Because we are going to be plotting the results, we use the `draws_pd` method."
+    "Because we are going to be plotting the results with plotnine, we use the `draws_pd` method to extract the model parameters `alpha`, `beta`, and `sigma`."
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "9e4c872c",
+   "id": "3781d33a",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -703,7 +704,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "7d0c83a6",
+   "id": "1e0037a4",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -715,7 +716,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "8350618b",
+   "id": "22aafd3e",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -728,7 +729,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "74999c7a",
+   "id": "5ffc0a68",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -740,7 +741,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "e99d5185",
+   "id": "4e5314d4",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -758,7 +759,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "fb9ac61e",
+   "id": "114086ca",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -776,12 +777,23 @@
     "     alpha=0.5,\n",
     " )\n",
     " + geom_point(no_pool_alpha_pd, aes(x='county', y='mean'))\n",
-    " + scale_x_discrete(limits=pop_desc['county']) \n",
+    " + geom_hline(yintercept=pool_stats.alpha[1], color='darkorange', size=1.5)\n",
+    " + scale_x_discrete(limits=pop_desc['county'])\n",
+    " + ggtitle(\"No pooling model estimates for alpha (basement log_radon level)\")\n",
+    " + ylab(\"central 67% interval\") + xlab(\"ordered by observations per county, desc\")\n",
     " + flip_xlabels\n",
     " + theme(figure_size=(12,4)) \n",
     ")"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "b6808b62",
+   "metadata": {},
+   "source": [
+    "The complete pooling model ignores the county level variance.  The no-pooling model overstates the variance between counties."
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "d15f8442",
@@ -1046,15 +1058,15 @@
   },
   {
    "cell_type": "markdown",
-   "id": "dca8dfc3",
+   "id": "2b3d68e6",
    "metadata": {},
    "source": [
     "## Appendix C:  Stan tips"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "a60524e1",
+   "id": "06387fc5",
    "metadata": {},
    "source": [
     "* it uses the more efficient [`normal_id_glm` distribution](https://mc-stan.org/docs/functions-reference/normal-id-glm.html) instead of the `normal` distribution. The `normal_id_glm` distribution is optimized for simple linear regresion models and takes as arguments:\n",
@@ -1069,7 +1081,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "add9533a",
+   "id": "a324f22c",
    "metadata": {},
    "outputs": [],
    "source": []