Add more description to notebook in module 2

adchia · adchia · commit 4aeae0f34dff · 2022-05-17T18:27:20.000-04:00
Signed-off-by: Danny Chiao &lt;danny@tecton.ai&gt;
diff --git a/module_2/client/module_2_client.ipynb b/module_2/client/module_2_client.ipynb
@@ -7,6 +7,26 @@
     "# Retrieving on demand features"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Requirement already satisfied: Pygments in /Users/dannychiao/.pyenv/versions/3.8.10/envs/python-3.8/lib/python3.8/site-packages (2.11.2)\n",
+      "\u001b[33mWARNING: You are using pip version 22.0.3; however, version 22.1 is available.\n",
+      "You should consider upgrading via the '/Users/dannychiao/.pyenv/versions/3.8.10/envs/python-3.8/bin/python -m pip install --upgrade pip' command.\u001b[0m\u001b[33m\n",
+      "\u001b[0mNote: you may need to restart the kernel to use updated packages.\n"
+     ]
+    }
+   ],
+   "source": [
+    "%pip install Pygments"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -16,7 +36,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": 2,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -27,7 +47,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 3,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -46,12 +66,51 @@
    "metadata": {},
    "source": [
     "### model_v2 feature service\n",
-    "This one leverages dummy `val_to_add` and `val_to_add_2` request data. Request data must be passed in as part of the `entity_df`. This may come from the same source that includes your labels for your model."
+    "This one leverages dummy `val_to_add` and `val_to_add_2` request data. Request data must be passed in as part of the `entity_df`. This may come from the same source that includes your labels for your model.\n",
+    "\n",
+    "A quick reminder of the on demand feature view (ODFV) being used in this feature service:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "@on_demand_feature_view(\n",
+      "    sources=[driver_hourly_stats_view, val_to_add_request],\n",
+      "    schema=[\n",
+      "        Field(name=\"conv_rate_plus_val1\", dtype=Float64),\n",
+      "        Field(name=\"conv_rate_plus_val2\", dtype=Float64),\n",
+      "    ],\n",
+      ")\n",
+      "def transformed_conv_rate(inputs: pd.DataFrame) -> pd.DataFrame:\n",
+      "    df = pd.DataFrame()\n",
+      "    df[\"conv_rate_plus_val1\"] = inputs[\"conv_rate\"] + inputs[\"val_to_add\"]\n",
+      "    df[\"conv_rate_plus_val2\"] = inputs[\"conv_rate\"] + inputs[\"val_to_add_2\"]\n",
+      "    return df\n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "import dill\n",
+    "print(dill.source.getsource(store.get_on_demand_feature_view(\"transformed_conv_rate\").udf))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now let's retrieve historical features from this feature service. The transformation will happen on the fly after doing the point-in-time join to produce the `conv_rate_plus_val1` and `conv_rate_plus_val2` features"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": 30,
    "metadata": {},
    "outputs": [
     {
@@ -98,12 +157,53 @@
    "metadata": {},
    "source": [
     "### model_v3 feature service\n",
-    "This one generates geohash features"
+    "This one generates geohash features from latitudes and longitudes. This is useful for generating features relating to geographic regions (e.g. A geohash of `qfb9c3mw8hte` represents a sub-region within the region represented by a `qfb` geohash.)\n",
+    "\n",
+    "Let's look at the on demand feature view used in this feature service:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "@on_demand_feature_view(\n",
+      "    sources=[driver_daily_features_view],\n",
+      "    schema=[Field(name=f\"geohash_{i}\", dtype=String) for i in range(1, 7)],\n",
+      ")\n",
+      "def location_features_from_push(inputs: pd.DataFrame) -> pd.DataFrame:\n",
+      "    import pygeohash as gh\n",
+      "\n",
+      "    df = pd.DataFrame()\n",
+      "    df[\"geohash\"] = inputs.apply(lambda x: gh.encode(x.lat, x.lon), axis=1).astype(\n",
+      "        \"string\"\n",
+      "    )\n",
+      "\n",
+      "    for i in range(1, 7):\n",
+      "        df[f\"geohash_{i}\"] = df[\"geohash\"].str[:i].astype(\"string\")\n",
+      "    return df\n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(dill.source.getsource(store.get_on_demand_feature_view(\"location_features_from_push\").udf))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now we retrieve features. This will compute the `geohash_X` features"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 31,
    "metadata": {},
    "outputs": [
     {
@@ -162,12 +262,12 @@
    "metadata": {},
    "source": [
     "### model_v2 feature service\n",
-    "This one leverages dummy `val_to_add` and `val_to_add_2` request data so this is passed into the `entity_rows` parameter"
+    "This one leverages dummy `val_to_add` and `val_to_add_2` request data so this is passed into the `entity_rows` parameter. The on demand transformation is executed on the fly and combines in this request data with the pre-computed `conv_rate` feature in the online store."
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": 32,
    "metadata": {},
    "outputs": [
     {
@@ -202,7 +302,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 11,
+   "execution_count": 33,
    "metadata": {},
    "outputs": [
     {
