{ "cells": [ { "cell_type": "markdown", "id": "dd59fccd", "metadata": { "papermill": { "duration": 0.003701, "end_time": "2025-12-04T16:45:56.340937", "exception": false, "start_time": "2025-12-04T16:45:56.337236", "status": "completed" }, "tags": [] }, "source": [ "# Kaggle diabetes prediction challenge: Submission template\n", "\n", "Our lesson 20: classification activity will use an active Kaggle playground competition. Playground competitions run every month and highlight interesting/approachable datasets for the ML community to practice skills on a variety of ML problem types without the pressure, complexity and long duration of prize money competitions. \n", "\n", "It just so happens that this month's playground competition is a classification challenge! See the full competition details here: [Playground Series - Season 5, Episode 12: Diabetes Prediction Challenge](https://www.kaggle.com/competitions/playground-series-s5e12)\n", "\n", "This notebook template will help you make your first Kaggle competition submission. It is pre-filled with code to load the competition data and output random predictions (see section 5. Submission below). This will score ~0.50 on the Kaggle public leaderboard. Your job is to improve that score with EDA, clever feature engineering and good model optimization. Good luck!\n", "\n", "## How to submit on Kaggle\n", "\n", "**1. Upload this notebook to Kaggle:**\n", " - Create a Kaggle account and log in to [kaggle](https://www.kaggle.com)\n", " - Click '+ Create' in the left navigation menu\n", " - Select 'Import Notebook' and upload this file (or link to GitHub)\n", "\n", "**2. Start the notebook:**\n", " - Find your uploaded notebook in your Kaggle account under 'Code'\n", " - Click on the notebook to open it\n", " - The notebook will open in edit mode - you can now run cells and make changes\n", "\n", "**3. Add the competition dataset:**\n", " - From the notebook environment, in the right sidebar, click the 'Input' tab\n", " - Click '+ Add Input' → filter by 'Competition Datasets'\n", " - Find 'Diabetes Prediction Challenge' and click the '+' icon\n", " - Note: You must join the competition first (click 'Join Competition' on the competition page)\n", "\n", "**4. Access the data:**\n", " - Once added, the data is available at `/kaggle/input/playground-series-s5e12/`\n", " - Files: `train.csv` (training data with labels), `test.csv` (test data without labels), `sample_submission.csv` (submission format example)\n", "\n", "**5. Make your submission:**\n", " - Your notebook must output test set predictions to `submission.csv` in the correct format\n", " - Go to 'Submit to competition' tab in the right sidebar and click 'Submit'\n", "\n", "You may see warnings when running on Kaggle due to inconsistencies in installed package versions between your environment and Kaggle. If you are using a virtual environment already, install this [kaggle_requirements.txt](https://github.com/gperdrizet/FSA_devops/blob/main/notebooks/unit3/lesson_20/kaggle_requirements.txt):\n", "\n", "```\n", "pip install --force-reinstall kaggle_requirements.txt\n", "```\n", "\n", "This is working for me with Python 3.12. It contains a slightly newer version of scikit-learn than is found on Kaggle. Update in the Kaggle environment by going to 'Add-ons' -> 'Install Dependencies' and adding:\n", "\n", "```\n", "pip install scikit-learn==1.5.2\n", "```\n", "\n", ">**Note:** This notebook uses a `KAGGLE` flag (under 'Run >configuration') to switch between Kaggle and local file paths. Set it >to `True` when running on Kaggle, or `False` when running locally.\n", "\n", "## Notebook set-up\n", "\n", "### Imports" ] }, { "cell_type": "code", "execution_count": null, "id": "04d85a3b", "metadata": { "execution": { "iopub.execute_input": "2025-12-04T16:45:56.348530Z", "iopub.status.busy": "2025-12-04T16:45:56.348220Z", "iopub.status.idle": "2025-12-04T16:45:58.407545Z", "shell.execute_reply": "2025-12-04T16:45:58.406198Z" }, "papermill": { "duration": 2.065596, "end_time": "2025-12-04T16:45:58.409770", "exception": false, "start_time": "2025-12-04T16:45:56.344174", "status": "completed" }, "tags": [] }, "outputs": [], "source": [ "# Standard library imports\n", "from pathlib import Path\n", "\n", "# Third party imports\n", "import pandas as pd" ] }, { "cell_type": "markdown", "id": "0f3904c9", "metadata": {}, "source": [ "### Run configuration" ] }, { "cell_type": "code", "execution_count": 2, "id": "20afe8a3", "metadata": {}, "outputs": [], "source": [ "# Set to True when running on Kaggle, False when running locally\n", "KAGGLE = False" ] }, { "cell_type": "markdown", "id": "2ae53a2b", "metadata": { "papermill": { "duration": 0.002851, "end_time": "2025-12-04T16:45:58.415832", "exception": false, "start_time": "2025-12-04T16:45:58.412981", "status": "completed" }, "tags": [] }, "source": [ "### Data loading" ] }, { "cell_type": "code", "execution_count": null, "id": "d1a2421c", "metadata": { "execution": { "iopub.execute_input": "2025-12-04T16:45:58.423485Z", "iopub.status.busy": "2025-12-04T16:45:58.423107Z", "iopub.status.idle": "2025-12-04T16:46:02.306776Z", "shell.execute_reply": "2025-12-04T16:46:02.305525Z" }, "papermill": { "duration": 3.889301, "end_time": "2025-12-04T16:46:02.308493", "exception": false, "start_time": "2025-12-04T16:45:58.419192", "status": "completed" }, "tags": [] }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
idagealcohol_consumption_per_weekphysical_activity_minutes_per_weekdiet_scoresleep_hours_per_dayscreen_time_hours_per_daybmiwaist_to_hip_ratiosystolic_bp...genderethnicityeducation_levelincome_levelsmoking_statusemployment_statusfamily_history_diabeteshypertension_historycardiovascular_historydiagnosed_diabetes
00311457.76.86.133.40.93112...FemaleHispanicHighschoolLower-MiddleCurrentEmployed0001.0
11502735.76.55.823.80.83120...FemaleWhiteHighschoolUpper-MiddleNeverEmployed0001.0
223231588.57.49.124.10.8395...MaleHispanicHighschoolLower-MiddleNeverRetired0000.0
33543774.67.09.226.60.83121...FemaleWhiteHighschoolLower-MiddleCurrentEmployed0101.0
44541555.76.25.128.80.90108...MaleWhiteHighschoolUpper-MiddleNeverRetired0101.0
\n", "

5 rows × 26 columns

\n", "
" ], "text/plain": [ " id age alcohol_consumption_per_week physical_activity_minutes_per_week \\\n", "0 0 31 1 45 \n", "1 1 50 2 73 \n", "2 2 32 3 158 \n", "3 3 54 3 77 \n", "4 4 54 1 55 \n", "\n", " diet_score sleep_hours_per_day screen_time_hours_per_day bmi \\\n", "0 7.7 6.8 6.1 33.4 \n", "1 5.7 6.5 5.8 23.8 \n", "2 8.5 7.4 9.1 24.1 \n", "3 4.6 7.0 9.2 26.6 \n", "4 5.7 6.2 5.1 28.8 \n", "\n", " waist_to_hip_ratio systolic_bp ... gender ethnicity education_level \\\n", "0 0.93 112 ... Female Hispanic Highschool \n", "1 0.83 120 ... Female White Highschool \n", "2 0.83 95 ... Male Hispanic Highschool \n", "3 0.83 121 ... Female White Highschool \n", "4 0.90 108 ... Male White Highschool \n", "\n", " income_level smoking_status employment_status family_history_diabetes \\\n", "0 Lower-Middle Current Employed 0 \n", "1 Upper-Middle Never Employed 0 \n", "2 Lower-Middle Never Retired 0 \n", "3 Lower-Middle Current Employed 0 \n", "4 Upper-Middle Never Retired 0 \n", "\n", " hypertension_history cardiovascular_history diagnosed_diabetes \n", "0 0 0 1.0 \n", "1 0 0 1.0 \n", "2 0 0 0.0 \n", "3 1 0 1.0 \n", "4 1 0 1.0 \n", "\n", "[5 rows x 26 columns]" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Set file paths based on environment\n", "if KAGGLE:\n", "\n", " # Kaggle paths - data is in /kaggle/input/\n", " train_df_path = '/kaggle/input/playground-series-s5e12/train.csv'\n", " test_df_path = '/kaggle/input/playground-series-s5e12/test.csv'\n", "\n", "else:\n", "\n", " # Otherwise, load data from course GitHub repository\n", " train_df_path = 'https://gperdrizet.github.io/FSA_devops/assets/data/unit3/diabetes_prediction_train.csv'\n", " test_df_path = 'https://gperdrizet.github.io/FSA_devops/assets/data/unit3/diabetes_prediction_test.csv'\n", "\n", "# Load the training and testing datasets\n", "train_df = pd.read_csv(train_df_path)\n", "test_df = pd.read_csv(test_df_path)\n", "\n", "# Display first few rows of training data\n", "train_df.head()" ] }, { "cell_type": "code", "execution_count": 4, "id": "c9edb967", "metadata": { "execution": { "iopub.execute_input": "2025-12-04T16:46:02.316341Z", "iopub.status.busy": "2025-12-04T16:46:02.316028Z", "iopub.status.idle": "2025-12-04T16:46:02.579217Z", "shell.execute_reply": "2025-12-04T16:46:02.577176Z" }, "papermill": { "duration": 0.26922, "end_time": "2025-12-04T16:46:02.581019", "exception": false, "start_time": "2025-12-04T16:46:02.311799", "status": "completed" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "RangeIndex: 700000 entries, 0 to 699999\n", "Data columns (total 26 columns):\n", " # Column Non-Null Count Dtype \n", "--- ------ -------------- ----- \n", " 0 id 700000 non-null int64 \n", " 1 age 700000 non-null int64 \n", " 2 alcohol_consumption_per_week 700000 non-null int64 \n", " 3 physical_activity_minutes_per_week 700000 non-null int64 \n", " 4 diet_score 700000 non-null float64\n", " 5 sleep_hours_per_day 700000 non-null float64\n", " 6 screen_time_hours_per_day 700000 non-null float64\n", " 7 bmi 700000 non-null float64\n", " 8 waist_to_hip_ratio 700000 non-null float64\n", " 9 systolic_bp 700000 non-null int64 \n", " 10 diastolic_bp 700000 non-null int64 \n", " 11 heart_rate 700000 non-null int64 \n", " 12 cholesterol_total 700000 non-null int64 \n", " 13 hdl_cholesterol 700000 non-null int64 \n", " 14 ldl_cholesterol 700000 non-null int64 \n", " 15 triglycerides 700000 non-null int64 \n", " 16 gender 700000 non-null object \n", " 17 ethnicity 700000 non-null object \n", " 18 education_level 700000 non-null object \n", " 19 income_level 700000 non-null object \n", " 20 smoking_status 700000 non-null object \n", " 21 employment_status 700000 non-null object \n", " 22 family_history_diabetes 700000 non-null int64 \n", " 23 hypertension_history 700000 non-null int64 \n", " 24 cardiovascular_history 700000 non-null int64 \n", " 25 diagnosed_diabetes 700000 non-null float64\n", "dtypes: float64(6), int64(14), object(6)\n", "memory usage: 138.9+ MB\n" ] } ], "source": [ "# Display dataset information (columns, dtypes, non-null counts)\n", "train_df.info()" ] }, { "cell_type": "markdown", "id": "c2f83f1b", "metadata": { "papermill": { "duration": 0.003317, "end_time": "2025-12-04T16:46:02.588198", "exception": false, "start_time": "2025-12-04T16:46:02.584881", "status": "completed" }, "tags": [] }, "source": [ "## 1. EDA" ] }, { "cell_type": "code", "execution_count": 5, "id": "1b49dac8", "metadata": { "execution": { "iopub.execute_input": "2025-12-04T16:46:02.597068Z", "iopub.status.busy": "2025-12-04T16:46:02.596748Z", "iopub.status.idle": "2025-12-04T16:46:02.601201Z", "shell.execute_reply": "2025-12-04T16:46:02.600004Z" }, "papermill": { "duration": 0.011581, "end_time": "2025-12-04T16:46:02.603066", "exception": false, "start_time": "2025-12-04T16:46:02.591485", "status": "completed" }, "tags": [] }, "outputs": [], "source": [ "# Your code here..." ] }, { "cell_type": "markdown", "id": "15d1a762", "metadata": { "papermill": { "duration": 0.003202, "end_time": "2025-12-04T16:46:02.609520", "exception": false, "start_time": "2025-12-04T16:46:02.606318", "status": "completed" }, "tags": [] }, "source": [ "## 2. Feature engineering" ] }, { "cell_type": "code", "execution_count": 6, "id": "715276e5", "metadata": { "execution": { "iopub.execute_input": "2025-12-04T16:46:02.617258Z", "iopub.status.busy": "2025-12-04T16:46:02.616925Z", "iopub.status.idle": "2025-12-04T16:46:02.621182Z", "shell.execute_reply": "2025-12-04T16:46:02.620037Z" }, "papermill": { "duration": 0.010201, "end_time": "2025-12-04T16:46:02.622943", "exception": false, "start_time": "2025-12-04T16:46:02.612742", "status": "completed" }, "tags": [] }, "outputs": [], "source": [ "# Your code here..." ] }, { "cell_type": "markdown", "id": "0a0e0c1b", "metadata": { "papermill": { "duration": 0.002954, "end_time": "2025-12-04T16:46:02.629151", "exception": false, "start_time": "2025-12-04T16:46:02.626197", "status": "completed" }, "tags": [] }, "source": [ "## 3. Model building" ] }, { "cell_type": "code", "execution_count": 7, "id": "abdb3c80", "metadata": { "execution": { "iopub.execute_input": "2025-12-04T16:46:02.636717Z", "iopub.status.busy": "2025-12-04T16:46:02.636400Z", "iopub.status.idle": "2025-12-04T16:46:02.641384Z", "shell.execute_reply": "2025-12-04T16:46:02.640372Z" }, "papermill": { "duration": 0.010911, "end_time": "2025-12-04T16:46:02.643109", "exception": false, "start_time": "2025-12-04T16:46:02.632198", "status": "completed" }, "tags": [] }, "outputs": [], "source": [ "# Your code here..." ] }, { "cell_type": "markdown", "id": "530931ae", "metadata": { "papermill": { "duration": 0.002984, "end_time": "2025-12-04T16:46:02.649294", "exception": false, "start_time": "2025-12-04T16:46:02.646310", "status": "completed" }, "tags": [] }, "source": [ "## 4. Model evaluation" ] }, { "cell_type": "code", "execution_count": 8, "id": "fddaa21e", "metadata": { "execution": { "iopub.execute_input": "2025-12-04T16:46:02.656583Z", "iopub.status.busy": "2025-12-04T16:46:02.656295Z", "iopub.status.idle": "2025-12-04T16:46:02.661034Z", "shell.execute_reply": "2025-12-04T16:46:02.660017Z" }, "papermill": { "duration": 0.010313, "end_time": "2025-12-04T16:46:02.662563", "exception": false, "start_time": "2025-12-04T16:46:02.652250", "status": "completed" }, "tags": [] }, "outputs": [], "source": [ "# Your code here..." ] }, { "cell_type": "markdown", "id": "4a4303c1", "metadata": { "papermill": { "duration": 0.003006, "end_time": "2025-12-04T16:46:02.668933", "exception": false, "start_time": "2025-12-04T16:46:02.665927", "status": "completed" }, "tags": [] }, "source": [ "## 5. Submission" ] }, { "cell_type": "code", "execution_count": 9, "id": "edf67e48", "metadata": { "execution": { "iopub.execute_input": "2025-12-04T16:46:02.676497Z", "iopub.status.busy": "2025-12-04T16:46:02.676229Z", "iopub.status.idle": "2025-12-04T16:46:02.971039Z", "shell.execute_reply": "2025-12-04T16:46:02.970066Z" }, "papermill": { "duration": 0.300682, "end_time": "2025-12-04T16:46:02.972781", "exception": false, "start_time": "2025-12-04T16:46:02.672099", "status": "completed" }, "tags": [] }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
iddiagnosed_diabetes
07000001
17000010
27000021
37000031
47000040
\n", "
" ], "text/plain": [ " id diagnosed_diabetes\n", "0 700000 1\n", "1 700001 0\n", "2 700002 1\n", "3 700003 1\n", "4 700004 0" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Make random predictions for submission by sampling from the training labels\n", "# Note: Replace this with your model's predictions!\n", "predictions = train_df['diagnosed_diabetes'].sample(n=test_df.shape[0], random_state=42).astype(int)\n", "prediction_ids = test_df['id'].astype(int)\n", "\n", "# Create submission DataFrame with required format: id, diagnosed_diabetes\n", "submission_df = pd.DataFrame({\n", " 'id': prediction_ids.values,\n", " 'diagnosed_diabetes': predictions.values\n", "})\n", "\n", "# Determine output path based on environment\n", "if KAGGLE:\n", "\n", " # On Kaggle, save to current directory for submission\n", " submission_file = 'submission.csv'\n", "\n", "else:\n", "\n", " # Locally, save to ../data/ directory\n", " # Create directory if it doesn't exist\n", " data_dir = Path('../data')\n", " data_dir.mkdir(parents=True, exist_ok=True)\n", " submission_file = data_dir / 'submission.csv'\n", "\n", "# Save submission file and display preview\n", "submission_df.to_csv(submission_file, index=False)\n", "submission_df.head()" ] } ], "metadata": { "kaggle": { "accelerator": "none", "dataSources": [ { "databundleVersionId": 14272474, "sourceId": 91723, "sourceType": "competition" } ], "dockerImageVersionId": 31192, "isGpuEnabled": false, "isInternetEnabled": true, "language": "python", "sourceType": "notebook" }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.12" }, "papermill": { "default_parameters": {}, "duration": 12.320018, "end_time": "2025-12-04T16:46:03.599457", "environment_variables": {}, "exception": null, "input_path": "__notebook__.ipynb", "output_path": "__notebook__.ipynb", "parameters": {}, "start_time": "2025-12-04T16:45:51.279439", "version": "2.6.0" } }, "nbformat": 4, "nbformat_minor": 5 }