How to check models f1 score using cross validation in Python?

This recipe helps you check models f1 score using cross validation in Python

Recipe Objective

After training a model we need a measure to check its performance, their are many scoring metric on which we can score the model's performance. Out of many metric we will be using f1 score to measure our models performance. We will also be using cross validation to test the model on multiple sets of data.

This data science python source code does the following:
1. Classification metrics used for validation of model.
2. Performs train_test_split to seperate training and testing dataset
3. Implements CrossValidation on models and calculating the final result using "F1 Score" method.

So this is the recipe on How we can check model's f1-score using cross validation in Python.

Get Closer To Your Dream of Becoming a Data Scientist with 70+ Solved End-to-End ML Projects

Step 1 - Import the library

from sklearn.model_selection import cross_val_score from sklearn.tree import DecisionTreeClassifier from sklearn.datasets import make_classification

We have imported various modules from differnt libraries such as cross_val_score, DecisionTreeClassifier and make_classification.

Step 2 - Setting up the Data

We are generating a dataset with make_classification function which will generate a classification dataset as per the passed parameters. X, y = make_classification(n_samples = 10000, n_features = 3, n_informative = 3, n_redundant = 0, n_classes = 2, random_state = 42)

Explore More Data Science and Machine Learning Projects for Practice. Fast-Track Your Career Transition with ProjectPro

Step 3 - Model and its accuracy

We are using DecisionTreeClassifier as a model to train the data. We are training the model with cross_validation which will train the data on different training set and it will calculate f1 score for all the test train split. We are printing the f1 score for all the splits in cross validation and we are also printing mean and standard deviation of f1 score. dec_tree = DecisionTreeClassifier() print(cross_val_score(dec_tree, X, y, scoring="f1", cv = 7)) mean_score = cross_val_score(dec_tree, X, y, scoring="f1", cv = 7).mean() std_score = cross_val_score(dec_tree, X, y, scoring="f1", cv = 7).std() print(mean_score) print(std_score) So the output comes as

[0.92254013 0.91392582 0.93802817 0.92426367 0.93614035 0.92210526
 0.9260539 ]

0.9257145721528974

0.006172506932493186

Join Millions of Satisfied Developers and Enterprises to Maximize Your Productivity and ROI with ProjectPro - Read ProjectPro Reviews Now!

Download Materials


What Users are saying..

profile image

Ray han

Tech Leader | Stanford / Yale University
linkedin profile url

I think that they are fantastic. I attended Yale and Stanford and have worked at Honeywell,Oracle, and Arthur Andersen(Accenture) in the US. I have taken Big Data and Hadoop,NoSQL, Spark, Hadoop... Read More

Relevant Projects

Learn How to Build a Linear Regression Model in PyTorch
In this Machine Learning Project, you will learn how to build a simple linear regression model in PyTorch to predict the number of days subscribed.

End-to-End Speech Emotion Recognition Project using ANN
Speech Emotion Recognition using RAVDESS Audio Dataset - Build an Artificial Neural Network Model to Classify Audio Data into various Emotions like Sad, Happy, Angry, and Neutral

Build a Logistic Regression Model in Python from Scratch
Regression project to implement logistic regression in python from scratch on streaming app data.

Multi-Class Text Classification with Deep Learning using BERT
In this deep learning project, you will implement one of the most popular state of the art Transformer models, BERT for Multi-Class Text Classification

Data Analysis of Working Capital Management using Tableau
In this Data Analysis Project using Tableau, you will focus on optimizing working capital by analyzing receivables and payables data using Tableau and build actionable dashboards.

Personalized Medicine: Redefining Cancer Treatment
In this Personalized Medicine Machine Learning Project you will learn to classify genetic mutations on the basis of medical literature into 9 classes.

Walmart Sales Forecasting Data Science Project
Data Science Project in R-Predict the sales for each department using historical markdown data from the Walmart dataset containing data of 45 Walmart stores.

Hands-On Approach to Causal Inference in Machine Learning
In this Machine Learning Project, you will learn to implement various causal inference techniques in Python to determine, how effective the sprinkler is in making the grass wet.

Forecasting Business KPI's with Tensorflow and Python
In this machine learning project, you will use the video clip of an IPL match played between CSK and RCB to forecast key performance indicators like the number of appearances of a brand logo, the frames, and the shortest and longest area percentage in the video.

Time Series Project to Build a Multiple Linear Regression Model
Learn to build a Multiple linear regression model in Python on Time Series Data