Python Data Cleaning Cookbook: Detect and remove dirty data and extract key insights with pandas, machine learning and ChatGPT, Spark, and more

Learn  the intricacies of data description, issue identification, and  practical problem-solving, armed with essential techniques and expert  tips.

Key Features

• Get to grips with various data cleaning techniques to reveal key insights.
• Manipulate data of different complexities to shape them into the right form according to your business needs..
• Clean,  monitor, and validate large data volumes to diagnose problems using  cutting-edge methodologies including Machine learning and AI.

Book Description

Jumping  into data analysis without proper data cleaning will certainly lead to  incorrect results. The Python Data Cleaning Cookbook will show you tools  and techniques for cleaning and handling data with Python for better  outcomes. You will begin by getting familiar with the shape of data by  using practices that can be deployed routinely with most data sources.

Fully  updated to the latest version of Python and all relevant tools, this  book will teach you how to manipulate data to get it into a useful form.  The current edition emphasizes advanced techniques like machine  learning and AI-specific approaches to data cleaning along with the  conventional ones. You will learn how to filter and summarize data to  gain insights and better understand what makes sense and what does not,  along with discovering how to operate on data to address the issues  you've identified. Next, you'll cover recipes for using supervised  learning and Naive Bayes analysis to identify unexpected values and  classification errors and generate visualizations for exploratory data  analysis (EDA) to identify unexpected values. Finally, you'll build  functions and classes that you can reuse without modification when you  have new data.

By the end of this Data Cleaning book, you'll know how to clean data and diagnose problems within it.

What you will learn

• Find out how to read and analyze data from a variety of sources
• Produce summaries of the attributes of datasets, columns, and rows
• Filter data and select columns of interest that satisfy given criteria
• Address messy data issues, including working with dates and missing values
• Improve your productivity in Python pandas by using method chaining
• Use visualizations to gain additional insights and identify potential data issues
• Enhance your ability to learn what is going on in your data
• Build user-defined functions and classes to automate data cleaning

Who This Book Is For

This  book is for anyone looking for ways to handle messy, duplicate, and  poor data using different Python tools and techniques. The book takes a  recipe-based approach to help you to learn how to clean and manage data  with practical examples. Working knowledge of Python programming is all  you need to get the most out of the book.

Table of Contents

1. Anticipating Data Cleaning Issues when Importing Tabular Data into Pandas
2. Anticipating Data Cleaning Issues when Importing HTML, JSON, and streaming into Pandas
3. Taking the Measure of Your Data
4. Identifying Missing Values and Outliers in Subsets of Data
5. Using Visualizations for the Identification of Unexpected Values
6. Cleaning and Exploring Data with Series Operations
7. Working with Missing Data
8. Fixing Messy Data When Aggregating
9. Addressing Data Issues When Combining Data Frames
10. Tidying and Reshaping Data
11. Automate Data Cleaning with User-Defined Functions and Classes