Pandas for beginners: cleaning and analysing data with Python

By Axyra Nova Editorial · 13 October 2026 · 5 min read · Data and AI

Pandas is a Python library for working with tables of data, similar to spreadsheets but far more powerful and repeatable. Analysts use it to load data from files and databases, clean messy values, combine datasets and summarise results. If you already know some spreadsheet work or SQL, pandas will feel familiar once you learn its vocabulary.

This guide introduces pandas for beginners, covering the operations you will use most often.

Why use pandas

Compared with spreadsheets, pandas lets you:

Work with larger datasets comfortably.
Repeat the same cleaning and analysis steps automatically.
Combine data from many files and sources.
Keep a clear record of every step in code.
Move on to visualisation, statistics and machine learning in the same language.

The official pandas documentation includes a getting started guide and a full reference.

Setting up

You can use pandas in several ways:

Online notebooks, such as those provided by Kaggle, need no installation. Kaggle's pandas course runs entirely in the browser.
A local install of Python with pandas, often through a distribution that includes Jupyter notebooks.

Notebooks are ideal for learning because you can run code step by step and see results immediately. Kaggle Learn also offers a short free Python course if you need a refresher before starting on pandas.

The two main structures

DataFrame: a table with rows and named columns.
Series: a single column.

Most work involves DataFrames.

Loading data

import pandas as pd

orders = pd.read_csv("orders.csv")

Pandas can also read Excel files, JSON and SQL databases.

Inspecting data

Always look at your data before analysing it:

orders.head()        # first five rows
orders.shape         # number of rows and columns
orders.info()        # column names, types and missing values
orders.describe()    # summary statistics for numeric columns

These commands reveal problems early: wrong types, missing values, unexpected columns.

Selecting and filtering

Select columns:

orders[["order_id", "region", "amount"]]

Filter rows:

large_lagos = orders[(orders["region"] == "Lagos") & (orders["amount"] > 10000)]

Cleaning data

Real data is messy. Common cleaning steps:

Fix data types

orders["order_date"] = pd.to_datetime(orders["order_date"])
orders["amount"] = pd.to_numeric(orders["amount"], errors="coerce")

Standardise text

orders["region"] = orders["region"].str.strip().str.title()

Handle missing values

orders.isna().sum()                 # count missing values per column
orders = orders.dropna(subset=["amount"])
orders["phone"] = orders["phone"].fillna("not provided")

Remove duplicates

orders = orders.drop_duplicates(subset=["order_id"])

Decide carefully how to treat missing or odd values. Dropping rows can bias results; document what you did and why.

Creating new columns

orders["month"] = orders["order_date"].dt.to_period("M")
orders["is_large"] = orders["amount"] > 50000

Grouping and summarising

Grouping is where pandas becomes powerful, much like a pivot table:

by_region = (
    orders.groupby("region")
    .agg(orders=("order_id", "count"), revenue=("amount", "sum"))
    .sort_values("revenue", ascending=False)
)

For a pivot-style summary:

pd.pivot_table(orders, index="region", columns="month", values="amount", aggfunc="sum")

Combining datasets

Merge tables on a shared column, similar to a SQL join:

customers = pd.read_csv("customers.csv")
combined = orders.merge(customers, on="customer_id", how="left")

Check row counts before and after merging. An unexpected increase usually means duplicate keys.

Exporting results

by_region.to_csv("revenue_by_region.csv")
by_region.to_excel("revenue_by_region.xlsx")

Thinking in columns, not cells

The biggest shift for people coming from spreadsheets is that pandas works on whole columns at once. In a spreadsheet, you might write a formula in one cell and drag it down a thousand rows. In pandas, you write one line that applies to the entire column, and it runs on every row together.

This has two benefits. It is much faster on large datasets, and it avoids the hidden errors that creep into spreadsheets when one cell's formula differs from the rest. If you find yourself writing a loop to go through rows one by one, there is usually a column-based way to do the same thing that is simpler and quicker.

Reading error messages

Beginners often feel stuck when an error appears. Python error messages look intimidating, but the most useful information is usually in the last line. Some common ones:

KeyError: the column name you used does not exist. Check spelling, capital letters and extra spaces with orders.columns.
TypeError: an operation does not suit the data type, such as adding text to numbers. Check types with orders.dtypes.
SettingWithCopyWarning: you may be changing a copy of a filtered table rather than the original. Create filtered tables explicitly with .copy() when you plan to modify them.

Searching the exact error message, together with the word pandas, usually leads to a clear explanation.

Quick charts

Pandas can draw simple charts directly, which is useful for checking data before building formal reports:

by_region["revenue"].plot(kind="bar")

A quick bar or line chart often reveals problems that tables hide, such as a sudden gap in dates or one category far larger than expected.

A worked example

A volunteer at a community food bank received monthly spreadsheets of donations from several collection points. Each file used slightly different column names and spellings of food categories, and totals took hours to compile by hand.

She wrote a short pandas notebook that:

Loaded every monthly file from a folder.
Renamed columns to a standard set.
Standardised category names and removed blank rows.
Combined everything into one DataFrame.
Summarised donations by month, location and category.
Exported a summary to Excel for the trustees.

Each new month now took minutes: add the file, rerun the notebook. The trustees could also see trends that had been invisible in separate spreadsheets.

Good habits

Keep raw data unchanged; save cleaned data separately.
Write cleaning steps in order, with short comments.
Check results against a small sample you can verify by hand.
Use clear variable names.
Restart and run the whole notebook from the top before sharing, to make sure it works in order.

Practice project ideas

The fastest way to learn is to answer real questions with real data. Some starting points:

Your own spending: export a few months of bank or mobile money transactions, categorise them and find where your money goes each month.
A public dataset: choose a topic you care about, such as transport, weather or football results, and write five questions before you start.
A small business problem: with permission, analyse a friend's sales or booking records to find their busiest days and most popular services.

For each project, keep the notebook tidy, explain your steps in short notes, and finish with a clear summary of what you found. A finished project like this makes a strong portfolio piece.

Where pandas fits

Pandas pairs naturally with SQL for extracting data and with visualisation libraries for charts. If you are new to data, start with SQL for data analysis and Excel pivot tables, then move to pandas. Our learning platform lists free courses in a sensible order, including Kaggle's Python course.

Frequently asked questions

Do I need to know Python before pandas?

Basic Python helps: variables, lists, functions and loops. A short introductory course is enough to start.

Is pandas suitable for very large datasets?

Pandas handles datasets that fit in memory well. For very large data, analysts often use databases or tools designed for scale.

What is the difference between pandas and Excel?

Excel is interactive and visual; pandas is code-based and repeatable. Many analysts use both.

How long does it take to learn pandas?

The core operations can be learned in a few weeks with regular practice on real data.