Quick answer
Learn pandas DataFrame operations for SQL Server data: inspect, filter, sort, clean missing values, convert dtypes, group process data and prepare Excel reports.
Architecture: pandas Filter + Clean + Analyze
Code-Based Architecture
pandas Filter + Clean + Analyze
1
SQL Rows
Read SQL result set
pd.read_sql(...)2
DataFrame
Tabular Python data
df3
Inspect
Check shape, columns, dtypes
df.info()4
Clean
Handle null/type issues
fillna / astype5
Analyze
Filter, group, aggregate
groupby / mean6
Report
Send clean data forward
to_excel(...)SQL Rows→DataFrame→Inspect→Clean→Analyze→Report
1. Inspect the DataFrame first
print(df.head())
print(df.shape)
print(df.columns)
print(df.dtypes)
print(df.info())2. Filter furnace records
sqf2 = df[df["SQF_No"] == 2]
high_temp = df[df["Temp_Act"] > 900]
print(high_temp.head())3. Sort latest events
latest = df.sort_values("DT", ascending=False)
print(latest.head(10))4. Clean missing values
print(df.isna().sum())
df["Fan_Status"] = df["Fan_Status"].fillna("UNKNOWN")
df = df.dropna(subset=["DT"])5. Convert dtypes safely
df["DT"] = pd.to_datetime(df["DT"], errors="coerce")
df["SQF_No"] = pd.to_numeric(df["SQF_No"], errors="coerce")
df["Temp_Act"] = pd.to_numeric(df["Temp_Act"], errors="coerce")6. Group and analyze process data
summary = ( df.groupby("SQF_No", dropna=False) .agg(Records=("SQF_No", "size"), Avg_Temp=("Temp_Act", "mean"), Max_Temp=("Temp_Act", "max")) .reset_index()
)
print(summary)Frequently Asked Questions
What is a pandas DataFrame?
A DataFrame is a two-dimensional labeled table used to manipulate and analyze structured data in Python.
Why check df.dtypes after reading SQL Server?
The SQL driver and missing values can influence pandas dtypes, which affects calculations, filtering and Excel formatting.
What does groupby do?
groupby divides rows into groups by one or more keys and lets you calculate aggregates such as count, mean, minimum or maximum.