fix DataFrame fillna for pandas 3.x compatibility#855
Conversation
pandas 3.0 raises TypeError when filling NaN in a numeric column
with an empty string, breaking the common df.fillna("", inplace=True)
pattern used throughout the MagIC writer and lab-instrument
converters. Replace with df = df.astype(object).fillna("") at every
call site so the result is consistent across pandas 2.x and 3.x.
Two chained-assignment sites in convert_2_magic.py were already
silently broken under pandas 3's copy-on-write rules; the rewrite
fixes those too.
Fixes #840.
Risks
Review testing
|
Test resultsTested by running
Confirms the PR resolves the pandas-3.x regression while remaining compatible with pandas 2.x. |
Spot-check: convert_2_magic with example dataRan four converters touched by the PR end-to-end with each pandas version, using
All four ran cleanly and produced identical record counts under both pandas 2.3.3 and pandas 3.0.2.
|
pandas 3.0 raises TypeError when filling NaN in a numeric column with an empty string, breaking the common
df.fillna("", inplace=True)pattern used throughout the MagIC writer and lab-instrument converters. Replace withdf = df.astype(object).fillna("")at every call site so the result is consistent across pandas 2.x and 3.x.Two chained-assignment sites in convert_2_magic.py were already silently broken under pandas 3's copy-on-write rules; the rewrite fixes those too.
Fixes #840.