Checklist

• I have searched the existing issues for similar issues.
• I added a very descriptive title to this issue.
• I have provided sufficient information below to help reproduce this issue.

Summary

According to https://docs.streamlit.io/develop/api-reference/data/st.column_config/st.column_config.textcolumn (applies to other column types as well) the default value for empty cells can be specified.

Consider a st.data_editor with num_rows="dynamic". New rows with empty cells appear not take on the default value of None but are set to math.nan instead which is unexpected.
To be clear, default values which are different from None are correctly applied.

Curiously, math.nan has the property that it evaluates as true in an if statement, which is opposite of None (the stated default value) which evaluates as false. This subtle difference caused me a lot of headache because I used to be able to detect empty cells by if cell_value: statements. Now, I have to detect empty cells like so if not math.isnan(cell_value): which is much less readable.

Apparently, this behavior was introduced by a change in pandas for versions at or above 3.0.0.

Reproducible Code Example

import streamlit as st

data = st.data_editor(
  [
    {"Text": "Row 1"},
  ],
  column_config={
    "Text": st.column_config.TextColumn(None, default=None),
  },
  num_rows="dynamic",
)

def dump_content():
    st.write(data)

st.button("Dump content", on_click=dump_content)

Steps To Reproduce

1. Install streamlit and pandas>=3.0.0
2. Run app
3. Add new row to table
4. Click button Dump content
5. Observe NaN value in output for empty row

Expected Behavior

Current Behavior

Is this a regression?

• Yes, this used to work in a previous version.

Debug info

• Streamlit version: 1.56
• Python version: 3.13.7
• Operating System: Ubuntu
• Browser: Chrome
  Bug is only observed with pandas>=3.0.0.

Additional Information

No response