Numpy/Pandas clean way to check if a specific value is NaN

  • A+
Category:Languages

How can I check if a given value is NaN?

e.g. if (a == np.NaN) (doesn't work)

Before you downvote, please note that:

  • Numpy's isnan method throws errors with data types like string
  • Pandas docs only provide methods to drop rows containing NaNs, or ways to check if/when DataFrame contains NaNs. I'm asking about checking if a specific value is NaN.
  • Relevant Stackoverflow questions and Google search results seem to be about checking "if any value is NaN" or "which values in a DataFrame"

There must be a clean way to check if a given value is NaN?


You can use the inate property that NaN != NaN

so a == a will return False if a is NaN

This will work even for strings

Example:

In[52]: s = pd.Series([1, np.NaN, '', 1.0]) s  Out[52]:  0      1 1    NaN 2        3      1 dtype: object   for val in s:     print(val==val) True False True True 

This can be done in a vectorised manner:

In[54]: s==s  Out[54]:  0     True 1    False 2     True 3     True dtype: bool 

but you can still use the method isnull on the whole series:

In[55]: s.isnull()  Out[55]:  0    False 1     True 2    False 3    False dtype: bool 

UPDATE

As noted by @piRSquared if you compare None==None this will return True but pd.isnull will return True so depending on whether you want to treat None as NaN you can still use == for comparison or pd.isnull if you want to treat None as NaN

Comment

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen: