Check for NaN in Pandas DataFrame (examples included)

Here are 4 ways to check for NaN in Pandas DataFrame:

(1) Check for NaN under a single DataFrame column:

df['your column name'].isnull().values.any()

(2) Count the NaN under a single DataFrame column:

df['your column name'].isnull().sum()

(3) Check for NaN under an entire DataFrame:

df.isnull().values.any()

(4) Count the NaN under an entire DataFrame:

df.isnull().sum().sum()

Examples of checking for NaN in Pandas DataFrame

(1) Check for NaN under a single DataFrame column

In the following example, we’ll create a DataFrame with a set of numbers and 3 NaN values:

import pandas as pd
import numpy as np

numbers = {'set_of_numbers': [1,2,3,4,5,np.nan,6,7,np.nan,8,9,10,np.nan]}
df = pd.DataFrame(numbers,columns=['set_of_numbers'])
print (df)

You’ll now see the DataFrame with the 3 NaN values:

Check for NaN in Pandas DataFrame

You can then use the following template in order to check for NaN under a single DataFrame column:

df['your column name'].isnull().values.any()

For our example, the DataFrame column is ‘set_of_numbers.’

And so, the code to check whether a NaN value exists under the ‘set_of_numbers’ column is as follows:

import pandas as pd
import numpy as np

numbers = {'set_of_numbers': [1,2,3,4,5,np.nan,6,7,np.nan,8,9,10,np.nan]}
df = pd.DataFrame(numbers,columns=['set_of_numbers'])

check_for_nan = df['set_of_numbers'].isnull().values.any()
print (check_for_nan)

Run the code, and you’ll get ‘True’ which confirms the existence of NaN values under the DataFrame column:

True Statement

And if you want to get the actual breakdown of the instances where NaN values exist, then you may remove .values.any() from the code. So the complete syntax to get the breakdown would look as follows:

import pandas as pd
import numpy as np

numbers = {'set_of_numbers': [1,2,3,4,5,np.nan,6,7,np.nan,8,9,10,np.nan]}
df = pd.DataFrame(numbers,columns=['set_of_numbers'])

check_for_nan = df['set_of_numbers'].isnull()
print (check_for_nan)

You’ll now see the 3 instances of the NaN values:

NaN values

Here is another approach where you can get all the instances where a NaN value exists:

import pandas as pd
import numpy as np

numbers = {'set_of_numbers': [1,2,3,4,5,np.nan,6,7,np.nan,8,9,10,np.nan]}
df = pd.DataFrame(numbers,columns=['set_of_numbers'])

df.loc[df['set_of_numbers'].isnull(),'value_is_NaN'] = 'Yes'
df.loc[df['set_of_numbers'].notnull(), 'value_is_NaN'] = 'No'

print (df)

You’ll now see a new column (called ‘value_is_NaN’), which indicates all the instances where a NaN value exists:

Check for NaN in Pandas DataFrame

(2) Count the NaN under a single DataFrame column

You can apply this syntax in order to count the NaN values under a single DataFrame column:

df['your column name'].isnull().sum()

Here is the syntax for our example:

import pandas as pd
import numpy as np

numbers = {'set_of_numbers': [1,2,3,4,5,np.nan,6,7,np.nan,8,9,10,np.nan]}
df = pd.DataFrame(numbers,columns=['set_of_numbers'])

count_nan = df['set_of_numbers'].isnull().sum()
print ('Count of NaN: ' + str(count_nan))

You’ll then get the count of 3 NaN values:

Count of NaN

And here is another approach to get the count:

import pandas as pd
import numpy as np

numbers = {'set_of_numbers': [1,2,3,4,5,np.nan,6,7,np.nan,8,9,10,np.nan]}
df = pd.DataFrame(numbers,columns=['set_of_numbers'])

df.loc[df['set_of_numbers'].isnull(),'value_is_NaN'] = 'Yes'
df.loc[df['set_of_numbers'].notnull(), 'value_is_NaN'] = 'No'

count_nan = df.loc[df['value_is_NaN']=='Yes'].count()
print (count_nan)

As before, you’ll get the count of 3 instances of NaN values:

Value in NaN

(3) Check for NaN under an entire DataFrame

Now let’s add a second column into the original DataFrame. This column would include another set of numbers with NaN values:

import pandas as pd
import numpy as np

numbers = {'first_set_of_numbers': [1,2,3,4,5,np.nan,6,7,np.nan,8,9,10,np.nan],
           'second_set_of_numbers': [11,12,np.nan,13,14,np.nan,15,16,np.nan,np.nan,17,np.nan,19]}
df = pd.DataFrame(numbers,columns=['first_set_of_numbers','second_set_of_numbers'])

print (df)

Run the code, and you’ll get 8 instances of NaN values across the entire DataFrame:

Ho to Check for NaN in Pandas DataFrame

You can then apply this syntax in order to verify the existence of NaN values under the entire DataFrame:

df.isnull().values.any()

For our example:

import pandas as pd
import numpy as np

numbers = {'first_set_of_numbers': [1,2,3,4,5,np.nan,6,7,np.nan,8,9,10,np.nan],
           'second_set_of_numbers': [11,12,np.nan,13,14,np.nan,15,16,np.nan,np.nan,17,np.nan,19]}
df = pd.DataFrame(numbers,columns=['first_set_of_numbers','second_set_of_numbers'])

check_nan_in_df = df.isnull().values.any()
print (check_nan_in_df)

Once you run the code, you’ll get ‘True’ which confirms the existence of NaN values in the DataFrame:

True

You can get a further breakdown by removing .values.any() from the code:

import pandas as pd
import numpy as np

numbers = {'first_set_of_numbers': [1,2,3,4,5,np.nan,6,7,np.nan,8,9,10,np.nan],
           'second_set_of_numbers': [11,12,np.nan,13,14,np.nan,15,16,np.nan,np.nan,17,np.nan,19]}
df = pd.DataFrame(numbers,columns=['first_set_of_numbers','second_set_of_numbers'])

check_nan_in_df = df.isnull()
print (check_nan_in_df)

Here is the result of the breakdown:

NaN values check

(4) Count the NaN under an entire DataFrame

You may now use this template to count the NaN values under the entire DataFrame:

df.isnull().sum().sum()

Here is the code for our example:

import pandas as pd
import numpy as np

numbers = {'first_set_of_numbers': [1,2,3,4,5,np.nan,6,7,np.nan,8,9,10,np.nan],
           'second_set_of_numbers': [11,12,np.nan,13,14,np.nan,15,16,np.nan,np.nan,17,np.nan,19]}
df = pd.DataFrame(numbers,columns=['first_set_of_numbers','second_set_of_numbers'])

count_nan_in_df = df.isnull().sum().sum()
print ('Count of NaN: ' + str(count_nan_in_df))

You’ll then get the total count of 8:

Count of values

And if you want to get the count of NaN by column, then you may use this code:

import pandas as pd
import numpy as np

numbers = {'first_set_of_numbers': [1,2,3,4,5,np.nan,6,7,np.nan,8,9,10,np.nan],
           'second_set_of_numbers': [11,12,np.nan,13,14,np.nan,15,16,np.nan,np.nan,17,np.nan,19]}
df = pd.DataFrame(numbers,columns=['first_set_of_numbers','second_set_of_numbers'])

count_nan_in_df = df.isnull().sum()
print (count_nan_in_df)

And here is the result:

nan

You just saw how to check for NaN in Pandas DataFrame. Alternatively you may: