Ordinal Encoding Sklearn.Preprocessing error

Using OrdinalEncoder library, I came across this error:
Input contains NaN, infinity or a value too large for dtype('float64') (Link below)

The article says

This method will not work if your feature has NaN values. Those need to be addressed prior to running .fit_transform


My question is how to fix this. assuming that the data does not have NaN values.