Parquet: Fix notNaN for columns missing from files

[yangshangqing95]

Fix #17632

Fix an NPE when a notNaN predicate references a column that has an initial default but is absent from an older Parquet file.

Treat missing null metadata conservatively and add regression coverage for columns absent from Parquet files.

[RussellSpitzer]

I think we should match the pattern in the other checks and just relocate if (mayContainNulls) to under the "hasNonDictPages"

The reason being the "isFallback.get(id) " would essentially be doing the "is this column there" check for us.

@Override
public <T> Boolean notNaN(BoundReference<T> ref) {
  int id = ref.fieldId();

  Boolean hasNonDictPage = isFallback.get(id);
  if (hasNonDictPage == null || hasNonDictPage) {
    return ROWS_MIGHT_MATCH;
  }

  if (mayContainNulls.get(id)) {
    return ROWS_MIGHT_MATCH;
  }

  Set<T> dictionary = dict(id, comparatorForNaNPredicate(ref));
  return dictionary.stream().allMatch(NaNUtil::isNaN) ? ROWS_CANNOT_MATCH : ROWS_MIGHT_MATCH;
}

This is basically the same as all the others

public <T> Boolean isNaN(BoundReference<T> ref) {
  int id = ref.fieldId();
  
Boolean hasNonDictPage = isFallback.get(id);
  if (hasNonDictPage == null || hasNonDictPage) {
    return ROWS_MIGHT_MATCH;
  }
  
Set<T> dictionary = dict(id, comparatorForNaNPredicate(ref));
  return dictionary.stream().anyMatch(NaNUtil::isNaN) ? ROWS_MIGHT_MATCH : ROWS_CANNOT_MATCH;
}

[yangshangqing95]

good point, done.

[RussellSpitzer]

+1 on Tests, Thanks for covering these!

[RussellSpitzer]

Thanks @yangshangqing95 !

[yangshangqing95]

Thanks @RussellSpitzer for reviewing!