Skip to content

Test rast1 mask can throw exceptions during raster operations #198

@SorooshMani-NOAA

Description

@SorooshMani-NOAA

For some of the unit tests we create 3 rasters (rast1, rast2 and rast3). The first one is created using a normal numpy array and the latter two using numpy masked arrays. While performing operations like fill_nodata on the raster object rast1 can run into an issue due to the mask value which is not None, but also not an array!

rast_z_1 = np.ones_like(rast_xy[0]) * 10
rast_z_2 = np.ma.MaskedArray(
np.ones_like(rast_xy[0]) * 10,
mask=np.random.random(size=rast_xy[0].shape) < 0.2,
fill_value=np.nan
)
rast_z_3 = np.ma.MaskedArray(
np.ones_like(rast_xy[0]) * 10,
mask=np.random.random(size=rast_xy[0].shape) < 0.2,
fill_value=-1e+15
)

Looking at it rast object created from rast1 path using PDB we see:

> p rast.src.read(masked=True)
masked_array(
  data=[[[10., 10., 10., ..., 10., 10., 10.],
         [10., 10., 10., ..., 10., 10., 10.],
         [10., 10., 10., ..., 10., 10., 10.],
         ...,
         [10., 10., 10., ..., 10., 10., 10.],
         [10., 10., 10., ..., 10., 10., 10.],
         [10., 10., 10., ..., 10., 10., 10.]]],
  mask=False,
  fill_value=1e+20)
> p rast.src.read(masked=True).mask
np.False_

Which results in:

ERROR: test_fill_nodata (tests.api.test_pickle.TestRasterPickling.test_fill_nodata)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/smani/workarea/sandbox/ocsmesh/tests/api/test_pickle.py", line 145, in test_fill_nodata
    rast.fill_nodata()
  File "/home/smani/workarea/sandbox/ocsmesh/ocsmesh/raster.py", line 925, in fill_nodata
    fillnodata(self.src.read(window=window, masked=True)),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/smani/miniforge3/envs/stormwf/lib/python3.11/site-packages/rasterio/env.py", line 410, in wrapper
    return f(*args, **kwds)
           ^^^^^^^^^^^^^^^^
  File "/home/smani/miniforge3/envs/stormwf/lib/python3.11/site-packages/rasterio/fill.py", line 72, in fillnodata
    return _fillnodata(image, mask, max_search_distance, smoothing_iterations)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "rasterio/_fill.pyx", line 33, in rasterio._fill._fillnodata
  File "rasterio/_io.pyx", line 2231, in rasterio._io.MemoryDataset.__init__
ValueError: arr must be 2D or 3D array

Note that here the array refers to the mask array and not the data. If one uses rast2 in this case, the test works fine.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions