model for instance segmentation

[MattiaRigi97]

Hi,

First of all, thank you for the work you have done. Really great.

I wanted to ask if the instance segmentation model will also be available.

Thanks,
Mattia

[calebrob6]

Hey @MattiaRigi97 -- we didn't train any instance segmentation models for the paper (but we provide the masks in case you want to try!). Our current inference process of "run semantic segmentation" then "polygonize" gives the same effect as instance segmentation however.

[fangzp]

Hi, just adding on to this—looking at the polygonize function, I had some questions about the way the mask is converted to polygons using rasterio. Does rasterio simply convert each connected component into a polygon? What if there are two fields which are 'touching' (sharing some portion of a boundary), would that be turned into one polygon? If one were doing boundary detection for example, wouldn't this require connecting the fragmented contours of the boundary prediction, cleaning up small or spurious connected components, and then determining what's "inside" versus "outside" the field contour?

From what I can understand instance segmentation of complete closed-field polygons would probably provide fairly different results from semantic, not to mention panoptic segmentation, and it would take nontrivial work to get from the prediction heatmaps from Fig 15 of the paper to usable polygons e.g. for doing spatial statistics of field distribution and size.

Apologies for the length of this follow-up!

[cholmes]

Thanks for the questions @fangzp. We've been actively working on polygonize, so it's a bit in flux, but I'll answer from the latest.

One general point to make though is that we're quite open to other polygonize algorithms. To start we're going for 'something decent', and hoping others are inspired to improve upon it - this is the same approach for the baseline models. See #55 and feel free to comment there with ideas, it'd be awesome to put.

Does rasterio simply convert each connected component into a polygon?

Roughly, yes. It's a relatively simple approach.

What if there are two fields which are 'touching' (sharing some portion of a boundary), would that be turned into one polygon?

I've mostly been using the three class one, where there's a 'boundary' and a 'field', and it just 'joins' if the field is touching, not the boundary. I'm not actually sure what happens with the two class one - I'll try to test it out next.

If one were doing boundary detection for example, wouldn't this require connecting the fragmented contours of the boundary prediction, cleaning up small or spurious connected components, and then determining what's "inside" versus "outside" the field contour?

Yup, ideally :) We added a very small bit of clean-up, though simpler than what you're describing - just removing the disconnected bits by a configurable amount (default 500 meters / ~5 pixels), and an option to just remove holes. Definitely would love contributions for better polygonization algorithms. The main goal for the first one was just to get into vector space and do fiboa output, for GIS people to get tangible results. The core ML team was also discussing deep learning models that would go straight to vector output, which would also be interesting to explore.

From what I can understand instance segmentation of complete closed-field polygons would probably provide fairly different results from semantic, not to mention panoptic segmentation, and it would take nontrivial work to get from the prediction heatmaps from Fig 15 of the paper to usable polygons e.g. for doing spatial statistics of field distribution and size.

I'll defer to the ML experts on this one, but yes, there's certainly more to do to get to polygons that are usable enough for accurate spatial statistics of field distribution and size. Our hope to start is to provide the foundations so everyone can innovate much more on these more interesting problems without having to reinvent all the data gathering & harmonization stuff each time, and without forcing everyone to make all their own tooling so the data can be used by 'normal' geospatial people. Our hope is that many people can collaborate on each of the parts, and the overall outputs get better and better as we all work together.