Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"gdal vector filter": add --where and --fields; add "gdal vector select" #11794

Merged
merged 8 commits into from
Feb 11, 2025

Conversation

rouault
Copy link
Member

@rouault rouault commented Feb 4, 2025

No description provided.

@rouault rouault added funded through GSP Work funded through the GDAL Sponsorship Program gdal_cli Anything related to the new 3.11 "gdal" CLI frontend labels Feb 4, 2025
@rouault rouault added this to the 3.11.0 milestone Feb 4, 2025
@rouault rouault force-pushed the gdal_vector_filter branch from 0671c17 to 7d406c7 Compare February 4, 2025 21:13
doc/source/programs/gdal_vector_filter.rst Outdated Show resolved Hide resolved
doc/source/programs/gdal_vector_filter.rst Outdated Show resolved Hide resolved
private:
bool m_bIsOK = true;
OGRLayer *const m_poSrcLayer;
OGRFeatureDefn *const m_poFeatureDefn = nullptr;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we use a unique pointer here or is it already using the self made share ptr with reference/dereference?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • m_poSrcLayer transformed into a reference
  • for m_poFeatureDefn, we can't do much about that since OGRFeatureDefn includes its own ref counting mechanism. Changing that (or studying if we can...) is I believe in the todo list of @dbaston. I've some doubts we can do that in the GDAL 3.x series

@dbaston
Copy link
Member

dbaston commented Feb 10, 2025

I would consider putting --fields in a separate command (select ?) that could be chained with filter, or not. (A bit like the verbs in dplyr, I guess)

@rouault rouault changed the title gdal vector filter: add --where and --fields; add documentation "gdal vector filter": add --where and --fields; add "gdal vector select" Feb 11, 2025
@rouault
Copy link
Member Author

rouault commented Feb 11, 2025

I would consider putting --fields in a separate command (select ?)

done: "gdal vector select" added

@jratike80
Copy link
Collaborator

Does --fields support EXCLUDE?

@rouault
Copy link
Member Author

rouault commented Feb 11, 2025

Does --fields support EXCLUDE?

I've added an --exclude switch

doc/source/programs/gdal_vector_select.rst Outdated Show resolved Hide resolved
doc/source/programs/gdal_vector_select.rst Outdated Show resolved Hide resolved
doc/source/programs/migration_guide_to_gdal_cli.rst Outdated Show resolved Hide resolved
A field is only selected once, even if mentioned several times in the list.

Geometry fields can also be specified in the list. If the source layer has
no explicit name for the geometry field, ``_ogr_geometry_`` must be used to
Copy link
Member

@dbaston dbaston Feb 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think needing to include _ogr_geometry_ explicitly might be surprising for users. I can imagine including it by default unless an --exclude-geom argument is passed. Would be curious what others think.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, the R sf package behaves this way (some discussion at r-spatial/sf#121).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think needing to include _ogr_geometry_ explicitly might be surprising for users. I can imagine including it by default unless an --exclude-geom argument is passed

that's a good question. I'm not sure either. One thing that is certain is that the current difference in behavior between OGRSQL where "SELECT" automatically selects the geometry field (unless excluded thanks to your recent work on EXCLUDE) compared to all other SQL dialects where the geometry field must be explicitly selected surprises users a lot.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ticketed at #11839

m_poFeatureDefn->Dereference();
}

bool IncludeFields(const std::vector<std::string> &selectedFields,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only thought I have on the implementation is whether the processing could be handled by creating an in-memory VRT that could either be materialized or not, depending on the output format. I don't think anything about this interface precludes that, so I could look into it later on as part of #10066.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment could apply to #11827

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. Regarding VRT'ization of all of this, I believe both GDAL and OGR VRT will have to be extended to support accepting a datasource and a pipeline string, rather than trying to extend the capabilities of the VRT drivers themselves to support all algorithms, which would involve duplicating all the code.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ticketed at #11837

autotest/utilities/test_gdalalg_vector_select.py Outdated Show resolved Hide resolved
@rouault rouault merged commit 51ee293 into OSGeo:master Feb 11, 2025
37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
funded through GSP Work funded through the GDAL Sponsorship Program gdal_cli Anything related to the new 3.11 "gdal" CLI frontend
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants