Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOGE-106] Instance aggregate/search limit=None means maximum #2121

Open
wants to merge 11 commits into
base: master
Choose a base branch
from
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,11 @@ Changes are grouped as follows
- Support for the `/simulators/models` and `/simulators/models/revisions` API endpoints.
- Support for the `/simulators` and `/simulators/integration` API endpoints.

## [7.73.5] - 2025-02-26
### Fixed
- The `client.data_modeling.instances.aggregate()` method now correctly returns maximum, 1000, results when setting
the `lmit` parameter to `None`, `-1`, or `math.inf`.

## [7.73.4] - 2025-02-24
### Fixed
- An issue with `DatapointsAPI.retrieve_latest` and usage of `instance_id` when using `ignore_unknown_ids=True`
Expand Down
25 changes: 17 additions & 8 deletions cognite/client/_api/data_modeling/instances.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,15 +71,15 @@
)
from cognite.client.data_classes.data_modeling.views import View
from cognite.client.data_classes.filters import _BASIC_FILTERS, Filter, _validate_filter
from cognite.client.utils._auxiliary import load_yaml_or_json
from cognite.client.utils._auxiliary import is_unlimited, load_yaml_or_json
from cognite.client.utils._concurrency import ConcurrencySettings
from cognite.client.utils._identifier import DataModelingIdentifierSequence
from cognite.client.utils._retry import Backoff
from cognite.client.utils._text import random_string
from cognite.client.utils.useful_types import SequenceNotStr

if TYPE_CHECKING:
from cognite.client import CogniteClient
from cognite.client import ClientConfig, CogniteClient

_FILTERS_SUPPORTED: frozenset[type[Filter]] = _BASIC_FILTERS.union(
{filters.Nested, filters.HasData, filters.MatchAll, filters.Overlaps, filters.InstanceReferences}
Expand Down Expand Up @@ -164,6 +164,10 @@ def _load(data: dict, cognite_client: CogniteClient | None = None) -> NodeApply
class InstancesAPI(APIClient):
_RESOURCE_PATH = "/models/instances"

def __init__(self, config: ClientConfig, api_version: str | None, cognite_client: CogniteClient) -> None:
super().__init__(config, api_version, cognite_client)
self._AGGREGATE_LIMIT = 1000
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't these be class variables instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, that could be. Consistently in the SDK we have limits as instance
attributes. Suggest that if we move these two class variables, we do it everywhere, which should in that case be a separate PR.


@overload
def __call__(
self,
Expand Down Expand Up @@ -1202,7 +1206,7 @@ def aggregate(
target_units: list[TargetUnit] | None = None,
space: str | SequenceNotStr[str] | None = None,
filter: Filter | dict[str, Any] | None = None,
limit: int = DEFAULT_LIMIT_READ,
limit: int | None = DEFAULT_LIMIT_READ,
) -> AggregatedNumberedValue: ...

@overload
Expand All @@ -1217,7 +1221,7 @@ def aggregate(
target_units: list[TargetUnit] | None = None,
space: str | SequenceNotStr[str] | None = None,
filter: Filter | dict[str, Any] | None = None,
limit: int = DEFAULT_LIMIT_READ,
limit: int | None = DEFAULT_LIMIT_READ,
) -> list[AggregatedNumberedValue]: ...

@overload
Expand All @@ -1232,7 +1236,7 @@ def aggregate(
target_units: list[TargetUnit] | None = None,
space: str | SequenceNotStr[str] | None = None,
filter: Filter | dict[str, Any] | None = None,
limit: int = DEFAULT_LIMIT_READ,
limit: int | None = DEFAULT_LIMIT_READ,
) -> InstanceAggregationResultList: ...

def aggregate(
Expand All @@ -1246,7 +1250,7 @@ def aggregate(
target_units: list[TargetUnit] | None = None,
space: str | SequenceNotStr[str] | None = None,
filter: Filter | dict[str, Any] | None = None,
limit: int = DEFAULT_LIMIT_READ,
limit: int | None = DEFAULT_LIMIT_READ,
) -> AggregatedNumberedValue | list[AggregatedNumberedValue] | InstanceAggregationResultList:
"""`Aggregate data across nodes/edges <https://developer.cognite.com/api/v1/#tag/Instances/operation/aggregateInstances>`_

Expand All @@ -1260,7 +1264,8 @@ def aggregate(
target_units (list[TargetUnit] | None): Properties to convert to another unit. The API can only convert to another unit if a unit has been defined as part of the type on the underlying container being queried.
space (str | SequenceNotStr[str] | None): Restrict instance aggregate query to the given space (or list of spaces).
filter (Filter | dict[str, Any] | None): Advanced filtering of instances.
limit (int): Maximum number of instances to return. Defaults to 25.
limit (int | None): Maximum number of instances to return. Defaults to 25. Will return the maximum number
of results (1000) if set to None, -1, or math.inf.

Returns:
AggregatedNumberedValue | list[AggregatedNumberedValue] | InstanceAggregationResultList: Node or edge aggregation results.
Expand All @@ -1282,7 +1287,11 @@ def aggregate(

self._validate_filter(filter)
filter = self._merge_space_into_filter(instance_type, space, filter)
body: dict[str, Any] = {"view": view.dump(camel_case=True), "instanceType": instance_type, "limit": limit}
body: dict[str, Any] = {
"view": view.dump(camel_case=True),
"instanceType": instance_type,
"limit": self._AGGREGATE_LIMIT if is_unlimited(limit) or limit is None else limit,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"limit": self._AGGREGATE_LIMIT if is_unlimited(limit) or limit is None else limit,
"limit": self._AGGREGATE_LIMIT if is_unlimited(limit) else limit,

ref its implementation:

def is_unlimited(limit: float | int | None) -> bool:
    return limit in {None, -1, math.inf}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The limit is None is just for mypy.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My mypy is happy without it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok then it is me that did not trust mypy

}
is_single = isinstance(aggregates, (dict, MetricAggregation))
aggregate_seq: Sequence[MetricAggregation | dict] = (
[aggregates] if isinstance(aggregates, (dict, MetricAggregation)) else aggregates
Expand Down
2 changes: 1 addition & 1 deletion cognite/client/_version.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
from __future__ import annotations

__version__ = "7.73.4"
__version__ = "7.73.5"

__api_subversion__ = "20230101"
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[tool.poetry]
name = "cognite-sdk"

version = "7.73.4"
version = "7.73.5"

description = "Cognite Python SDK"
readme = "README.md"
Expand Down