Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow trigger side filtering on aggregation keys #1149

Merged
merged 23 commits into from
Feb 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
05d7a24
Filter Aggregatable values
ThomasQuesadilla Jan 22, 2024
6339470
Spec change first draft
ThomasQuesadilla Jan 23, 2024
b8e5c9c
Merge branch 'main' into tquintanilla-agg-value-filter
ThomasQuesadilla Jan 23, 2024
80db5cd
adds tests
ThomasQuesadilla Jan 23, 2024
dc9f92a
updates struct call to ignore warnings
ThomasQuesadilla Jan 23, 2024
3c9be18
formatting
ThomasQuesadilla Jan 24, 2024
d4e0ace
Renames struct, clean up algorithms
ThomasQuesadilla Jan 24, 2024
8ff952b
nits
ThomasQuesadilla Jan 24, 2024
282caf1
key identifer constant fix
ThomasQuesadilla Jan 24, 2024
bca1a16
Aligns with dedup key algo
ThomasQuesadilla Jan 25, 2024
cd0e5e2
adds sub algorithm, refactoring
ThomasQuesadilla Jan 25, 2024
9f44657
fix empty list, var
ThomasQuesadilla Jan 26, 2024
582ef72
Apply suggestions from code review
ThomasQuesadilla Jan 26, 2024
b810420
specified individual element of aggregation values
ThomasQuesadilla Jan 26, 2024
139c1e5
formatting
ThomasQuesadilla Jan 26, 2024
6169620
Merge branch 'main' into tquintanilla-agg-value-filter
ThomasQuesadilla Jan 31, 2024
680462f
Update filter name
ThomasQuesadilla Jan 31, 2024
bb35870
Update explainer with new registration option
ThomasQuesadilla Feb 7, 2024
97fa376
renames, minor changes to explainer
ThomasQuesadilla Feb 12, 2024
dcc661c
update aggregatable data check
ThomasQuesadilla Feb 12, 2024
f4f3d4c
removes ordered qualifier
ThomasQuesadilla Feb 12, 2024
49ab7ff
link values to definition
ThomasQuesadilla Feb 12, 2024
f1ff596
removes ordered qualifier
ThomasQuesadilla Feb 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 28 additions & 3 deletions AGGREGATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -172,9 +172,6 @@ The scheme above will generate the following abstract histogram contributions:
value: 1664
}]
```
Note: The `filters` field will still apply to aggregatable reports, and each
dict in `aggregatable_trigger_data` can still optionally have filters applied
to it just like for event-level reports.

Note: the above scheme was used to maximize the [contribution
budget](#contribution-bounding-and-budgeting) and optimize utility in the face
Expand All @@ -185,6 +182,34 @@ true_agg_campaign_counts = raw_agg_campaign_counts / (L1 / 2)
true_agg_geo_value = 1024 * raw_agg_geo_value / (L1 / 2)
```

Note: The `filters` field will still apply to aggregatable reports, and each
dict in `aggregatable_trigger_data` can still optionally have filters applied
to it just like for event-level reports.

Note: The `aggregatable_values` field may also be specified as a list of
dictionaries, where each dictionary contains a dictionary `values` of key-value
pairs as outlined above as well as optional `filters` and `not_filters` fields,
allowing trigger registrations to customize how values are contributed to keys
depending on source filter data. If multiple list entries have filters that
match the source's filters, only the first entry and its corresponding values
will be used.

For example:
```jsonc
{
...,
"aggregatable_values": [
{
"values": {
"campaignCounts": 32768,
"geoValue": 1664
},
"filters": {"source_type": ["navigation"]}
}
]
}
```

Trigger registration will accept an optional field
`aggregatable_deduplication_keys` which will be used to deduplicate multiple
triggers containing the same `deduplication_key` for a single source with selective filtering.
Expand Down
121 changes: 90 additions & 31 deletions index.bs
Original file line number Diff line number Diff line change
Expand Up @@ -797,6 +797,20 @@ An aggregatable trigger data is a [=struct=] with the following items:

</dl>

<h3 dfn-type=dfn>Aggregatable values configuration</h3>

An aggregatable values configuration is a [=struct=] with the following items:

<dl dfn-for="aggregatable values configuration">
: <dfn>values</dfn>
:: A [=map=] whose [=map/key|keys=] are [=strings=] and whose [=map/value|values=] are non-negative 32-bit integers.
: <dfn>filters</dfn>
:: A [=list=] of [=filter configs=].
: <dfn>negated filters</dfn>
:: A [=list=] of [=filter configs=].

</dl>

<h3 dfn-type=dfn>Aggregatable dedup key</h3>

An aggregatable dedup key is a [=struct=] with the following items:
Expand Down Expand Up @@ -868,9 +882,8 @@ An attribution trigger is a [=struct=] with the following items:
:: A [=set=] of [=event-level trigger configuration=].
: <dfn>aggregatable trigger data</dfn>
:: A [=list=] of [=aggregatable trigger data=].
: <dfn>aggregatable values</dfn>
:: An [=ordered map=] whose [=map/key|keys=] are [=strings=] and whose
[=map/value|values=] are non-negative 32-bit integers.
: <dfn>aggregatable values configurations</dfn>
:: A [=list=] of [=aggregatable values configuration=].
: <dfn>aggregatable dedup keys</dfn>
:: A [=list=] of [=aggregatable dedup key=].
: <dfn>verifications</dfn>
Expand Down Expand Up @@ -1196,9 +1209,9 @@ controls the maximum [=map/size=] of an [=attribution source=]'s

<dfn>Max length per aggregation key identifier</dfn> is a positive integer that controls
the maximum [=string/length=] of an [=attribution source=]'s [=attribution source/aggregation keys=]'s
[=map/keys=], an [=attribution trigger=]'s [=attribution trigger/aggregatable values=]'s [=map/keys=],
and an [=aggregatable trigger data=]'s [=aggregatable trigger data/source keys=]'s [=set/items=].
Its value is 25.
[=map/keys=], an [=attribution trigger=]'s [=attribution trigger/aggregatable values configurations=]'s [=list/item=]'s
[=aggregatable values configuration/values=]'s [=map/keys=], and an [=aggregatable trigger data=]'s
[=aggregatable trigger data/source keys=]'s [=set/items=]. Its value is 25.

<dfn>Default trigger data cardinality</dfn> is a [=map=] that
controls the valid range of [=event-level trigger configuration/trigger data=].
Expand Down Expand Up @@ -2399,18 +2412,51 @@ To <dfn>parse aggregatable trigger data</dfn> given an [=ordered map=] |map|:
1. [=list/Append=] |aggregatableTrigger| to |aggregatableTriggerData|.
1. Return |aggregatableTriggerData|.

To <dfn>parse aggregatable key-values</dfn> given a [=map=] |map|:

1. [=map/iterate|For each=] |key| → |value| of |map|:
1. If |key|'s [=string/length=] is greater than the [=max length per aggregation key identifier=],
return null.
1. If |value| is not an integer, return null.
1. If |value| is less than or equal to 0, return null.
1. If |value| is greater than [=allowed aggregatable budget per source=], return null.
1. Return |map|.

To <dfn>parse aggregatable values</dfn> given an [=ordered map=] |map|:

1. If |map|["`aggregatable_values`"] does not [=map/exist=], return «[]».
1. If |map|["`aggregatable_values`"] does not [=map/exist=], return a new [=list/is empty|empty=] [=list=].
1. Let |values| be |map|["`aggregatable_values`"].
1. If |values| is not an [=ordered map=], return null.
1. [=map/iterate|For each=] |key| → |value| of |values|:
1. If |key|'s [=string/length=] is greater than the [=max length per aggregation key identifier=],
return null.
1. If |value| is not an integer, return null.
1. If |value| is less than or equal to 0, return null.
1. If |value| is greater than [=allowed aggregatable budget per source=], return null.
1. Return |values|.
1. If |values| is not an [=ordered map=] or a [=list=], return null.
1. Let |aggregatableValuesConfigurations| be a [=list=] of [=aggregatable values configurations=], initially empty.
1. If |values| is a [=map=]:
1. Let |aggregatableKeyValues| be the result of running [=parse aggregatable key-values=] with |values|.
1. If |aggregatableKeyValues| is null, return null.
1. Let |aggregatableValuesConfiguration| be a new [=aggregatable values configuration=] with the items:
: [=aggregatable values configuration/values=]
:: |aggregatableKeyValues|
: [=aggregatable values configuration/filters=]
:: «»
: [=aggregatable values configuration/negated filters=]
:: «»
1. [=list/Append=] |aggregatableValuesConfiguration| to |aggregatableValuesConfigurations|.
1. Return |aggregatableValuesConfigurations|.
1. [=list/iterate|For each=] |value| of |values|:
1. If |value| is not a [=map=], return null.
1. If |value|["`values`"] does not [=map/exist=], return null.
1. Let |aggregatableKeyValues| be the result of running [=parse aggregatable key-values=] with |value|["`values`"].
1. If |aggregatableKeyValues| is null, return null.
1. Let |filterPair| be the result of running [=parse a filter pair=] with
|value|.
1. If |filterPair| is null, return null.
1. Let |aggregatableValuesConfiguration| be a new [=aggregatable values configuration=] with the items:
: [=aggregatable values configuration/values=]
:: |aggregatableKeyValues|
: [=aggregatable values configuration/filters=]
:: |filterPair|[0]
: [=aggregatable values configuration/negated filters=]
:: |filterPair|[1]
1. [=list/Append=] |aggregatableValuesConfiguration| to |aggregatableValuesConfigurations|.
1. Return |aggregatableValuesConfigurations|.

To <dfn>parse aggregatable dedup keys</dfn> given an [=ordered map=] |map|:

Expand Down Expand Up @@ -2450,8 +2496,8 @@ and a [=moment=] |triggerTime|:
1. Let |aggregatableTriggerData| be the result of running [=parse aggregatable trigger data=]
with |value|.
1. If |aggregatableTriggerData| is null, return null.
1. Let |aggregatableValues| be the result of running [=parse aggregatable values=] with |value|.
1. If |aggregatableValues| is null, return null.
1. Let |aggregatableValuesConfigurations| be the result of running [=parse aggregatable values=] with |value|.
1. If |aggregatableValuesConfigurations| is null, return null.
1. Let |aggregatableDedupKeys| be the result of running [=parse aggregatable dedup keys=]
with |value|.
1. If |aggregatableDedupKeys| is null, return null.
Expand Down Expand Up @@ -2503,8 +2549,8 @@ and a [=moment=] |triggerTime|:
:: |eventTriggers|
: [=attribution trigger/aggregatable trigger data=]
:: |aggregatableTriggerData|
: [=attribution trigger/aggregatable values=]
:: |aggregatableValues|
: [=attribution trigger/aggregatable values configurations=]
:: |aggregatableValuesConfigurations|
: [=attribution trigger/aggregatable dedup keys=]
:: |aggregatableDedupKeys|
: [=attribution trigger/verifications=]
Expand Down Expand Up @@ -2649,6 +2695,21 @@ Given an [=attribution trigger=] |trigger|, an [=attribution source=]

<h3 algorithm id="creating-aggregatable-contributions">Creating aggregatable contributions</h3>

To <dfn>create [=aggregatable contributions=] from [=attribution source/aggregation keys=] and aggregatable [=aggregatable values configuration/values=]</dfn>
given a [=map=] |aggregationKeys| and a [=map=] |aggregatableValues|,
run the following steps:

1. Let |contributions| be an empty [=list=].
1. [=map/iterate|For each=] |id| → |key| of |aggregationKeys|:
1. If |aggregatableValues|[|id|] does not [=map/exist=], [=iteration/continue=].
1. Let |contribution| be a new [=aggregatable contribution=] with the items:
: [=aggregatable contribution/key=]
:: |key|
: [=aggregatable contribution/value=]
:: |aggregatableValues|[|id|]
1. [=list/Append=] |contribution| to |contributions|.
1. Return |contributions|.

To <dfn>create [=aggregatable contributions=]</dfn> given an [=attribution source=] |source| and an
[=attribution trigger=] |trigger|, run the following steps:

Expand All @@ -2663,17 +2724,15 @@ To <dfn>create [=aggregatable contributions=]</dfn> given an [=attribution sourc
1. If |aggregationKeys|[|sourceKey|] does not [=map/exist=], [=iteration/continue=].
1. [=map/Set=] |aggregationKeys|[|sourceKey|] to |aggregationKeys|[|sourceKey|] bitwise-OR |triggerData|'s
[=aggregatable trigger data/key piece=].
1. Let |aggregatableValues| be |trigger|'s [=attribution trigger/aggregatable values=].
1. Let |contributions| be a new empty [=list=].
1. [=map/iterate|For each=] |id| → |key| of |aggregationKeys|:
1. If |aggregatableValues|[|id|] does not [=map/exist=], [=iteration/continue=].
1. Let |contribution| be a new [=aggregatable contribution=] with the items:
: [=aggregatable contribution/key=]
:: |key|
: [=aggregatable contribution/value=]
:: |aggregatableValues|[|id|]
1. [=list/Append=] |contribution| to |contributions|.
1. Return |contributions|.
1. Let |aggregatableValuesConfigurations| be |trigger|'s [=attribution trigger/aggregatable values configurations=].
1. [=list/iterate|For each=] |aggregatableValuesConfiguration| of |aggregatableValuesConfigurations|:
1. If the result of running [=match an attribution source against filters and negated filters=] with
|source|, |aggregatableValuesConfiguration|'s [=aggregatable values configuration/filters=],
|aggregatableValuesConfiguration|'s [=aggregatable values configuration/negated filters=], and
|trigger|'s [=attribution trigger/trigger time=] is true:
1. Return the result of running [=create aggregatable contributions from aggregation keys and aggregatable values=] with
|aggregationKeys| and |aggregatableValuesConfiguration|'s [=aggregatable values configuration/values=].
1. Return a new [=list/is empty|empty=] [=list=].

<h3 id="can-source-create-aggregatable-contributions">Can source create aggregatable contributions</h3>

Expand Down Expand Up @@ -3019,7 +3078,7 @@ To <dfn>check if an [=attribution trigger=] contains aggregatable data</dfn> giv
run the following steps:

1. If |trigger|'s [=attribution trigger/aggregatable trigger data=] is not [=list/is empty|empty=], return true.
1. If |trigger|'s [=attribution trigger/aggregatable values=] is not [=map/is empty|empty=], return true.
1. If any of |trigger|'s [=attribution trigger/aggregatable values configurations=]'s [=aggregatable values configuration/values=] is not [=list/is empty|empty=], return true.
1. Return false.

To <dfn noexport>trigger attribution</dfn> given an [=attribution trigger=] |trigger|, run the following steps:
Expand Down
72 changes: 70 additions & 2 deletions ts/src/header-validator/trigger.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,13 @@ const testCases: jsontest.TestCase<Trigger>[] = [
sourceKeys: new Set(['x']),
},
],
aggregatableValues: new Map([['x', 5]]),
aggregatableValuesConfigurations: [
{
values: new Map([['x', 5]]),
positive: [],
negative: [],
},
],
debugKey: 5n,
debugReporting: true,
eventTriggerData: [
Expand Down Expand Up @@ -118,6 +124,27 @@ const testCases: jsontest.TestCase<Trigger>[] = [
],
}),
},
{
name: 'aggregatable-values-list-with-filters',
json: `{
"aggregatable_values": [
{
"values": {
"a": 1
},
"filters": [{"g": []}, {"h": []}],
"not_filters": [{"g": []}, {"h": []}]
},
{
"values": {
"b": 2
},
"filters": [{"i": []}, {"j": []}],
"not_filters": [{"i": []}, {"j": []}]
}
]
}`,
},
{
name: 'or-filters',
json: `{
Expand Down Expand Up @@ -320,7 +347,7 @@ const testCases: jsontest.TestCase<Trigger>[] = [
expectedErrors: [
{
path: ['aggregatable_values'],
msg: 'must be an object',
msg: 'must be an object or a list',
},
],
},
Expand Down Expand Up @@ -364,7 +391,47 @@ const testCases: jsontest.TestCase<Trigger>[] = [
},
],
},
{
name: 'aggregatable-values-list-values-field-missing',
json: `{
"aggregatable_values": [
{
"a": 1
}
]
}`,
expectedErrors: [
{
path: ['aggregatable_values', 0, 'values'],
msg: 'required',
},
],
expectedWarnings: [
{
msg: 'unknown field',
path: ['aggregatable_values', 0, 'a'],
},
],
},
{
name: 'aggregatable-values-list-wrong-type',
json: `{
"aggregatable_values": [
{
"values": []
}
]
}`,
expectedErrors: [
{
path: ['aggregatable_values', 0, 'values'],
msg: 'must be an object',
},
],
},

// TODO(apasel422): Uncomment once respective function is updated.
/*
{
name: 'inconsistent-aggregatable-keys',
json: `{
Expand Down Expand Up @@ -398,6 +465,7 @@ const testCases: jsontest.TestCase<Trigger>[] = [
},
],
},
*/

{
name: 'debug-reporting-wrong-type',
Expand Down
Loading
Loading