Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

5 add safe guards and warnings when requesting more then 10k entries #12

Conversation

e-kotov
Copy link
Owner

@e-kotov e-kotov commented Feb 1, 2025

The mi_data() function has been enhanced to perform duplicate and error checks on the data retrieved from the API. In the event of duplicates or errors, it identifies the specific variable for which insufficient filters were applied and provides informative error messages with suggestions for improvement. Examples of this functionality are provided below.

test - duplicates only in ghs_smod

library(mapineqr)

x_source <- "ghs_smod"
y_source <- "ookla"
x_filters <- list()
y_filters <- list(quarter = "4", network_type = "fixed", direction = "download")
year <- 2020
level <- "3"
limit <- 2500
xy <- mi_data(
 x_source = x_source, y_source = y_source,
 year = year, level = level,
 x_filters = x_filters, y_filters = y_filters,
 limit = limit
)
Error in `mi_data()`:
! The API returned duplicate values for some geographic regions. This may indicate that not all necessary filters were specified.

For the 'x' variable (source: 'ghs_smod'):
  The following filter fields (with multiple available options) were not specified: indicator
You can review available filters by running:
  mi_source_filters(source_name = 'ghs_smod', year = 2020, level = '3')

test - duplicates only in ookla

library(mapineqr)

x_source <- "ghs_smod"
y_source <- "ookla"
x_filters <- list(indicator = "WATER GRID CELL")
y_filters <- list(quarter = "4", direction = "download")
year <- 2020
level <- "3"
limit <- 2500
xy <- mi_data(
 x_source = x_source, y_source = y_source,
 year = year, level = level,
 x_filters = x_filters, y_filters = y_filters,
 limit = limit
)
Error in `mi_data()`:
! The API returned duplicate values for some geographic regions. This may indicate that not all necessary filters were specified.

For the 'y' variable (source: 'ookla'):
  The following filter fields (with multiple available options) were not specified: network_type
You can review available filters by running:
  mi_source_filters(source_name = 'ookla', year = 2020, level = '3')

test - duplicates in both data sources

library(mapineqr)

x_source <- "ghs_smod"
y_source <- "ookla"
x_filters <- list()
y_filters <- list(network_type = "fixed", direction = "download")
year <- 2020
level <- "3"
limit <- 2500
xy <- mi_data(
 x_source = x_source, y_source = y_source,
 year = year, level = level,
 x_filters = x_filters, y_filters = y_filters,
 limit = limit
)
Error in `mi_data()`:
! The API returned duplicate values for some geographic regions. This may indicate that not all necessary filters were specified.

For the 'x' variable (source: 'ghs_smod'):
  The following filter fields (with multiple available options) were not specified: indicator
You can review available filters by running:
  mi_source_filters(source_name = 'ghs_smod', year = 2020, level = '3')

For the 'y' variable (source: 'ookla'):
  The following filter fields (with multiple available options) were not specified: quarter
You can review available filters by running:
  mi_source_filters(source_name = 'ookla', year = 2020, level = '3')

test - multiple variables for a single indicator

library(mapineqr)

mi_data(
  x_source = "TGS00010",
  year = 2020,
  level = "2"
)
Error in `mi_data()`:
! The API returned duplicate values for some geographic regions. This may indicate that not all necessary filters were specified.

For the 'x' variable (source: 'TGS00010'):
  The following filter fields (with multiple available options) were not specified: isced11, sex
You can review available filters by running:
  mi_source_filters(source_name = 'TGS00010', year = 2020, level = '2')

@e-kotov e-kotov linked an issue Feb 1, 2025 that may be closed by this pull request
@e-kotov e-kotov merged commit 13575c8 into main Feb 1, 2025
6 checks passed
@e-kotov e-kotov deleted the 5-add-safe-guards-and-warnings-when-requesting-more-then-10k-entries branch February 1, 2025 22:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

add safe guards and warnings when requesting more then 10k entries
1 participant