rdhs fails to download data for countries outside of Africa

[skotikot]

Session Info

R version 4.2.3 (2023-03-15 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 22621)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.utf8  LC_CTYPE=English_United States.utf8    LC_MONETARY=English_United States.utf8 LC_NUMERIC=C                          
[5] LC_TIME=English_United States.utf8    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] dplyr_1.1.1         reshape_0.8.9       stringr_1.5.0       readr_2.1.4         haven_2.5.2         rnaturalearth_0.3.2 sf_1.0-12           sp_1.6-0           
[9] rdhs_0.7.6         

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.10        plyr_1.8.8         compiler_4.2.3     pillar_1.9.0       getPass_0.2-2      forcats_1.0.0      class_7.3-21       tools_4.2.3        digest_0.6.31     
[10] jsonlite_1.8.4     lifecycle_1.0.3    tibble_3.2.1       lattice_0.20-45    pkgconfig_2.0.3    rlang_1.1.0        DBI_1.1.3          cli_3.6.1          rstudioapi_0.14   
[19] curl_5.0.0         e1071_1.7-13       storr_1.2.5        httr_1.4.5         generics_0.1.3     vctrs_0.6.1        rappdirs_0.3.3     hms_1.1.3          tidyselect_1.2.0  
[28] classInt_0.4-9     grid_4.2.3         glue_1.6.2         R6_2.5.1           fansi_1.0.4        tzdb_0.3.0         magrittr_2.0.3     units_0.8-1        KernSmooth_2.23-20
[37] utf8_1.2.3         stringi_1.7.12     proxy_0.4-27       brio_1.1.3

</I have pasted my sessioninfo() above. My code works perfectly except for one thing. I have access to dhs surveys and GPS data for all countries (global), but only a subset of African countries is being successfully downloaded. I am supplying a list of country codes that includes some that are outside of Africa. For any countries out of Africa, I get the warning that "These requested datasets are not available from your DHS login credentials". Is this a problem with rdhs or with dhs data approval?>

[jeffeaton]

Hello,
Many thanks for reporting this. Could you please provide R code to reproduce the issue you are having and the R output that you are seeing from this?

Thanks,
Jeff

[skotikot]

Hi Jeff,
Thank you so much for your response. Below is my code. For you to be able to run it you would need dhs credentials, and an approved project.

The last part of the code on "questions" is the one that reproduces the warning I am getting. For your reference, here is the exact warning I am getting;

These requested datasets are not available from your DHS login credentials:

BDKR4JFL.ZIP, BDKR51FL.ZIP, BDKR61FL.ZIP, BDKR72FL.ZIP, BDKR7RFL.ZIP, KHKR42FL.zip, COKR72FL.ZIP, DRKR4BFL.ZIP, DRKR52FL.zip, DRKR61FL.ZIP, EGKR5AFL.ZIP, EGKR61FL.ZIP, GYKR5IFL.ZIP, HTKR42FL.zip, HNKR62FL.ZIP, IAKR52FL.ZIP, IAKR74FL.ZIP, IAKR7DFL.ZIP, IDKR42FL.ZIP, IDKR51FL.ZIP, IDKR63FL.ZIP, IDKR71FL.ZIP, PEKR5AFL.ZIP, PEKR5IFL.ZIP, PEKR61FL.ZIP, PEKR6AFL.ZIP, PEKR6IFL.ZIP, phkr41fl.zip, PHKR52FL.ZIP, PHKR61FL.ZIP, PHKR71FL.ZIP, TLKR61FL.ZIP, TLKR71FL.ZIP, VNKR41FL.ZIP, YEKR61FL.ZIP, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA

Please request permission for these datasets from the DHS website to be able to download them

Here is the full code:
`
library(rdhs)
library(sp)
library(sf)
library(rnaturalearth)
library(haven)
library(readr)
library(stringr)
library(reshape)
library(dplyr)

#-------------------------------------------------------------------------------------------------------------------------------------
#set up dhs credentials necessary to download the data

set_rdhs_config(email = "enter email address", # email address of account holder authorized to download dhs data
project = "enter approved project name", #approved dhs project name
password_prompt = TRUE,
config_path = "rdhs.json",
cache_path = "Downloads",
global = FALSE,
timeout = 10000)

#-------------------------------------------------------------------------------------------------------------------------------------

Variables of interest

allCodes_KR = c("hhid","caseid","hw70","hw72") #Children's file
allCodes_IR = c("hhid","caseid","v133","v414n") #Women file
allCodes_HR = c("hhid","caseid","hv270","hv271","hv226","hv244","hv246","hv243c","hv243d") # Household
allCodes_MR = c("hhid","mcaseid","mv716","mv717") # Men

allCodes <- list(allCodes_KR,allCodes_IR,allCodes_HR,allCodes_MR)

FileTypes <- c('KR', 'IR', 'HR', 'MR')

#=====================================================================================================================================

#Define countries we need data for.

dhsCountries_wMangrove <- c("AO", "BD", "BJ", "BR", "BU", "KH", "CM", "CO", "KM", "CD", "CI", "DR", "EC", "EG", "EK", "ER", "GA,", "GM", "GH", "GN", "GY",
"HT", "HN", "IA", "ID", "KK" ,"KE" ,"KY" ,"MR" ,"MX", "MB", "MZ", "MM" ,"NM" ,"NC", "NI" ,"NG", "OS", "PK", "PG", "PE", "PH",
"WS" ,"ST", "SN", "SL" ,"ZA" ,"LK", "SD", "TZ" ,"TH", "TL", "TT", "VN", "YE", "BU", "DR", "EK" ,"IA" ,"KK", "MB" ,"NM" ,"OS")

#-------------------------------------------------------------------------------------------------------------------------------------

#Search for all surveys containing data that matches our set criteria.
survs <- dhs_surveys(indicatorIds = c('CN_NUTS_C_HAM','CN_NUTS_C_WHM','CN_NUTS_C_WAM','CN_NUTS_C_WH',#children survey file
'HC_WIXQ_P_GNI','HC_CKFL_H_WOD','HC_CKFL_H_CHR','EM_OCCP_M_MNU','EM_OCCP_M_AGR','EM_OCCP_M_OTH',#hh
'HC_AGON_H_LND', 'HC_AGON_H_ANM', 'HC_TRNS_H_CRT', 'HC_TRNS_H_BTM', #hh
'CN_ANMC_C_ANY',# hh member
'ED_EDUC_W_MYR'), # Women
countryIds = dhsCountries_wMangrove,
surveyType = "DHS",
surveyYearStart=2000) #2000 to present

#-------------------------------------------------------------------------------------------------------------------------------------

i=1

#Find the datasets for download.
datasets <- dhs_datasets(surveyIds = survs$SurveyId,
fileFormat = "flat",
fileType = FileTypes[i])

questions <- search_variables(datasets$FileName, variables = allCodes[[i]], reformat=TRUE,
essential_variables = c(allCodes[[i]][3:length(allCodes[[i]])]))
`

[OJWatson]

Hi there,

Thank you for the issue and sending an update (apologies for the delay). With this, can you confirm that you have access (as far as rdhs sees it) for one of these datasets. To do this could you post the result of the following once you have set up a config:

d <- rdhs::get_available_datasets(clear_cache = TRUE)
"KHKR42FL.zip" %in% d$FileName

If you have access that should return TRUE