Adapter Setup & Execution

How to verify the adapter configuration settings and run the local adapter.

Adding the Schema File

Prior to running the adapter for the first time place the agency specific schema mapping file in the ./deposit folder located in the root directory. All schema files, similar to CSV files MUST be added to the ./deposit folder in the root directory.

cp ~/file_location/fusd_02_10_2020.json ~/adapter-location/deposit

For first time use, if the ./deposit folder does not yet exist. Run the script below and then kill the terminal session.

The ./deposit and ./results directory will be auto-generated after the following script has been executed for the first time (e.g. running the executable).

~/root_directory_location

./fresno-adapter

The ./deposit and ./results folders will persist after you kill the executable or shell session.

Verifying the Configuration

Prior to running the executable enter the root directory and open the ./Settings.toml to verify the project settings are configured properly.

~/root_directory

cat Settings.toml

The ./Settings.toml file should display the following information:

~/root_directory/settings_example.toml

attest_api_url = "https://sandbox.attest.id/v1"
deposit_directory = "./deposit"
results_directory = "./results"
api_key = "NJLPtNi5Ybm6N0BnxUYktLWH1yXTfOmmFuAhiPvteYpY9lzDy-kSvSz5sKOyY6YHxpQhZJo4MDCHbDsO2kfni0dC1JEOYGYEL_PajPA_cfhU8Oe1nUWiSftkri6nEeb-RP8lbDYwBqpHXXYj4lv1DjWXsSEPI60up0lSY3ohMu9VP-bQs0nul4iZBKntFezBXXr6PpHE38z6kAluC7al0Xk9AHvyhUmS-zpA9o4CewWrI8m5ejLThPejSeTO6RH4WVozFt6KwUAQuWubPJpC5C4KHdOS5wIe49ADEv1xR5ME2A9N.a2V5LWlkOmdhbmRhbGYw"

In this file, verify that the URL on the first line matches the following:

...
attest_api_url = "https://sandbox.attest.id/v1"
...

If the URL in the file does not match"https://sandbox.attest.id/v1", edit the file and change the URL to the one listed above. Make sure to only change the URL, do not change the variable in front of it, attest_api_url.

Running the Adapter

Running the adapter is simple and only requires entering the root directory and executing the following script:

~/root_directory

./fresno-adapter

Running this script will start the executable locally. If started successfully, the terminal should display the following start screen:

:-------------------------------------------------------------:
:.               Fresno Identity Resolution                  .:
:-------------------------------------------------------------:
: -> Agency ed25519 Public Key:                               :
:  atl0FG2JkPGhbvdyKgq1X1DjIAF3N308LalQwHBaUlg                :
:-------------------------------------------------------------:
:= Waiting for file to be dropped in the directory: "./deposit"

The adapter is now running locally and waiting for an import file to be dropped in the ./deposit folder

Depositing the Import File

Place the import file in the ./deposit folder located in the root directory:

cp ~/file_location/fusd_02_10_2020.csv ~/adapter-location/deposit

After depositing the import file the resolution adapter will read, cleanse and parse the records in the CSV import file.

Reading 4943 records for Identity Resolution:
██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ 4943/4943

After processing the records locally, the adapter will send records to the hosted resolution API in batches of 1,000:

Reading 4943 records for Identity Resolution:
██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ 4943/4943
Resolving 4943 records in batches of 1000:
░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 0/4943

Records will be matched/resolved against existing records in the database until the entire import file is fully resolved:

Reading 4943 records for Identity Resolution:
██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ 4943/4943
Resolving 4943 records in batches of 1000:
██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ 4943/4943
-> Completed identity resolution, check the results directory for files.

Implementors can upload additional import files to the local adapter or choose to kill the terminal session and executable running.

Batching records

For testing purposes, DO NOT load data sets in groups larger than 25,000-50,000 records.

The API and adapter can handle much more throughput, however, 300K+ records will take a fair bit of time to process and may be susceptible to a local network timeout.

Deposit Errors

1. Missing Schema

Depositing a CSV without a corresponding schema file will return the following error:

Issue processing file: The schema file "fusd_02_10_2020.json" for "fusd_02_10_2020.csv" does not exist, please ensure it exists before depositing the CSV.

2. Mismatched File Names

As noted in the naming conventions section a mismatch in the schema and import file name will return the following error:

Issue processing file: The schema file "fph_02_10_2020.json" for "fph_02_10_2020.csv" does not exist, please ensure it exists before depositing the CSV.

3. Stopping the service while running:

Stopping the adapter while running, will not impact the results of the identity resolution process. Each resolution output file will contain a timestamp at the end of the file name.