Using API Keys to Access TERN Data Services

1 How to use the API Key
- 1.1 1. Using API Swagger Dashboard
  - 1.1.1 Steps
- 1.2 2. Using the Curl command-line program
  - 1.2.1 2.1 Curl with API Key authentication type
  - 1.2.2 2.2 Curl with Basic authentication type
- 1.3 3. Using Wget command-line tool
- 1.4 4. Using ffmpeg
- 1.5 5. Using a Python Script

To create an API key, please refer to https://ternaus.atlassian.net/wiki/spaces/TERNSup/pages/2353496065.

How to use the API Key

The following sections present examples of how the API Key can be used to connect to any TERN portal to build your own applications (or scripts) to obtain the information you are interested in. (Furthermore, the examples given here are using the TERN EcoImages portal as an example portal in which a user could connect. However, the processes can be applied similarly to any TERN portal which has its API exposed to the public - see list of TERN Rest APIs here).

1. Using API Swagger Dashboard

For TERN portals that expose public APIs, Swagger API Dashboards that list all the available API methods are available to the user (see the list here). As an example here, we will use the TERN EcoImages portal using the EcoImages API Swagger Dashboard to list the packages the user has downloaded.

First generate the API key (as outlined in the section above, namely, How to get a TERN API Key). Then follow the following steps.

Steps

Using your internet browser, navigate to the TERN EcoImages API Swagger Dashboard. (See the figure below for the landing page of the API dashboard). Click on Authorize

In the ApiKeyAuth (apiKey) authentication section, paste your API Key in the Value field (2a). Click on Authorize button (2b). Then click on Close (2c)

(Optional). Alternatively, you may choose the BasicAuth (http, Basic) to authenticate. In this case, paste your API Key in the Username field (3a). Click on Authorize (3b). Then click on Close (3d)

To retrieve the packages, select the /packages GET method (4), then click on Try it out(4a), then click on Execute (4b)

You can then view the results of the first 10 records returned (5).

The steps outlined above can be done similarly to any of the other TERN API Swagger Dashboards (see here), to obtain specific data.

2. Using the Curl command-line program

To use the curl tool, make sure that your curl version supports OpenSSL.
Tip: curl version 7.81.0 should work

2.1 Curl with API Key authentication type

In your computer, open the terminal then type in: curl -H 'X-Api-Key: <apikey>' https://ecoimages.tern.org.au/api/whoami (e.g. curl -H 'X-Api-Key: fakebmRSFNPXSF5aiI7OjpUM1s6eiANQmgyKF8NJjRpZFJqSGMlPWlRVQlGKndoUzI4JXhkVSY/b2IL' https://ecoimages.tern.org.au/api/whoami
The output should be similar to what is shown below:

This step shows that you are able to reach the endpoint /api/whoami in the TERN EcoImages portal and that your API key is valid (because this endpoint requires a valid API key to allow you to retrieve your details). Subsequently, this means you can also use other publicly available endpoints from the TERN portals using the supplied API key.

2.2 Curl with Basic authentication type

In the terminal type: curl --user <apikey>:<password> <tern application endpoint>. Make sure that the <password> is blank! (e.g. curl --user FAKEAPIKEY123: https://ecoimages.tern.org.au/api/v1.0/packages)
The figures below demonstrate how you can retrieve a list of the packages you have downloaded from the TERN EcoImages Portal using curl with a basic authentication type method. The first figure below is the curl command. The subsequent figure is the JSON result of the packages.

Curl command

(Hint: If you have installed the jq command-line tool, you can pipe the output to format the response JSON string nicely [pretty print], as the below screenshot shows!)

The JSON result of the packages

3. Using Wget command-line tool

Wget is another tool you can use to connect to TERN portal REST APIs to access data. The following is an example of how you can use Wget to download acoustic data from TERN (e.g. download acoustic data from the Calperum site and sensor Calperum01 for the 2013 period):

The data from this period is about 118G, so it might take a while to download - nevertheless it gives an indication of what you can download).

3.1 In the terminal type in:

wget --mirror -np -e robots=off --user='<apikey>' --password='' https://data.tern.org.au/ecoacoustics/Calperum/Calperum01/2013/

and press Enter.

e.g.

wget --mirror -np -e robots=off --user='FAKE_APIKEY_MEN0aHJtSzZFd0FCTlZNMC5yd0BJaXY/bQlxOCE+Qn' --password='' https://data.tern.org.au/ecoacoustics/Calperum/Calperum01/audio_files/2013/

(Make sure the password is blank)

Parameter explanation:

--mirror: instruct wget to follow links and download every link encountered on the same site
--np: don’t follow links into parent directories
--e robotos=off: wget mirroring follows instructions in the robots.txt file. this option tells wget to ignore it

3.2 The data will be downloaded to the folder data.tern.org.au.

4. Using ffmpeg

ffmpeg is the Swiss army knife to work with audio and video data. It also works well with our acoustic data.

The following example extracts 30 seconds of audio data (starting from a specific offset), re-samples the audio to 22kHz, and stores the result as a WAV file. The total download for this example is about 17MB out of a ~800MB file.

# setup auth for ffmpeg 
APIKEY="your api key here"
AUTH=$(echo "apikey:$APIKEY" | base64)
ffmpeg \
  -ss 00:00:30 -t 00:00:30 \
  -multiple_requests 1 -seekable 1 \
  -headers "Authorization:  Basic $AUTH" \
  -i "https://data.tern.org.au/ecoacoustics/CapeTribulation/CapeTribulation01/audio_files/2018/01/capetrib-FNQ2-DRO_20180101_034910.flac" \
  -af aresample=resampler=soxr \
  -ar 22000 \
  test.wav

5. Using a Python Script

The same results can be achieved as in the above examples using a Python script (or a full-blown Python, Django, or React application). The following is an example of a small Python script to retrieve your details using the endpoint /api/whoami in the EcoImages portal:

import requests

API_KEY = "fakeRnZRTC5Nfk8/bjk1SSFNPXp5KSFF8NJjRpZFJqSGMlPWlRVQlGKndoUzI4JXhkVSY/b2IL"
URI = "https://ecoimages.tern.org.au/api/whoami"

headers = {
    "X-Api-Key": API_KEY
}
result = requests.get(URI, headers=headers)
print(result.json())

####Results#####
{'email': 'user@adelaide.edu.au', 'email_verified': True, 'family_name': 'FamilyName', 'given_name': 'User', 'id': '97e6e762-257f-4e31-8cb4-9a30d578194b', 'name': 'UserName', 'roles': ['admin'], 'scopes': ['email', 'profile']}

Another Python script example to retrieve a list of packages in the EcoImages portal

import requests

API_KEY = "fakeTUd5QnpwRnZRTC5Nfk8/bjk1SSFNJjRpZFJqSGMlPWlRVQlGKndoUzI4JXhkVSY/b2IL"
URI = "https://ecoimages.tern.org.au/api/v1.0/packages?page_size=1&page_num=1"

headers = {
    "X-Api-Key": API_KEY
}
result = requests.get(URI, headers=headers)
print(result.json())

###Results######
{"current_count": 1, 
    "packages": [
        {"can_download": True, 
            "can_mint_package": True, 
            "can_request_download": False, 
            "can_update_package": True, 
            "description": "Dete again", 
            "doi": "", 
            "doi_requested": False, 
            "download_file_format": "zip", 
            "eta": -1, 
            "expiration_date": "2022-01-26", 
            "is_my_package": True, 
            "online_status": "In Progress", 
            "package_id": "41e3ef57532011ecb2aae22c8c1c3fad", 
            "package_location": "/api/v1.0/packages/41e3ef57532011ecb2aae22c8c1c3fad", 
            "package_status": "Complete", 
            "progress": 100, 
            "query": {"from": 0, "image_type": ["ancillary.general"], "page_num": 1, "page_size": 8000}, 
            "request_date": "2021-12-02", 
            "title": "Test again delete initialized", 
            "zip_progress": 100, 
            "zip_status": "Complete"
            }
            ], 
            "total_packages": 18
        }

In the above examples, we have used the query parameters page_size=1&page_num=1 to return only one package. However, in your applications, you can use other values to return more packages and paginate based on the requirements of your application. The key idea here is to demonstrate that you can build your own applications as long as you have obtained a valid API key from TERN and include that key in the header of every request you make to a TERN service.

For more EcoImages Portal public API endpoints (used in the above demonstrations), please refer to the EcoImages production API dashboard https://ecoimages.tern.org.au/api/v1.0/ui#/ . With the endpoints listed in the API Swagger Dashboard, you can build your own extensive applications.

(Hint: The EcoImages portal: https://ecoimages.tern.org.au was built using API endpoints in https://ecoimages.tern.org.au/api/v1.0/ui#/ . Hence, having the API Key can be a powerful tool enabling you to build your own applications similar to the EcoImages portal [or just a small, targeted python scripts for your own requirements]. - see other Swagger APIs here)