Create and Use API Key to Access TERN Data Services

1 Purpose
2 How to get a TERN API Key
- 2.1 Steps
3 How to use the API Key
- 3.1 1. Using API Swagger Dashboard
  - 3.1.1 Steps
- 3.2 2. Using the Curl command-line program
  - 3.2.1 1.1 Curl with API Key authentication type
  - 3.2.2 1.2 Curl with Basic authentication type
- 3.3 3. Using Wget command-line tool
- 3.4 4. Using ffmpeg
- 3.5 5. Using a Python Script

Purpose

The API key is used to connect to TERN API endpoints from the user’s custom applications (or by using simple tools such as Curl, or WGET, or any of the TERN API Swagger Dashboards) to get/download TERN data the user is interested in. All requests made from custom applications must include the API key in the Authorization section of the header of each request.

Learn about what an API is here

How to get a TERN API Key

To generate an API Key, please visit the TERN Account portal at (https://account.tern.org.au) and Sign In. After Sign In, follow the steps below (see figure 1 and figure 2):

Steps

In the menu on the left, click Create API key menu link (1)
Enter the name of the API key in the API key name field (can be arbitrary, for your records - but it's mandatory) (2)
Click the button Request API Key (3)
Copy the generated API key in the API key field (4)

Figure 1

Figure 2

After generating the API key, copy the key and store it in a secure place (Note: The API key is only displayed once in the API Key Information page [see figure 2 above] - and will not be shown or accessible after this - for security reasons). if you lose the API Key you need to generate a new key.

How to use the API Key

The following sections present examples of how the API Key can be used to connect to any TERN portal to build your own applications (or scripts) to obtain the information you are interested in. (Furthermore, the examples given here are using the TERN EcoImages portal as an example portal in which a user could connect. However, the processes can be applied similarly to any TERN portal which has its API exposed to the public - see list of TERN Rest APIs here).

1. Using API Swagger Dashboard

For TERN portals that expose public APIs, Swagger API Dashboards that list all the available API methods are available to the user (see the list here). As an example here, we will use the TERN EcoImages portal using the EcoImages API Swagger Dashboard to list the packages the user has downloaded.

First generate the API key (as outlined in the section above, namely, How to get a TERN API Key). Then follow the following steps.

Steps

Using your internet browser, navigate to the TERN EcoImages API Swagger Dashboard. (See figure 1 for the landing page of the API dashboard). Click on Authorize

Figure 3

2. In the ApiKeyAuth (apiKey) authentication section, paste your API Key in the Value field (2a). Click on Authorize button (2b). Then click on Close (2c)

Figure 4

3. (Optional). Alternatively, you may choose the BasicAuth (http, Basic) to authenticate. In this case, paste your API Key in the Username field (3a). Click on Authorize (3b). Then click on Close (3d)

Figure 5

4. To retrieve the packages, select the /packages GET method (4), then click on Try it out(4a), then click on Execute (4b)

Figure 6

5. You can then view the results of the first 10 records returned (5).

Figure 7

The steps outlined above can be done similarly to any of the other TERN API Swagger Dashboards (see here), to obtain specific data.

2. Using the Curl command-line program

To use the curl tool, make sure that your curl version supports OpenSSL.
Tip: curl version 7.81.0 should work

1.1 Curl with API Key authentication type

In your computer, open the terminal then type in: curl -H 'X-Api-Key: <apikey>' https://ecoimages.tern.org.au/api/whoami (e.g. curl -H 'X-Api-Key: fakebmRSFNPXSF5aiI7OjpUM1s6eiANQmgyKF8NJjRpZFJqSGMlPWlRVQlGKndoUzI4JXhkVSY/b2IL' https://ecoimages.tern.org.au/api/whoami
The output should be similar to what is shown in Figure 7:

Figure 7

This step shows that you are able to reach the endpoint /api/whoami in the TERN EcoImages portal and that your API key is valid (because this endpoint requires a valid API key to allow you to retrieve your details). Subsequently, this means you can also use other publicly available endpoints from the TERN portals using the supplied API key.

1.2 Curl with Basic authentication type

In the terminal type: curl --user <apikey>:<password> <tern application endpoint>. Make sure that the <password> is blank! (e.g. curl --user FAKEAPIKEY123: https://ecoimages.tern.org.au/api/v1.0/packages)
Figures 8 and 9 below demonstrate how you can retrieve a list of the packages you have downloaded from the TERN EcoImages Portal using curl with a basic authentication type method. Figure 8 is the curl command. Figure 9 is the JSON result of the packages.

Figure 8

(Hint: If you have installed the jq command-line tool, you can pipe the output to format the response JSON string nicely [pretty print], as Figure 8 below shows!)

Figure 9

3. Using Wget command-line tool

Wget is another tool you can use to connect to TERN portal REST APIs to access data. The following is an example of how you can use Wget to download acoustic data from TERN (e.g. download acoustic data from the Calperum site and sensor Calperum01 for the 2013 period):

3.1 In the terminal type in:

wget --mirror -np -e robots=off --user='<apikey>' --password='' https://data.tern.org.au/ecoacoustics/Calperum/Calperum01/2013/

and press Enter.

e.g.

wget --mirror -np -e robots=off --user='FAKE_APIKEY_MEN0aHJtSzZFd0FCTlZNMC5yd0BJaXY/bQlxOCE+Qn' --password='' https://data.tern.org.au/ecoacoustics/Calperum/Calperum01/audio_files/2013/

(Make sure the password is blank)

Parameter explanation:

--mirror: instruct wget to follow links and download every link encountered on the same site
--np: don’t follow links into parent directories
--e robotos=off: wget mirroring follows instructions in the robots.txt file. this option tells wget to ignore it

3.2 The data will be downloaded to the folder data.tern.org.au.

4. Using ffmpeg

ffmpeg is the Swiss army knife to work with audio and video data. It also works well with our acoustic data.

The following example extracts 30 seconds of audio data (starting from a specific offset), re-samples the audio to 22kHz, and stores the result as a WAV file. The total download for this example is about 17MB out of a ~800MB file.

# setup auth for ffmpeg 
APIKEY="your api key here"
AUTH=$(echo "apikey:$APIKEY" | base64)
ffmpeg \
  -ss 00:00:30 -t 00:00:30 \
  -multiple_requests 1 -seekable 1 \
  -headers "Authorization:  Basic $AUTH" \
  -i "https://data.tern.org.au/ecoacoustics/CapeTribulation/CapeTribulation01/audio_files/2018/01/capetrib-FNQ2-DRO_20180101_034910.flac" \
  -af aresample=resampler=soxr \
  -ar 22000 \
  test.wav

5. Using a Python Script

The same results can be achieved as in the above examples using a Python script (or a full-blown Python, Django, or React application). The following is an example of a small Python script to retrieve your details using the endpoint /api/whoami in the EcoImages portal:

import requests

API_KEY = "fakeRnZRTC5Nfk8/bjk1SSFNPXp5KSFF8NJjRpZFJqSGMlPWlRVQlGKndoUzI4JXhkVSY/b2IL"
URI = "https://ecoimages.tern.org.au/api/whoami"

headers = {
    "X-Api-Key": API_KEY
}
result = requests.get(URI, headers=headers)
print(result.json())

####Results#####
{'email': 'user@adelaide.edu.au', 'email_verified': True, 'family_name': 'FamilyName', 'given_name': 'User', 'id': '97e6e762-257f-4e31-8cb4-9a30d578194b', 'name': 'UserName', 'roles': ['admin'], 'scopes': ['email', 'profile']}

Another Python script example to retrieve a list of packages in the EcoImages portal

import requests

API_KEY = "fakeTUd5QnpwRnZRTC5Nfk8/bjk1SSFNJjRpZFJqSGMlPWlRVQlGKndoUzI4JXhkVSY/b2IL"
URI = "https://ecoimages.tern.org.au/api/v1.0/packages?page_size=1&page_num=1"

headers = {
    "X-Api-Key": API_KEY
}
result = requests.get(URI, headers=headers)
print(result.json())

###Results######
{"current_count": 1, 
    "packages": [
        {"can_download": True, 
            "can_mint_package": True, 
            "can_request_download": False, 
            "can_update_package": True, 
            "description": "Dete again", 
            "doi": "", 
            "doi_requested": False, 
            "download_file_format": "zip", 
            "eta": -1, 
            "expiration_date": "2022-01-26", 
            "is_my_package": True, 
            "online_status": "In Progress", 
            "package_id": "41e3ef57532011ecb2aae22c8c1c3fad", 
            "package_location": "/api/v1.0/packages/41e3ef57532011ecb2aae22c8c1c3fad", 
            "package_status": "Complete", 
            "progress": 100, 
            "query": {"from": 0, "image_type": ["ancillary.general"], "page_num": 1, "page_size": 8000}, 
            "request_date": "2021-12-02", 
            "title": "Test again delete initialized", 
            "zip_progress": 100, 
            "zip_status": "Complete"
            }
            ], 
            "total_packages": 18
        }

For more EcoImages Portal public API endpoints (used in the above demonstrations), please refer to the EcoImages production API dashboard https://ecoimages.tern.org.au/api/v1.0/ui#/. With the endpoints listed in the API Swagger Dashboard, you can build your own extensive applications.