/
Using API Keys to Access TERN Data Services

Welcome to TERN Knowledge Base

Using API Keys to Access TERN Data Services

To create an API key, please refer to Creating API Keys.

How to use the API Key

The following sections present examples of how the API Key can be used to connect to any TERN portal to build your own applications (or scripts) to obtain the information you are interested in. (Furthermore, the examples given here are using the TERN EcoImages portal as an example portal in which a user could connect. However, the processes can be applied similarly to any TERN portal which has its API exposed to the public - see list of TERN Rest APIs here).

 

1. Using API Swagger Dashboard

For TERN portals that expose public APIs, Swagger API Dashboards that list all the available API methods are available to the user (see the list here). As an example here, we will use the TERN EcoImages portal using the EcoImages API Swagger Dashboard to list the packages the user has downloaded.

First generate the API key (as outlined in the section above, namely, How to get a TERN API Key). Then follow the following steps.

Steps

  1. Using your internet browser, navigate to the TERN EcoImages API Swagger Dashboard. (See the figure below for the landing page of the API dashboard). Click on Authorize

 

  1. In the ApiKeyAuth (apiKey) authentication section, paste your API Key in the Value field (2a). Click on Authorize button (2b). Then click on Close (2c)

 

  1. (Optional). Alternatively, you may choose the BasicAuth (http, Basic) to authenticate. In this case, paste your API Key in the Username field (3a). Click on Authorize (3b). Then click on Close (3d)

 

  1. To retrieve the packages, select the /packages GET method (4), then click on Try it out(4a), then click on Execute (4b)

  1. You can then view the results of the first 10 records returned (5).

 

The steps outlined above can be done similarly to any of the other TERN API Swagger Dashboards (see here), to obtain specific data.

 

2. Using the Curl command-line program

To use the curl tool, make sure that your curl version supports OpenSSL.
Tip: curl version 7.81.0 should work

2.1 Curl with API Key authentication type

  1. In your computer, open the terminal then type in: curl -H 'X-Api-Key: <apikey>' https://ecoimages.tern.org.au/api/whoami (e.g. curl -H 'X-Api-Key: fakebmRSFNPXSF5aiI7OjpUM1s6eiANQmgyKF8NJjRpZFJqSGMlPWlRVQlGKndoUzI4JXhkVSY/b2IL' https://ecoimages.tern.org.au/api/whoami

  2. The output should be similar to what is shown below:

 

This step shows that you are able to reach the endpoint /api/whoami in the TERN EcoImages portal and that your API key is valid (because this endpoint requires a valid API key to allow you to retrieve your details). Subsequently, this means you can also use other publicly available endpoints from the TERN portals using the supplied API key.

2.2 Curl with Basic authentication type

  1. In the terminal type: curl --user <apikey>:<password> <tern application endpoint>. Make sure that the <password> is blank! (e.g. curl --user FAKEAPIKEY123: https://ecoimages.tern.org.au/api/v1.0/packages)

  2. The figures below demonstrate how you can retrieve a list of the packages you have downloaded from the TERN EcoImages Portal using curl with a basic authentication type method. The first figure below is the curl command. The subsequent figure is the JSON result of the packages.

Curl command

(Hint: If you have installed the jq command-line tool, you can pipe the output to format the response JSON string nicely [pretty print], as the below screenshot shows!)

The JSON result of the packages

 

3. Using Wget command-line tool

Wget is another tool you can use to connect to TERN portal REST APIs to access data. The following is an example of how you can use Wget to download acoustic data from TERN (e.g. download acoustic data from the Calperum site and sensor Calperum01 for the 2013 period):

The data from this period is about 118G, so it might take a while to download - nevertheless it gives an indication of what you can download).

3.1 In the terminal type in:

wget --mirror -np -e robots=off --user='<apikey>' --password='' https://data.tern.org.au/ecoacoustics/Calperum/Calperum01/2013/

and press Enter.

e.g.

wget --mirror -np -e robots=off --user='FAKE_APIKEY_MEN0aHJtSzZFd0FCTlZNMC5yd0BJaXY/bQlxOCE+Qn' --password='' https://data.tern.org.au/ecoacoustics/Calperum/Calperum01/audio_files/2013/

 

(Make sure the password is blank)

Parameter explanation:

--mirror: instruct wget to follow links and download every link encountered on the same site
--np: don’t follow links into parent directories
--e robotos=off: wget mirroring follows instructions in the robots.txt file. this option tells wget to ignore it

 

3.2 The data will be downloaded to the folder data.tern.org.au.

4. Using ffmpeg

ffmpeg is the Swiss army knife to work with audio and video data. It also works well with our acoustic data.

The following example extracts 30 seconds of audio data (starting from a specific offset), re-samples the audio to 22kHz, and stores the result as a WAV file. The total download for this example is about 17MB out of a ~800MB file.

# setup auth for ffmpeg APIKEY="your api key here" AUTH=$(echo "apikey:$APIKEY" | base64) ffmpeg \ -ss 00:00:30 -t 00:00:30 \ -multiple_requests 1 -seekable 1 \ -headers "Authorization:  Basic $AUTH" \ -i "https://data.tern.org.au/ecoacoustics/CapeTribulation/CapeTribulation01/audio_files/2018/01/capetrib-FNQ2-DRO_20180101_034910.flac" \ -af aresample=resampler=soxr \ -ar 22000 \ test.wav

 

5. Using a Python Script

The same results can be achieved as in the above examples using a Python script (or a full-blown Python, Django, or React application). The following is an example of a small Python script to retrieve your details using the endpoint /api/whoami in the EcoImages portal:

import requests API_KEY = "fakeRnZRTC5Nfk8/bjk1SSFNPXp5KSFF8NJjRpZFJqSGMlPWlRVQlGKndoUzI4JXhkVSY/b2IL" URI = "https://ecoimages.tern.org.au/api/whoami" headers = { "X-Api-Key": API_KEY } result = requests.get(URI, headers=headers) print(result.json()) ####Results##### {'email': 'user@adelaide.edu.au', 'email_verified': True, 'family_name': 'FamilyName', 'given_name': 'User', 'id': '97e6e762-257f-4e31-8cb4-9a30d578194b', 'name': 'UserName', 'roles': ['admin'], 'scopes': ['email', 'profile']}

 

Another Python script example to retrieve a list of packages in the EcoImages portal

import requests API_KEY = "fakeTUd5QnpwRnZRTC5Nfk8/bjk1SSFNJjRpZFJqSGMlPWlRVQlGKndoUzI4JXhkVSY/b2IL" URI = "https://ecoimages.tern.org.au/api/v1.0/packages?page_size=1&page_num=1" headers = { "X-Api-Key": API_KEY } result = requests.get(URI, headers=headers) print(result.json()) ###Results###### {"current_count": 1, "packages": [ {"can_download": True, "can_mint_package": True, "can_request_download": False, "can_update_package": True, "description": "Dete again", "doi": "", "doi_requested": False, "download_file_format": "zip", "eta": -1, "expiration_date": "2022-01-26", "is_my_package": True, "online_status": "In Progress", "package_id": "41e3ef57532011ecb2aae22c8c1c3fad", "package_location": "/api/v1.0/packages/41e3ef57532011ecb2aae22c8c1c3fad", "package_status": "Complete", "progress": 100, "query": {"from": 0, "image_type": ["ancillary.general"], "page_num": 1, "page_size": 8000}, "request_date": "2021-12-02", "title": "Test again delete initialized", "zip_progress": 100, "zip_status": "Complete" } ], "total_packages": 18 }

In the above examples, we have used the query parameters page_size=1&page_num=1 to return only one package. However, in your applications, you can use other values to return more packages and paginate based on the requirements of your application. The key idea here is to demonstrate that you can build your own applications as long as you have obtained a valid API key from TERN and include that key in the header of every request you make to a TERN service.

For more EcoImages Portal public API endpoints (used in the above demonstrations), please refer to the EcoImages production API dashboard https://ecoimages.tern.org.au/api/v1.0/ui#/ . With the endpoints listed in the API Swagger Dashboard, you can build your own extensive applications.

 

(Hint: The EcoImages portal: https://ecoimages.tern.org.au was built using API endpoints in https://ecoimages.tern.org.au/api/v1.0/ui#/ . Hence, having the API Key can be a powerful tool enabling you to build your own applications similar to the EcoImages portal [or just a small, targeted python scripts for your own requirements]. - see other Swagger APIs here)

Another Python example below shows how to open a remote Cloud‑Optimised GeoTIFF (COG) hosted on data.tern.org.au using rasterio/GDAL with a TERN API key (X-Api-Key). It sets environment variables to avoid 401 and rate‑limiting issues, then connects and prints basic raster metadata such as CRS, bounds, resolution, and band count.

""" ======================================================== Read a Cloud-Optimised GeoTIFF (COG) from TERN ======================================================== What this script does: Connects to the TERN data portal (data.tern.org.au) and opens a raster file to display basic information such as coordinate system, bounds, and resolution. Known issue — 401 Unauthorised error: Some users encounter a 401 error with the standard rasterio approach. This is caused by two things: 1. Wrong authentication format: TERN requires the header "X-Api-Key: YOUR_KEY". The standard "Authorization: Bearer YOUR_KEY" format does NOT work with TERN. 2. Rate limiting: GDAL sends several rapid connection attempts when opening a remote file. The TERN server may see this as abuse and temporarily block the request. How this script fixes it: - Uses the correct X-Api-Key header format - Disables the extra HEAD request GDAL sends before the main connection (GDAL_HTTP_USE_HEAD=NO) - Skips an unnecessary directory scan (GDAL_DISABLE_READDIR_ON_OPEN=YES) - Waits 5 seconds before connecting to avoid triggering rate limits from previous attempts Requirements: pip install rasterio ======================================================== """ import time import rasterio from rasterio.env import Env # --------------------------------------------------------------- # STEP 1: Paste your TERN API key between the quotes below # Get your key at: https://account.tern.org.au # --------------------------------------------------------------- API_KEY = "paste_your_api_key_here" # --------------------------------------------------------------- # STEP 2: Paste the URL of the file you want to open # --------------------------------------------------------------- COG_URL = ( "/vsicurl/https://data.tern.org.au/xxxxx/xxxxxx.cog.tif" ) # --------------------------------------------------------------- # STEP 3: Run the script — no changes needed below this line # --------------------------------------------------------------- print("Connecting to TERN... (waiting 5 seconds)") time.sleep(5) with Env( GDAL_HTTP_HEADERS=f"X-Api-Key: {API_KEY}", GDAL_HTTP_USE_HEAD="NO", GDAL_DISABLE_READDIR_ON_OPEN="YES", ): try: with rasterio.open(COG_URL) as src: print("Success! Here is your file info:") print(f" CRS: {src.crs}") print(f" Bounds: {src.bounds}") print(f" Resolution: {src.res}") print(f" Band count: {src.count}") except rasterio.errors.RasterioIOError as e: print(f"Could not open the file: {e}") print("Please check your API key and try again.")

6. Using GDAL library and CLI tools

Any tools or libraries using GDAL can configure GDAL to use API keys.

# A small optimisation. This prevents GDAL from scanning the source directory for # GDAL side car files, which improves the time it takes to open a raster file. export GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR # setup API KEY export GDAL_HTTP_AUTH=BASIC export GDAL_HTTP_USERPWD="apikey:FAKE_APIKEY_MEN0aHJtSzZFd0FCTlZNMC5yd0BJaXY/bQlxOCE+Qn" # after this you should be able to use gdal cli tools or in fact an tool / library that # uses GDAL underneath. gdalinfo /vsicurl/https://data.tern.org.au/gov/qld/fractional_cover_v3/landsat/fractional_cover/seasonal/qld/lztmre_qld_m202403202405_dp1a2.tif

7. Setup .netrc file

Many tools and libraries support reading a .netrc which contains credentials for specific domains.

# .netrc contents to read data from data.tern.org.au machine data.tern.org.au login apikey password FAKE_APIKEY_MEN0aHJtSzZFd0FCTlZNMC5yd0BJaXY/bQlxOCE+Qn machine ecoimages.tern.org.au login apikey password FAKE_APIKEY_MEN0aHJtSzZFd0FCTlZNMC5yd0BJaXY/bQlxOCE+Qn

With this file in place tools like gdal, curl or libraries like python-requests and many more can read the credentials from this file on demand

Windows notes:

On windows this file is usually called _netrc. (Windows prefers underscore prefix).

The usual location on Windows is %USERPROFILE%\_netrc. This usually translates to something like C:\Users\Username\_netrc.Some tools require the HOME variable to be set though. e.g. setx HOME %USERPROFILE%.

 

# curl example # -n tells curl to use .netrc file curl -n https://ecoimages.tern.org.au/api/whoami # gdal example (will pick up .netrc automatically) gdalinfo /vsicurl/https://data.tern.org.au/gov/qld/fractional_cover_v3/landsat/fractional_cover/seasonal/qld/lztmre_qld_m202403202405_dp1a2.tif # same for WGET (wget will use .netrc credentials in case the first attempt without credentials returns # a 401 Unauthorized Error) wget https://data.tern.org.au/edu/curtin/soil_carbon_sequestration/C_Amax.tif

8. Small script to download whole folder from data.tern.org.au

There are multiple easy to use command line scripts to download whole folder structures from data.tern.org.au. The recommended tools are wget and lftp.

The following scripts assume that a .netrc file has been setup to handle authentication.

# using wget # -m ... mirror the site structure # -e robots=off ... ignore robots.txt file # -np ... don't follow parent links # -nH ... don't create host based folder # --cut-dirs=2 skips avoids creating `./edu/curtin/` locally. # --reject "index.html" ... skip any index html pages wget -m -e robots=off -np -nH --cut-dirs=2 --reject "index.html*" https://data.tern.org.au/edu/curtin/soil_carbon_sequestration/ ./soil_carbon_sequestration/

Same can be achieved using lftp

# username as in .netrc needs to be supplied in url # target folder (if desired) needs to set as well. lftp -c "mirror --parallel=5 -X 'index.html*' https://apikey@data.tern.org.au/edu/curtin/soil_carbon_sequestration/ ./soil_carbon_sequestration/"

 

Provide your feedback about the experience with Knowledge base