EcoPlots API search queries

Data query

One of the strengths of EcoPlots is how powerful the query system is. Data are stored as atomic elements (observations) allowing to filter by 0 to many facets.

Before the user starts to define queries, it is important to understand how the underlying system has been built. Values for every search category store a URI (Uniform Resource Identifier), instead of any human-readable value. In EcoPlots case, this URI is actually a resolvable URL identifying a unique concept or instance of a class.

"http://linked.data.gov.au/dataset/ausplots" is the identifier of 1 instance of the class Dataset, in the example, TERN Surveillance Monitoring.

"http://linked.data.gov.au/def/tern-cv/b311c0d3-4a1a-4932-a39c-f5cdc1afa611"is the identifier of the class Concept, in the example, the Concept “plant occurrence” that belongs to the Concept Scheme “Feature types”.

This design offers multiple benefits (e.g., a URI is a unique and immutable identifier for a specific concept, we can easily modify the vocabulary changing the label, the description or any other property of the concept without having to update the data). The only drawback is that URIs are usually not human readable.

Query example:

"query": {
    "dataset": [
        "http://linked.data.gov.au/dataset/ausplots"
    ],
    "feature_type": [
        "http://linked.data.gov.au/def/tern-cv/b311c0d3-4a1a-4932-a39c-f5cdc1afa611"
    ],
    "observed_property": [
        "http://linked.data.gov.au/def/tern-cv/0bbd7fcd-0782-4efc-96e6-1f0f7669c655"
    ]
}

All our URIs are resolvable using the HTTP protocol, meaning that you can paste the URI into your browser and retrieve all the details of the element.

There are different ways you can discover our vocabularies (i.e., possible values in queries):

1. Inspecting the TERN vocabularies, and using the vocabulary URI.

2. Using the /discover/{field} endpoint. This endpoint returns a set of key (label) - value (uri) entries with all the possible values for every facet.

3. Using the Ecoplots Portal as a data discovery tool. Then copy the used query and use it in your API request.

The filter query must be sent in the request body (content type JSON) along with the pagination parameters, it desired.

{
    "page_number": 1, # optional
    "page_size": 50, # optional
    "query": { # mandatory, but query can be empty
        "dataset": [
            "http://linked.data.gov.au/dataset/ausplots"
        ]
    }
}

{
    "page_number": 1,
    "page_size": 50,
    "query": {}
}

{
    "query": {}
}

{
    "page_number": 1,
    "query": {}
}

{}

{
    "page_number": 1,
    "page_size": 50
}

Available facets in the query object

Notice that all values (even if it is 1 single value) must be enclosed between square brackets.

"dataset": [
    "http://linked.data.gov.au/dataset/ausplots"
]

"dataset": [
    "http://linked.data.gov.au/dataset/ausplots",
    "http://linked.data.gov.au/dataset/corveg"
]

"dataset": "http://linked.data.gov.au/dataset/ausplots"

Adding an empty list is a valid request but the response will be always empty.

"dataset": []

Region

Filter data whose site is located within a specific region.

Example 1 - Data collected in the Bioregion Cape York Peninsula

"region": [
  "http://linked.data.gov.au/dataset/bioregion/CYP"
]

Example 2 - Data collected in the Queensland state

"region": [
      "http://linked.data.gov.au/dataset/asgs2016/stateorterritory/3"
    ]

Example 3 - Data collected either in Tasmania or in Cape York Peninsula

"region": [
      "http://linked.data.gov.au/dataset/asgs2016/stateorterritory/6",
      "http://linked.data.gov.au/dataset/bioregion/CYP"
    ]

Dataset

Filter data by dataset.

Example - Data from “Tern Surveillance Monitoring” or “Williams Wet Tropics Vertebrates”

"dataset": [
  "http://linked.data.gov.au/dataset/ausplots",
  "http://linked.data.gov.au/dataset/wet-tropics-vertebrate"
]

Site

Retrieve data from 1 or more ecological monitoring sites.

Example - Retrieve all the data from Tern Surveillance Monitoring site nsabbs0001

"site_id": [
  "http://linked.data.gov.au/dataset/ausplots/site-nsabbs0001"
]

Site Visit

Retrieve data from 1 or more ecological monitoring sites.

Example - Retrieve all the data from Tern Surveillance Monitoring sitevisit 58580

"site_visit_id": [
  "http://linked.data.gov.au/dataset/ausplots/sv-58580"
]

Feature type (feature of interest)

EcoPlots observations are categorised in Feature of Interest (FOI). Please see the FOI vocabularies for more detail.

Example - Retrieve all the data from Landform feature of interest

"feature_type": [
  "http://linked.data.gov.au/def/tern-cv/2cf3ed29-440e-4a50-9bbc-5aab30df9fcd"
]

Parameter (observable property)

Each observation in EcoPlots is measuring an observable property (or parameter). Please see the Parameter vocabularies for more detail.

Example - Get all the data from Crown cover parameter.

"observed_property": [
  "http://linked.data.gov.au/def/tern-cv/83ac0278-51cd-4d1a-8e05-7c1fc68cf020"
]

Date

in addition to all the vocabularised facets, you can filter data based on a temporal resolution.

"date_from": "2006-01-01",
"date_to": "2021-07-30"

Spatial

Statial queries in EcoPlots allows three different formats: GeoJSON, Well-Known Text (WKT) and Bounding Box. The spatial query will do an “intersect” between the given geometry and the geometry in the data.

# WKT example
"spatial": "POINT(139.21370 -34.35821)"

# GEOJSON example
"spatial": {
    "type": "multipoint",
    "coordinates": [
        [
            139.213700000,
            -34.358210000
        ],
        [
            150,
            -35
        ]
    ]
}

# Bounding Box (BBOX) example
"spatial": [139.2, -34.35, 140.0, -35.00]

Pagination

The EcoPlots API allows users to paginate their data request in order to retrieve small chunks of data at a time. This is great to discover the data, to inspect the shape of the response and to adjust the desired query to narrow the result subset.

In order to control the amount of computational resources needed to paginate over millions of elements, a limit of 10,000 has been established. This means that even if the count of the query (i.e., the total amount of elements) is greater than 10,000, you will be only able to paginate over and retrieve the first 10,000.

Once the query has been defined, you can remove pagination to retrieve the full set of elements.

{
    "page_number": 1,
    "page_size": 50,
    "query": {
        "dataset": [
            "http://linked.data.gov.au/dataset/ausplots"
        ],
        "feature_type": [
            "http://linked.data.gov.au/def/tern-cv/b311c0d3-4a1a-4932-a39c-f5cdc1afa611"
        ],
        "observed_property": [
            "http://linked.data.gov.au/def/tern-cv/0bbd7fcd-0782-4efc-96e6-1f0f7669c655"
        ]
    }
}

Adding pagination to your request is as easy as include the “page_number” and “page_size” parameters to the request body.

Response headers are returned with information about the pagination:

Pagination-Page: 1
Pagination-Size: 50
Pagination-Limit: 200 # Max number of page available. Notice that it could be smaller than the real number of pages due to the 10,000 limitation.
Pagination-Count: 11827 # Total amount of elements

Request example:

curl -X POST "https://ecoplots-test.tern.org.au/api/v1.0/observations?dformat=ndjson" -H "accept: */*" -H "Authorization: apikey-v1 ZjF6eUpHaE1uRDFwNmRIUi5UcHVkQVUMVHFoYn5QcU1MeF4KVydnISwtclR3dHpZaSg7dHVFeSgocVcNLEZHQ0ozeSMiUGk/TzRHUClCU0he" -H "Content-Type: application/json" -d "{\"page_num\":1,\"page_size\":50,\"query\":{\"dataset\":[\"http://linked.data.gov.au/dataset/ausplots\"],\"feature_type\":[\"http://linked.data.gov.au/def/tern-cv/b311c0d3-4a1a-4932-a39c-f5cdc1afa611\"],\"observed_property\":[\"http://linked.data.gov.au/def/tern-cv/0bbd7fcd-0782-4efc-96e6-1f0f7669c655\"]}}"

Using Ecoplots portal to auto-generate queries

The easiest method to create valid queries to be used in the API requests is building the query visually and dynamically through the EcoPlots Portal.

You can play around with the faceted search that the portal provides and inspect the results in the data table. Once your search has been refined, you can access the query (shown below) and use it in the API request.

1. Define the filter via the facets menu.

2. Inspect the returned data. Polish your query changing the filter values dynamically and clicking in the “Show results” button.

3. Click on the API button.

4. Copy the query dictionary using the copy button and use it directly in any API request.