...
In order to control the number of fields (as ES has some limitations to mainly avoid “mapping explosion”, as well as internal limits in Lucene), this simple command count the number of fields in an index.
Using wc from mapping
curl -s -XGET https://es-test.tern.org.au/plotdata_ecoplots-data_no_nested/_mapping?pretty --header 'Authorization: Basic TOKEN' | grep type | wc -l
...
For this testing, all original indices have been transformed and merged into only 1 index, but the real data would be: 1 index per FOI and dataset.
Using _field_caps API
curl -s -XGET "https://es-test.tern.org.au/plotdata_ecoplots-data_no_nested/_field_caps?fields=instr_att*" | jq '.fields|length'
At 1/11/2021 with 1 full dataset ingested, the total number of fields is 290.