Incremental Export
Background
Incremental export allows for export of entities after a specified timestamp. This allows for synchronization capability to be built as it makes payloads lighter.
Export Options
New fetchType added to indicate incremental export. This option can be used with any matchType. When fetchType is incremental, it is necessary to specify the changeMarker option for incremental export to function, else full export will be performed.
{
"itemsToExport": [
{ "typeName": "hive_db", "uniqueAttributes": { "qualifiedName": "stocks@cl1" } }
],
"options": {
"fetchType": "incremental",
"changeMarker": 10000
}
}
Getting Change Marker
The very first call to export with fetchType set to incremental should be made with changeMarker set to 0. This will perform a full export. The AtlasExportResult will have the changeMarker set to a value. This is the value that should be used for subsequent call to Export.