Data Hub Release - June 27 2022 - v-0.7.263-stable

The latest release can be found at: v-0.7.263-stable

Highlights

Union Dataset Source

Data hub jobs now can use union dataset source. Union dataset sources allow to consume multiple datasets sequentially. Jobs using this new source type will trace progress for all consumed datasets as well, enabling incremental runs.

Go 1.18

Data hub is now built with the latest version of go, giving a performance boost of up to 20% to it's ARM build.

Safer backups

Data hub's built-in backup manager now has an added safety feature: it will only overwrite backup locations that are either empty or written to by the same data hub instance previosly. This prevents multiple datahubs from accidently overwriting each others backups.

The full set of releases can be seen here.

To get started with the MIMIRO data hub check out the github repository readme and the user documentation. If you have questions about best practice and use of the data hub then please get in contact.