New typescript library

We are happy to announce a new typescript library for writing datahub transformation scripts with strict typing.

Start writing new typescript transforms

Read more on how to get started on the library information page

Migration from javascript

Migration to typescript requires a few additions to the syntax, but the logic will remain mostly untouched. Let's walk through the migration process for one file to illustrate both the pieces that need to be added, and also how this helps writing safer scripts.

As an example we use this javascript transform file.

function transform_entities(entities) {
    var prefix = GetNamespacePrefix("http://example.com/ex/");
    entities.forEach(function (e) {
        if (GetProperty(e, prefix, "name").substring(0, 3) === "Old") {
            SetDeleted(e, true);
        }
    });
    return entities;
}

Lets rename the file to converted.ts, and add the mandatory import statement. The function transform_entities also needs a typed signature in typescript, so we add type annotations.

import * as dh from "datahub-tslib/datahub";

export function transform_entities(entities: dh.Entity[]): dh.Entity[] {
    var prefix = GetNamespacePrefix("http://example.com/ex/");
    entities.forEach(function (e) {
        if (GetProperty(e, prefix, "name").substring(0, 3) === "Old") {
            SetDeleted(e, true);
        }
    });
    return entities;
}

The tt transpiler which comes with the library package can tell us if our file is valid. Let's run npx tt converted.ts.

The output indicates that there are still problems.

converted.ts:4:18 - error TS2304: Cannot find name 'GetNamespacePrefix'.

4     var prefix = GetNamespacePrefix("http://example.com/ex/");
                   ~~~~~~~~~~~~~~~~~~

tt's strict ruleset does point out that there is no global GetNamespacePrefix function. Which is true, we imported the datahub module under the dh alias. We see already that typescript can catch spelling mistakes for us.

So let's prefix all calls to builtin datahub functions with dh.

import * as dh from "datahub-tslib/datahub";

export function transform_entities(entities: dh.Entity[]): dh.Entity[] {
    var prefix = dh.GetNamespacePrefix("http://example.com/ex/");
    entities.forEach(function (e) {
        if (dh.GetProperty(e, prefix, "name").substring(0, 3) === "Old") {
            dh.SetDeleted(e, true);
        }
    });
    return entities;
}

Now tt reports new issues:

converted.ts:6:13 - error TS2531: Object is possibly 'null'.

6         if (dh.GetProperty(e, prefix, "name").substring(0, 3) === "Old") {
              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

converted.ts:6:47 - error TS2339: Property 'substring' does not exist on type 'PropertyValue'. Property 'substring' does not exist on type 'number'.

6         if (dh.GetProperty(e, prefix, "name").substring(0, 3) === "Old") {
                                                ~~~~~~~~~

Here the typechecking catches potential runtime bugs for us. The original transform would only work for entities that fulfill certain assumptions: Entities have a name property, and the value of name is a string.

If we encounter a differently shaped entity, the transform would fail. To be fair, this is a good pattern for many use cases: program the happy path and fail hard when exceptions occur. But this should be defined behaviour, not accidental behaviour.

Typescript now forces us to explicitly express what we want to do with entities that fall outside the expected shape. We must add null checks and type assertions. And if we still want the transform to fail, we must throw errors explicitly.

In our example here, we don't want to fail. We just want to keep entities without a name, or where the name value is not a string, unmodified.

import * as dh from "datahub-tslib/datahub";

export function transform_entities(entities: dh.Entity[]): dh.Entity[] {
    const prefix = GetNamespacePrefix("http://example.com/ex/");
    entities.forEach(function (e) {
        const name = dh.GetProperty(e, prefix, "name");
        if (name !== null && typeof name === 'string' && name.substring(0,3) ==="Old") {
            dh.SetDeleted(e, true);
        }
    });
    return entities;
}

Now we have migrated an existing javascript transformation script to typescript, and made it a little safer already along the way.