Skip to main content
Version: v5.3

Additional script worker functions

Overview​

In Platform version 5.3, the script worker has been significantly improved with the addition of the following set of powerful new functions:

  • scriptInput
  • fetch
  • uploadToFilesvc
  • getApplicationConfig
  • setResponseProperty
  • getResponseProperties

Refer to the sections below for a description of each function.

Function signature and inputs​

Function: scriptInput​

async function myScript(scriptInput, libraries, ctx) {}

scriptInput

{
  "params": {
    "nsfilter": "MyNamespace",
    "HTTP_METHOD": "GET"
    // ...any other query/body params the caller passed in
  },

  "script": {
    "_userType": "iaf_obj_model_api",
    "_scriptName": "extractAssetById"
  },

  "context": { /* same object also passed to you as `ctx`, see below */ }
  // ...any additional caller-supplied fields, e.g. "assetid": "123"

}

ctx​

Context: The request/session context object (Ctx type from @dtplatform/platform-api). Pass this straight through as the ctx argument to any libraries PlatformApi.* / libraries.IafScriptEngine.* call that needs one.

{

  "id": "3981f7f4-89c9-4985-8b1d-3098d07650ba",
  "_namespaces": ["MyNamespace"],

  "principalInfo": {
    "_userid": "fb3d9f13-8b63-49a9-9fdf-13713b52fbbe",
    "_userName": "[email protected]",
    "_email": "[email protected]",
    "_firstName": "First",
    "_lastName": "Last",
    "_systemUser": false

  },

  // optional, present only if the caller set them:

  "noAuth": false,        // true = skip auth on outbound platform-api calls
  "authScheme": "Bearer",
  "authToken": "...",
  "storage": {}           // free-form session storage

}

Return value​

Whatever your script function returns (or resolves to, if async) becomes the response and is wrapped as { _executionStatus: 'success', response }.

If your script throws an error, it is caught and wrapped as { _executionStatus: 'failure', response: <error string> } before it is returned.

Sandbox convenience functions​

These three functions are specific to the script worker and are not part of any libraries. They are always in scope.

The functions are:

  • fetch
  • uploadToFilesvc
  • getApplicationConfig

Function: fetch​

fetch(url, requestOptions?, resultsWithHeaders?)

This function makes an outbound HTTP(S) request from the host process  and returns the parsed response. If the URL points at a .zip file, then the host will download the file and automatically unzips the file, instead of returning a fetch response.

Parameters​

ParametersTypeDescription
urlString, requiredAbsolute URL to fetch, or a URL ending in .zip or a similar zip extension to have the host download and extract it into the job scratch space.
requestOptionsObject, optionalStandard fetch options, such as method, headers, and body.
resultsWithHeadersBoolean, optionalWhen true, returns { headers, status, statusText, url, body } instead of just the parsed body.

Returns: Promise<any> — parsed JSON if the response's content-type is application/json, otherwise raw text. Alternatively { headers, status, ... } envelope if resultsWithHeaders was set; or { relative: <path> } for zip URLs (the path where the zip file was extracted, relative to the job's scratch folder (usable with libraries.fs.*)).

Code example​

async function myScript(scriptInput, libraries, ctx) {

    // simple GET, JSON auto-parsed
    const data = await fetch('https://api.example.com/widgets')

    // POST with headers/body, and get status/headers back too

    const res = await fetch('https://api.example.com/widgets', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ name: 'new widget' })

    }, true)

    console.log(res.status, res.body)

    // a .zip URL is downloaded and extracted on the host; you get back
    // the relative path to the extracted contents

    const { relative } = await fetch('https://files.example.com/export.zip')
    const files = await libraries.fs.readdirSync(relative)

    return data

}

Function: uploadToFilesvc(filePath, ctx, opts?)​

This function uploads a file from the job's local scratch space to the File Service, using a resumable upload.

Parameters​

ParametersTypeDescription
filePathString, requiredRelative path to a file.
ctxObject, requiredUsed for ctx._namespaces[0] as the upload's target namespace.
optsObject, optionalExtra upload options (for example filename) forwarded to addFileResumable.

Returns: Promise<Object> — the created File object.

Code example​

async function myScript(scriptInput, libraries, ctx) {

    await libraries.fs.writeFileSync('./report.csv', 'id,name\n1,foo\n')

    const uploadedFile = await uploadToFilesvc('./report.csv', ctx)

    return { fileId: uploadedFile._id }
}

Function: getApplicationConfig()​

This function returns the platform-api client's current environment configuration (service origins for Passport, File, Graphics, Datasource, ObjectModel, Metrics, Notification, AI, and Workflow services).

Parameters​

None.

Returns: Promise<Object>, for example { passportServiceOrigin, itemServiceOrigin, fileServiceOrigin, ... }.

Code example​

async function myScript(scriptInput, libraries, ctx) {

    const config = await getApplicationConfig()

    console.log('File service origin:', config.fileServiceOrigin)

    return config

}

Response-metadata globals​

This category includes two functions which are not part of any libraries. These are raw isolated globals, used to influence the HTTP response envelope which the call ultimately sees.

Function: setResponseProperty(key, value)​

This function sets a property on the response envelope. STATUS_CODE is validated to be a number in the range: 100–599. Any key is rejected if the total serialized size of all response properties would exceed the response size.

Code example​

async function myScript(scriptInput, libraries, ctx) {

    setResponseProperty('STATUS_CODE', 201)
    setResponseProperty('X-Custom-Header', 'created')

    return { id: 'abc123' }

}

Function: getResponseProperties()​

This function returns everything set so far via setResponseProperty.

async function myScript(scriptInput, libraries, ctx) {

    setResponseProperty('X-Trace-Id', ctx.id)
    const props = await getResponseProperties()
    console.log(props) // { 'X-Trace-Id': '...' }

    return {}
}

Utility libraries​

These are a set of bundled JavaScript libraries. See details below.

libraries.*PackageNotes
_lodash-esFull lodash. VERSION property is stripped.
momentmoment-timezoneNote: exposed under the key moment, not the moment timezone, despite being the timezone-aware build.
dayjsdayjs
mingomingoMongoDB-style in-memory query engine. Useful for filtering arrays with Mongo query syntax.
ajvajvJSON schema validator
chronochrono-nodeNatural-language date parsing

Code example​

async function myScript(scriptInput, libraries, ctx) {

    const { _, moment, dayjs, mingo, ajv, chrono } = libraries
    const grouped = _.groupBy(scriptInput.items, 'type')
    const now = moment().tz('America/New_York').format()
    const inTwoDays = dayjs().add(2, 'day').toISOString()
    const matches = new mingo.Query({ status: 'active' }).find(scriptInput.items).all()
    const validate = new ajv.Ajv().compile({ type: 'object', required: ['id'] })
    const parsedDate = chrono.parseDate('next Friday')

    return { grouped, now, inTwoDays, matches, isValid: validate({ id: 1 }) }
}

Sandboxed file system: libraries.fs​

In these libraries all paths must be relative (start with ./). They are rewritten to /<jobId>/... before use. Anything that resolves outside that per-job folder, or ends in .sh, is rejected with the message "Not allowed to access this file!!".

FunctionDescription
createReadStream(file, options?)Readable stream for file.
createWriteStream(file, options?)Writable stream for file.
existsSync(file)Boolean — whether file exists.
unlink(file)Async delete.
unlinkSync(file)Sync-style delete.
mkdirsSync(dir, options?)Create directory and any missing parents.
readFileSync(file, options?)Read file's contents.
readdirSync(dir)List a directory's contents.
rmSync(file, options?)Remove a file/directory (options.recursive for directories).
writeFileSync(file, data, options?)Write data (string/buffer) to file.
rmdirSync(dir, options?)Remove a directory.

Code example​

async function myScript(scriptInput, libraries, ctx) {

    const { fs } = libraries
    const file = './out/report.txt'

    await fs.mkdirsSync('./out')
    await fs.writeFileSync(file, 'hello world')

    if (await fs.existsSync(file)) {

        const content = await fs.readFileSync(file)
        console.log(content)

    }

    await fs.unlinkSync(file)
    return { done: true }

}

libraries.path and libraries.uuidv4​

These two library functions work as follows:

  • libraries.path — Node's built-in path module, proxied method-by-method (join, resolve, dirname, basename, extname, parse, format, relative, normalize, isAbsolute, ...).

  • libraries.uuidv4() — generates a random UUID v4 string. 

Code example​

async function myScript(scriptInput, libraries, ctx) {

    const { path, uuidv4 } = libraries
    const filePath = path.join('./out', `${uuidv4()}.json`)

    return { filePath }
}

libraries.CoreUtils​

This library acts as a wrapper around @dtplatform/core-utils.

FunctionDescription
CoreUtils.CSVParser.parse(content, options)Parses CSV text into rows/objects.
CoreUtils.GridParser.parse(grid, options)Parses tabular "grid" data.
CoreUtils.GridParser.setOptions(obj, options)Merge parser options onto obj.
CoreUtils.GridParser.extend(dest)Extend dest with grid-parser helpers.
CoreUtils.DataXlsxRe-exported as-is from @dtplatform/core-utils for XLSX handling.
CoreUtils.UtilRe-exported as-is from @dtplatform/core-utils.

Code example​

async function myScript(scriptInput, libraries, ctx) {

    const { CoreUtils } = libraries
    const rows = CoreUtils.CSVParser.parse('id,name\n1,foo\n2,bar', { header: true })

    return { rows }

}

libraries.ModelFileReader​

These libraries function as a wrapper around script worker's internal model-file handling. For example, downloading or unzipping model files, batchlet extraction, and small or large file uploads.

FunctionDescription
downloadAndUnzipModelFile(params, ctx)Downloads and extracts a model archive.
getModelBatchlet(filePath, orchRunId)Reads a model "batchlet" from a downloaded model.
getSgpkBatchlet(filePath, orchRunId)Reads an .sgpk batchlet.
uploadSmallFile(params)Uploads a small file in one shot.
uploadLargeFile(params)Uploads a large file (chunked/resumable).
calculateDataSize(name, data, orchRunId)Computes the on-disk size of data.

Code example​

async function myScript(scriptInput, libraries, ctx) {

    const { ModelFileReader } = libraries
    const { path: modelPath } = await ModelFileReader.downloadAndUnzipModelFile(

        { url: scriptInput.modelUrl }, ctx

    )

    const batchlet = await ModelFileReader.getModelBatchlet(modelPath, scriptInput.orchRunId)

    return { batchlet }

}