Additional script worker functions
Overview
In Platform version 5.3, the script worker has been significantly improved with the addition of the following set of powerful new functions:
scriptInputfetchuploadToFilesvcgetApplicationConfigsetResponsePropertygetResponseProperties
Refer to the sections below for a description of each function.
Function signature and inputs
Function: scriptInput
async function myScript(scriptInput, libraries, ctx) {}
scriptInput
{
"params": {
"nsfilter": "MyNamespace",
"HTTP_METHOD": "GET"
// ...any other query/body params the caller passed in
},
"script": {
"_userType": "iaf_obj_model_api",
"_scriptName": "extractAssetById"
},
"context": { /* same object also passed to you as `ctx`, see below */ }
// ...any additional caller-supplied fields, e.g. "assetid": "123"
}
ctx
Context: The request/session context object (Ctx type from @dtplatform/platform-api). Pass this straight through as the ctx argument to any libraries PlatformApi.* / libraries.IafScriptEngine.* call that needs one.
{
"id": "3981f7f4-89c9-4985-8b1d-3098d07650ba",
"_namespaces": ["MyNamespace"],
"principalInfo": {
"_userid": "fb3d9f13-8b63-49a9-9fdf-13713b52fbbe",
"_firstName": "First",
"_lastName": "Last",
"_systemUser": false
},
// optional, present only if the caller set them:
"noAuth": false, // true = skip auth on outbound platform-api calls
"authScheme": "Bearer",
"authToken": "...",
"storage": {} // free-form session storage
}
Return value
Whatever your script function returns (or resolves to, if async) becomes the response and is wrapped as { _executionStatus: 'success', response }.
If your script throws an error, it is caught and wrapped as
{ _executionStatus: 'failure', response: <error string> } before it is returned.
Sandbox convenience functions
These three functions are specific to the script worker and are not part of any libraries. They are always in scope.
The functions are:
fetchuploadToFilesvcgetApplicationConfig
Function: fetch
fetch(url, requestOptions?, resultsWithHeaders?)
This function makes an outbound HTTP(S) request from the host process and returns the parsed response. If the URL points at a .zip file, then the host will download the file and automatically unzips the file, instead of returning a fetch response.
Parameters
| Parameters | Type | Description |
|---|---|---|
url | String, required | Absolute URL to fetch, or a URL ending in .zip or a similar zip extension to have the host download and extract it into the job scratch space. |
requestOptions | Object, optional | Standard fetch options, such as method, headers, and body. |
resultsWithHeaders | Boolean, optional | When true, returns { headers, status, statusText, url, body } instead of just the parsed body. |
Returns:
Promise<any> — parsed JSON if the response's content-type is application/json, otherwise raw text. Alternatively { headers, status, ... } envelope if resultsWithHeaders was set; or { relative: <path> } for zip URLs (the path where the zip file was extracted, relative to the job's scratch folder (usable with libraries.fs.*)).
Code example
async function myScript(scriptInput, libraries, ctx) {
// simple GET, JSON auto-parsed
const data = await fetch('https://api.example.com/widgets')
// POST with headers/body, and get status/headers back too
const res = await fetch('https://api.example.com/widgets', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ name: 'new widget' })
}, true)
console.log(res.status, res.body)
// a .zip URL is downloaded and extracted on the host; you get back
// the relative path to the extracted contents
const { relative } = await fetch('https://files.example.com/export.zip')
const files = await libraries.fs.readdirSync(relative)
return data
}
Function: uploadToFilesvc(filePath, ctx, opts?)
This function uploads a file from the job's local scratch space to the File Service, using a resumable upload.
Parameters
| Parameters | Type | Description |
|---|---|---|
filePath | String, required | Relative path to a file. |
ctx | Object, required | Used for ctx._namespaces[0] as the upload's target namespace. |
opts | Object, optional | Extra upload options (for example filename) forwarded to addFileResumable. |
Returns: Promise<Object> — the created File object.
Code example
async function myScript(scriptInput, libraries, ctx) {
await libraries.fs.writeFileSync('./report.csv', 'id,name\n1,foo\n')
const uploadedFile = await uploadToFilesvc('./report.csv', ctx)
return { fileId: uploadedFile._id }
}
Function: getApplicationConfig()
This function returns the platform-api client's current environment configuration (service origins for Passport, File, Graphics, Datasource, ObjectModel, Metrics, Notification, AI, and Workflow services).
Parameters
None.
Returns: Promise<Object>, for example { passportServiceOrigin, itemServiceOrigin, fileServiceOrigin, ... }.
Code example
async function myScript(scriptInput, libraries, ctx) {
const config = await getApplicationConfig()
console.log('File service origin:', config.fileServiceOrigin)
return config
}
Response-metadata globals
This category includes two functions which are not part of any libraries. These are raw isolated globals, used to influence the HTTP response envelope which the call ultimately sees.
Function: setResponseProperty(key, value)
This function sets a property on the response envelope. STATUS_CODE is validated to be a number in the range: 100–599. Any key is rejected if the total serialized size of all response properties would exceed the response size.
Code example
async function myScript(scriptInput, libraries, ctx) {
setResponseProperty('STATUS_CODE', 201)
setResponseProperty('X-Custom-Header', 'created')
return { id: 'abc123' }
}
Function: getResponseProperties()
This function returns everything set so far via setResponseProperty.
async function myScript(scriptInput, libraries, ctx) {
setResponseProperty('X-Trace-Id', ctx.id)
const props = await getResponseProperties()
console.log(props) // { 'X-Trace-Id': '...' }
return {}
}
Utility libraries
These are a set of bundled JavaScript libraries. See details below.
| libraries.* | Package | Notes |
|---|---|---|
_ | lodash-es | Full lodash. VERSION property is stripped. |
moment | moment-timezone | Note: exposed under the key moment, not the moment timezone, despite being the timezone-aware build. |
dayjs | dayjs | |
mingo | mingo | MongoDB-style in-memory query engine. Useful for filtering arrays with Mongo query syntax. |
ajv | ajv | JSON schema validator |
chrono | chrono-node | Natural-language date parsing |
Code example
async function myScript(scriptInput, libraries, ctx) {
const { _, moment, dayjs, mingo, ajv, chrono } = libraries
const grouped = _.groupBy(scriptInput.items, 'type')
const now = moment().tz('America/New_York').format()
const inTwoDays = dayjs().add(2, 'day').toISOString()
const matches = new mingo.Query({ status: 'active' }).find(scriptInput.items).all()
const validate = new ajv.Ajv().compile({ type: 'object', required: ['id'] })
const parsedDate = chrono.parseDate('next Friday')
return { grouped, now, inTwoDays, matches, isValid: validate({ id: 1 }) }
}
Sandboxed file system: libraries.fs
In these libraries all paths must be relative (start with ./). They are rewritten to /<jobId>/... before use. Anything that resolves outside that per-job folder, or ends in .sh, is rejected with the message "Not allowed to access this file!!".
| Function | Description |
|---|---|
createReadStream(file, options?) | Readable stream for file. |
createWriteStream(file, options?) | Writable stream for file. |
existsSync(file) | Boolean — whether file exists. |
unlink(file) | Async delete. |
unlinkSync(file) | Sync-style delete. |
mkdirsSync(dir, options?) | Create directory and any missing parents. |
readFileSync(file, options?) | Read file's contents. |
readdirSync(dir) | List a directory's contents. |
rmSync(file, options?) | Remove a file/directory (options.recursive for directories). |
writeFileSync(file, data, options?) | Write data (string/buffer) to file. |
rmdirSync(dir, options?) | Remove a directory. |
Code example
async function myScript(scriptInput, libraries, ctx) {
const { fs } = libraries
const file = './out/report.txt'
await fs.mkdirsSync('./out')
await fs.writeFileSync(file, 'hello world')
if (await fs.existsSync(file)) {
const content = await fs.readFileSync(file)
console.log(content)
}
await fs.unlinkSync(file)
return { done: true }
}
libraries.path and libraries.uuidv4
These two library functions work as follows:
-
libraries.path— Node's built-in path module, proxied method-by-method(join, resolve, dirname, basename, extname, parse, format, relative, normalize, isAbsolute, ...). -
libraries.uuidv4()— generates a random UUID v4 string.
Code example
async function myScript(scriptInput, libraries, ctx) {
const { path, uuidv4 } = libraries
const filePath = path.join('./out', `${uuidv4()}.json`)
return { filePath }
}
libraries.CoreUtils
This library acts as a wrapper around @dtplatform/core-utils.
| Function | Description |
|---|---|
CoreUtils.CSVParser.parse(content, options) | Parses CSV text into rows/objects. |
CoreUtils.GridParser.parse(grid, options) | Parses tabular "grid" data. |
CoreUtils.GridParser.setOptions(obj, options) | Merge parser options onto obj. |
CoreUtils.GridParser.extend(dest) | Extend dest with grid-parser helpers. |
CoreUtils.DataXlsx | Re-exported as-is from @dtplatform/core-utils for XLSX handling. |
CoreUtils.Util | Re-exported as-is from @dtplatform/core-utils. |
Code example
async function myScript(scriptInput, libraries, ctx) {
const { CoreUtils } = libraries
const rows = CoreUtils.CSVParser.parse('id,name\n1,foo\n2,bar', { header: true })
return { rows }
}
libraries.ModelFileReader
These libraries function as a wrapper around script worker's internal model-file handling. For example, downloading or unzipping model files, batchlet extraction, and small or large file uploads.
| Function | Description |
|---|---|
downloadAndUnzipModelFile(params, ctx) | Downloads and extracts a model archive. |
getModelBatchlet(filePath, orchRunId) | Reads a model "batchlet" from a downloaded model. |
getSgpkBatchlet(filePath, orchRunId) | Reads an .sgpk batchlet. |
uploadSmallFile(params) | Uploads a small file in one shot. |
uploadLargeFile(params) | Uploads a large file (chunked/resumable). |
calculateDataSize(name, data, orchRunId) | Computes the on-disk size of data. |
Code example
async function myScript(scriptInput, libraries, ctx) {
const { ModelFileReader } = libraries
const { path: modelPath } = await ModelFileReader.downloadAndUnzipModelFile(
{ url: scriptInput.modelUrl }, ctx
)
const batchlet = await ModelFileReader.getModelBatchlet(modelPath, scriptInput.orchRunId)
return { batchlet }
}