Replace essential module system with isFatal errors and bootstrap libraries

conf/config.schema.json

@@ -24,6 +24,22 @@
       "type": "string",
       "isDirectory": true,
       "default": "$ROOT/APP_DATA/temp"
+    },
+    "logLevels": {
+      "description": "Configures which log levels should be shown. Accepts generic levels, module-specific levels and not logic (e.g. 'debug', 'debug.core' and '!debug' respectively). Set to an empty array to mute all logging.",
+      "type": "array",
+      "items": { "type": "string" },
+      "default": ["error", "warn", "success", "info", "debug", "verbose"]
+    },
+    "showLogTimestamp": {
+      "description": "Whether to prepend a timestamp to each log message",
+      "type": "boolean",
+      "default": true
+    },
+    "defaultLang": {
+      "description": "The default language used by the server",
+      "type": "string",
+      "default": "en"
     }
   }
 }

conf/deprecated-lang.json

@@ -0,0 +1,4 @@
+{
+  "_module": "adapt-authoring-lang",
+  "defaultLang": "adapt-authoring-core.defaultLang"
+}

conf/deprecated-logger.json

@@ -0,0 +1,7 @@
+{
+  "_module": "adapt-authoring-logger",
+  "levels": "adapt-authoring-core.logLevels",
+  "showTimestamp": "adapt-authoring-core.showLogTimestamp",
+  "mute": null,
+  "dateFormat": null
+}

docs/configure-environment.md

@@ -0,0 +1,78 @@
+# Configuring your environment
+> For a list of all supported configuration options, see [this page](configuration).
+
+The authoring tool has been built to allow for multiple configurations for different system environments (e.g. testing, production, development).
+
+## Set up your environment
+
+To configure your tool for a specific environment, you must create a config file in `/conf` named according to the env value your system will be using (e.g. `dev.config.js`, `production.config.js`, `helloworld.config.js`). We recommend sticking to something short like `dev`, or `test`, but it's up to you what you name these; just make sure to set the environment variable to the same.
+
+> The `NODE_ENV` environment variable is used to determine the current environment, so make sure that this is set appropriately when running the application:
+
+Express.js has a number of [performance enhancing features](https://expressjs.com/en/advanced/best-practice-performance.html#set-node_env-to-production) which are only enabled when the NODE_ENV is set to `production`, so we strongly recommend you use this for your production env name.
+
+### Creating your config
+
+Each config file is a JavaScript file which exports a single object. Within this file, settings are grouped by module:
+
+```Javascript
+export default {
+  'modulename': {
+    // settings
+  }
+};
+```
+
+See [this page](configuration) for a complete list of all configuration options.
+
+####
+For convenience, we've bundled a script which will generate a new config file for you automatically.
+
+You can do this by running the following:
+```bash
+npx at-confgen [NODE_ENV]
+```
+
+> If you choose to include the default settings in your configuration, please be aware that once set, these values will not be updated if the defaults change in the future. It is advised therefore that you leave out any settings that you don't wish to change.
+
+See the [Bin scripts](binscripts#at-confgen) page for more information, included supported flags.
+
+### Setting your 'env'
+
+You can do this temporarily using the following:
+
+**Bash/Mac OS Terminal**:
+```bash
+$ NODE_ENV=dev npm start
+```
+**Windows Powershell/Command Prompt**:
+```bash
+> set NODE_ENV=dev | npm start
+```
+
+Please see the documentation for your own operating system for instructions on how to save environment variables in a more permanent way.
+
+## Using environment variables for configuration
+
+As an alternative to config files, any configuration option can be set via an environment variable using the following naming convention:
+
+```
+ADAPT_AUTHORING_<MODULE>__<property>
+```
+
+- The module name is converted to uppercase with underscores replacing hyphens (e.g. `adapt-authoring-server` becomes `ADAPT_AUTHORING_SERVER`)
+- A double underscore (`__`) separates the module name from the property name
+- The property name is kept in its original camelCase format
+
+For example:
+
+| Environment variable | Config equivalent |
+| --- | --- |
+| `ADAPT_AUTHORING_SERVER__host` | `adapt-authoring-server.host` |
+| `ADAPT_AUTHORING_SERVER__port` | `adapt-authoring-server.port` |
+| `ADAPT_AUTHORING_MONGODB__connectionUri` | `adapt-authoring-mongodb.connectionUri` |
+| `ADAPT_AUTHORING_AUTH_LOCAL__saltRounds` | `adapt-authoring-auth-local.saltRounds` |
+
+Values are parsed as JSON where possible, so non-string types like numbers and booleans can be set directly (e.g. `ADAPT_AUTHORING_SERVER__port=5678`).
+
+Any environment variables that do not start with `ADAPT_AUTHORING_` are available under the `env` namespace (e.g. `NODE_ENV` becomes `env.NODE_ENV`).

docs/error-handling.md

@@ -0,0 +1,49 @@
+# Error handling
+Handling errors correctly is a key aspect of writing stable software. This guide will give some tips on how you should deal with errors, as well as the utilities available to make error handling simpler.
+
+Before going into specifics, it would be useful to discuss application errors in general terms. The errors you experience are likely to fall into one of the following broad groups:
+- **Initialisation errors**: i.e. problems during start-up
+- **General server errors**: errors which occur outside of user requests, possibly during automated tasks
+- **User errors**: errors which are a direct result of a user request
+
+You will need to deal with each category of error differently. Below are some general tips on handling each type of error.
+
+## Initialisation errors
+Any errors which occur during initialisation should be captured and logged as appropriate. Depending on the type of error, it may or may not be considered fatal to your code.
+
+Some examples:
+- For a database-handler module, failing to connect to the database would be considered a fatal error, as no further actions can be executed. In this case, the code should perform any clean-up and exit.
+- For a configuration module, failing to load the user configuration file may not be fatal if the application can run without it (e.g. with default settings). In this case the error should be logged, but the code can continue to initialise post-error.
+- For a module which attempts to load a specific file in each module connected to the core system, failing to load a single configuration file may not be an error as such, but rather an expected outcome if the configuration file in question is not something that's required to be defined for every module. In this case, the code can continue and it may not even be necessary to log a message.
+
+## General server errors
+'General server errors' is a broad category which covers other errors that don't take place at either initialisation or as a result of direct user action. Again, depending on the specific error, these may or may not be fatal.
+
+Some examples:
+- For a database-handler module, disconnecting from the database is an expected error, and can be handled and rectified easily.
+
+## User errors
+User errors are any errors which are caused as a direct result of a user performing an action incorrectly. It is even *more* critical with user errors that the error is as specific and descriptive as possible, as the response needs to be informative and instructive to the user that caused the error. Failing to do so will result in an unpleasant user experience.
+
+Some examples:
+- A user uploads a file in an invalid format. This definitely isn't a fatal error, as the code can continue post-error. The returned error should inform the user of the issue, as well as how it can be rectified.
+
+## Defining errors
+Depending on the kinds of error that you're dealing with in your code, it may be useful to include a set of custom error definitions specific to your code.
+
+Defining useful errors is a critical part of any software system. The error registry makes it easy to define errors for your own modules, and make use of errors defined in other modules.
+
+## Catching errors
+
+## Throwing errors
+As mentioned above, it is preferable to catch errors internally in your code and re-throw these errors.
+
+The error registry acts as a central store for all errors defined in the system, and errors can be accessed and thrown from here. For convenience, the errors registry is available directly as a property of the main App instance via `app.errors`:
+
+```js
+try {
+  // do something here
+} catch(e) {
+  throw this.app.errors.MY_ERROR
+}
+```

docs/plugins/configuration.js

@@ -0,0 +1,75 @@
+import fs from 'fs'
+import path from 'path'
+
+export default class Configuration {
+  async run () {
+    const schemas = this.loadSchemas()
+    this.contents = Object.keys(schemas).sort()
+    this.manualFile = 'configuration.md'
+    this.replace = {
+      CODE_EXAMPLE: this.generateCodeExample(schemas),
+      LIST: this.generateList(schemas)
+    }
+  }
+
+  loadSchemas () {
+    const schemas = {}
+    Object.values(this.app.dependencies).forEach(c => {
+      const confDir = path.join(c.rootDir, 'conf')
+      try {
+        schemas[c.name] = JSON.parse(fs.readFileSync(path.join(confDir, 'config.schema.json')))
+      } catch (e) {}
+    })
+    return schemas
+  }
+
+  generateCodeExample (schemas) {
+    let output = '```javascript\nexport default {\n'
+    this.contents.forEach((name) => {
+      const schema = schemas[name]
+      output += `  '${name}': {\n`
+      Object.entries(schema.properties).forEach(([attr, config]) => {
+        const required = schema.required && schema.required.includes(attr)
+        if (config.description) output += `    // ${config.description}\n`
+        output += `    ${attr}: ${this.defaultToMd(config)}, // ${config.type}, ${required ? 'required' : 'optional'}\n`
+      })
+      output += '  },\n'
+    })
+    output += '};\n```'
+    return output
+  }
+
+  generateList (schemas) {
+    let output = ''
+
+    this.contents.forEach(dep => {
+      const schema = schemas[dep]
+      output += `<h3 id="${dep}" class="dep">${dep}</h3>\n\n`
+      output += '<div class="options">\n'
+      Object.entries(schema.properties).forEach(([attr, config]) => {
+        const required = schema.required && schema.required.includes(attr)
+        output += '<div class="attribute">\n'
+        output += `<div class="title"><span class="main">${attr}</span> (${config.type || ''}, ${required ? 'required' : 'optional'})</div>\n`
+        output += '<div class="inner">\n'
+        output += `<div class="description">${config.description}</div>\n`
+        if (!required) {
+          output += `<div class="default"><span class="label">Default</span>: <pre class="no-bg">${this.defaultToMd(config)}</pre></div>\n`
+        }
+        output += '</div>\n'
+        output += '</div>\n'
+      })
+      output += '</div>'
+      output += '\n\n'
+    })
+
+    return output
+  }
+
+  /**
+   * Returns a string formatted nicely for markdown
+   */
+  defaultToMd (config) {
+    const s = JSON.stringify(config.default, null, 2)
+    return s?.length < 75 ? s : s?.replaceAll('\n', '\n    ')
+  }
+}

docs/plugins/configuration.md

@@ -0,0 +1,14 @@
+# Configuration reference
+This page lists all configuration options supported by the [core bundle](coremodules) of Adapt authoring modules. For details on how to set up your configuration, including using environment variables, see [Configuring your environment](configure-environment).
+
+{{{TABLE_OF_CONTENTS}}}
+
+## Quick reference
+See below for an overview of all available configuration options.
+
+{{{CODE_EXAMPLE}}}
+
+## Complete reference
+See below for a full list of available configuration options.
+
+{{{LIST}}}

docs/plugins/errors.js

@@ -0,0 +1,22 @@
+export default class Errors {
+  async run () {
+    this.manualFile = 'errorsref.md'
+    this.contents = Object.keys(this.app.errors)
+    this.replace = { ERRORS: this.generateMd() }
+  }
+
+  generateMd () {
+    return Object.keys(this.app.errors).reduce((md, k) => {
+      const e = this.app.errors[k]
+      return `${md}\n| \`${e.code}\` | ${e.meta.description} | ${e.statusCode} | <ul>${this.dataToMd(e.meta.data)}</ul> |`
+    }, '| Error code | Description | HTTP status code | Supplemental data |\n| - | - | :-: | - |')
+  }
+
+  dataToMd (data) {
+    if (!data) return ''
+    return Object.entries(data).reduce((acc, [k, v]) => {
+      const nested = typeof v === 'object' ? this.dataToMd(v) : v
+      return `${acc}<li>\`${k}\`: ${nested}</li>`
+    }, '')
+  }
+}

docs/plugins/errorsref.md

@@ -0,0 +1,9 @@
+# Errors Reference
+
+This page documents all errors which are likely to be thrown in the system, along with the appropriate HTTP status code and any supplemental data which is stored with the error.
+
+Supplemental data can be used at the point that errors are translated to provide more context to a specific error. All data stored with an error can be assumed to be a primitive type for easy printing.
+
+{{{TABLE_OF_CONTENTS}}}
+
+{{{ERRORS}}}

errors/errors.json

@@ -1,4 +1,84 @@
 {
+  "FILE_SYNTAX_ERROR": {
+    "data": {
+      "path": "Path to the invalid file",
+      "message": "The error message"
+    },
+    "description": "File contains a syntax error",
+    "statusCode": 500
+  },
+  "DEP_ALREADY_LOADED": {
+    "data": { "module": "The module name" },
+    "description": "Module has already been loaded",
+    "statusCode": 500
+  },
+  "DEP_FAILED": {
+    "data": { "module": "The module name" },
+    "description": "Required dependency failed to load",
+    "statusCode": 500,
+    "isFatal": true
+  },
+  "DEP_INVALID_EXPORT": {
+    "data": { "module": "The module name" },
+    "description": "Module must export a class as its default export",
+    "statusCode": 500
+  },
+  "DEP_MISSING": {
+    "data": { "module": "The module name" },
+    "description": "Required module is not installed",
+    "statusCode": 500,
+    "isFatal": true
+  },
+  "DEP_NO_ONREADY": {
+    "data": { "module": "The module name" },
+    "description": "Module must define an onReady function",
+    "statusCode": 500
+  },
+  "DEP_TIMEOUT": {
+    "data": { "module": "The module name", "timeout": "The timeout in ms" },
+    "description": "Module load exceeded timeout",
+    "statusCode": 500
+  },
+  "FUNC_DISABLED": {
+    "data": {
+      "name": "The name of the function"
+    },
+    "description": "Function has been disabled",
+    "statusCode": 500
+  },
+  "FUNC_NOT_OVERRIDDEN": {
+    "data": {
+      "name": "The name of the function"
+    },
+    "description": "Function must be overridden in child class",
+    "statusCode": 500
+  },
+  "INVALID_PARAMS": {
+    "data": {
+      "params": "The invalid params"
+    },
+    "description": "Invalid parameters have been provided",
+    "statusCode": 400
+  },
+  "LOAD_ERROR": {
+    "description": "Config failed to load",
+    "statusCode": 500
+  },
+  "NOT_FOUND": {
+    "data": {
+      "id": "An identifier for the missing item",
+      "type": "Type of the missing item"
+    },
+    "description": "Requested item could not be found",
+    "statusCode": 404
+  },
+  "SERVER_ERROR": {
+    "description": "Generic server error",
+    "statusCode": 500,
+    "data": {
+      "error": "The original error"
+    }
+  },
   "SPAWN": {
     "data": {
       "cmd": "The command",
@@ -7,5 +87,12 @@
     },
     "description": "Error occurred spawning command",
     "statusCode": 500
+  },
+  "UNKNOWN_LANG": {
+    "data": {
+      "lang": "language"
+    },
+    "description": "unknown language",
+    "statusCode": 400
   }
 }