Skip to content

feat!: modernize sqsd for Node 26 #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Akhoris wants to merge 1 commit into
base: master
Choose a base branch
from
Open

feat!: modernize sqsd for Node 26 #2

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .dockerignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
node_modules/
.git/
.agents/
.codex/
.env
.dockerignore
.gitignore
Dockerfile
README.md
test/
npm-debug.log*
*.log
*.log.txt
18 changes: 8 additions & 10 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,17 +1,15 @@
FROM node:21-alpine
FROM node:26-alpine

MAINTAINER Aleksandr Popov <mogadanez@gmail.com>
LABEL maintainer="Aleksandr Popov <mogadanez@gmail.com>"

# Create sqsd directory
WORKDIR /
RUN mkdir /sqsd
WORKDIR /sqsd

# Copy sqsd source including
COPY ./ /sqsd
# Install dependencies (reproducible install from package-lock.json)
COPY package*.json ./
RUN npm ci --omit=dev

# Install dependencies
RUN npm install
# Copy sqsd source
COPY ./ /sqsd

# Run sqsd
CMD ["node", "run-cli.js"]
CMD ["node", "run-cli.js"]
16 changes: 12 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ But even more important! **We have "dockerized" it** so that you can use it as a
Following are detailed instructions of configuration and usage with and without Docker. Any changes, suggestions or Forks are welcome!

## Technologies / Environments Used
- Node.js 4+
- AWS Node SDK 2.2.26+
- Node.js 26.x
- AWS SDK for JavaScript v3 (`@aws-sdk/client-sqs`)

## Usage

Expand All @@ -34,6 +34,13 @@ Following are detailed instructions of configuration and usage with and without
#### Local development
To execute the program, clone down the repository, navigate to it with a terminal and run `npm start`

To run the local verification suite:

```bash
npm run check
npm test
```

#### As global command-line tool

To run as global command-line tool, you can either install globally from NPM or clone down the repository and run `npm link`
Expand Down Expand Up @@ -86,11 +93,12 @@ Run the program with the `--help` flag to see the full list of accepted argument
| -q, --queue-url | `SQSD_QUEUE_URL` | - | no | Your queue URL. |
| --queue-name | `SQSD_QUEUE_NAME` | - | no | The name of the queue. Fetch from queue URL if blank |
| --endpoint-url | `SQSD_ENDPOINT_URL` | - | no | Your endpoint URL if you using a fake sqs. |
| --ssl-enabled | `SQSD_SSL_ENABLED` | `true` | no | To enable ssl or not. |
| --ssl-enabled | `SQSD_SSL_ENABLED` | `true` | no | Deprecated (no-op since AWS SDK v3). SSL is now determined by the scheme (`http`/`https`) of `--endpoint-url`. |
| -m, --max-messages | `SQSD_MAX_MESSAGES_PER_REQUEST` | `10` (max: `10`) | no | Max number of messages to retrieve per request. |
| -d, --daemonized | `SQSD_RUN_DAEMONIZED` | `0` | no | Whether to continue running with empty queue (0,no,false is no, 1,yes,true is yes) |
| -s, --sleep | `SQSD_SLEEP_SECONDS` | `0` | no | Number of seconds to wait after polling empty queue when daemonized |
| --wait-time | `SQSD_WAIT_TIME_SECONDS` | `20` (max: `20`) | no | Long polling wait time when querying the queue. |
| --shutdown-timeout | `SQSD_SHUTDOWN_TIMEOUT` | `SQSD_WORKER_TIMEOUT + SQSD_WAIT_TIME_SECONDS * 1000 + 5000` | no | Max time to wait for in-flight messages during shutdown, ms. Use `0` to disable forced shutdown. |
| -w, --web-hook | `SQSD_WORKER_HTTP_URL` | - | yes | Web url address to your service. |
| --content-type | `SQSD_WORKER_HTTP_REQUEST_CONTENT_TYPE` | `application/json` | no | Message MIME Type. |
| --concurrency | `SQSD_WORKER_CONCURRENCY` | 3 | no | Number of concurrent http request to worker service |
Expand All @@ -104,7 +112,7 @@ Use this run configuration when your worker is running in another container or i

cd /your/sqsd/local/path
docker build -t someImageName .
docker run -e -e SQSD_WORKER_HTTP_URL=http://someRemoteHost/someRemotePath someImageName
docker run -e SQSD_WORKER_HTTP_URL=http://someRemoteHost/someRemotePath someImageName

**Remember that if you are running your worker on your Docker host's instance, you cannot use `localhost` as the worker host path since the `localhost` in this case will be the container's address, not your host's. Use linked containers instead**

Expand Down
98 changes: 77 additions & 21 deletions cli.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
#!/usr/bin/env node
'use strict';
const path = require('path');
const program = require('commander');
const _ = require('lodash');
const { program } = require('commander');



Expand All @@ -12,38 +11,90 @@ function increaseVerbosity(v, total) {
}
var pkg = require('./package.json');

function parseOptionalInteger(value) {
if (value === undefined || value === null || value === '') {
return undefined;
}

const normalized = String(value).trim();
if (!normalized) {
return undefined;
}

const parsed = Number(normalized);
return Number.isInteger(parsed) ? parsed : undefined;
}

function parseOptionalBoolean(value) {
if (value === undefined || value === null || value === '') {
return undefined;
}

const normalized = String(value).trim().toLowerCase();
if (normalized in { "1":1, "yes":1, "true":1 }) {
return true;
}

if (normalized in { "0":1, "no":1, "false":1 }) {
return false;
}

return undefined;
}

function mergeDefined() {
const result = {};

for (let i = 0; i < arguments.length; ++i) {
const source = arguments[i];
Object.keys(source).forEach((key) => {
if (source[key] !== undefined) {
result[key] = source[key];
}
});
}

return result;
}

program.version(pkg.version)
.option('-w, --web-hook [value]', 'The webhook url to which messages from queue be posted. Required' )
.option('-q, --queue-url [value]', 'Your queue URL.')
.option('--queue-name [value]', 'The name of the queue. Fetched from queue URL if empty')
.option('--queue-name [value]', 'The name of the queue. Fetched from queue URL if empty.')
.option('--access-key-id [value]', 'Your AWS Access Key. Leave empty if use IAM roles.')
.option('--secret-access-key [value]', 'Your AWS Secret Access Key. Leave empty if use IAM roles.')
.option('--endpoint-url [value]', 'Your endpoint Url if your using a fake service. Leave empty if using Amazon sqs.')
.option('--endpoint-url [value]', 'Endpoint URL when using a fake SQS service. Leave empty when using Amazon SQS.')
.option('-r, --region [value]', 'The region name of the AWS SQS queue')
.option('-m, --max-messages [value]', 'Max number of messages to retrieve per request.',parseInt )
.option('-m, --max-messages [value]', 'Max number of messages to retrieve per SQS request.',parseInt )
.option('-d, --daemonized ', 'Whether to continue running with empty queue' )
.option('-s, --sleep [value]', 'Number of seconds to wait after polling empty queue when daemonized', parseInt)
.option('-t, --timeout [value]', 'Timeout for waiting response from worker, ms' )
.option('--worker-health-url [value]', 'Url for checking that worker is running, useful when running in linked containers and worker needs some time to up' )
.option('--worker-health-wait-time [value]', 'Timeout for waiting while worker become health, ms', parseInt)
.option('--wait-time [value]', 'Long polling wait time when querying the queue.', parseInt)
.option('--content-type [value]', 'Long polling wait time when querying the queue.' )
.option('--concurrency [value]', 'Long polling wait time when querying the queue.', parseInt, 3 )
.option('--shutdown-timeout [value]', 'Max time to wait for in-flight messages during shutdown, ms. Use 0 to disable forced shutdown.', parseInt)
.option('--content-type [value]', 'Content-Type header sent to the worker.' )
.option('--concurrency [value]', 'Max number of concurrent worker HTTP requests.', parseInt, 3 )
.option('--user-agent [value]', 'User agent', "sqsd" )
.option('--env [value]', 'Path to .env file to load environment variables from. Optional', '.env')
.option('--ssl-enabled [value]', 'To enable ssl or not. Default is true')
.option('--ssl-enabled [value]', 'Deprecated no-op. Endpoint protocol is determined by the URL scheme.')
.option('-v, --verbose', 'A value that can be increased', increaseVerbosity, 0)


process.argv[1] = 'sqsd';
program.parse(process.argv);

// commander >=7 exposes parsed options through opts() instead of attaching
// them directly to the program instance.
var opts = program.opts();

var defaults = {
region: "us-east-1"
, maxMessages: 10
, daemonized: false
, sleep: 0
, waitTime: 20
, shutdownTimeout: undefined
, userAgent: "sqsd"
, contentType: 'application/json'
, concurrency: 3
Expand All @@ -55,33 +106,39 @@ var defaults = {

const dotenv = require('dotenv');
dotenv.config({
silent: true,
path: path.resolve(__dirname, program.env)
quiet: true,
path: path.resolve(process.cwd(), opts.env || '.env')
});

var envParams = { accessKeyId: process.env.AWS_ACCESS_KEY_ID
, secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
, sessionToken: process.env.AWS_SESSION_TOKEN
, region: process.env.SQSD_QUEUE_REGION_NAME || process.env.AWS_DEFAULT_REGION
, queueUrl: process.env.SQSD_QUEUE_URL
, maxMessages: process.env.SQSD_MAX_MESSAGES_PER_REQUEST
, daemonized: (process.env.SQSD_RUN_DAEMONIZED || "") in { "1":1, "yes":1, "true":1 }
, sleep: process.env.SQSD_SLEEP_SECONDS
, waitTime: process.env.SQSD_WAIT_TIME_SECONDS
, maxMessages: parseOptionalInteger(process.env.SQSD_MAX_MESSAGES_PER_REQUEST)
, daemonized: parseOptionalBoolean(process.env.SQSD_RUN_DAEMONIZED)
, sleep: parseOptionalInteger(process.env.SQSD_SLEEP_SECONDS)
, waitTime: parseOptionalInteger(process.env.SQSD_WAIT_TIME_SECONDS)
, shutdownTimeout: parseOptionalInteger(process.env.SQSD_SHUTDOWN_TIMEOUT)
, webHook: process.env.SQSD_WORKER_HTTP_URL
, userAgent: process.env.SQSD_WORKER_USER_AGENT
, contentType: process.env.SQSD_WORKER_HTTP_REQUEST_CONTENT_TYPE
, concurrency: process.env.SQSD_WORKER_CONCURRENCY
, timeout: process.env.SQSD_WORKER_TIMEOUT
, concurrency: parseOptionalInteger(process.env.SQSD_WORKER_CONCURRENCY)
, timeout: parseOptionalInteger(process.env.SQSD_WORKER_TIMEOUT)
, workerHealthUrl: process.env.SQSD_WORKER_HEALTH_URL
, workerHealthWaitTime: process.env.SQSD_WORKER_HEALTH_WAIT_TIME
, workerHealthWaitTime: parseOptionalInteger(process.env.SQSD_WORKER_HEALTH_WAIT_TIME)
, endpointUrl: process.env.SQSD_ENDPOINT_URL
, queueName: process.env.SQSD_QUEUE_NAME
, sslEnabled: process.env.SQSD_SSL_ENABLED
, sslEnabled: parseOptionalBoolean(process.env.SQSD_SSL_ENABLED)
}

var extractedCliArgs = _.pick(program, Object.keys(envParams));
var mergedParams = _.defaults(extractedCliArgs, envParams, defaults);
var extractedCliArgs = {};
Object.keys(envParams).forEach((key) => {
if (Object.prototype.hasOwnProperty.call(opts, key)) {
extractedCliArgs[key] = opts[key];
}
});
var mergedParams = mergeDefined(defaults, envParams, extractedCliArgs);

if (!mergedParams.webHook) {
console.log ( "--web-hook is required")
Expand All @@ -91,7 +148,6 @@ if (!mergedParams.webHook) {


const debug = require('debug')("sqsd");
const error = require('debug')('sqsd:error');
const sqsd = require('./lib/index').SQSProcessor;

console.log('SQSD v' + pkg.version);
Expand Down
Loading